Intelligent glasses control method and device, intelligent glasses and storage medium

Abstract

The invention discloses a control method and device of intelligent glasses, the intelligent glasses and a storage medium, the method is applied to the intelligent glasses, the intelligent glasses comprise a camera module, a driving mechanism and a motion sensor, the driving mechanism is used for adjusting the posture of the camera module, and the motion sensor is used for monitoring head posture data of a user; the method comprises the following steps: determining a photographing compensation angle of a camera module, wherein the photographing compensation angle is determined according to one of head posture data and an adjustment instruction sent by a control terminal; and controlling the driving mechanism to adjust the posture of the camera module according to the photographing compensation angle. By means of the mode, the camera shooting visual angle deviation caused by the head movement in the photographing picture previewing process of the user can be effectively counteracted, it is ensured that the camera shooting module always keeps the stable view finding angle, and therefore real-time view finding of the non-screen photographing glasses in the previewing stage is achieved; and the shooting experience, the shooting precision and the shooting efficiency of the user during use are improved.

Description

Technical Field

[0001] The present application relates to the technical field of wearable devices, and in particular to a control method and device for smart glasses, smart glasses, and a storage medium. Background Art

[0002] Currently, screenless camera glasses lack a real-time framing function, so users cannot intuitively obtain the framing situation during the shooting process, resulting in a poor shooting experience. Although there are solutions that use a supporting control terminal (such as a smart watch) as a framing preview terminal, the existing camera glasses will have the problem of changing the camera angle during the preview process, and cannot achieve true real-time framing. Summary of the Invention

[0003] The main purpose of this application is to provide a control method and device for smart glasses, smart glasses and storage medium, aiming to solve the technical problem of the prior art that the camera angle of screenless camera glasses changes during the preview of the photo.

[0004] To achieve the above objectives, the present application proposes a control method for smart glasses, the method being applied to the smart glasses, the smart glasses comprising a camera module, a drive mechanism, and a motion sensor, the drive mechanism being used to adjust the posture of the camera module, and the motion sensor being used to monitor the user's head posture data; The method comprises: determining a photographing compensation angle of the camera module, wherein the photographing compensation angle is determined according to one of the head posture data and an adjustment instruction sent by the control terminal; The driving mechanism is controlled to adjust the posture of the camera module according to the shooting compensation angle.

[0005] In one embodiment, the step of determining the shooting compensation angle of the camera module includes: Obtain the reference head pose corresponding to the target photographic field of view; Calculating a posture deviation of a pitch angle according to the reference head posture and the head posture data to determine a head posture deviation; When the absolute value of the head posture deviation is greater than the deviation threshold and the duration of the deviation is greater than the preset duration, the shooting compensation angle of the camera module is determined according to the head posture deviation.

[0006] In one embodiment, before the step of obtaining a reference head posture corresponding to the target photographing field of view, the method further includes: Upon receiving the target locking instruction sent by the control terminal, parsing the target locking instruction and determining the instruction generation time of the target locking instruction; Obtaining the target head posture collected by the motion sensor at the time of generating the instruction; The target head posture is used as the reference head posture corresponding to the target photographing field of view.

[0007] In one embodiment, the step of determining the shooting compensation angle of the camera module includes: Upon receiving an adjustment instruction sent by the control terminal, determining an adjustment operation distance according to the adjustment instruction; Determining an adjustment angle of the camera module in a pitch angle dimension by calculating according to the adjustment operation distance and a target mapping function, wherein the target mapping function includes at least an adjustment mapping coefficient; The shooting compensation angle of the camera module is determined according to the adjustment angle corresponding to the pitch angle dimension of the camera module.

[0008] In one embodiment, before the step of determining the adjustment operating distance according to the adjustment instruction when the adjustment instruction sent by the control terminal is received, the method further includes: When the control terminal enters the preview mode, obtaining the initial image captured by the camera module; compressing the initial image according to the target resolution to generate a preview compressed image corresponding to the initial image; The preview compressed image is sent to the control terminal for display, so that the control terminal generates an adjustment instruction and feeds back the adjustment instruction according to the user's adjustment operation distance.

[0009] In one embodiment, after the step of controlling the driving mechanism to adjust the posture of the camera module according to the shooting compensation angle, the method further includes: When a photographing trigger instruction sent by the control terminal is detected, the camera module is controlled to capture the fine-tuned image according to the target photographing parameters; When image acquisition is completed and a preview mode closing signal sent by the control terminal is received, the driving mechanism is controlled to be closed, and the posture of the camera module is controlled to be restored to an initial state.

[0010] In one embodiment, before the step of determining the shooting compensation angle of the camera module, the method further includes: Upon receiving a communication connection request sent by a control terminal, performing a whitelist check according to the terminal identification of the control terminal to obtain a pairing check result of the control terminal; Performing a validity check on the communication connection request to obtain a validity check result of the communication connection request; When both the pairing verification result and the legitimacy verification result are verified to be passed, respond to the communication connection request and establish a communication connection with the control terminal.

[0011] In addition, to achieve the above-mentioned purpose, the present application also proposes a control device for smart glasses, the control device for smart glasses comprising: a processing module, configured to determine a photographing compensation angle of the camera module, wherein the photographing compensation angle is determined based on one of the head posture data and an adjustment instruction sent by the control terminal; The control module is used to control the driving mechanism to adjust the posture of the camera module according to the shooting compensation angle.

[0012] In addition, to achieve the above-mentioned purpose, the present application also proposes a pair of smart glasses, which include: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the control method of the smart glasses as described above.

[0013] In addition, to achieve the above-mentioned purpose, the present application also proposes a storage medium, which is a computer-readable storage medium. A computer program is stored on the storage medium, and when the computer program is executed by the processor, the steps of the control method of the smart glasses as described above are implemented.

[0014] In addition, to achieve the above-mentioned purpose, the present application also provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, it implements the steps of the control method of the smart glasses as described above.

[0015] The method of the present application is applied to smart glasses, which include a camera module, a driving mechanism and a motion sensor, wherein the driving mechanism is used to adjust the posture of the camera module, and the motion sensor is used to monitor the user's head posture data; the method includes: determining the shooting compensation angle of the camera module, wherein the shooting compensation angle is determined according to the head posture data and one of the adjustment instructions sent by the control terminal; and controlling the driving mechanism to adjust the posture of the camera module according to the shooting compensation angle. In the above manner, by monitoring the head posture data or using the adjustment instruction to determine the shooting compensation angle, and controlling the driving mechanism to adjust the posture of the camera module, it is possible to effectively offset the camera angle deviation caused by the head movement of the user during the preview of the shooting picture, and ensure that the camera module always maintains a stable framing angle, thereby realizing real-time framing of the screenless camera glasses in the preview stage, and improving the shooting experience, shooting accuracy and shooting efficiency of the user when using it. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0017] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0018] Figure 1 A flowchart of the first embodiment of the control method of the smart glasses of the present application is provided; Figure 2 This is a schematic diagram of the overall structure of the smart glasses provided in Example 1 of this application; Figure 3 This is a schematic diagram of the partial structure of the smart glasses provided in Example 1 of this application; Figure 4 A flowchart of the second embodiment of the control method of the smart glasses of the present application is provided; Figure 5 This is a schematic diagram of the interaction process provided for Example 2 of this application; Figure 6 A flowchart of the third embodiment of the control method of the smart glasses of the present application is provided; Figure 7 This is a schematic diagram of the compensation process provided for Example 3 of this application; Figure 8 This is a schematic diagram of the module structure of the control device of the smart glasses according to an embodiment of the present application; Figure 9 Schematic diagram of the device structure of the hardware operating environment involved in the control method of smart glasses in an embodiment of the present application; Description of Figure Numbers: FPC cable 1, CMOS chip 2, lens barrel and lens 3, IMU chip 4, PCB 5, PCB bracket 6; Transmission gear 7, servo motor 8.

[0019] The purpose, features and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0020] It should be understood that the specific embodiments described herein are merely used to explain the technical solutions of the present application and are not intended to limit the present application.

[0021] In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.

[0022] The main solution of the embodiment of the present application is: determining the shooting compensation angle of the camera module, and the shooting compensation angle is determined according to the head posture data and one of the adjustment instructions sent by the control terminal; controlling the driving mechanism to adjust the posture of the camera module according to the shooting compensation angle.

[0023] Currently, screenless camera glasses lack a real-time framing function, so users cannot intuitively obtain the framing situation during the shooting process, resulting in a poor shooting experience. Although there are solutions that use a supporting control terminal (such as a smart watch) as a framing preview terminal, the existing camera glasses will have the problem of changing camera angle during the preview process.

[0024] This application provides a solution that monitors head posture data or uses adjustment instructions to determine the shooting compensation angle, and controls the driving mechanism to adjust the posture of the camera module. It can effectively offset the camera angle deviation caused by the user's head movement during the preview of the photo, ensuring that the camera module always maintains a stable framing angle, thereby realizing real-time framing of screenless camera glasses during the preview stage, and improving the user's shooting experience, shooting accuracy and shooting efficiency.

[0025] It should be noted that the execution subject of this embodiment can be a computing service device with data processing, network communication, and program execution functions, such as a tablet computer, personal computer, mobile phone, etc., or a pair of smart glasses capable of performing the above functions. The following uses smart glasses as an example to illustrate this embodiment and the following embodiments.

[0026] Based on this, the embodiment of the present application provides a control method for smart glasses, referring to Figure 1 , Figure 1 This is a flow chart of the first embodiment of the control method of the smart glasses of the present application.

[0027] In this embodiment, the method is applied to smart glasses, which include a camera module, a drive mechanism, and a motion sensor. The drive mechanism is used to adjust the posture of the camera module, and the motion sensor is used to monitor the user's head posture data. The control method of the smart glasses includes steps S10 to S20: Step S10: determining a photographing compensation angle of the camera module, wherein the photographing compensation angle is determined according to one of the head posture data and an adjustment instruction sent by the control terminal.

[0028] It should be noted that the smart glasses in this embodiment refer to screenless glasses that can be worn directly by the user and capable of taking photos. Smart glasses include but are not limited to a camera module, a drive mechanism, a motion sensor, and the glasses themselves. The camera module is the core component used to capture images or videos and includes components such as a lens and a lens barrel. Its posture (e.g., shooting angle, pitch angle, and rotation angle) directly determines the framing range and shooting angle. The drive mechanism is the actuator used to physically adjust the camera module's posture. It can drive the camera module in different directions based on control signals, thereby changing its shooting angle. The motion sensor is used to sense changes in the user's head motion state, such as rotation, tilt, and displacement, and converts the corresponding posture information into quantified head posture data. In this embodiment, an IMU (Inertial Measurement Unit) can be used as the motion sensor, but other sensors with similar functions can also be used, and this embodiment is not limited to this.

[0029] It can be understood that in this embodiment, the driving mechanism, the camera module and the motion sensor are all located at the same side of the temple of the smart glasses, and the camera module is adjacent to the driving mechanism. The overall structure of the smart glasses is as follows: Figure 2 As shown, A is a module including a driving mechanism, a camera module and a motion sensor. The specific structure of module A including the positional relationship between the three is as follows: Figure 3 As shown, the driving mechanism specifically includes a servo motor 8 and a transmission gear 7; the camera module specifically includes a CMOS (Complementary Metal-Oxide-Semiconductor) chip 2, a PCB (Printed Circuit Board) 5, a PCB bracket 6, an FPC (Flexible Printed Circuit) flexible cable 1, a lens and a lens barrel 3, and the motion sensor is the IMU chip 4 in the figure.

[0030] In a specific implementation, head posture data is real-time head movement information collected by motion sensors, including but not limited to the pitch angle, heading angle, and roll angle of the user's head. The pitch angle is the angle of rotation of the user's head around the horizontal axis, used to describe the movement of raising or lowering the head. In this embodiment, the user's pitch angle is the focus.

[0031] It is understood that the control terminal is an independent device that establishes a communication connection with the smart glasses. It has the function of previewing the images captured by the smart glasses in real time and can realize remote control of the smart glasses' photo operation. The control terminal can be a wearable device including but not limited to a smart watch, or other devices can be used, and this embodiment is not limited to this.

[0032] In a specific implementation, the user can actively initiate an adjustment instruction through the interactive interface of the control terminal or the physical controls on the control terminal. After the adjustment instruction is transmitted to the smart glasses via the communication link, the smart glasses adjust the posture of the camera module based on the adjustment instruction.

[0033] It should be noted that when the control terminal enters the preview mode, it will send a preview mode activation signal to the smart glasses. When the smart glasses receive the activation signal, they turn on the preview function of the smart watch, activate the driving mechanism, and enter step S10.

[0034] It is understandable that after the driving mechanism of the smart glasses is activated, there are two ways of automatic compensation and manual compensation during the photo-taking process. The specific process of automatic compensation is: when the user enters the photo preview mode through the control terminal, the user is in the process of previewing the photo through the control terminal, or due to other factors, the actual framing direction of the camera module will be offset relative to the initial locked target. In order to maintain the stability of the framing, the user's head posture data is monitored in real time through the motion sensor, and the head posture data is compared with the reference posture corresponding to the locked target photo field of view, and the corresponding compensation angle is determined based on the comparison result, and the posture of the camera module is adjusted based on the corresponding compensation angle.

[0035] In the specific implementation, the manual compensation process is as follows: when the user needs to actively adjust the framing direction, the user issues an adjustment instruction through the control terminal, and the control terminal forwards the adjustment instruction to the smart terminal through the communication link. The smart terminal determines the corresponding compensation angle based on the operating parameters attached to the adjustment instruction, and adjusts the posture of the camera module based on the corresponding compensation angle.

[0036] In a specific implementation, the photo compensation angle is a specific angle correction parameter calculated to maintain the stability of the camera module's framing angle or to adjust the framing direction to the target posture according to user instructions. In this embodiment, the photo compensation angle uses the camera module's pitch angle as the main control dimension, and may also include other dimensions, which is not limited in this embodiment.

[0037] Step S20: Control the driving mechanism to adjust the posture of the camera module according to the shooting compensation angle.

[0038] It should be noted that the smart glasses send a control signal to the drive mechanism based on the determined compensation angle for shooting. The drive mechanism then mechanically rotates the camera module to the corresponding angle, ultimately achieving precise adjustment of the camera module's posture. For example, if the user lowers their head, causing the head posture data to show a 20° change in pitch, the drive mechanism will reverse the camera module according to the calculated 20° compensation angle to maintain the original viewing angle.

[0039] It's understandable that during the photo-taking process, the smart glasses can determine the compensation angle through either automatic or manual compensation, with the manual compensation process taking precedence over the automatic compensation process. The smart glasses will immediately initiate the compensation process when they detect any of the following triggering conditions: receiving an adjustment command forwarded by the control terminal; or when there's a significant deviation between the real-time monitored head posture data and the baseline posture corresponding to the locked-on target field of view. This approach prioritizes active user control while also compensating for unconscious posture changes through automatic compensation.

[0040] In a feasible implementation manner, before step S10, steps A11 to A13 may also be included: Step A11: upon receiving a communication connection request sent by a control terminal, performing a whitelist check according to the terminal identification of the control terminal to obtain a pairing check result of the control terminal.

[0041] It should be noted that the terminal identifier refers to the displacement identification information of the control terminal, such as the Bluetooth device address of the control terminal, the unique identifier of the physical network card, etc. The whitelist is a pre-configured list of trusted devices for the smart trap, including the terminal identifiers of legitimate control terminals allowed to establish connections with the smart glasses. The whitelist can be preset at the factory or manually added by the user in the management interface of the smart glasses.

[0042] It is understood that upon receiving a communication connection request from a control terminal, the legitimacy of the control terminal's identity is verified using the whitelist and the terminal identifier of the control terminal to determine whether the control terminal belongs to the range of trusted devices allowed to establish a connection by the smart glasses. In this embodiment, the specific process of whitelist verification is as follows: extracting the terminal identifier of the control terminal carried in the communication connection request and comparing it with the locally stored whitelist. If the terminal identifier exists in the whitelist, the pairing verification result is verification passed; if the terminal identifier does not exist in the whitelist, the pairing verification result is verification failed.

[0043] Step A12: performing a validity check on the communication connection request to obtain a validity check result of the communication connection request.

[0044] It should be noted that when the smart glasses receive a communication connection request, they further verify its compliance with pre-set security rules and interaction protocol requirements. This validation includes, but is not limited to, ensuring that the requested protocol version is compatible with the version supported by the smart glasses and that the timestamp in the request is within the valid time window.

[0045] It is understood that the smart glasses parse the complete content of the communication connection request, check the compliance of each communication connection request according to the pre-set legality verification rules, and ultimately output a legality verification result indicating whether the communication connection request is legal. If the communication connection request is legal, the legality verification result is "passed"; if the communication connection request is illegal, the legality verification result is "failed".

[0046] Step A13: When both the pairing verification result and the legitimacy verification result are verified to be passed, respond to the communication connection request and establish a communication connection with the control terminal.

[0047] It should be noted that only when both the pairing verification and the legitimacy verification results are verified successfully will the smart glasses respond to the communication connection request and establish a communication connection with the control terminal to enable subsequent command transmission and data exchange. Communication connection methods include but are not limited to BLE (Bluetooth Low Energy) and WIFI (Wireless Fidelity).

[0048] It is understandable that when any one of the pairing verification result and the legitimacy verification result is a verification failure result, no response is made to the communication connection request.

[0049] In a feasible implementation manner, after step S20, steps B11 and B12 may be further included: Step B11: When a photographing trigger instruction sent by the control terminal is detected, the camera module is controlled to capture the fine-tuned image according to the target photographing parameters.

[0050] It should be noted that a photo trigger command is a shooting control signal initiated by the control terminal and generated by the user operating the control terminal's interactive interface or physical controls on the control terminal. The photo trigger command generated by the user operating the control terminal's interactive interface may include but is not limited to the following: clicking a photo button on the interactive interface; performing a preset gesture corresponding to the photo; and performing voice commands.

[0051] It is understood that the target photography parameters are pre-set photography configuration information that ensures that the generated image meets the expected quality requirements, including but not limited to image resolution and focus distance. After receiving the photography trigger command from the control terminal, the smart glasses control the camera module to initiate the image acquisition process according to the target photography parameters, ultimately outputting high-resolution image data reflecting the target photography field of view.

[0052] In specific implementations, in addition to the above methods, users can also directly operate the body of the smart glasses (such as physical controls, touch areas), gesture interaction, and voice interaction to send photo commands to the smart glasses, and control the camera module to start the image acquisition process according to the target photo parameters.

[0053] It should be noted that when the camera module collects high-resolution image data reflecting the target photographic field of view, the image data will be sent to the control terminal for display.

[0054] Step B12: When image acquisition is completed and a preview mode closing signal sent by the control terminal is received, the driving mechanism is controlled to be closed, and the posture of the camera module is controlled to be restored to an initial state.

[0055] In a specific implementation, the preview mode shutdown signal is a control command sent by the control terminal when the user actively ends the photo preview function. When the smart glasses complete image acquisition and receive the preview mode shutdown signal, they control the drive mechanism to shut down and control the camera module to return to its default posture before the photo operation (i.e., the camera module's posture returns to a pre-set default value), and the smart glasses enter a low-power sleep state.

[0056] It should be noted that when the smart glasses receive the preview mode shutdown signal sent by the control terminal, even if the image acquisition is not completed, the smart glasses control the driving mechanism to shut down and control the posture of the camera module to restore to the default posture when the photo operation is not performed, and the smart glasses enter a low-power sleep state.

[0057] It is understood that, in addition to the above-mentioned circumstances, when the smart glasses receive a preview mode activation signal sent by the control terminal and determine that the control terminal has entered the preview mode, the smart glasses turn on the preview function and activate the drive mechanism. After the drive mechanism is activated, the smart glasses continuously monitor the user's interaction status. If no control instructions are received from the control terminal for a period of time (for example, 60 seconds), and the user does not send a photo-taking instruction through interaction with the smart glasses, it is determined that no operation has occurred during the timeout. The smart glasses control the drive mechanism to shut down and control the camera module's posture to return to the default posture when no photo-taking operation is performed. The smart glasses then enter a low-power sleep state.

[0058] In this embodiment, the smart glasses include a camera module, a driving mechanism and a motion sensor, the driving mechanism is used to adjust the posture of the camera module, and the motion sensor is used to monitor the user's head posture data; the method includes: determining the shooting compensation angle of the camera module, the shooting compensation angle is determined according to the head posture data and one of the adjustment instructions sent by the control terminal; controlling the driving mechanism to adjust the posture of the camera module according to the shooting compensation angle. In the above manner, by monitoring the head posture data or using the adjustment instruction to determine the shooting compensation angle, and controlling the driving mechanism to adjust the posture of the camera module, it is possible to effectively offset the camera angle deviation caused by the head movement of the user during the preview of the shooting picture, ensure that the camera module always maintains a stable framing angle, thereby realizing real-time framing of the screenless camera glasses in the preview stage, and improving the shooting experience, shooting accuracy and shooting efficiency of the user when using it.

[0059] Based on the first embodiment of the present application, in the second embodiment of the present application, the same or similar contents as those in the above embodiment 1 can be referred to the above introduction and will not be described in detail later. Figure 4 , the step S10 further includes steps S11 to S13: Step S11, obtaining a reference head posture corresponding to the target photographing field of view.

[0060] It should be noted that the target field of view refers to the specific range of the image that the user locks on through the control terminal or the smart glasses determine based on user behavior (such as gaze duration) and that they wish to capture. When capturing images, the smart glasses must keep the target field of view stable within the camera module's framing area. While the user locks on the target field of view, the smart glasses will simultaneously record the user's head posture data captured by the motion sensor at that moment and use it as the baseline head posture, including but not limited to parameters such as the head's pitch angle and heading angle.

[0061] Step S12: Calculate the posture deviation of the pitch angle according to the reference head posture and the head posture data to determine the head posture deviation.

[0062] It should be noted that when the preview function is enabled on the smart glasses, the current view captured by the camera module is processed in real time into a low-resolution compressed image. The smart glasses then send this low-resolution compressed image to the control terminal's display interface for display. During this process, the user can freely move their head to view the real-time preview image on the control terminal's display interface, thereby evaluating the shooting angle and composition. The motion sensor built into the smart glasses continuously collects data on the user's head posture.

[0063] It can be understood that the smart glasses extract the pitch angle parameters from the baseline head posture through the posture solution algorithm, and extract the real-time pitch angle parameters from the current head posture data, and calculate the angle difference Δθ_head between the real-time pitch angle parameters and the pitch angle parameters in the baseline head posture. The angle difference is the user's head posture deviation Δθ_head.

[0064] Step S13, when the absolute value of the head posture deviation is greater than the deviation threshold and the deviation duration is greater than the preset duration, determining the shooting compensation angle of the camera module according to the head posture deviation.

[0065] It should be noted that the deviation threshold T is a preset critical value of the pitch angle posture deviation (for example, 15°); the preset duration is a preset minimum duration threshold (for example, 0.3s), which is used to confirm the stability of the posture deviation; the deviation duration refers to the duration during which the head posture deviation is greater than the deviation threshold.

[0066] It is understood that if the absolute value of the head posture deviation, |Δθ_head|, is greater than the deviation threshold and the deviation persists for longer than a preset duration, the camera module's pitch compensation angle is determined based on the head posture deviation. The specific method for determining the camera module's pitch compensation angle is: compensation angle Δθ_cam = Function (Δθ_head).

[0067] In practice, when using smart glasses to take photos, the camera module's pitch and roll angles can be adjusted by the user automatically rotating their head. This adjustment does not affect the preview image on the control terminal. Therefore, the camera module's shooting compensation angle is primarily controlled by the pitch compensation angle.

[0068] In a feasible implementation manner, before step S11, steps C11 to C13 may also be included: Step C11: upon receiving the target locking instruction sent by the control terminal, parsing the target locking instruction and determining the instruction generation time of the target locking instruction.

[0069] It should be noted that the target lock command refers to a control signal actively triggered by the user through the control terminal, which is used to instruct the smart glasses to lock onto the current target field of view they wish to capture. The target lock command can be generated by, but is not limited to, the user tapping the display interface of the control terminal, issuing a preset voice command, or triggering it through a specific gesture.

[0070] It is understandable that the instruction generation time refers to the specific time point when the target lock instruction is created, which is used to accurately associate the head posture data collected by the motion sensor at the same time.

[0071] Step C12: Acquire the target head posture collected by the motion sensor at the time of generating the instruction.

[0072] Step C13: Using the target head posture as a reference head posture corresponding to the target photographing field of view.

[0073] It should be noted that the smart glasses extract the target head pose that strictly corresponds to the command generation time from the data stream collected by the motion sensor. The target head pose is the head pose data recorded when the user locks on the target camera field of view. Ultimately, the target head pose collected at the time of command generation is used as the reference head pose corresponding to the target camera field of view.

[0074] It is understandable that, taking the control terminal as a smart watch as an example, the interaction process between the smart watch and the smart glasses is as follows: Figure 5 As shown, after the smartwatch and smart glasses establish a communication connection, control commands can be transmitted between them with low latency. If the smartwatch enters preview mode, it sends a corresponding activation signal to the smart glasses. The smart glasses can transmit a compressed preview image corresponding to the camera module's viewfinder to the smartwatch's display interface with low latency, and the smartwatch will display a real-time preview image on the display interface. At the same time, the smartwatch will prompt the user to look at the target field of view through voice or information displayed on the display interface. The user can lock the target field of view by tapping the display interface (i.e., the smartwatch screen) and record the baseline head posture Δθ_target at that moment.

[0075] This embodiment obtains a baseline head pose corresponding to the target photographic field of view; calculates the pitch angle deviation based on the baseline head pose and the head pose data to determine the head pose deviation; and, if the absolute value of the head pose deviation is greater than a deviation threshold and the deviation persists for longer than a preset duration, determines the camera module's photographic compensation angle based on the head pose deviation. This approach accurately determines the photographic compensation angle to offset the field of view deviation caused by head movement and ensure a stable target photographic field of view.

[0076] Based on the first embodiment and / or the second embodiment of the present application, in the third embodiment of the present application, the same or similar contents as those in the above embodiments can be referred to the above introduction and will not be described in detail later. Figure 6 , step S10, further comprising steps S21 to S23: Step S21 : upon receiving an adjustment instruction sent by the control terminal, determining an adjustment operation distance according to the adjustment instruction.

[0077] It should be noted that upon receiving an adjustment instruction from the control terminal, the adjustment instruction is parsed to determine the adjustment operation distance. The specific manifestations of the adjustment operation distance include, but are not limited to, any of the following: the vertical distance generated when a user slides on the display interface of the control terminal; the actual physical displacement generated by the movement of the physical controls of the control terminal by the user; and other manifestations.

[0078] Step S22: Calculate based on the adjustment operation distance and the target mapping function to determine the adjustment angle of the camera module corresponding to the pitch angle dimension, wherein the target mapping function at least includes an adjustment mapping coefficient.

[0079] It should be noted that the target mapping function is a function model used to establish a mathematical relationship between the adjustment operating distance and the compensation angle of the camera module; wherein, the adjustment mapping coefficient is used to define the proportional relationship between the adjustment operating distance and the adjustment angle. In this embodiment, the adjustment mapping coefficient is set to k = 0.1° / pixel, and can also be adjusted according to needs, which is not limited in this embodiment.

[0080] It is understood that by substituting the adjustment operating distance into the target mapping function for calculation, the compensation angle of the camera module at the pitch angle can be obtained. In this embodiment, the specific formula of the target mapping function is: Δθ_manual = k × Δy, where Δθ_manual is the compensation angle of the camera module at the pitch angle, and Δy is the adjustment operating distance. The adjustment angle corresponding to the pitch angle dimension refers to the compensation angle of the camera module at the pitch angle.

[0081] Step S23: determining a shooting compensation angle of the camera module according to an adjustment angle corresponding to the pitch angle dimension of the camera module.

[0082] It should be noted that the calculated adjustment angle corresponding to the camera angle in the pitch angle dimension is used as the camera compensation angle of the camera module for subsequent camera compensation.

[0083] In a feasible implementation manner, before step S21, steps D11 to D13 may also be included: Step D11: When the control terminal enters the preview mode, the initial image captured by the camera module is obtained.

[0084] It should be noted that after receiving the preview mode start signal sent by the control terminal, the smart terminal determines that the control terminal enters the preview mode. At this time, the smart terminal will turn on the preview function and obtain the initial picture captured by the camera module in real time.

[0085] Step D12: compress the initial image according to the target resolution to generate a preview compressed image corresponding to the initial image.

[0086] It should be noted that the target resolution is a pre-set image resolution. To balance preview clarity and transmission efficiency, the target resolution is lower than the resolution of the initial image. The smart glasses downsample the initial image and further compress the downsampled image according to the target resolution, generating a smaller, less data-intensive preview compressed image. In addition to the above methods, the initial image can also be directly compressed according to the target resolution, which is not limited in this embodiment.

[0087] Step D13: sending the preview compressed image to the control terminal for display, so that the control terminal generates an adjustment instruction and feeds back the adjustment instruction according to the user's adjustment operation distance.

[0088] It should be noted that the smart glasses send the preview compressed image to the control terminal, and the control terminal displays the preview compressed image in real time. When the user observes the preview compressed image on the display interface of the control terminal, he can initiate adjustment operations according to actual needs. The control terminal generates corresponding adjustment instructions based on the user's adjustment operation distance and sends them to the smart glasses.

[0089] It is understandable that in this embodiment, the smart glasses can enter the automatic compensation process and the manual compensation process in the process of taking pictures. The specific process is as follows: Figure 7 The following example is provided for illustrative purposes only. The parameters shown in the figure are examples only and are not intended to be limiting. After determining the baseline head pose corresponding to the target field of view, the smart glasses continuously monitor head pose data using a motion sensor and calculate the pitch angle deviation Δθ_head between the head pose data and the baseline head pose. If the absolute value of the deviation |Δθ_head| is greater than 15° and persists for 0.3 seconds, the camera module's shooting compensation angle Δθ_cam is determined based on the pitch angle deviation Δθ_head, and the camera module is compensated according to the shooting compensation angle. A preview of the updated compensated image is then sent to the control terminal. The control terminal then sends an adjustment command to the smart glasses. The smart glasses then determine the camera module's pitch compensation angle Δθ_manual based on the adjustment operation distance Δy in the adjustment command. Based on the camera module's pitch compensation angle Δθ_manual, the smart glasses then determine the camera module's shooting compensation angle Δθ_cam+, and the camera module is compensated according to the shooting compensation angle. And send a preview of the updated fine-tuned screen to the control terminal. The user clicks the photo button through the control terminal, and the control terminal sends a photo trigger instruction. At this time, the smart glasses control the camera module to take high-resolution photos.

[0090] This embodiment, upon receiving an adjustment instruction from a control terminal, determines the adjustment operation distance based on the adjustment instruction; calculates the corresponding adjustment angle of the camera module in the pitch dimension based on the adjustment operation distance and a target mapping function, where the target mapping function includes at least an adjustment mapping coefficient; and determines the camera module's shooting compensation angle based on the corresponding adjustment angle of the camera module in the pitch dimension. This ensures the accuracy and real-time performance of the adjustment action, improving the photography experience.

[0091] It should be noted that the above examples are only used to understand the present application and do not constitute a limitation on the control method of the smart glasses of the present application. More simple transformations based on this technical concept are all within the scope of protection of the present application.

[0092] This application also provides a control device for smart glasses, please refer to Figure 8 , the control device of the smart glasses includes: The processing module 10 is used to determine the shooting compensation angle of the camera module, where the shooting compensation angle is determined according to one of the head posture data and the adjustment instruction sent by the control terminal.

[0093] The control module 20 is used to control the driving mechanism to adjust the posture of the camera module according to the shooting compensation angle.

[0094] Optionally, the processing module 10 is further configured to: Obtain the reference head pose corresponding to the target photographic field of view; Calculating a posture deviation of a pitch angle according to the reference head posture and the head posture data to determine a head posture deviation; When the absolute value of the head posture deviation is greater than the deviation threshold and the duration of the deviation is greater than the preset duration, the shooting compensation angle of the camera module is determined according to the head posture deviation.

[0095] Optionally, the processing module 10 is further configured to: Upon receiving the target locking instruction sent by the control terminal, parsing the target locking instruction and determining the instruction generation time of the target locking instruction; Obtaining the target head posture collected by the motion sensor at the time of generating the instruction; The target head posture is used as the reference head posture corresponding to the target photographing field of view.

[0096] Optionally, the processing module 10 is further configured to: Upon receiving an adjustment instruction sent by the control terminal, determining an adjustment operation distance according to the adjustment instruction; Determining an adjustment angle of the camera module in a pitch angle dimension by calculating according to the adjustment operation distance and a target mapping function, wherein the target mapping function includes at least an adjustment mapping coefficient; The shooting compensation angle of the camera module is determined according to the adjustment angle corresponding to the pitch angle dimension of the camera module.

[0097] Optionally, the processing module 10 is further configured to: When the control terminal enters the preview mode, obtaining the initial image captured by the camera module; compressing the initial image according to the target resolution to generate a preview compressed image corresponding to the initial image; The preview compressed image is sent to the control terminal for display, so that the control terminal generates an adjustment instruction and feeds back the adjustment instruction according to the user's adjustment operation distance.

[0098] Optionally, the control module 20 is further configured to: When a photographing trigger instruction sent by the control terminal is detected, the camera module is controlled to capture the fine-tuned image according to the target photographing parameters; When image acquisition is completed and a preview mode closing signal sent by the control terminal is received, the driving mechanism is controlled to be closed, and the posture of the camera module is controlled to be restored to an initial state.

[0099] Optionally, the processing module 10 is further configured to: Upon receiving a communication connection request sent by a control terminal, performing a whitelist check according to the terminal identification of the control terminal to obtain a pairing check result of the control terminal; Performing a validity check on the communication connection request to obtain a validity check result of the communication connection request; When both the pairing verification result and the legitimacy verification result are verified to be passed, respond to the communication connection request and establish a communication connection with the control terminal.

[0100] The control device for smart glasses provided in this application utilizes the control method for smart glasses described in the aforementioned embodiments, resolving the prior art technical issue of changes in the viewing angle of screenless camera glasses during preview of a photo. Compared to the prior art, the control device for smart glasses provided in this application achieves the same beneficial effects as the control method for smart glasses described in the aforementioned embodiments. Other technical features of the control device for smart glasses are the same as those disclosed in the aforementioned embodiments and are not further elaborated upon here.

[0101] The present application provides smart glasses, which include: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the control method of the smart glasses in the above-mentioned embodiment 1.

[0102] Reference below Figure 9 , which shows a schematic diagram of the structure of smart glasses suitable for implementing the embodiments of the present application. The smart glasses in the embodiments of the present application may include, but are not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), in-vehicle terminals (such as in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 9 The smart glasses shown are merely an example and should not limit the functions and scope of use of the embodiments of the present application.

[0103] like Figure 9 As shown, smart glasses may include a processing device 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes based on programs stored in a read-only memory (ROM) 1002 or programs loaded from a storage device 1003 into a random access memory (RAM) 1004. RAM 1004 also stores various programs and data required for the operation of the smart glasses. Processing device 1001, ROM 1002, and RAM 1004 are interconnected via a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems may be connected to I / O interface 1006: input devices 1007, such as a touchscreen, touchpad, keyboard, mouse, image sensor, microphone, accelerometer, gyroscope, etc.; output devices 1008, such as a liquid crystal display (LCD), speaker, vibrator, etc.; storage device 1003, such as a magnetic tape or hard disk; and communication device 1009. The communication device 1009 can allow the smart glasses to communicate with other devices wirelessly or wired to exchange data. Although the figure shows smart glasses with various systems, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems can be implemented or have instead.

[0104] In particular, according to the embodiments disclosed in the present application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, the embodiments disclosed in the present application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program comprising program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network via a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.

[0105] The smart glasses provided in this application utilize the control method for smart glasses in the aforementioned embodiment, resolving the technical issue in the prior art whereby the viewing angle of screenless camera glasses changes during preview of a photo. Compared to the prior art, the beneficial effects of the smart glasses provided in this application are the same as those of the control method for smart glasses in the aforementioned embodiment, and the other technical features of the smart glasses are the same as those disclosed in the aforementioned embodiment, and are not further elaborated here.

[0106] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0107] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

[0108] The present application provides a computer-readable storage medium having computer-readable program instructions (ie, a computer program) stored thereon, and the computer-readable program instructions are used to execute the control method of the smart glasses in the above-mentioned embodiment.

[0109] The computer-readable storage medium provided herein may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, systems, or devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to, an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, system, or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including, but not limited to, wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.

[0110] The computer-readable storage medium may be included in the smart glasses, or may exist independently without being incorporated into the smart glasses.

[0111] The above-mentioned computer-readable storage medium carries one or more programs. When the above-mentioned one or more programs are executed by the smart glasses, the smart glasses are enabled to: determine the photo compensation angle of the camera module, and the photo compensation angle is determined based on the head posture data and one of the adjustment instructions sent by the control terminal; and control the driving mechanism to adjust the posture of the camera module according to the photo compensation angle.

[0112] Computer program code for performing the operations of the present application may be written in one or more programming languages, or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0113] The flow charts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present application. In this regard, each box in the flow chart or block diagram can represent a module, program segment or a part of code, and the module, program segment or a part of code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order than that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented by a dedicated hardware-based system that performs the specified function or operation, or can be implemented by a combination of dedicated hardware and computer instructions.

[0114] The modules described in the embodiments of the present application may be implemented in software or hardware, wherein the name of a module does not necessarily limit the unit itself.

[0115] The computer-readable storage medium provided in this application stores computer-readable program instructions (i.e., a computer program) for executing the control method for the smart glasses described above. This computer-readable storage medium can address the technical issue of the prior art causing the viewing angle of screenless camera glasses to change during preview of a photo. Compared to the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as those of the control method for the smart glasses provided in the aforementioned embodiments, and are not further elaborated here.

[0116] The present application also provides a computer program product, comprising a computer program, which implements the steps of the control method of the smart glasses as described above when the computer program is executed by a processor.

[0117] The computer program product provided in this application can solve the technical problem of the prior art causing the viewing angle of screenless camera glasses to change during the preview of a photo. Compared with the prior art, the beneficial effects of the computer program product provided in this application are the same as those of the control method of the smart glasses provided in the above-mentioned embodiment, and will not be elaborated here.

[0118] The above description is only part of the embodiments of the present application and does not limit the patent scope of the present application. All equivalent structural transformations made by using the contents of the present application specification and drawings under the technical concept of the present application, or direct / indirect application in other related technical fields are included in the patent protection scope of the present application.

Claims

1. A method for controlling smart glasses, characterized in that: The method is applied to smart glasses, which include a camera module, a drive mechanism, and a motion sensor. The drive mechanism is used to adjust the posture of the camera module, and the motion sensor is used to monitor the user's head posture data. The method comprises: determining a photographing compensation angle of the camera module, wherein the photographing compensation angle is determined according to one of the head posture data and an adjustment instruction sent by the control terminal; The driving mechanism is controlled to adjust the posture of the camera module according to the shooting compensation angle.

2. The method according to claim 1, wherein The step of determining the shooting compensation angle of the camera module includes: Obtain the reference head pose corresponding to the target photographic field of view; Calculating a posture deviation of a pitch angle according to the reference head posture and the head posture data to determine a head posture deviation; When the absolute value of the head posture deviation is greater than the deviation threshold and the duration of the deviation is greater than the preset duration, the shooting compensation angle of the camera module is determined according to the head posture deviation.

3. The method according to claim 2, wherein Before the step of obtaining the reference head posture corresponding to the target photographing field of view, the method further includes: Upon receiving the target locking instruction sent by the control terminal, parsing the target locking instruction and determining the instruction generation time of the target locking instruction; Obtaining the target head posture collected by the motion sensor at the time of generating the instruction; The target head posture is used as the reference head posture corresponding to the target photographing field of view.

4. The method according to claim 1, wherein The step of determining the shooting compensation angle of the camera module includes: Upon receiving an adjustment instruction sent by the control terminal, determining an adjustment operation distance according to the adjustment instruction; Determining an adjustment angle of the camera module in a pitch angle dimension by calculating according to the adjustment operation distance and a target mapping function, wherein the target mapping function includes at least an adjustment mapping coefficient; The shooting compensation angle of the camera module is determined according to the adjustment angle corresponding to the pitch angle dimension of the camera module.

5. The method according to claim 4, wherein Before the step of determining the adjustment operating distance according to the adjustment instruction when the adjustment instruction sent by the control terminal is received, the method further includes: When the control terminal enters the preview mode, obtaining the initial image captured by the camera module; compressing the initial image according to the target resolution to generate a preview compressed image corresponding to the initial image; The preview compressed image is sent to the control terminal for display, so that the control terminal generates an adjustment instruction and feeds back the adjustment instruction according to the user's adjustment operation distance.

6. The method according to any one of claims 1 to 5, characterized in that After the step of controlling the driving mechanism to adjust the posture of the camera module according to the shooting compensation angle, the method further includes: When a photographing trigger instruction sent by the control terminal is detected, the camera module is controlled to capture the fine-tuned image according to the target photographing parameters; When image acquisition is completed and a preview mode closing signal sent by the control terminal is received, the driving mechanism is controlled to be closed, and the posture of the camera module is controlled to be restored to an initial state.

7. The method according to any one of claims 1 to 5, characterized in that Before the step of determining the compensation angle of the camera module, the method further includes: Upon receiving a communication connection request sent by a control terminal, performing a whitelist check according to the terminal identification of the control terminal to obtain a pairing check result of the control terminal; Performing a validity check on the communication connection request to obtain a validity check result of the communication connection request; When both the pairing verification result and the legitimacy verification result are verified to be passed, respond to the communication connection request and establish a communication connection with the control terminal.

8. A control device for smart glasses, characterized in that: The control device of the smart glasses includes: a processing module, configured to determine a photographing compensation angle of the camera module, wherein the photographing compensation angle is determined based on one of the head posture data and an adjustment instruction sent by the control terminal; The control module is used to control the driving mechanism to adjust the posture of the camera module according to the shooting compensation angle.

9. A pair of smart glasses, characterized in that: The smart glasses include: a memory, a processor, and a control program for the smart glasses stored in the memory and executable on the processor, wherein the control program for the smart glasses is configured to implement the steps of the control method for the smart glasses according to any one of claims 1 to 7.

10. A storage medium, characterized in that: The storage medium stores a control program for the smart glasses, and when the control program for the smart glasses is executed by the processor, the steps of the method for controlling the smart glasses according to any one of claims 1 to 7 are implemented.

Images