Control methods, devices, smart glasses, and storage media for smart glasses

Abstract

This application discloses a control method, device, smart glasses, and storage medium for smart glasses. The method is applied to smart glasses, which include a camera module, a driving mechanism, and a motion sensor. The driving mechanism is used to adjust the posture of the camera module, and the motion sensor is used to monitor the user's head posture data. The method includes: determining a photo compensation angle for the camera module, the photo compensation angle being determined based on one of the head posture data and an adjustment command sent by a control terminal; and controlling the driving mechanism to adjust the posture of the camera module according to the photo compensation angle. Through this method, the camera viewing angle shift caused by the user's head movements during the preview of the photo can be effectively offset, ensuring that the camera module always maintains a stable viewing angle. This enables real-time framing during the preview stage of the screenless camera glasses, improving the user's shooting experience, shooting accuracy, and shooting efficiency.

Description

Technical Field

[0001] This application relates to the field of wearable device technology, and more particularly to control methods, devices, smart glasses, and storage media for smart glasses. Background Technology

[0002] Currently, screenless camera glasses lack real-time framing capabilities, preventing users from intuitively obtaining the framing information during shooting, resulting in a poor shooting experience. Although some solutions use a matching control terminal (such as a smartwatch) as a framing preview terminal, existing camera glasses suffer from changes in camera angle during preview, failing to achieve true real-time framing. Summary of the Invention

[0003] The main objective of this application is to provide a control method, device, smart glasses, and storage medium for smart glasses, aiming to solve the technical problem that the camera angle of screenless camera glasses changes during the preview of the photographed image in the prior art.

[0004] To achieve the above objectives, this application proposes a control method for smart glasses. The method is applied to smart glasses, which include a camera module, a drive mechanism, and a motion sensor. The drive mechanism is used to adjust the posture of the camera module, and the motion sensor is used to monitor the user's head posture data.

[0005] The method includes:

[0006] The image compensation angle of the camera module is determined based on one of the head posture data and the adjustment command sent by the control terminal.

[0007] The drive mechanism is controlled to adjust the posture of the camera module according to the image compensation angle.

[0008] In one embodiment, the step of determining the image compensation angle of the camera module includes:

[0009] Obtain the baseline head posture corresponding to the target's field of view in the photograph;

[0010] The head posture deviation is determined by calculating the pitch angle based on the reference head posture and the head posture data.

[0011] If the absolute value of the head posture deviation is greater than the deviation threshold and the duration of the deviation is greater than the preset duration, the camera module's photo compensation angle is determined based on the head posture deviation.

[0012] In one embodiment, before the step of obtaining the reference head pose corresponding to the target's field of view, the method further includes:

[0013] Upon receiving a target locking command from the control terminal, the target locking command is parsed to determine the command generation time.

[0014] The motion sensor acquires the target head posture at the instruction generation time;

[0015] The target head posture is used as the reference head posture corresponding to the target's field of view.

[0016] In one embodiment, the step of determining the image compensation angle of the camera module includes:

[0017] Upon receiving an adjustment instruction from the control terminal, the adjustment operation distance is determined based on the adjustment instruction;

[0018] Based on the adjustment operation distance and the target mapping function, the adjustment angle of the camera module in the pitch angle dimension is determined, wherein the target mapping function includes at least an adjustment mapping coefficient;

[0019] The image compensation angle of the camera module is determined based on the adjustment angle corresponding to the pitch angle dimension.

[0020] In one embodiment, before the step of determining the adjustment operation distance according to the adjustment instruction sent by the control terminal, the method further includes:

[0021] When the control terminal enters preview mode, the initial image captured by the camera module is obtained;

[0022] The initial image is compressed according to the target resolution to generate a preview compressed image corresponding to the initial image;

[0023] The preview compressed image is sent to the control terminal for display, so that the control terminal can generate adjustment instructions based on the user's adjustment operation distance and provide feedback.

[0024] In one embodiment, after the step of controlling the drive mechanism to adjust the posture of the camera module according to the image compensation angle, the method further includes:

[0025] When the camera module detects a photo-taking trigger command sent by the control terminal, it is controlled to capture the finely adjusted image according to the target photo-taking parameters.

[0026] Upon completion of image acquisition and receipt of a preview mode shutdown signal from the control terminal, the drive mechanism is shut down, and the camera module's posture is restored to its initial state.

[0027] In one embodiment, before the step of determining the image compensation angle of the camera module, the method further includes:

[0028] Upon receiving a communication connection request from the control terminal, a whitelist verification is performed based on the terminal identifier of the control terminal to obtain the pairing verification result of the control terminal.

[0029] The validity of the communication connection request is verified to obtain the validity verification result of the communication connection request;

[0030] When both the pairing verification result and the legality verification result pass the verification, the system responds to the communication connection request and establishes a communication connection with the control terminal.

[0031] Furthermore, to achieve the above objectives, this application also proposes a control device for smart glasses, the control device for smart glasses comprising:

[0032] The processing module is used to determine the image compensation angle of the camera module, which is determined based on one of the head posture data and the adjustment command sent by the control terminal.

[0033] The control module is used to control the drive mechanism to adjust the posture of the camera module according to the image compensation angle.

[0034] In addition, to achieve the above objectives, this application also proposes a smart glasses, the smart glasses comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the control method for the smart glasses as described above.

[0035] In addition, to achieve the above objectives, this application also proposes a storage medium, which is a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, it implements the steps of the control method for smart glasses as described above.

[0036] In addition, to achieve the above objectives, this application also provides a computer program product, which includes a computer program that, when executed by a processor, implements the steps of the smart glasses control method described above.

[0037] The method of this application is applied to smart glasses, which include a camera module, a driving mechanism, and a motion sensor. The driving mechanism is used to adjust the posture of the camera module, and the motion sensor is used to monitor the user's head posture data. The method includes: determining the photo compensation angle of the camera module, the photo compensation angle being determined based on one of the head posture data and an adjustment command sent by a control terminal; and controlling the driving mechanism to adjust the posture of the camera module according to the photo compensation angle. By monitoring head posture data or using adjustment commands to determine the photo compensation angle, and controlling the driving mechanism to adjust the posture of the camera module, the method effectively counteracts the camera perspective shift caused by the user's head movements during the preview of the photo, ensuring that the camera module maintains a stable framing angle. This achieves real-time framing during the preview stage of the screenless camera glasses, improving the user's shooting experience, shooting accuracy, and shooting efficiency. Attached Figure Description

[0038] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0039] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0040] Figure 1 This is a flowchart illustrating an embodiment of the control method for smart glasses in this application.

[0041] Figure 2 This is a schematic diagram of the overall structure of the smart glasses provided in Embodiment 1 of this application;

[0042] Figure 3 This is a partial structural diagram of the smart glasses provided in Embodiment 1 of this application;

[0043] Figure 4 This is a flowchart illustrating a second embodiment of the control method for smart glasses in this application.

[0044] Figure 5 This is a schematic diagram of the interaction process provided in Embodiment 2 of this application;

[0045] Figure 6 This is a flowchart illustrating Embodiment 3 of the control method for smart glasses in this application.

[0046] Figure 7 This is a schematic diagram of the compensation process provided in Embodiment 3 of this application;

[0047] Figure 8 This is a schematic diagram of the module structure of the control device for the smart glasses according to an embodiment of this application;

[0048] Figure 9 This is a schematic diagram of the device structure of the hardware operating environment involved in the control method of smart glasses in the embodiments of this application;

[0049] Explanation of icon numbers:

[0050] 1. FPC flexible flat cable; 2. CMOS chip; 3. Lens barrel and lens; 4. IMU chip; 5. PCB; 6. PCB bracket;

[0051] 7. Transmission gears; 8. Servo motor.

[0052] The purpose, features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0053] It should be understood that the specific embodiments described herein are merely illustrative of the technical solutions of this application and are not intended to limit this application.

[0054] To better understand the technical solution of this application, a detailed description will be provided below in conjunction with the accompanying drawings and specific implementation methods.

[0055] The main solution of this application embodiment is: determining the photo compensation angle of the camera module, wherein the photo compensation angle is determined based on one of the head posture data and the adjustment command sent by the control terminal; and controlling the drive mechanism to adjust the posture of the camera module according to the photo compensation angle.

[0056] Currently, screenless camera glasses lack real-time framing capabilities, preventing users from intuitively obtaining the framing information during shooting, resulting in a poor shooting experience. Although some solutions use a matching control terminal (such as a smartwatch) as a framing preview terminal, existing camera glasses suffer from changes in camera angle during preview.

[0057] This application provides a solution that, by monitoring head posture data or using adjustment commands to determine the photo compensation angle, and controlling the drive mechanism to adjust the posture of the camera module, can effectively counteract the camera perspective shift caused by the user's head movements during the preview of the photo, ensuring that the camera module always maintains a stable framing angle. This enables real-time framing of the screenless photo glasses during the preview stage, improving the user's shooting experience, shooting accuracy, and shooting efficiency.

[0058] It should be noted that the executing entity in this embodiment can be a computing service device with data processing, network communication, and program execution functions, such as a tablet computer, personal computer, or mobile phone, or a smart glasses device capable of performing the above functions. The following description uses smart glasses as an example to illustrate this embodiment and the subsequent embodiments.

[0059] Based on this, embodiments of this application provide a control method for smart glasses, referring to... Figure 1 , Figure 1 This is a flowchart illustrating the first embodiment of the control method for smart glasses according to this application.

[0060] In this embodiment, the method is applied to smart glasses, which include a camera module, a driving mechanism, and a motion sensor. The driving mechanism is used to adjust the posture of the camera module, and the motion sensor is used to monitor the user's head posture data. The control method of the smart glasses includes steps S10~S20:

[0061] Step S10: Determine the image compensation angle of the camera module. The image compensation angle is determined based on one of the head posture data and the adjustment command sent by the control terminal.

[0062] It should be noted that the smart glasses in this embodiment refer to screenless glasses that can be worn directly by the user and are capable of taking photos. The smart glasses include, but are not limited to, a camera module, a drive mechanism, a motion sensor, and the glasses themselves. The camera module is the core component for capturing images or videos, including lenses, lens barrels, and other parts. Its posture (e.g., shooting angle, pitch angle, and rotation angle) directly determines the framing range and shooting angle. The drive mechanism is an execution component used to physically adjust the posture of the camera module. It can drive the camera module to rotate in different directions according to control signals, thereby changing its shooting angle. The motion sensor is used to sense changes in the user's head movement, such as rotation, tilt, and displacement, and converts the corresponding posture information into quantified head posture data. In this embodiment, an IMU (Inertial Measurement Unit) can be used as the motion sensor, or other sensors with similar functions can be used; this embodiment does not impose any limitations on this.

[0063] It is understood that in this embodiment, the drive mechanism, camera module, and motion sensor are all located on the same temple of the smart glasses, and the camera module is adjacent to the drive mechanism. The overall structure of the smart glasses is as follows: Figure 2 As shown, module A contains a drive mechanism, a camera module, and a motion sensor. The specific structure of module A, including the positional relationships between the three components, is as follows: Figure 3As shown, the drive mechanism specifically includes a servo motor 8 and a transmission gear 7; the camera module specifically includes a CMOS (Complementary Metal-Oxide-Semiconductor) chip 2, a PCB (Printed Circuit Board) 5, a PCB bracket 6, an FPC (Flexible Printed Circuit) cable 1, a lens and a lens barrel 3, and the motion sensor is the IMU chip 4 shown in the figure.

[0064] In the specific implementation, head posture data is user head movement information collected in real time by motion sensors, including but not limited to the user's head pitch angle, yaw angle, and roll angle. Among them, the pitch angle is the angle at which the user's head rotates up and down around the horizontal axis, used to describe head-up or head-down movements. In this embodiment, the focus is on the user's pitch angle.

[0065] It is understood that the control terminal is an independent device that establishes a communication connection with the smart glasses. It has the function of real-time previewing of images captured by the smart glasses and can remotely control the smart glasses' photo-taking operation. The control terminal can be a wearable device including but not limited to smartwatches, or other devices; this embodiment does not impose any restrictions on this.

[0066] In practice, users can initiate adjustment commands through the interactive interface of the control terminal or through physical controls on the control terminal. After the adjustment commands are transmitted to the smart glasses via the communication link, the smart glasses adjust the posture of the camera module based on the adjustment commands.

[0067] It should be noted that when the control terminal enters preview mode, it sends a preview mode enable signal to the smart glasses. Upon receiving the enable signal, the smart glasses activate the smartwatch's preview function, at which point the drive mechanism is activated, and the process proceeds to step S10.

[0068] Understandably, after activating the drive mechanism, smart glasses employ both automatic and manual compensation methods during the photo-taking process. The automatic compensation process is as follows: When the user enters the photo preview mode via the control terminal, the actual framing direction of the camera module may shift relative to the initially locked target due to factors during the preview process. To maintain framing stability, motion sensors monitor the user's head posture data in real time, compare this data with the baseline posture corresponding to the locked target's field of view, determine the appropriate compensation angle based on the comparison result, and adjust the camera module's posture accordingly.

[0069] In practice, the manual compensation process is as follows: when the user needs to actively adjust the framing direction, the user issues an adjustment command through the control terminal. The control terminal forwards the adjustment command to the smart terminal through the communication link. The smart terminal determines the corresponding compensation angle based on the operation parameters attached to the adjustment command, and adjusts the posture of the camera module based on the corresponding compensation angle.

[0070] In a specific implementation, the photo compensation angle is a specific angle correction parameter calculated to maintain the stability of the camera module's framing angle or to adjust the framing direction to the target posture according to user instructions. In this embodiment, the photo compensation angle is mainly controlled by the pitch angle of the camera module, but other dimensions may also be included. This embodiment does not limit this.

[0071] Step S20: Control the drive mechanism to adjust the posture of the camera module according to the photo compensation angle.

[0072] It should be noted that the smart glasses send control signals to the drive mechanism based on the determined image compensation angle. The drive mechanism then uses mechanical movement to rotate the camera module by the corresponding angle, ultimately achieving precise adjustment of the camera module's posture. For example, when the user tilts their head down, causing a 20° change in the head posture display, the drive mechanism will adjust the camera module in the opposite direction according to the calculated 20° compensation angle to maintain the original framing angle.

[0073] Understandably, during the photo-taking process, smart glasses can determine the photo compensation angle either through an automatic compensation process or a manual compensation process, but the priority between the two is manual compensation > automatic compensation. The smart glasses will immediately initiate the photo compensation process when either of the following conditions is detected: receiving an adjustment command forwarded from the control terminal; or when the deviation between the real-time monitored head posture data and the baseline posture corresponding to the locked target's field of view is too large. This approach ensures the priority of user-initiated control while compensating for the impact of unconscious posture changes through an automatic compensation mechanism.

[0074] In one feasible implementation, steps A11 to A13 may be included before step S10:

[0075] Step A11: Upon receiving a communication connection request from the control terminal, perform a whitelist verification based on the terminal identifier of the control terminal to obtain the pairing verification result of the control terminal.

[0076] It should be noted that the terminal identifier refers to the displacement identification information of the control terminal, such as the Bluetooth device address and unique physical network card identifier of the control terminal. The whitelist is a pre-configured list of trusted devices for the smart trap, including a set of terminal identifiers of legitimate control terminals allowed to establish connections with the smart glasses. The whitelist can be preset at the factory or manually added by the user in the management interface of the smart glasses.

[0077] Understandably, upon receiving a communication connection request from the control terminal, the whitelist and the control terminal's terminal identifier are used to verify the legitimacy of the control terminal's identity, determining whether the control terminal belongs to the trusted device range allowed to establish connections with the smart glasses. In this embodiment, the specific process of whitelist verification is as follows: extract the control terminal's terminal identifier carried in the communication connection request, compare it with the locally stored whitelist. If the terminal identifier exists in the whitelist, the pairing verification result is a successful verification; if the terminal identifier does not exist in the whitelist, the pairing verification result is a failed verification.

[0078] Step A12: Perform a validity check on the communication connection request to obtain the validity check result of the communication connection request.

[0079] It should be noted that when the smart glasses receive a communication connection request, they will further verify the compliance of the request, determining whether it meets the preset security rules and interaction protocol requirements. The verification rules include, but are not limited to: the request protocol version being compatible with the version supported by the smart glasses; and the timestamp carried in the request being within a valid time window.

[0080] Understandably, the smart glasses parse the complete content of the communication connection request, check the compliance of each request according to preset validity rules, and finally output the validity check result of whether the communication connection request is valid. When the communication connection request is valid, the validity check result is "validation passed"; when the communication connection request is invalid, the validity check result is "validation failed".

[0081] Step A13: When both the pairing verification result and the legality verification result are passed, respond to the communication connection request and establish a communication connection with the control terminal.

[0082] It should be noted that the smart glasses will only respond to the communication connection request and establish a communication connection with the control terminal when both the pairing verification result and the validity verification result pass, thus enabling subsequent command transmission and data interaction. The communication connection methods include, but are not limited to, BLE (Bluetooth Low Energy) and WIFI (Wireless Fidelity) connections.

[0083] Understandably, if either the pairing verification result or the validity verification result fails, the communication connection request will not be responded to.

[0084] In one possible implementation, after step S20, steps B11-B12 may also be included:

[0085] Step B11: When the camera trigger command sent by the control terminal is detected, the camera module is controlled to acquire the finely adjusted image according to the target image capture parameters.

[0086] It should be noted that the photo-taking trigger command is a shooting control signal actively initiated by the control terminal, generated through user operation of the control terminal's interactive interface or physical controls on the control terminal. When the photo-taking trigger command is generated through the user operation of the control terminal's interactive interface, it includes, but is not limited to, the following forms: clicking the photo button on the interactive interface; the user executing a preset gesture or voice command corresponding to taking a photo.

[0087] Understandably, the target image capture parameters are pre-set shooting configuration information to ensure that the generated image meets the expected quality requirements, including but not limited to image resolution and focus distance. After receiving the image capture trigger command from the control terminal, the smart glasses control the camera module to start the image acquisition process according to the target image capture parameters, and finally output high-resolution image data reflecting the target's field of view.

[0088] In practice, in addition to the methods mentioned above, users can also directly operate the smart glasses themselves (e.g., physical controls, touch areas), use gestures, or use voice to send a photo-taking command to the smart glasses and control the camera module to start the image acquisition process according to the target photo-taking parameters.

[0089] It should be noted that when the camera module acquires high-resolution image data that reflects the target's field of view, it will send the image data to the control terminal for display.

[0090] Step B12: After image acquisition is completed and a preview mode shutdown signal is received from the control terminal, the drive mechanism is shut down, and the camera module's posture is restored to its initial state.

[0091] In the specific implementation, the preview mode shutdown signal is a control command sent by the control terminal when the user actively ends the photo preview function. When the smart glasses have completed image acquisition and received the preview mode shutdown signal, the control drive mechanism is turned off, and the camera module's posture is restored to the default posture when no photo-taking operation was performed (i.e., the camera module's posture is restored to the preset default value), and the smart glasses enter a low-power sleep state.

[0092] It should be noted that when the smart glasses receive a preview mode off signal from the control terminal, even if image acquisition has not been completed, the smart glasses will control the drive mechanism to shut down and restore the camera module's posture to the default posture when no photo-taking operation is performed, and the smart glasses will enter a low-power sleep state.

[0093] Understandably, aside from the above situations, when the smart glasses receive a preview mode enable signal from the control terminal and confirm that the control terminal has entered preview mode, the smart glasses activate the preview function, at which point the drive mechanism is activated. After activating the drive mechanism, the smart glasses continuously monitor the user's interaction status. If no control commands are received from the control terminal within a certain period (e.g., 60 seconds), and the user does not send a photo-taking command through interaction with the smart glasses, it is determined that no operation has occurred within the time limit. The smart glasses then control the drive mechanism to shut down and restore the camera module's posture to the default posture when no photo-taking operation was performed. The smart glasses then enter a low-power sleep state.

[0094] This embodiment utilizes smart glasses comprising a camera module, a driving mechanism, and a motion sensor. The driving mechanism adjusts the posture of the camera module, and the motion sensor monitors the user's head posture data. The method includes: determining a photo compensation angle for the camera module, the photo compensation angle being determined based on either the head posture data or an adjustment command sent by a control terminal; and controlling the driving mechanism to adjust the posture of the camera module according to the photo compensation angle. By monitoring head posture data or using adjustment commands to determine the photo compensation angle, and controlling the driving mechanism to adjust the posture of the camera module, this effectively counteracts the camera perspective shift caused by the user's head movements during the preview process, ensuring the camera module maintains a stable framing angle. This achieves real-time framing during the preview stage of the screenless camera glasses, improving the user's shooting experience, shooting accuracy, and shooting efficiency.

[0095] Based on the first embodiment of this application, in the second embodiment of this application, the content that is the same as or similar to that in the first embodiment described above can be referred to the above description, and will not be repeated hereafter. Based on this, please refer to... Figure 4 Step S10 further includes steps S11 to S13:

[0096] Step S11: Obtain the reference head pose corresponding to the target's field of view.

[0097] It should be noted that the target field of view for photography refers to the specific area of ​​the image that the user locks onto via the control terminal or that the smart glasses determine based on user behavior (such as gaze duration) and the desired image to be captured. When acquiring images, the smart glasses must keep the target field of view stable within the camera module's framing area. Simultaneously, while the user locks onto the target field of view, the smart glasses record the user's head posture data collected by the motion sensor at that moment, using this data as a reference head posture, including but not limited to parameters such as pitch and yaw angles.

[0098] Step S12: Calculate the attitude deviation of the pitch angle based on the reference head posture and the head posture data to determine the head posture deviation.

[0099] It's important to note that when the preview function is enabled, the camera module processes the current view captured by the smart glasses into a low-resolution compressed image in real time. The smart glasses then send this low-resolution compressed image to the control terminal's display interface for display. During this process, the user can freely move their head and view the real-time preview through the control terminal's display interface to evaluate the shooting angle and composition. The smart glasses' built-in motion sensors continuously collect the user's head posture data.

[0100] Understandably, smart glasses use a posture calculation algorithm to extract pitch angle parameters from a baseline head posture and extract real-time pitch angle parameters from the current head posture data. They then calculate the angle difference Δθ_head between the real-time pitch angle parameters and the pitch angle parameters in the baseline head posture. This angle difference is the user's head posture deviation Δθ_head.

[0101] Step S13: If the absolute value of the head posture deviation is greater than the deviation threshold and the duration of the deviation is greater than the preset duration, determine the shooting compensation angle of the camera module based on the head posture deviation.

[0102] It should be noted that the deviation threshold T is a preset pitch angle attitude deviation critical value (e.g., 15°); the preset duration is a preset minimum duration threshold (e.g., 0.3s) used to confirm the stability of the attitude deviation; the deviation duration refers to the duration during which the head attitude deviation is greater than the deviation threshold.

[0103] Understandably, when the absolute value of the head posture deviation |Δθ_head| is greater than the deviation threshold and the duration of the deviation is greater than the preset duration, the compensation angle of the camera module in the pitch angle is determined based on the head posture deviation. The specific method for determining the compensation angle of the camera module in the pitch angle is: compensation angle Δθ_cam = Function(Δθ_head).

[0104] In practical implementation, when taking photos through smart glasses, the yaw and roll angles of the camera module can be adjusted by the user automatically turning their head. This head movement caused by adjusting the yaw and roll angles does not affect the preview screen on the control terminal. Therefore, the camera module's photo compensation angle is primarily controlled by the compensation angle in the pitch angle.

[0105] In one feasible implementation, steps C11 to C13 may be included before step S11:

[0106] Step C11: Upon receiving the target locking instruction sent by the control terminal, the target locking instruction is parsed to determine the instruction generation time of the target locking instruction.

[0107] It should be noted that a target lock command refers to a control signal actively triggered by the user through the control terminal, used to instruct the smart glasses to lock onto the target field of view to be captured. Target lock commands can be generated in ways including, but not limited to, by the user tapping the display interface of the control terminal, issuing a preset voice command, or triggering it through a specific gesture.

[0108] It is understandable that the instruction generation time refers to the specific time point at which the target locking instruction is created, which is used to accurately correlate the head posture data collected by the motion sensor at the same moment.

[0109] Step C12: Obtain the target head posture acquired by the motion sensor at the instruction generation time.

[0110] Step C13: The target head pose is used as the reference head pose corresponding to the target's field of view.

[0111] It should be noted that the smart glasses extract the target head pose from the data stream collected by the motion sensor, which strictly corresponds to the command generation time. The target head pose is the head pose data recorded synchronously when the user locks onto the target's field of view for photography. Ultimately, the target head pose collected at the command generation time is used as the baseline head pose corresponding to the target's field of view for photography.

[0112] Understandably, taking a smartwatch as the control terminal as an example, the interaction process between a smartwatch and smart glasses is as follows: Figure 5As shown, after the smartwatch and smart glasses establish a communication connection, they can transmit control commands with low latency. If the smartwatch enters preview mode, it will send a corresponding enable signal to the smart glasses. The smart glasses can transmit a compressed preview image corresponding to the viewfinder image from the camera module to the smartwatch's display interface with low latency, and the smartwatch will display the real-time preview image on the display interface. At the same time, the smartwatch will prompt the user to look at the target field of view through voice or information displayed on the display interface. The user can lock the target field of view by tapping the display interface (i.e., the smartwatch screen), and at the same time record the reference head posture Δθ_target at this moment.

[0113] This embodiment obtains a reference head posture corresponding to the target's field of view; calculates the posture deviation of the pitch angle based on the reference head posture and the head posture data to determine the head posture deviation; when the absolute value of the head posture deviation is greater than a deviation threshold and the duration of the deviation is greater than a preset duration, the camera module's image compensation angle is determined based on the head posture deviation. Through this method, the image compensation angle can be accurately obtained to offset the field of view shift caused by head movement, ensuring the stability of the target's field of view.

[0114] Based on the first and / or second embodiments of this application, in the third embodiment of this application, the content that is the same as or similar to the above embodiments can be referred to the above description, and will not be repeated hereafter. Based on this, please refer to... Figure 6 Step S10 also includes steps S21 to S23:

[0115] Step S21: Upon receiving an adjustment instruction from the control terminal, determine the adjustment operation distance based on the adjustment instruction.

[0116] It should be noted that upon receiving an adjustment command from the control terminal, the command is parsed to determine the adjustment operation distance. The specific forms of this adjustment operation distance include, but are not limited to, any of the following: the vertical distance generated when the user slides on the control terminal's display interface; the actual physical displacement generated by the movement of the physical controls on the control terminal, and other forms.

[0117] Step S22: Calculate the adjustment angle of the camera module in the pitch angle dimension based on the adjustment operation distance and the target mapping function, wherein the target mapping function includes at least the adjustment mapping coefficient.

[0118] It should be noted that the target mapping function is a function model used to establish the mathematical relationship between the adjustment operation distance and the compensation angle of the camera module; wherein, the adjustment mapping coefficient is used to define the proportional relationship between the adjustment operation distance and the adjustment angle. In this embodiment, the adjustment mapping coefficient is set to k = 0.1° / pixel, but it can also be adjusted according to the requirements. This embodiment does not impose any restrictions on this.

[0119] Understandably, by substituting the adjustment operation distance into the target mapping function for calculation, the compensation angle of the camera module in the pitch angle can be obtained. In this embodiment, the specific formula of the target mapping function is: Δθ_manual = k×Δy, where Δθ_manual is the compensation angle of the camera module in the pitch angle, and Δy is the adjustment operation distance. The adjustment angle corresponding to the pitch angle dimension refers to the compensation angle of the camera module in the pitch angle.

[0120] Step S23: Determine the image compensation angle of the camera module based on the adjustment angle corresponding to the pitch angle dimension.

[0121] It should be noted that the calculated camera angle in the pitch angle dimension is used as the camera module's photo compensation angle for subsequent photo compensation.

[0122] In one feasible implementation, steps D11 to D13 may be included before step S21:

[0123] Step D11: With the control terminal in preview mode, acquire the initial image captured by the camera module.

[0124] It should be noted that after receiving the preview mode activation signal sent by the control terminal, the smart terminal determines that the control terminal has entered the preview mode. At this time, the smart terminal will enable the preview function and obtain the initial image captured in real time by the camera module.

[0125] Step D12: Compress the initial image according to the target resolution to generate a preview compressed image corresponding to the initial image.

[0126] It should be noted that the target resolution is a pre-set image resolution. To balance preview clarity and transmission efficiency, the target resolution is lower than the resolution of the initial image. The smart glasses downsample the initial image and further compress the downsampled image according to the target resolution, generating a smaller, lower-data-volume compressed preview image. Besides the above method, the initial image can also be compressed directly according to the target resolution; this embodiment does not impose any restrictions on this approach.

[0127] Step D13: Send the preview compressed image to the control terminal for display, so that the control terminal can generate adjustment instructions and provide feedback based on the user's adjustment operation distance.

[0128] It should be noted that the smart glasses send a preview compressed image to the control terminal, which displays the preview compressed image in real time. After the user observes the preview compressed image on the display interface of the control terminal, they can initiate an adjustment operation according to actual needs. The control terminal generates a corresponding adjustment command based on the user's adjustment operation distance and sends it to the smart glasses.

[0129] It is understood that in this embodiment, the smart glasses can sequentially enter an automatic compensation process and a manual compensation process during the photo-taking process, as detailed below. Figure 7 The following example illustrates the concept; the parameters shown are for illustrative purposes only and are not intended to limit the scope of the device. After determining the baseline head posture corresponding to the target's field of view, the smart glasses continuously monitor head posture data using motion sensors and calculate the pitch angle deviation Δθ_head between the head posture data and the baseline head posture. If the absolute value of the deviation |Δθ_head| is greater than 15° and the duration reaches 0.3s, the camera module's image compensation angle Δθ_cam is determined based on the pitch angle deviation Δθ_head, and the camera module is compensated accordingly. A preview of the updated compensated image is then sent to the control terminal. The control terminal sends an adjustment command to the smart glasses. The smart glasses determine the camera module's pitch compensation angle Δθ_manual based on the adjustment operation distance Δy in the command, and determine the camera module's image compensation angle Δθ_cam+ based on the pitch compensation angle Δθ_manual, and then compensate the camera module accordingly. The system sends a preview image of the updated, fine-tuned screen to the control terminal. The user then clicks the camera button on the control terminal, which sends a photo-taking trigger command. At this point, the smart glasses control the camera module to take a high-resolution photo.

[0130] This embodiment determines the adjustment operation distance based on the adjustment command sent by the control terminal; calculates the adjustment angle of the camera module in the pitch angle dimension based on the adjustment operation distance and a target mapping function, wherein the target mapping function includes at least an adjustment mapping coefficient; and determines the shooting compensation angle of the camera module based on the adjustment angle of the camera module in the pitch angle dimension. This method ensures the accuracy and real-time performance of the adjustment action, improving the shooting experience.

[0131] It should be noted that the above examples are only for understanding this application and do not constitute a limitation on the control method of the smart glasses in this application. Any simple modifications based on this technical concept are within the protection scope of this application.

[0132] This application also provides a control device for smart glasses; please refer to [reference needed]. Figure 8 The control device for the smart glasses includes:

[0133] The processing module 10 is used to determine the photo compensation angle of the camera module, which is determined based on one of the head posture data and the adjustment command sent by the control terminal.

[0134] The control module 20 is used to control the drive mechanism to adjust the posture of the camera module according to the photo compensation angle.

[0135] Optionally, the processing module 10 is further configured to:

[0136] Obtain the baseline head posture corresponding to the target's field of view in the photograph;

[0137] The head posture deviation is determined by calculating the pitch angle based on the reference head posture and the head posture data.

[0138] If the absolute value of the head posture deviation is greater than the deviation threshold and the duration of the deviation is greater than the preset duration, the camera module's photo compensation angle is determined based on the head posture deviation.

[0139] Optionally, the processing module 10 is further configured to:

[0140] Upon receiving a target locking command from the control terminal, the target locking command is parsed to determine the command generation time.

[0141] The motion sensor acquires the target head posture at the instruction generation time;

[0142] The target head posture is used as the reference head posture corresponding to the target's field of view.

[0143] Optionally, the processing module 10 is further configured to:

[0144] Upon receiving an adjustment instruction from the control terminal, the adjustment operation distance is determined based on the adjustment instruction;

[0145] Based on the adjustment operation distance and the target mapping function, the adjustment angle of the camera module in the pitch angle dimension is determined, wherein the target mapping function includes at least an adjustment mapping coefficient;

[0146] The image compensation angle of the camera module is determined based on the adjustment angle corresponding to the pitch angle dimension.

[0147] Optionally, the processing module 10 is further configured to:

[0148] When the control terminal enters preview mode, the initial image captured by the camera module is obtained;

[0149] The initial image is compressed according to the target resolution to generate a preview compressed image corresponding to the initial image;

[0150] The preview compressed image is sent to the control terminal for display, so that the control terminal can generate adjustment instructions based on the user's adjustment operation distance and provide feedback.

[0151] Optionally, the control module 20 is further configured to:

[0152] When the camera module detects a photo-taking trigger command sent by the control terminal, it is controlled to capture the finely adjusted image according to the target photo-taking parameters.

[0153] Upon completion of image acquisition and receipt of a preview mode shutdown signal from the control terminal, the drive mechanism is shut down, and the camera module's posture is restored to its initial state.

[0154] Optionally, the processing module 10 is further configured to:

[0155] Upon receiving a communication connection request from the control terminal, a whitelist verification is performed based on the terminal identifier of the control terminal to obtain the pairing verification result of the control terminal.

[0156] The validity of the communication connection request is verified to obtain the validity verification result of the communication connection request;

[0157] When both the pairing verification result and the legality verification result pass the verification, the system responds to the communication connection request and establishes a communication connection with the control terminal.

[0158] The control device for smart glasses provided in this application, employing the control method for smart glasses in the above embodiments, can solve the technical problem in the prior art where the camera angle of screenless camera glasses changes during the preview of the photographed image. Compared with the prior art, the beneficial effects of the control device for smart glasses provided in this application are the same as those of the control method for smart glasses provided in the above embodiments, and other technical features in the control device for smart glasses are the same as those disclosed in the methods of the above embodiments, and will not be repeated here.

[0159] This application provides a smart glasses, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, which are executed by the at least one processor to enable the at least one processor to perform the control method of the smart glasses in the first embodiment described above.

[0160] The following is for reference. Figure 9 The diagram illustrates a structural schematic suitable for implementing smart glasses in the embodiments of this application. The smart glasses in the embodiments of this application may include, but are not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 9 The smart glasses shown are merely an example and should not be construed as limiting the functionality and scope of use of the embodiments of this application.

[0161] like Figure 9 As shown, the smart glasses may include a processing device 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1002 or a program loaded from a storage device 1003 into a random access memory (RAM) 1004. The RAM 1004 also stores various programs and data required for the operation of the smart glasses. The processing device 1001, ROM 1002, and RAM 1004 are interconnected via a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems can be connected to the I / O interface 1006: input devices 1007 including, for example, a touchscreen, touchpad, keyboard, mouse, image sensor, microphone, accelerometer, gyroscope, etc.; output devices 1008 including, for example, a liquid crystal display (LCD), speaker, vibrator, etc.; storage devices 1003 including, for example, magnetic tape, hard disk, etc.; and communication devices 1009. The communication device 1009 allows the smart glasses to communicate wirelessly or wiredly with other devices to exchange data. While the figures show smart glasses with various systems, it should be understood that implementing or having all of the systems shown is not required. More or fewer systems may be implemented alternatively.

[0162] Specifically, according to the embodiments disclosed in this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from storage device 1003, or installed from ROM 1002. When the computer program is executed by processing device 1001, it performs the functions defined in the methods of the embodiments disclosed in this application.

[0163] The smart glasses provided in this application, employing the control method of the smart glasses in the above embodiments, can solve the technical problem in the prior art where the camera angle of screenless camera glasses changes during the preview of the photographed image. Compared with the prior art, the beneficial effects of the smart glasses provided in this application are the same as those of the control method of the smart glasses provided in the above embodiments, and other technical features of the smart glasses are the same as those disclosed in the method of the previous embodiment, and will not be repeated here.

[0164] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.

[0165] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0166] This application provides a computer-readable storage medium having computer-readable program instructions (i.e., a computer program) stored thereon, the computer-readable program instructions being used to execute the control method for smart glasses in the above embodiments.

[0167] The computer-readable storage medium provided in this application may be, for example, a USB flash drive, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, system, or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.

[0168] The aforementioned computer-readable storage medium may be included in the smart glasses; or it may exist independently and not assembled into the smart glasses.

[0169] The aforementioned computer-readable storage medium carries one or more programs that, when executed by the smart glasses, cause the smart glasses to: determine the image compensation angle of the camera module, the image compensation angle being determined based on one of the head posture data and the adjustment command sent by the control terminal; and control the drive mechanism to adjust the posture of the camera module according to the image compensation angle.

[0170] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a Local Area Network (LAN) or a Wide Area Network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0171] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0172] The modules described in the embodiments of this application can be implemented in software or hardware. The names of the modules do not necessarily limit the functionality of the unit itself.

[0173] The readable storage medium provided in this application is a computer-readable storage medium that stores computer-readable program instructions (i.e., a computer program) for executing the control method of the above-described smart glasses. This solves the technical problem in the prior art where the camera angle of screenless camera glasses changes during the preview of the photographed image. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as those of the control method of the smart glasses provided in the above embodiments, and will not be repeated here.

[0174] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the control method for smart glasses as described above.

[0175] The computer program product provided in this application solves the technical problem that the camera angle of screenless camera glasses changes during the preview of the captured image in the prior art. Compared with the prior art, the beneficial effects of the computer program product provided in this application are the same as those of the control method for smart glasses provided in the above embodiments, and will not be repeated here.

[0176] The above description is only a part of the embodiments of this application and does not limit the patent scope of this application. All equivalent structural transformations made under the technical concept of this application and using the contents of the specification and drawings of this application, or direct / indirect applications in other related technical fields, are included in the patent protection scope of this application.

Claims

1. A control method of smart glasses, characterized by, The method is applied to smart glasses, the smart glasses are screen-free glasses, the smart glasses include a camera module, a driving mechanism and a motion sensor, the driving mechanism is used to adjust the posture of the camera module, and the motion sensor is used to monitor head posture data of a user; The method comprises: In the case that the control terminal enters a preview mode, an initial picture collected by the camera module is acquired; wherein the control terminal is an independent device that establishes a communication connection with the smart glasses; The initial picture is compressed according to a target resolution, a preview compressed picture corresponding to the initial picture is generated, and the preview compressed picture is sent to the control terminal for display, so that the control terminal determines that the framing direction of the camera module is adjusted to a specific angle correction parameter corresponding to a target posture based on an adjustment operation of a user and outputs a corresponding adjustment instruction to the smart glasses; The actual framing direction of the camera module is determined according to the head posture data, and a photographing compensation angle is generated; The photographing compensation angle of the camera module is determined, and the photographing compensation angle is determined according to the head posture data or the adjustment instruction sent by the control terminal; The driving mechanism is controlled to adjust the posture of the camera module according to the photographing compensation angle.

2. The method of claim 1, wherein, The step of determining the photographing compensation angle of the camera module comprises: A reference head posture corresponding to a target photographing field of view is acquired; A head posture deviation is determined by performing a pitch angle posture deviation calculation according to the reference head posture and the head posture data; In the case that the absolute value of the head posture deviation is greater than a deviation threshold value and the deviation duration is greater than a preset duration, the photographing compensation angle of the camera module is determined according to the head posture deviation.

3. The method of claim 2, wherein, Before the step of acquiring the reference head posture corresponding to the target photographing field of view, the method further comprises: When the target locking instruction sent by the control terminal is received, the target locking instruction is analyzed to determine the instruction generation time of the target locking instruction; A target head posture collected by the motion sensor at the instruction generation time is acquired; The target head posture is taken as the reference head posture corresponding to the target photographing field of view.

4. The method of claim 1, wherein, The step of determining the photographing compensation angle of the camera module comprises: In the case that the adjustment instruction sent by the control terminal is received, the adjustment operation distance is determined according to the adjustment instruction; The adjustment angle corresponding to the camera module in the pitch angle dimension is determined by performing calculation according to the adjustment operation distance and a target mapping function, wherein the target mapping function at least comprises an adjustment mapping coefficient; The photographing compensation angle of the camera module is determined according to the adjustment angle corresponding to the camera module in the pitch angle dimension.

5. The method of any one of claims 1 to 4, wherein, After the step of controlling the driving mechanism to adjust the posture of the camera module according to the photographing compensation angle, the method further comprises: When the photographing trigger instruction sent by the control terminal is detected, the camera module is controlled to collect the fine-tuned picture according to the target photographing parameter. In a case that the image acquisition is completed and a closing signal of the preview mode sent by the control terminal is received, the driving mechanism is controlled to be closed, and the posture of the camera module is controlled to be restored to an initial state.

6. The method of any one of claims 1 to 4, wherein, Before the step of determining the photographing compensation angle of the camera module, the method further includes: When receiving a communication connection request sent by the control terminal, performing white list verification according to a terminal identifier of the control terminal to obtain a pairing verification result of the control terminal; Performing legality verification on the communication connection request to obtain a legality verification result of the communication connection request; When the pairing verification result and the legality verification result are both passed, responding to the communication connection request to establish a communication connection between the control terminal and the smart glasses.

7. A control device of smart glasses, characterized by, The control device of the smart glasses includes: The processing module is configured to determine a photographing compensation angle of a corresponding camera module based on the control method of the smart glasses according to any one of claims 1-6; The control module is configured to control the driving mechanism to adjust the posture of the camera module according to the photographing compensation angle.

8. An intelligent eyewear, characterized in that, The smart glasses include a memory, a processor, and a control program of the smart glasses stored in the memory and executable on the processor, and the control program of the smart glasses is configured to implement the steps of the control method of the smart glasses according to any one of claims 1-6.

9. A storage medium, characterized by The storage medium stores a control program of the smart glasses, and the control program of the smart glasses, when executed by the processor, implements the steps of the control method of the smart glasses according to any one of claims 1-6.

Images