Wearing reminder method, smart glasses, system and readable storage medium

Abstract

This application discloses a wearing reminder method, smart glasses, a system, and a readable storage medium, relating to the field of eyewear technology and applied to smart glasses. The smart glasses include a camera module. The method includes: acquiring an eye image captured by the camera module; identifying the current position coordinates of multiple eye feature points based on the eye image; obtaining reference position coordinates of the multiple eye feature points under a pre-calibrated standard wearing position; comparing the current position coordinates with the reference position coordinates to obtain a wearing deviation parameter of the smart glasses; and outputting a prompt message when the wearing deviation parameter is greater than or equal to a preset threshold to remind the user to wear the smart glasses in a standard posture. This application enables users to promptly perceive whether the wearing position has shifted during the wearing of smart glasses.

Description

Technical Field

[0001] This application relates to the field of eyewear technology, and more particularly to a wearing reminder method, smart glasses, system, and readable storage medium. Background Technology

[0002] With the technological advancements in smart wearable devices, smart glasses have become widely used in daily consumption and vision control. Given the current prevalence of myopia among children and adolescents, defocus lenses, which can effectively inhibit myopia progression, are often combined with smart glasses to create vision protection products. The correct wearing posture of smart glasses directly affects their functionality, especially for products equipped with defocus lenses. Improper wearing can reduce the effectiveness of myopia control; therefore, detecting and reminding users of the correct wearing position is a crucial requirement.

[0003] Traditional methods for detecting the position of glasses include: first, subjective judgment, where users adjust the position of their glasses by observing in a mirror or by feeling; second, mechanical positioning, which involves setting physical markers or positioning structures on the eyeglass frame and aligning them with facial features; and third, external sensor methods, which use external cameras or depth sensors to capture images of the user's face and detect the position of the glasses through image processing or 3D reconstruction.

[0004] However, all of the above methods have significant limitations: subjective judgment relies on personal perception, making it difficult for users to detect minute shifts; mechanical positioning increases structural complexity and cannot adapt to individual facial differences; external sensor methods require additional equipment, limiting application scenarios and failing to provide real-time feedback during user wear. Therefore, existing technologies generally lack a means to allow users to promptly ascertain whether their wearing position is correct during wear. Thus, enabling users to promptly perceive any shifts in the wearing position while wearing smart glasses has become a pressing technical problem to be solved. Summary of the Invention

[0005] The main purpose of this application is to provide a wearing reminder method, smart glasses, system and readable storage medium, aiming to solve the technical problem of how to enable users to perceive whether the wearing position has shifted in a timely manner during the wearing of smart glasses.

[0006] To achieve the above objectives, this application provides a wearing reminder method applied to smart glasses, the smart glasses including a camera module, the wearing reminder method comprising the following steps:

[0007] Acquire the eye image captured by the camera module; Based on the eye image, identify the current position coordinates of multiple eye feature points, and obtain the reference position coordinates of multiple eye feature points under the pre-calibrated standard wearing position; The current position coordinates are compared with the reference position coordinates to obtain the wearing deviation parameter of the smart glasses. When the wearing deviation parameter is greater than or equal to a preset threshold, a prompt message is output to remind the user to wear the smart glasses in a standard posture.

[0008] In one embodiment, the step of comparing the current position coordinates with the reference position coordinates to obtain the wearing deviation parameters of the smart glasses includes: Perform point set registration between the point set formed by the current position coordinates and the point set formed by the reference position coordinates, and solve for the rigid body transformation matrix from the reference position coordinates to the current position coordinates. The translation vector and rotation angle are decomposed from the rigid body transformation matrix; The wearing deviation parameters of the smart glasses are determined based on the translation vector and the rotation angle.

[0009] In one embodiment, the rigid body transformation matrix is ​​a transformation matrix in a three-dimensional spatial coordinate system. The three-dimensional spatial coordinate system is established with the user's head as a reference, with the X-axis pointing horizontally to the right, the Y-axis pointing horizontally forward, and the Z-axis pointing vertically upward. The step of determining the wearing deviation parameters of the smart glasses based on the translation vector and the rotation angle includes: Extract the horizontal offset, vertical offset, and forward / backward offset from the translation vector, wherein the horizontal offset corresponds to the displacement in the X-axis direction, the vertical offset corresponds to the displacement in the Z-axis direction, and the forward / backward offset corresponds to the displacement in the Y-axis direction. Extract the forward tilt angle around the X-axis and the left and right tilt angle around the Y-axis from the rotation angle; The horizontal offset, the vertical offset, the forward and backward offset, the forward tilt angle, and the left and right tilt angle are used as the wearing deviation parameters of the smart glasses.

[0010] In one embodiment, after the step of comparing the current position coordinates with the reference position coordinates to obtain the wearing deviation parameter of the smart glasses, the method further includes: Each of the aforementioned wearing deviation parameters is compared with its corresponding preset threshold. If any of the wearing deviation parameters is greater than or equal to the corresponding preset threshold, an adjustment suggestion is determined based on the wearing deviation parameter that is greater than or equal to the corresponding preset threshold, wherein the adjustment suggestion includes the adjustment direction and the adjustment amount; The adjustment suggestions are delivered via visual, auditory, or vibration methods.

[0011] In one embodiment, the step of obtaining the reference position coordinates of multiple eye feature points under a pre-calibrated standard wearing position includes: A standard eye image is obtained by capturing the user's eye image when the user is in a standard wearing posture; Multiple eye feature points are identified from the standard eye image, and the two-dimensional coordinates of the eye feature points in the standard eye image are obtained; The two-dimensional coordinates are converted into three-dimensional coordinates in a preset three-dimensional spatial coordinate system and stored as reference position coordinates.

[0012] In one embodiment, the camera module includes an infrared camera and an RGB camera, and the step of converting the two-dimensional coordinates into three-dimensional coordinates in a preset three-dimensional spatial coordinate system includes: Acquire a first type of feature point and a second type of feature point from a plurality of eye feature points, wherein the first type of feature point is an eye feature point identified based on an infrared image acquired by the infrared camera, and the second type of feature point is an eye feature point identified based on a visible light image acquired by the RGB camera. Initial estimates of the three-dimensional coordinates are set for each of the aforementioned eye feature points; Establish the projection relationship from 3D spatial coordinates to image coordinates based on camera intrinsic parameters and distortion model; Construct an objective function, which includes a first term and a second term, wherein the first term is the weighted sum of squared projection errors of each of the first type of feature points, and the second term is the weighted sum of squared projection errors of each of the second type of feature points. The projection error refers to the deviation between the theoretical image coordinates calculated by the projection relationship based on the current estimated three-dimensional coordinates of the feature points and the actual image coordinates of the eye feature points in the corresponding image. With the objective function as the goal, the estimated three-dimensional coordinates of each eye feature point are iteratively updated, and the converged three-dimensional coordinates are used as the reference position coordinates of the eye feature points.

[0013] In one embodiment, the eye feature points include at least three feature points selected from the pupil center point, eyeball center point, eyelid key point, inner canthus point, outer canthus point, and iris center point, wherein the eyelid key point includes the eyelid edge point and / or the midpoint of the eyelid fitting contour.

[0014] In addition, to achieve the above objectives, this application also provides smart glasses, which include a camera module and a processor; The camera module is used to capture images of the eyes; The processor is used to perform the steps of the wearing reminder method described above.

[0015] In addition, to achieve the above objectives, this application also provides a wearing reminder system, which includes smart glasses and an edge device that are connected in communication, and the smart glasses include a camera module; The smart glasses are used to capture eye images and send the eye images to the edge device; The edge device is used to perform the steps of the wearing reminder method described above.

[0016] In addition, to achieve the above objectives, this application also provides a readable storage medium, which is a computer-readable storage medium, on which a computer program is stored, and the computer program is executed by a processor to implement the steps of the wearing reminder method as described above.

[0017] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the wearing reminder method described above.

[0018] One or more technical solutions proposed in this application have at least the following technical effects: This application embodiment acquires eye images through the camera module built into the smart glasses themselves, without the need for additional external equipment. Utilizing the built-in characteristics of the camera module, it directly obtains eye-related image information and identifies the current coordinates of multiple eye feature points based on these images. Simultaneously, it retrieves the reference coordinates of multiple eye feature points under a pre-calibrated standard wearing position, comparing the current coordinates with the reference coordinates to quantify the wearing deviation parameters of the smart glasses. This coordinate comparison and deviation quantification mechanism allows for the precise capture of minute shifts during the wearing process of the smart glasses. It does not rely on the user's personal perception, nor does it require additional physical positioning structures. It avoids both the perceptual limitations of subjective judgment methods and the structural defects of mechanical positioning methods. Furthermore, relying on the built-in camera module for real-time acquisition and comparison overcomes the dependence on additional equipment and the limitations of application scenarios associated with external sensor methods. When the wearing position shifts and the deviation reaches a preset threshold, a timely prompt is output, allowing the user to be aware of the wearing status in real time. Therefore, the embodiments of this application realize continuous monitoring and active feedback of the wearing position during the wearing of smart glasses, enabling users to perceive in real time whether their wearing status is in the standard position, ensuring that optical components such as defocus lenses are always accurately aligned with the eyeball, thereby ensuring the stable performance of its myopia prevention and control function. Attached Figure Description

[0019] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0020] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 This is a flowchart illustrating the first embodiment of the wearing reminder method of this application; Figure 2 This is a schematic diagram illustrating the process for determining wearing deviation parameters in one embodiment of the wearing reminder method of this application; Figure 3 This is a schematic diagram of the eyeglass structure of the smart glasses in this application; Figure 4 This is a schematic diagram of the system architecture of the wearing reminder system in this application; Figure 5 This is a schematic diagram of the hardware operating environment of the wearing reminder method device in the embodiments of this application.

[0022] The purpose, features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0023] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0024] With the widespread use of electronic products and the increasing intensity of study and work in modern society, myopia has become one of the major problems affecting the visual health of the global public. According to statistics from the World Health Organization, more than 2 billion people worldwide suffer from myopia, and the incidence rate is increasing year by year. Among myopia correction technologies, defocus lenses, as a new type of optical correction method, can effectively slow down the growth of the axial length of the eye by forming a specific defocus signal in the periphery of the retina, thereby controlling the progression of myopia.

[0025] However, the therapeutic effect of defocus lenses is highly dependent on the accuracy of the relative position between the lens and the eyeball. Studies have shown that when the center of the lens deviates from the center of the pupil by more than 2 mm, the defocus effect is significantly weakened and may even have a negative impact on visual health. Therefore, ensuring that users wear their glasses correctly in the standard posture is crucial for maximizing the therapeutic function of defocus lenses.

[0026] Currently, the detection of the position of glasses wearing mainly relies on the following methods: (1) Subjective judgment method: Users adjust the position of the glasses by observing in the mirror or by feeling. This method is highly subjective, lacks quantitative standards, and makes it difficult to guarantee the accuracy of wearing.

[0027] (2) Mechanical positioning method: Physical markers or positioning structures are set on the eyeglass frame to achieve positioning by aligning with facial features. This method is structurally complex and increases the weight and manufacturing cost of the eyeglasses.

[0028] (3) External sensor method: This method uses an external camera or depth sensor to acquire images of the user's face and then uses 3D reconstruction to detect the position of the glasses. This method requires additional equipment and its application scenarios are limited.

[0029] In summary, existing technologies generally suffer from insufficient accuracy, inconvenience of use, high cost, or limited application scenarios, making it difficult to meet the urgent need of modern functional glasses for accurate detection of wearing position, thus hindering users from knowing in a timely manner whether their current wearing position is standard.

[0030] To address the aforementioned issues, the main solution of this application is as follows: acquiring eye images captured by the camera module on the smart glasses; identifying the current position coordinates of multiple eye feature points based on the eye images, and obtaining reference position coordinates of the multiple eye feature points under a pre-calibrated standard wearing position; comparing the current position coordinates with the reference position coordinates to obtain the wearing deviation parameter of the smart glasses; and outputting a prompt message when the wearing deviation parameter is greater than or equal to a preset threshold, reminding the user to wear the smart glasses in a standard posture.

[0031] This application acquires eye images using the camera module built into the smart glasses themselves, without the need for additional external equipment. Utilizing the built-in features of the camera module, it directly obtains eye-related image information and identifies the current coordinates of multiple eye feature points based on these images. Simultaneously, it retrieves the reference coordinates of multiple eye feature points under a pre-calibrated standard wearing position, comparing the current coordinates with the reference coordinates to quantify the wearing deviation parameters of the smart glasses. This coordinate comparison and deviation quantification mechanism allows for the precise capture of minute shifts during smart glasses wear. It eliminates the need for user perception and additional physical positioning structures, thus avoiding the perceptual limitations of subjective judgment methods and the structural defects of mechanical positioning methods. Furthermore, relying on the built-in camera module for real-time acquisition and comparison overcomes the dependence on additional equipment and the limitations of application scenarios associated with external sensor methods. When the wearing position shifts and the deviation reaches a preset threshold, a timely prompt is output, allowing the user to be aware of the wearing status in real time. Therefore, this application realizes continuous monitoring and active feedback of the wearing position during the wearing of smart glasses, enabling users to perceive in real time whether their wearing status is in the standard position, ensuring that optical components such as defocus lenses are always accurately aligned with the eyeball, thereby ensuring the stable performance of its myopia prevention and control function.

[0032] It should be noted that the execution subject of the various embodiments of the wearing reminder method of this application can be an edge device with data processing, network communication and program running functions, such as a tablet computer, personal computer, mobile phone, server, etc., or a smart glasses that can achieve the above functions. The various embodiments of the wearing reminder method of this application do not impose specific limitations on this.

[0033] Based on this, this application proposes a wearing reminder method according to a first embodiment. In this embodiment, it is applied to smart glasses, which include a camera module. (Refer to...) Figure 1 As shown, the wearing reminder method includes the following steps S10~S30: Step S10: Obtain the eye image captured by the camera module; The camera module may include one or more miniature cameras, typically embedded inside the frame of smart glasses, such as above the lenses, on either side of the nose bridge, or at the front of the temples, in a concealed position facing the eyeball, to ensure clear and stable acquisition of images of the user's eye area.

[0034] When the user is wearing smart glasses and the device is powered on, the camera module can automatically start shooting at a preset sampling frequency, such as once per second or once per minute, or trigger the acquisition when it detects that the user's head movement is becoming stable, in order to obtain eye images containing structures such as the iris, pupil, and corner of the eye.

[0035] Step S20: Based on the eye image, identify the current position coordinates of multiple eye feature points, and obtain the reference position coordinates of multiple eye feature points under the pre-calibrated standard wearing position; Multiple representative eye feature points can be extracted from the acquired eye images using image recognition algorithms. These feature points include the pupil center point, eyeball center point, key eyelid points, inner canthus point, outer canthus point, and iris center point. The key eyelid points include the eyelid edge point and / or the midpoint of the fitted eyelid contour. To ensure the accuracy of subsequent comparisons, this embodiment selects at least three non-collinear eye feature points to form a feature point set.

[0036] During feature point recognition, appropriate detection algorithms can be selected based on different feature point types. For example, for pupil center point detection, one or a combination of the following methods can be used: Preferably, a deep learning-based method can be used to train a convolutional neural network or semantic segmentation network (such as U-Net) to directly output the pupil region mask or center point coordinates; alternatively, traditional image processing methods can be used, taking advantage of the pupil's characteristic of appearing as a high-contrast dark spot under infrared light, to locate the pupil contour and calculate the center coordinates through threshold segmentation, edge detection, ellipse fitting, and other steps; alternatively, a fusion method of image processing and neural networks can be used, combining traditional feature extraction and deep learning classifiers to achieve high-precision pupil key point localization. For eyelid key point detection, preferably, a semantic segmentation network can be used to segment the eyelid region and extract edge contour points; alternatively, facial key point detection algorithms (such as Dlib, MediaPipe, etc.) can be used to extract upper and lower eyelid contour points to help determine the fit between the glasses and the eyes.

[0037] It should be noted that the position coordinates can be selected as two-dimensional or three-dimensional coordinates according to the actual application requirements: two-dimensional coordinates can be the coordinates in the image pixel coordinate system; three-dimensional coordinates can be obtained by combining the depth perception function of the camera module to achieve accurate representation of the spatial position of eye feature points.

[0038] Reference position coordinates are coordinate data calibrated and stored after the camera module pre-captures eye images and identifies common feature points when the user wears the smart glasses in a standard posture. These coordinates are consistent with the type and dimension of the current position coordinates. This calibration process can be performed during the first wear of the smart glasses or when the user actively calibrates them, to adapt to the individual facial features of different users and ensure the accuracy of the comparison benchmark. A standard wearing posture is generally defined as a state where the glasses are worn in the optical design position, the lens optical center is aligned with the pupil, and the frame is horizontal and fits well against the face.

[0039] Step S30: Compare the current position coordinates with the reference position coordinates to obtain the wearing deviation parameter of the smart glasses. When the wearing deviation parameter is greater than or equal to a preset threshold, output a prompt message to remind the user to wear the smart glasses in a standard posture.

[0040] By mathematically comparing two sets of coordinate points, the spatial positional difference between the actual wearing state and the standard wearing state is quantified. This comparison is essentially a point set registration problem—by finding the optimal spatial transformation relationship to align the current coordinate point set as closely as possible with the reference coordinate point set, the physically meaningful wearing deviation is decoupled from the transformation parameters. Based on the comparison result of the calculated deviation parameters and a preset threshold, a corresponding feedback mechanism can be automatically triggered to convey the wearing status information to the user in a perceptible form.

[0041] The wearing deviation parameter may specifically include at least one of the following: horizontal offset, vertical offset, front-to-back offset, forward tilt angle, and left-to-right tilt angle. Among them: Horizontal offset refers to the lateral displacement of the center of the left and right pupils relative to the standard position; Vertical offset refers to the longitudinal displacement of the center of the left and right pupils relative to the standard position; Anterior-posterior offset refers to the change in distance between the eyeball and the lens; The forward tilt angle refers to the angle at which the glasses are tilted in a vertical plane, reflecting whether the frame slides down the bridge of the nose. The left and right tilt angle refers to the angular deviation caused by the relative height difference between the left and right temples.

[0042] The aforementioned deviation parameters are compared with preset thresholds. These preset thresholds can be pre-set according to the specific requirements of the functional eyeglasses; for example, a horizontal offset threshold of 2mm, a vertical offset threshold of 2mm, and a forward tilt angle threshold of 3°. This threshold range matches the optical tolerance requirements of the defocused lens. When any deviation parameter reaches or exceeds the corresponding threshold, it is determined that the current wearing position has deviated from the standard state, which may affect the control effect of the defocused lens or the accuracy of eye tracking.

[0043] At this time, the smart glasses can output a prompt message. The prompt message can take one or more of the following forms: Visual cues: Display deviation diagrams and text prompts on the glasses' display screen or the screen of the accompanying device; Voice prompts: The system will announce the current wearing deviation and specific adjustment suggestions, such as "Please lower the left temple of the glasses by about 5mm" or "Please adjust the glasses slightly to the right." Vibration alert: The built-in vibration motor in the temples indicates the direction and degree of deviation through different vibration modes.

[0044] Once the user receives the prompt, they can actively adjust their wearing posture until the deviation parameter falls below the threshold. Through this feedback mechanism, real-time monitoring and proactive reminders of the wearing position of the smart glasses are achieved, effectively solving the problems of insufficient positioning accuracy, low detection efficiency, lack of personalized adaptation, and lack of real-time feedback in traditional detection methods. This ensures that optical components such as defocused lenses are always in a precisely aligned working state, thereby guaranteeing the stable performance of the functional glasses.

[0045] Based on the first embodiment of this application, in the second embodiment of this application, the content that is the same as or similar to that in the first embodiment described above can be referred to the above description and will not be repeated hereafter. Based on this, the step of comparing the current position coordinates with the reference position coordinates to obtain the wearing deviation parameters of the smart glasses includes: Step A10: Perform point set registration between the point set formed by the current position coordinates and the point set formed by the reference position coordinates, and solve for the rigid body transformation matrix from the reference position coordinates to the current position coordinates. Based on the identified multiple eye feature points, the current position coordinates are constructed as a current point set P = {p1, p2, ..., pn}, and the reference position coordinates are constructed as a reference point set Q = {q1, q2, ..., qn}, where n is the number of feature points and n ≥ 3 to ensure the degrees of freedom of spatial transformation. To solve the spatial transformation relationship between the two sets of point sets, a point set registration algorithm is used for optimal matching.

[0046] Specifically, point set registration algorithms such as Procrustes analysis and the iterative nearest point algorithm can be used. Preferably, given the known correspondence between point sets, the iterative nearest point algorithm is used to solve the problem. Through iterative optimization, the rigid body transformation matrix T that minimizes the sum of squared distances between the current point set P and the reference point set Q after transformation is found. The rigid body transformation matrix T is a 4×4 homogeneous transformation matrix, with the form T=[R t; 0 1], where R is a 3×3 rotation matrix and t is a 3×1 translation vector. This matrix completely describes the spatial rigid body motion required to transform from the reference position coordinates to the current position coordinates.

[0047] Step A20: Decompose the translation vector and rotation angle from the rigid body transformation matrix; The obtained rigid body transformation matrix is ​​decomposed. The translation vector t = [t x , t y , t z ] The elements in the first three rows of the fourth column of the matrix can be directly read, representing the translation along the x-axis (usually left-right), y-axis (usually up-down), and z-axis (usually forward-backward) of the smart glasses coordinate system. The rotation matrix R needs to be further calculated as the rotation angles around the coordinate axes: based on the structure of the rotation matrix, the rotation angle φ (roll angle) around the x-axis, the rotation angle θ (pitch angle) around the y-axis, and the rotation angle ψ (yaw angle) around the z-axis can be solved using trigonometric functions. Thus, the six-degree-of-freedom spatial transformation parameters, containing three translational components and three rotational components, are obtained from the rigid body transformation matrix.

[0048] Step A30: Determine the wearing deviation parameters of the smart glasses based on the translation vector and the rotation angle.

[0049] The six-degree-of-freedom spatial transformation parameters obtained from the decomposition are transformed into wearing deviation parameters with clear physical meaning. Position offset parameters can be determined based on the translation vector t: the horizontal offset corresponds to t. x , represents the displacement of the glasses in the left-right direction; the vertical offset corresponds to t. y , represents the vertical displacement of the glasses; the forward / backward offset corresponds to t. z This indicates the change in distance between the eyeball and the lens.

[0050] Based on the rotation angle, attitude offset parameters can be determined: the forward tilt angle corresponds to the rotation angle φ around the x-axis, reflecting the degree of forward and backward tilt of the glasses in the vertical plane; the left and right tilt angle corresponds to the rotation angle θ around the y-axis, reflecting the angular deviation caused by the relative height difference between the left and right temples; the horizontal twist angle corresponds to the rotation angle ψ around the z-axis, reflecting the degree of left and right twist of the glasses in the horizontal plane.

[0051] The above six parameters or their combinations can constitute the wearing deviation parameters, which are used to compare with the preset threshold to determine whether the current wearing position deviates from the standard state.

[0052] In one possible implementation, the rigid body transformation matrix is ​​a transformation matrix in a three-dimensional spatial coordinate system. This three-dimensional spatial coordinate system is established with the user's head as a reference, with the X-axis pointing horizontally to the right, the Y-axis pointing horizontally forward, and the Z-axis pointing vertically upward. The step of determining the wearing deviation parameters of the smart glasses based on the translation vector and the rotation angle includes: Step B10: Extract the horizontal offset, vertical offset, and forward / backward offset from the translation vector, wherein the horizontal offset corresponds to the displacement in the X-axis direction, the vertical offset corresponds to the displacement in the Z-axis direction, and the forward / backward offset corresponds to the displacement in the Y-axis direction. "Using the user's head as a reference" means that the established three-dimensional spatial coordinate system is associated with the user's head and moves synchronously with the user's head movement. In other words, this coordinate system is a relative coordinate system defined on the user's head. When the user's head is in a natural, upright posture, the three axes of the coordinate system are defined as follows: the X-axis points horizontally to the user's right side, parallel to the line connecting the eyes; the Y-axis points horizontally to the user's front, consistent with the direction of the face; and the Z-axis points vertically to the top of the head, consistent with the direction of the vertical line from the forehead to the top of the head.

[0053] In this coordinate system, the spatial position and posture changes of smart glasses, as wearable devices worn on the head, can be accurately described by the translation and rotation around the three axes mentioned above. The advantage of this definition is that all wearing deviation parameters are referenced to the user's own anatomy, eliminating the interference of overall head movement on the detection results. This ensures that the measured offset truly reflects the change in the relative position between the glasses and the eyeballs, rather than the absolute movement of the head in space.

[0054] Translation vector t=[t x , t y , t z ] Each component in t directly corresponds to the displacement along the three axes in the defined three-dimensional coordinate system. Where t x This represents the displacement along the X-axis, specifically the horizontal left-right shift of the glasses relative to the head—a positive value indicates a rightward shift, and a negative value indicates a leftward shift; t z This represents the displacement along the Z-axis, specifically the vertical shift of the glasses relative to the head—a positive value indicates an upward shift, and a negative value indicates a downward shift; t y This represents the displacement along the Y-axis, that is, the forward and backward shift of the glasses relative to the head—a positive value indicates that the glasses shift forward (away from the eyeball), and a negative value indicates that they shift backward (closer to the eyeball).

[0055] Therefore, three position offset parameters with clear physical meaning are extracted from the translation vector.

[0056] Step B20: Extract the forward tilt angle around the X-axis and the left and right tilt angle around the Y-axis from the rotation angle; Based on the three rotation angles calculated from the rotation matrix R, and combined with the spatial coordinate system defined in this embodiment, each rotation angle can be assigned a clear physical meaning. Specifically, the rotation angle φ around the X-axis represents the rotation of the glasses around the left-right axis in the vertical plane, i.e., the forward tilt angle of the glasses—this angle reflects whether the frame slides back and forth on the bridge of the nose, causing it to tilt upwards or downwards; the rotation angle θ around the Y-axis represents the rotation of the glasses around the front-back axis in the coronal plane, i.e., the left-right tilt angle of the glasses—this angle reflects the lateral tilt of the frame caused by the relative height difference between the left and right temples. The rotation angle ψ around the Z-axis represents the rotation of the glasses around the vertical axis in the horizontal plane, i.e., the horizontal torsion angle of the glasses, and can be included in the wearing deviation parameters depending on the actual application requirements.

[0057] Step B30: The horizontal offset, the vertical offset, the front-to-back offset, the forward tilt angle, and the left-to-right tilt angle are used as the wearing deviation parameters of the smart glasses.

[0058] Extracted horizontal offset t x Vertical offset t z Front and rear offset t y The five parameters, along with the forward tilt angle φ and the left and right tilt angle θ, together constitute the wearing deviation parameter set of this embodiment. The above five parameters comprehensively describe the spatial offset state of the smart glasses relative to the standard wearing position from different dimensions: three displacement parameters reflect the positional changes of the frame in three-dimensional space, and two angle parameters reflect the posture changes of the frame.

[0059] This embodiment constructs a complete five-dimensional wearing deviation parameter system by decomposing the translation vector into horizontal offset, vertical offset, and forward / backward offset, and the rotation angle into forward tilt angle and left / right tilt angle. This decomposition transforms the originally abstract rigid body transformation matrix into spatial displacement and attitude parameters with clear physical meaning: the horizontal and vertical offsets directly characterize the alignment between the lens optical center and the pupil center; the forward / backward offset reflects the change in the key optical parameter of lens-eye distance; and the forward tilt angle and left / right tilt angle accurately depict the posture deflection of the frame caused by nose bridge slippage or temple height difference. Thus, the five parameters comprehensively cover the six-degree-of-freedom spatial offset state of the smart glasses relative to the head from both position and attitude perspectives. They can independently reflect small deviations in a single direction, and can also combine to present complex comprehensive offset situations. This provides clear and multi-dimensional quantitative basis for subsequent item-by-item comparison with preset thresholds, thereby ensuring that optical components such as defocused lenses are always in a working state of precise alignment with the eyeball.

[0060] Based on the first and / or second embodiments of this application, in the third embodiment of this application, the content that is the same as or similar to the first and second embodiments described above can be referred to the above description and will not be repeated hereafter. On this basis, after the step of comparing the current position coordinates with the reference position coordinates to obtain the wearing deviation parameters of the smart glasses, the method further includes: Step C10: Compare each of the wearing deviation parameters with the corresponding preset threshold; Threshold ranges are pre-defined for different types of wearing deviation parameters. These thresholds can be calibrated based on the optical tolerance requirements of the defocused lens, the accuracy requirements of the eye-tracking system, and clinical control standards. For example, the thresholds for horizontal and vertical offset can be set to ±2mm, the threshold for anterior-posterior offset can be set to ±1mm, and the thresholds for forward tilt and left-right tilt can be set to ±3°.

[0061] The determined wearing deviation parameters—such as horizontal offset t—are used to determine the wearing deviation parameters. x Vertical offset t2, forward and backward offset t The forward tilt angle φ and the left and right tilt angle θ are compared with the corresponding preset thresholds one by one to determine whether each parameter is within the allowable range, thereby identifying the specific deviation type that exceeds the tolerance limit in the current wearing state.

[0062] Step C20: If any of the wearing deviation parameters is greater than or equal to the corresponding preset threshold, then an adjustment suggestion is determined based on the wearing deviation parameters that are greater than or equal to the corresponding preset threshold, wherein the adjustment suggestion includes the adjustment direction and the adjustment amount; When any wearing deviation parameter is detected to reach or exceed its preset threshold, it is determined that there is an offset in the current wearing position that needs to be corrected. At this time, based on the specific value and sign of the parameter exceeding the limit, combined with the defined coordinate system direction rules, a targeted adjustment suggestion is generated. For example, if the horizontal offset t x If the value is positive and exceeds the limit, it indicates that the glasses are shifted to the right. The adjustment suggestion is to move the glasses to the left by approximately |t. x |mm”; If the vertical offset t2 is negative and exceeds the limit, it means the glasses are too low, and it is recommended to “raise the glasses by about |t2| millimeters”; If the forward tilt angle φ is positive and exceeds the limit, it means the frame is tilted forward and downward, and it is recommended to “lower the back of the temples appropriately” or “raise the nose pads appropriately”; If the left and right tilt angle θ is negative and exceeds the limit, it means the left temple is higher than the right, and it is recommended to “lower the left temple by about |θ| degrees” or “raise the right temple”.

[0063] When multiple parameters exceed their limits simultaneously, a combined adjustment suggestion can be generated, such as "Please move the entire pair of glasses 2 millimeters to the left, and lower the left temple by 1.5 degrees." In this way, the adjustment suggestion not only informs the user of the deviation, but also guides the user on how to correct it in a quantitative way, making the feedback information actionable.

[0064] Step C30: Output the adjustment suggestions via vision, voice, or vibration.

[0065] The generated adjustment suggestions will be output in a form that is perceptible to the user. The output method can be selected or combined according to the hardware configuration and usage scenario of the smart glasses: in the visual prompt mode, deviation diagrams, text descriptions and adjustment instructions can be displayed on the display screen of the smart glasses or the interface of the accompanying mobile app, such as using arrow animations to indicate the direction of movement and indicating the specific adjustment amount; In voice prompt mode, adjustment suggestions are broadcast through the built-in miniature speaker or bone conduction headphones of the smart glasses, such as "The glasses are 2 mm to the left, please adjust to the right", so that users can receive the information without looking at the screen; In vibration alert mode, a linear vibration motor embedded in the temple indicates the direction and degree of deviation with different frequencies, intensities and vibration sequences. For example, vibration on the left temple indicates that adjustment to that side is needed, and the vibration intensity is positively correlated with the amount of deviation.

[0066] Through the diverse feedback methods described above, users can obtain timely and accurate adjustment suggestions and perform corresponding operations in different usage scenarios until the wearing deviation parameters fall back to the threshold range, thus forming a closed-loop mechanism of "detection-judgment-feedback-adjustment".

[0067] In one possible implementation, the step of obtaining the reference position coordinates of the plurality of eye feature points under a pre-calibrated standard wearing position includes: Step D10: When in the standard wearing posture, acquire the user's eye image to obtain a standard eye image; The standard wearing posture refers to the state in which the smart glasses are worn in accordance with the optical design, the optical center of the lenses is aligned with the pupil, and the frame is horizontal and fits well against the face. In this state, the built-in camera module of the smart glasses captures images of the user's eyes, obtaining one or more clear images of the eyes, including structures such as the pupil, iris, corner of the eye, and eyelids, as standard eye images.

[0068] Furthermore, to ensure the accuracy of the calibration data, multiple frames of images can be acquired during the calibration process and the frame with the best imaging quality can be selected, or the recognition results of multiple frames can be averaged to reduce the random error caused by a single acquisition.

[0069] Step D20: Identify multiple eye feature points from the standard eye image and obtain the two-dimensional coordinates of the eye feature points in the standard eye image; The same image recognition algorithm as in step S20 can be used to process the standard eye image, locate and extract multiple eye feature points from the image, and ensure that the type, number and recognition algorithm of the feature points are consistent with those in the subsequent real-time detection stage to ensure comparability between the coordinate point clusters. Each feature point is assigned two-dimensional pixel coordinates (u, v) in the current standard image coordinate system, and the two-dimensional coordinates of multiple feature points together constitute the initial coordinate point set under standard wearing conditions.

[0070] Step D30: Convert the two-dimensional coordinates into three-dimensional coordinates in a preset three-dimensional spatial coordinate system and store them as reference position coordinates.

[0071] The obtained two-dimensional pixel coordinates are converted into three-dimensional coordinates in a preset three-dimensional spatial coordinate system. This three-dimensional spatial coordinate system is established with the user's head as the reference, where the X-axis points horizontally to the right, the Y-axis points horizontally forward, and the Z-axis points vertically upward.

[0072] The conversion process can be calculated by combining the internal parameters of the camera module (such as focal length and principal point coordinates) and external parameters (such as the position and attitude of the camera relative to the head). If the camera module has depth perception capabilities (such as binocular stereo vision, structured light, or time-of-flight ranging), it can directly obtain the depth information of feature points in space and reconstruct three-dimensional spatial coordinates by combining two-dimensional coordinates.

[0073] The resulting three-dimensional coordinates serve as the reference coordinates for each eye feature point in a standard wearing posture. These coordinates can be stored in the smart glasses' local memory as benchmark data for coordinate comparison during subsequent real-time detection. This calibration process establishes a personalized reference benchmark for each user, linked to the head coordinate system, providing a reliable basis for accurate quantification of wearing deviations.

[0074] In one possible implementation, the camera module includes an infrared camera and an RGB camera, and the step of converting the two-dimensional coordinates into three-dimensional coordinates in a preset three-dimensional spatial coordinate system includes: Step E10: Obtain a first type of feature point and a second type of feature point from the plurality of eye feature points, wherein the first type of feature point is an eye feature point identified based on the infrared image acquired by the infrared camera, and the second type of feature point is an eye feature point identified based on the visible light image acquired by the RGB camera. Infrared and RGB cameras can acquire images of the user's eye area simultaneously or in shifts. In infrared images, the pupil appears as a high-contrast dark spot under infrared light, making it easy to accurately locate feature points sensitive to changes in illumination, such as the pupil center, iris edge, pupil contour, and corneal reflection points. These feature points are classified as Class I feature points. In RGB images, areas such as the eyelids and corners of the eyes have rich color and texture information, making it easy to identify feature points with clear color and structural boundaries, such as the inner corner of the eye, outer corner of the eye, upper and lower eyelid edges, the midpoint of the eyelid fitted contour, and the junction of the iris and sclera. These feature points are classified as Class II feature points.

[0075] By fusing the recognition results from two types of cameras, the imaging characteristics of different spectral images can be fully utilized—infrared images provide high-contrast pupil localization information, while RGB images provide rich information on the periocular anatomy, thereby improving the robustness and accuracy of feature point extraction and laying a reliable data foundation for subsequent 3D reconstruction and wearing deviation analysis.

[0076] It should be noted that the first type of feature points and the second type of feature points are different types. While they may overlap, different sets of feature points are typically selected based on the advantages of their respective imaging modalities. By fusing the recognition results from both types of cameras, the imaging characteristics of different spectral images can be fully utilized, improving the robustness and accuracy of feature point extraction.

[0077] Step E20: Set initial estimates of the three-dimensional coordinates for each of the aforementioned eye feature points; For each eye feature point, an initial 3D coordinate estimate is assigned to it in a preset 3D spatial coordinate system. This initial estimate can be obtained in several ways: for example, it can be roughly located using a standard face model or an eye anatomy model; it can also be obtained by triangulation of the same feature points that appear in both types of images to obtain the initial depth, based on the parallax principle of binocular vision, if there is a known baseline distance between the infrared camera and the RGB camera; or it can be the projection of the feature point onto a reference plane (such as the lens plane) as the initial value.

[0078] Step E30: Establish the projection relationship from 3D spatial coordinates to image coordinates based on camera intrinsic parameters and distortion model; For both infrared and RGB cameras, their internal parameters are acquired, including focal length, principal point coordinates, and lens distortion coefficients. Based on these parameters, a projection relationship from three-dimensional spatial coordinates to two-dimensional image coordinates can be established.

[0079] This projection relationship describes the process by which a point in three-dimensional space, through the camera's optical imaging system, ultimately falls onto a specific pixel position on the image sensor. This process typically involves rigid body transformations between coordinate systems, perspective projection imaging, and corrections for lens distortion on the imaging position.

[0080] Using this projection relationship, given any three-dimensional spatial coordinates, its theoretically correct position in the camera image can be calculated.

[0081] Step E40: Construct an objective function, which includes a first term and a second term. The first term is the weighted sum of squared projection errors of each of the first type of feature points, and the second term is the weighted sum of squared projection errors of each of the second type of feature points. The projection error refers to the deviation between the theoretical image coordinates calculated based on the current estimated three-dimensional coordinates of the feature points through the projection relationship and the actual image coordinates of the eye feature points in the corresponding image. After completing the projection relationship modeling, an objective function is constructed to measure the accuracy of the current 3D coordinate estimation. This objective function consists of two parts: the first part corresponds to the first type of feature points (feature points recognized by infrared image recognition), and the second part corresponds to the second type of feature points (feature points recognized by RGB image recognition).

[0082] For each feature point, the deviation between the theoretical image coordinates obtained by projecting its current 3D coordinate estimate and the true image coordinates identified in the actual image is calculated; this deviation is the projection error. The first term of the objective function is the weighted sum of squared projection errors for all first-type feature points, and the second term is the weighted sum of squared projection errors for all second-type feature points.

[0083] The weights can be set based on the recognition confidence of each feature point—for example, the pupil center can be assigned a larger weight when the recognition confidence is high in the infrared image, and the corner of the eye can be assigned a larger weight when the recognition is stable in the RGB image. The entire objective function quantifies the overall reprojection error of the current 3D coordinate estimate across all imaging modalities.

[0084] Step E50: With the objective function as the goal, iteratively update the estimated three-dimensional coordinates of each eye feature point, and use the converged three-dimensional coordinates as the reference position coordinates of the eye feature points.

[0085] A nonlinear least squares optimization algorithm is used to iteratively solve the objective function. In each iteration, the estimated 3D coordinates of all feature points are adjusted based on the current objective function value and its changing trend to reduce the overall reprojection error.

[0086] This adjustment process aims to ensure that the estimated 3D coordinates of each feature point projected onto the image correspond as closely as possible to their actual identified positions in the infrared and RGB images. The iterative process continues until the objective function converges to a stable value—that is, when further coordinate adjustments can no longer significantly reduce the projection error. The resulting 3D coordinates are the optimal estimates of each eye feature point in the preset 3D spatial coordinate system. These converged 3D coordinates are then stored as the user's reference position coordinates.

[0087] Through the above-described fusion optimization process, this embodiment fully utilizes the complementary advantages of infrared and RGB images, achieving high-precision joint optimization of pupil features and eyelid and corner features during the 3D reconstruction process, providing more accurate and reliable benchmark data for subsequent wearing deviation detection.

[0088] Furthermore, this application also proposes a smart pair of glasses, referring to... Figure 3 As shown, the smart glasses include a camera module and a processor; The camera module is used to capture images of the eyes; The processor is used to perform the steps of the wearing reminder method described above.

[0089] Furthermore, this application also proposes a wearing reminder system, referring to... Figure 4 As shown, the wearing reminder system includes smart glasses and an edge device that are connected in communication, and the smart glasses include a camera module; The smart glasses are used to capture eye images and send the eye images to the edge device; The edge device is used to perform the steps of the wearing reminder method described above.

[0090] refer to Figure 5 It illustrates a structural schematic diagram suitable for implementing the edge device in the embodiments of this application. The edge device in the embodiments of this application may also include, but is not limited to, mobile phones, headphones, etc. Figure 5 The edge device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0091] like Figure 5As shown, the edge device may include a processing unit 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1002 or a program loaded from a storage device 1003 into a random access memory (RAM) 1004. The RAM 1004 also stores various programs and data required for the operation of the edge device. The processing unit 1001, ROM 1002, and RAM 1004 are interconnected via a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems can be connected to the I / O interface 1006: input devices 1007 including, for example, a touchscreen, touchpad, keyboard, mouse, image sensor, microphone, accelerometer, gyroscope, etc.; output devices 1008 including, for example, a liquid crystal display (LCD), speaker, vibrator, etc.; storage devices 1003 including, for example, magnetic tape, hard disk, etc.; and communication devices 1009. Communication device 1009 allows the edge device to communicate wirelessly or wiredly with other devices to exchange data. While the figure shows edge devices with various systems, it should be understood that implementation or possession of all the systems shown is not required. More or fewer systems may be implemented alternatively.

[0092] Specifically, according to the embodiments disclosed in this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from storage device 1003, or installed from ROM 1002. When the computer program is executed by processing device 1001, it performs the functions defined in the methods of the embodiments disclosed in this application.

[0093] The edge device provided in this application, employing the wearing reminder method described in the above embodiments, solves the technical problem of enabling users to promptly perceive whether the wearing position has shifted during the wearing of smart glasses. Compared with the prior art, the beneficial effects of the edge device provided in this application are the same as those of the wearing reminder method provided in the above embodiments, and other technical features of this edge device are the same as those disclosed in the previous embodiment method, and will not be repeated here.

[0094] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.

[0095] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0096] In addition, to achieve the above objectives, this application also provides a readable storage medium having computer-readable program instructions (i.e., a computer program) stored thereon, which are used to execute the wearing reminder method in the above embodiments.

[0097] The computer-readable storage medium provided in this application embodiment may be, for example, a USB flash drive, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections with one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, system, or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.

[0098] The aforementioned computer-readable storage medium may be included in the edge device; or it may exist independently and not assembled into the edge device.

[0099] The aforementioned computer-readable storage medium carries one or more programs, which, when executed by an edge device, cause the edge device to perform the process steps of any embodiment of the wear reminder method described above.

[0100] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a Local Area Network (LAN) or a Wide Area Network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0101] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0102] The modules described in the embodiments of this application can be implemented in software or hardware. The names of the modules do not necessarily limit the functionality of the modules themselves.

[0103] The readable storage medium provided in this application is a computer-readable storage medium that stores computer-readable program instructions (i.e., a computer program) for executing the above-described wearing reminder method. This solves the technical problem of enabling users to promptly perceive whether the wearing position has shifted during the wearing of smart glasses. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as those of the wearing reminder method provided in the above embodiments, and will not be repeated here.

[0104] Furthermore, this application also proposes a computer program product, including a computer program that, when executed by a processor, implements the steps of the wearing reminder method as described above.

[0105] The specific implementation method of the computer program product in this application is basically the same as the embodiments of the above-mentioned wearing reminder method, and will not be described again here.

[0106] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or system. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or system that includes that element.

[0107] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0108] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software sensor. This computer software sensor is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes several instructions to cause an edge device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0109] The above are merely preferred embodiments of this application and do not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

Claims

1. A method for reminding users to wear a product, characterized in that, Applied to smart glasses, the smart glasses including a camera module, the wearing reminder method includes the following steps: Acquire the eye image captured by the camera module; Based on the eye image, identify the current position coordinates of multiple eye feature points, and obtain the reference position coordinates of multiple eye feature points under the pre-calibrated standard wearing position; The current position coordinates are compared with the reference position coordinates to obtain the wearing deviation parameter of the smart glasses. When the wearing deviation parameter is greater than or equal to a preset threshold, a prompt message is output to remind the user to wear the smart glasses in a standard posture.

2. The wearing reminder method as described in claim 1, characterized in that, The step of comparing the current position coordinates with the reference position coordinates to obtain the wearing deviation parameters of the smart glasses includes: Perform point set registration between the point set formed by the current position coordinates and the point set formed by the reference position coordinates, and solve for the rigid body transformation matrix from the reference position coordinates to the current position coordinates. The translation vector and rotation angle are decomposed from the rigid body transformation matrix; The wearing deviation parameters of the smart glasses are determined based on the translation vector and the rotation angle.

3. The wearing reminder method as described in claim 2, characterized in that, The rigid body transformation matrix is ​​a transformation matrix in a three-dimensional spatial coordinate system. This three-dimensional spatial coordinate system is established with the user's head as the reference, with the X-axis pointing horizontally to the right, the Y-axis pointing horizontally forward, and the Z-axis pointing vertically upward. The step of determining the wearing deviation parameters of the smart glasses based on the translation vector and the rotation angle includes: Extract the horizontal offset, vertical offset, and forward / backward offset from the translation vector, wherein the horizontal offset corresponds to the displacement in the X-axis direction, the vertical offset corresponds to the displacement in the Z-axis direction, and the forward / backward offset corresponds to the displacement in the Y-axis direction. Extract the forward tilt angle around the X-axis and the left and right tilt angle around the Y-axis from the rotation angle; The horizontal offset, the vertical offset, the forward and backward offset, the forward tilt angle, and the left and right tilt angle are used as the wearing deviation parameters of the smart glasses.

4. The wearing reminder method as described in claim 1, characterized in that, After the step of comparing the current position coordinates with the reference position coordinates to obtain the wearing deviation parameters of the smart glasses, the method further includes: Each of the aforementioned wearing deviation parameters is compared with its corresponding preset threshold. If any of the wearing deviation parameters is greater than or equal to the corresponding preset threshold, an adjustment suggestion is determined based on the wearing deviation parameter that is greater than or equal to the corresponding preset threshold, wherein the adjustment suggestion includes the adjustment direction and the adjustment amount; The adjustment suggestions are delivered via visual, auditory, or vibration methods.

5. The wearing reminder method as described in claim 1, characterized in that, The step of obtaining the reference position coordinates of multiple eye feature points under the pre-calibrated standard wearing position includes: A standard eye image is obtained by capturing the user's eye image when the user is in a standard wearing posture; Multiple eye feature points are identified from the standard eye image, and the two-dimensional coordinates of the eye feature points in the standard eye image are obtained; The two-dimensional coordinates are converted into three-dimensional coordinates in a preset three-dimensional spatial coordinate system and stored as reference position coordinates.

6. The wearing reminder method as described in claim 5, characterized in that, The camera module includes an infrared camera and an RGB camera. The step of converting the two-dimensional coordinates into three-dimensional coordinates in a preset three-dimensional spatial coordinate system includes: Acquire a first type of feature point and a second type of feature point from a plurality of eye feature points, wherein the first type of feature point is an eye feature point identified based on an infrared image acquired by the infrared camera, and the second type of feature point is an eye feature point identified based on a visible light image acquired by the RGB camera. Initial estimates of the three-dimensional coordinates are set for each of the aforementioned eye feature points; Establish the projection relationship from 3D spatial coordinates to image coordinates based on camera intrinsic parameters and distortion model; Construct an objective function, which includes a first term and a second term, wherein the first term is the weighted sum of squared projection errors of each of the first type of feature points, and the second term is the weighted sum of squared projection errors of each of the second type of feature points. The projection error refers to the deviation between the theoretical image coordinates calculated by the projection relationship based on the current estimated three-dimensional coordinates of the feature points and the actual image coordinates of the eye feature points in the corresponding image. With the objective function as the goal, the estimated three-dimensional coordinates of each eye feature point are iteratively updated, and the converged three-dimensional coordinates are used as the reference position coordinates of the eye feature points.

7. The wearing reminder method according to any one of claims 1 to 6, characterized in that, The eye feature points include at least three of the following: pupil center point, eyeball center point, eyelid key point, inner canthus point, outer canthus point, and iris center point. The eyelid key point includes the eyelid edge point and / or the midpoint of the eyelid fitting contour.

8. A type of smart glasses, characterized in that, The smart glasses include a camera module and a processor; The camera module is used to capture images of the eyes; The processor is configured to perform the steps of the wearing reminder method as described in any one of claims 1 to 7.

9. A wearing reminder system, characterized in that, The wearing reminder system includes smart glasses and an edge device that are connected in communication, and the smart glasses include a camera module; The smart glasses are used to capture eye images and send the eye images to the edge device; The edge device is used to perform the steps of the wearing reminder method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a device control program, which, when executed by a processor, implements the steps of the wearing reminder method as described in any one of claims 1 to 7.

Images