Dynamic diopter detection method and system based on wearable module

Abstract

The invention provides a dynamic diopter detection method and system based on a wearable module, and relates to the field of myopia prevention and control, and the method comprises the steps: exciting light rays through a light source part of the wearable module to enable the light rays to enter eyeballs of a wearer, and obtaining first reflection light rays through diffuse reflection and refraction of the light rays through a retina and a crystalline lens; acquiring first video information including the first reflected light through an optical axis of a camera module of the wearable module; the first video information is uploaded to a server for training, a target model is obtained, second video information is collected and uploaded to the server, and a crystalline lens image is segmented from the second video information; and analyzing the crystalline lens image based on the target model to output diopter change trend information. After the target model corresponding to the wearer is obtained, the second video information continues to be collected, the crystalline lens image is segmented, diopter change trend information can be obtained by analyzing the crystalline lens image through the target model, and therefore the myopia condition of the wearer is obtained.

Description

Technical Field

[0001] This application relates to the technical field of myopia prevention and control, and specifically relates to a dynamic diopter detection method and system based on a wearable module. Background Art

[0002] The diopter is a unit used to measure the refractive power of the eye. The diopter reflects the degree of refraction of light after passing through the eye's refractive system. The diopter characterizes the eye's focusing ability on light. The diopter of a normal eye can accurately focus light on the retina, thus forming a clear image. However, in myopic patients, due to the excessive length of the anteroposterior diameter of the eyeball, or the excessive refractive power of the refractive system such as the cornea and lens, the light is focused in front of the retina, and the image formed on the retina becomes blurred. The abnormal diopter is the direct cause of this imaging abnormality. The eye diopter is an important indicator for measuring the degree of myopia. By performing diopter detection, it helps to understand the development of myopia, prevent and control myopia, and protect eye health.

[0003] In the related art, in the automatic refractometry, the eye is measured using an automatic refractometer. The automatic refractometer automatically calculates the diopter of the eye by measuring the refraction and scattering characteristics of the refractive medium of the eyeball on light. This method is relatively simple, but due to the differences in each person's lens, it is difficult to standardize the measurement environment, and the usage conditions of each person vary greatly, so the error of the detection result is usually large.

[0004] The current automatic refractometer is relatively easy to operate, but its flexibility still needs to be improved. Moreover, the current automatic refractometer has poor detection accuracy and is difficult to provide accurate detection for customers with great differences. The accuracy of the detection result of the dynamic diopter needs to be improved. Based on this, there is an urgent need to propose a dynamic diopter detection method and system based on a wearable module to flexibly, highly accurately, and generally detect and analyze the changing trend of the dynamic diopter. Summary of the Invention

[0005] Aiming at the deficiencies of the prior art, this application provides a dynamic diopter detection method and system based on a wearable module, which solves the problem that it is difficult for the current diopter detection equipment to balance operation flexibility, environmental adaptability, and detection accuracy.

[0006] To achieve the above objectives, this application is implemented through the following technical solutions:

[0007] In a first aspect, an embodiment of the present application provides a method for dynamically detecting refractive power based on a wearable module. The method for dynamically detecting refractive power based on a wearable module includes: exciting light through a light source component of the wearable module so that the light enters the wearer's eyeball, and obtaining a first reflected light through the diffuse reflection and refraction of the retina and the lens; collecting first video information containing the first reflected light through the optical axis of the camera module of the wearable module; uploading the first video information to a server for training to obtain a target model, where the target model is used to analyze the light reflected by the lens to obtain corresponding refractive power information; collecting second video information and uploading it to the server, and using OpenCV tools to segment multiple lens images from the second video information; where the second video information includes a second reflected light through the diffuse reflection and refraction of the retina and the lens; analyzing the lens images based on the target model to output refractive power change trend information; where the refractive power trend information is used to characterize the myopia development information of the wearer for myopia prevention and control.

[0008] According to the first aspect of the embodiment of the present application, the wearable module includes two light source components and two camera modules. One light source component and one camera module form a first acquisition group, and the other light source component and the other camera module form a second acquisition group; the first video information and the second video information are jointly collected through the first acquisition group and the second acquisition group; during the rotation of the wearer's eyeball, at least part of the lens is in the excitation light path area corresponding to the first acquisition group and the second acquisition group.

[0009] According to the first aspect of the embodiment of the present application, before analyzing the lens images based on the target model to output refractive power change trend information, the method for dynamically detecting refractive power based on a wearable module further includes the following steps: determining whether the performance of the user's terminal device meets a preset computing power standard, where the preset computing power standard is used to calibrate the computing power performance threshold for running the target model for data analysis; in the case where the terminal device does not meet the preset computing power standard, determining to analyze multiple lens images on the server; in the case where the terminal device meets the preset computing power standard, determining whether to analyze multiple lens images on the terminal device.

[0010] According to the first aspect of the embodiment of the present application, in the case where it is determined to analyze multiple lens images on the server, after analyzing the lens images based on the target model to output refractive power change trend information, the method for dynamically detecting refractive power based on a wearable module further includes the following steps: saving the dynamic refractive power change trend information as target refractive power information in a target format; sending the target refractive power information to the terminal device determined by the wearer; where the target format includes at least one of a document format, an image format, and a video format, and the terminal device includes at least one of a mobile phone, a tablet, and a computer.

[0011] According to the first aspect of the embodiments of the present application, when it is determined that multiple lens images are analyzed by a terminal device, the foregoing analysis of the lens images based on a target model to output diopter change trend information includes the following steps: sending the multiple lens images and the target model to the terminal device; after installing the target model on the terminal device, filtering and denoising the multiple lens images in the frequency domain using a low-pass filter through fast Fourier transform; performing image analysis and visualization processing based on the target model to obtain diopter change trend information.

[0012] According to the first aspect of the embodiments of the present application, the foregoing uploading of the first video information to a server for training to obtain a target model includes the following steps: extracting valid frames of the first video information as an initial sample set; extracting target images corresponding to the lens from each image information in the initial sample set to obtain a target sample set; building a model framework based on a preset semantic segmentation basic model by adding a convolutional layer, a pooling layer, and an upsampling layer; and training the model individually for each wearer in a multi-concurrent manner based on the model framework for the target sample set to determine a target model to adapt to the lens characteristics of multiple wearers.

[0013] According to the first aspect of the embodiments of the present application, the foregoing training the model individually for each wearer in a multi-concurrent manner based on the model framework for the target sample set to determine a target model to adapt to the lens characteristics of multiple wearers includes the following steps: dividing the target sample set into a training set and a test set, and shuffling the training set and the test set; using pre-trained weights to read and search for a preset GPU, and allocating a first computing power and a second computing power of the GPU; processing the shuffled training set and test set respectively according to the first computing power and the second computing power, and adjusting the number of samples in real time based on the model framework to synchronously perform training and test evaluation to obtain a target model.

[0014] In a second aspect, embodiments of the present application provide a dynamic diopter detection system based on a wearable module. The dynamic diopter detection system based on the wearable module includes an excitation module, a first acquisition module, a training module, a second acquisition module, and an analysis module.

[0015] Specifically, the excitation module is used to excite light through the light source component of the wearable module so that the light enters the wearer's eyeball, and the first reflected light is obtained through the diffuse reflection and refraction of the retina and the lens; the first acquisition module is used to collect the first video information containing the first reflected light through the optical axis of the camera module of the wearable module; the training module is used to upload the first video information to the server for training to obtain a target model, and the target model is used to analyze the light reflected by the lens to obtain corresponding diopter information; the second acquisition module is used to collect the second video information and upload it to the server, and use the OpenCV tool to segment multiple lens images from the second video information; wherein, the second video information includes the second reflected light through the diffuse reflection and refraction of the retina and the lens; the analysis module is used to analyze the lens images based on the target model to output the diopter change trend information; wherein, the diopter trend information is used to characterize the myopia development information of the wearer for myopia prevention and control.

[0016] In a third aspect, an embodiment of the present application provides an electronic device, which includes: a processor, a memory, and a program stored on the memory and executable on the processor. When the program is executed by the processor, it implements the dynamic diopter detection method based on the wearable module in the foregoing first aspect.

[0017] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by the processor, it implements the dynamic diopter detection method based on the wearable module in the foregoing first aspect.

[0018] The present application provides a dynamic diopter detection method and system based on a wearable module. Compared with the prior art, it has the following beneficial effects:

[0019] The present application performs dynamic diopter detection based on the wearable module. When the wearer uses the wearable module, light is excited through the light source component and enters the eyeball, and the first reflected light is obtained through the action of the retina and the lens. The first video information collected through the camera module can characterize the diopter information of the eye; the present application uploads the first video information to the server for personalized training to obtain a target model; after obtaining the target model corresponding to the wearer, continue to collect the second video information and segment the lens images, and the diopter change trend information can be obtained by analyzing the lens images through the target model, so as to obtain the myopia situation of the wearer; the wearable module is flexible to operate and is not limited to a specific environment. The present application trains the model based on specific first video information to match the wearer, and can perform accurate diopter detection on the wearer. Description of the Drawings

[0020] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0021] Figure 1 It is a schematic flow chart of a dynamic diopter detection method based on a wearable module provided by an embodiment of the present application;

[0022] Figure 2 It is an exemplary structural schematic diagram of a wearable module provided by an embodiment of the present application;

[0023] Figure 3 It is a structural schematic diagram of a dynamic diopter detection system based on a wearable module provided by an embodiment of the present application;

[0024] Figure 4 It is a structural schematic diagram of an electronic device provided by an embodiment of the present application.

[0025] Reference numerals: camera 1; light source component 2; network communication circuit board 3; rechargeable power supply 4; USB wire harness 5. Detailed implementation manners

[0026] To make the objectives, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention are clearly and completely described. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts fall within the scope of protection of the present invention.

[0027] It should be noted that in this article, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the term "comprising", "including" or any other variant thereof is intended to cover a non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements, but also includes other elements not expressly listed, or further includes elements inherent to such process, method, article or device. Without further limitations, the element defined by the statement "including one..." does not exclude the existence of additional identical elements in the process, method, article or device including the element.

[0028] Embodiments of the present application provide a method and system for dynamically detecting refractive power based on a wearable module, which solves the problem that current refractive power detection devices are difficult to balance operation flexibility, environmental adaptability, and detection accuracy.

[0029] The technical solutions in the embodiments of the present application for solving the above technical problems are generally as follows:

[0030] Refractive power is a unit used to measure the refractive ability of the eye. Refractive power reflects the degree of refraction of light after passing through the eye's refractive system (such as the cornea, lens, etc.). Refractive power characterizes the eye's focusing ability for light. The refractive power of a normal eye can accurately focus light on the retina, thus forming a clear image. In myopic patients, due to the excessive length of the anteroposterior axis of the eyeball (axial myopia), or the excessive refractive power of the refractive system such as the cornea and lens (refractive myopia), light is focused in front of the retina, and the image formed on the retina becomes blurred. The abnormal refractive power is the direct cause of this imaging abnormality. The refractive power of the eye is an important indicator for measuring the degree of myopia. By performing refractive power detection, it helps to understand the development of myopia, prevent and control myopia, and protect eye health.

[0031] In related technologies, refractive power detection mainly includes subjective optometry and objective optometry. Subjective optometry is divided into preliminary measurement by a computer optometer and precise measurement by a comprehensive optometer; in the preliminary measurement by a computer optometer, the subject sits in front of the computer optometer, places the chin on the bracket, and focuses the eyes on the target inside the optometer. The optometer emits light such as infrared rays, and based on measuring the refraction of light in the eye, the refractive power of the eye is initially obtained; in the precise measurement by a comprehensive optometer, on the basis of computer optometry, a comprehensive optometer is used for further precise measurement. The subject needs to wear the trial frame of the comprehensive optometer, and by changing different lens combinations, according to the clarity of the visual acuity chart and the imaging situation seen by oneself, feedback to the optometrist, and the optometrist adjusts the lens power to finally determine the accurate refractive power. This method requires the subjective cooperation and feedback of the subject, and the process is cumbersome.

[0032] Objective optometry includes retinoscopy and auto-refractor optometry. In retinoscopy, the optometrist projects light into the subject's eye with a retinoscope in a dark room, and judges the refractive state of the eye according to the light situation reflected by the retina. The optometrist determines the lens power to be added to achieve a neutral state by observing the movement direction, speed, and brightness of the light and shadow, etc., so as to obtain the refractive power. This method does not require the subjective judgment of the subject and is suitable for children, people who cannot cooperate with subjective optometry, or first-time optometry patients, but has high technical requirements for the optometrist.

[0033] In autorefraction, an autorefractor is used to measure the eyes. The autorefractor automatically calculates the refractive power of the eyes by measuring the refraction and scattering of light by the refractive media of the eyeballs. This method is relatively simple. However, due to the differences in each person's lens, it is difficult to standardize the measurement environment, and the usage conditions of each person vary greatly, so the error of the detection results is usually large.

[0034] In summary, the current refractive power detection methods have high requirements for the environment, equipment, and posture, and the detection flexibility is relatively low. Even though the current autorefractor is relatively easy to operate, its flexibility still needs to be improved. Moreover, the current autorefractor has poor detection accuracy and is difficult to provide accurate detection for customers with diverse conditions. The accuracy of the detection results of dynamic refractive power needs to be improved.

[0035] To better understand the above technical solution, the above technical solution will be described in detail below in conjunction with the accompanying drawings of the specification and specific implementation manners.

[0036] First, a dynamic refractive power detection method based on a wearable module provided by an embodiment of the present application will be introduced below.

[0037] The flowchart of a dynamic refractive power detection method based on a wearable module provided by an embodiment of the present application is shown as Figure 1 shown. The dynamic refractive power detection method based on the wearable module may include the following steps S110 - S150.

[0038] S110: Activate light through the light source component of the wearable module so that the light enters the eyeball of the wearer, and the first reflected light is obtained through the diffuse reflection and refraction of the light by the retina and the lens.

[0039] S120: Collect the first video information containing the first reflected light through the optical axis of the camera module of the wearable module.

[0040] S130: Upload the first video information to the server for training to obtain a target model, and the target model is used to analyze the light reflected by the lens to obtain the corresponding refractive power information.

[0041] S140: Collect the second video information and upload it to the server, and use the OpenCV tool to segment multiple lens images from the second video information; wherein, the second video information includes the second reflected light through the diffuse reflection and refraction of the retina and the lens.

[0042] S150: Analyze the lens images based on the target model to output the refractive power change trend information; wherein, the refractive power trend information is used to characterize the myopia development information of the wearer for myopia prevention and control.

[0043] The above is the specific implementation manner of a dynamic diopter detection method based on a wearable module provided by the embodiments of the present application. It can be understood that the present application performs dynamic diopter detection based on the wearable module. When the wearer uses the wearable module, light is excited by a light source member and enters the eyeball, and after acting through the retina and the lens, a first reflected light is obtained. By collecting corresponding first video information through a camera module, the diopter information of the eye can be characterized; the present application uploads the first video information to a server for personalized training to obtain a target model.

[0044] Based on this, after obtaining the target model corresponding to the wearer, the present application continues to collect second video information and segment the lens image. By analyzing the lens image through the target model, the diopter change trend information can be obtained, so as to obtain the myopia condition of the wearer; the wearable module is flexible in operation and use, not limited to a specific environment. The present application trains a model based on specific first video information to match the wearer, and can perform accurate diopter detection on the wearer, and then understand the myopia development information of the wearer for myopia prevention and control.

[0045] It should be noted that the present application proposes a dynamic diopter detection method based on a wearable module, which realizes dynamic diopter measurement all-weather, indoors and outdoors, regardless of the environment and posture. The present application calculates the change in diopter by measuring the change in the refractive index of the lens. Since the lens is in a real-time dynamic change state, the diopter measured by this method is a dynamic diopter. Since there are differences in the lenses of each person and it is difficult to standardize the measurement environment, and the usage conditions of each person are very different, this method can more accurately obtain the change trend of the dynamic diopter, rather than a numerical value.

[0046] It should also be noted that the wearable module in the present application can be either an AR glasses or exist in the form of other independent modules; the AR glasses can use the hardware of an eye-tracking glasses, and an infrared point light source is added to the frame, and the same image acquisition protocol is used. Please refer to Figure 2 , the independent module corresponding to the wearable module is composed of an imaging unit and an output unit. The imaging unit and the output unit are connected by an extended USB cable 5. The imaging unit includes a camera 1 and a light source member 2, and the output unit includes a network communication circuit board 3 and a rechargeable power supply 4. The usage scenarios of the independent module are installed on glasses, desks, bedside, walls, far-view screens, reading and writing desks, tablet computers, and laptop computers. The independent module exists independently in an external form, can be carried around, or can be fixed.

[0047] In one example, the light source component is a micro point light source; the wearable module includes a driving unit, the driving unit includes a driving chip, and the preset code of the driving chip includes an encryption part and a communication part to control the light source component to emit light. The driving chip can transcode and transmit the collected video information, and the video information includes the foregoing first video information and the foregoing second video information.

[0048] In some embodiments, the wearable module includes two light source components and two camera modules. One light source component and one camera module form a first acquisition group, and the other light source component and the other camera module form a second acquisition group. The first video information and the second video information are jointly collected by the first acquisition group and the second acquisition group; during the rotation of the wearer's eyeball, at least part of the lens is in the excitation light path regions corresponding to the first acquisition group and the second acquisition group.

[0049] In the embodiments of the present application, it can be understood that since the wearer's eyes are in a rotating state rather than a static state, the wearable module of the present application can be provided with two light source components and two camera modules. After the wearer wears the wearable module, at least part of the lens is in the excitation light path regions corresponding to the first acquisition group and the second acquisition group, which can ensure that the wearable module can collect effective data and reduce the dependence on a specific environment and a specific posture.

[0050] It should be noted that the wearable module is based on the principle of a slit lamp. When infrared light passes through the lens and enters the fundus of the eye, if the refractive power changes, then the focus of the light moves forward, and the more the focus moves, the greater the refractive power; when the light undergoes diffuse reflection in the fundus and then returns through the lens, the greater the refractive power, the wider the line.

[0051] Assuming an ideal state, infrared light enters from the optical axis of the eye, passes through the lens to the fundus, undergoes diffuse reflection and then refracts in the lens, and then passes through the optical axis of the infrared camera, and there is no loss of energy in all optical paths, and the camera can also fully receive it; assuming the eyeball is a standard sphere, then the emitted line should decrease evenly from the middle to both sides; take out the energy values of all pixel points in any area of the line, arrange them from large to small, and then fit the values into a straight line by the least squares method, then the slopes of any areas obtained should be equal.

[0052] It is understandable that the retina occupies about 70% of the fundus area. As long as light passes through the lens in any 70% area, effective diffuse reflection can occur. Only the total energy value of the reflected lines is relatively low, but the slope is the same as in the ideal state. That is to say, the diopter is not related to the brightness of the reflected light. Since the lens is always in a moving state relative to the wearable module, it cannot ensure that light can always pass through the lens, and thus effective parameters cannot be obtained. Therefore, the measurement is not continuous. Therefore, only by calculating the slope change of the energy in the fixed area of the reflected light can the corresponding change in diopter be obtained.

[0053] In some embodiments, before analyzing the lens image based on the target model to output the diopter change trend information, the dynamic diopter detection method based on the wearable module may further include the following steps:

[0054] S210. Determine whether the performance of the user's terminal device meets a preset computing power standard, where the preset computing power standard is used to calibrate the computing power performance threshold for running the target model for data analysis.

[0055] S220. In the case where the terminal device does not meet the preset computing power standard, determine to analyze multiple lens images on the server.

[0056] S230. In the case where the terminal device meets the preset computing power standard, determine whether to analyze multiple lens images on the terminal device.

[0057] In the embodiments of the present application, it can be understood that after obtaining the target model, it is possible to choose to analyze the lens image on the server, or according to the actual situation, when the terminal device has sufficient computing power, choose to analyze the lens image on the terminal device.

[0058] In some embodiments, in the case where it is determined to analyze multiple lens images on the server, after analyzing the lens image based on the target model to output the diopter change trend information, the dynamic diopter detection method based on the wearable module may further include the following steps: saving the dynamic diopter change trend information as target diopter information in a target format; sending the target diopter information to the terminal device determined by the wearer; where the target format includes at least one of a document format, an image format, and a video format, and the terminal device includes at least one of a mobile phone, a tablet, and a computer.

[0059] It can be understood that the detection result corresponding to the dynamic diopter change trend information can exist in the form of a document, or in the form of an image, or in the form of a video. The end user can choose different formats of the detection results to understand.

[0060] In some embodiments, when it is determined that multiple lens images are to be analyzed on a terminal device, the foregoing analysis of the lens images based on the target model is performed to output refractive power change trend information. That is, the foregoing S150 may include the following steps:

[0061] S310. Send multiple lens images and the target model to the terminal device;

[0062] S320. After installing the target model on the terminal device, use a low-pass filter in the frequency domain to filter and denoise the multiple lens images through fast Fourier transform;

[0063] S330. Perform image analysis based on the target model and perform visualization processing to obtain refractive power change trend information.

[0064] In the embodiments of the present application, it can be understood that for the valuable lens region of the present application, high frequencies are filtered in the frequency domain through fast Fourier transform using a low-pass filter to remove invalid noise points and flares; after obtaining the image analysis result through image analysis using the target model, the change curvature of the refractive power is visualized and plotted into a visualization chart to obtain refractive power change trend information.

[0065] In some embodiments, the foregoing uploading of the first video information to the server for training to obtain the target model, that is, the foregoing S130 may include the following steps:

[0066] S410. Extract the valid frames of the first video information as the initial sample set.

[0067] S420. Extract the target images corresponding to the lens from each image information in the initial sample set to obtain the target sample set.

[0068] S430. Based on the preset semantic segmentation basic model, add a convolutional layer, a pooling layer, and an upsampling layer to build a model framework.

[0069] S440. Based on the model framework, train the model separately for each wearer in a multi-concurrent manner for the target sample set to determine the target model to adapt to the lens characteristics of multiple wearers.

[0070] In the embodiments of the present application, it can be understood that the infrared point light source corresponding to the light source component interferes with the camera field of view of the camera module. The reason is that only by making the included angle region between the point light source and the camera optical path as large as possible, there will be a larger "illuminated" region when the eyeball moves, and the point light source can only be further brought closer to the eyeball.

[0071] Based on this, there may be flare in the video collected by the camera module. The reason for the appearance of flare is that only a small part of the light emitted by the point light source actually enters the fundus of the eye, most of it is reflected on the ocular surface, and a part enters the camera. Since the energy is relatively high, it is relatively bright. To ensure the accuracy of information analysis and processing, after collecting a video to obtain the first video information, the valid frames in the video are taken out as samples.

[0072] In some embodiments, based on the model framework described above, the model is trained separately for each wearer in a multi-concurrent manner for the target sample set to determine the target model to adapt to the lens characteristics of multiple wearers. That is, the foregoing S440 may specifically include the following steps:

[0073] S510. Divide the target sample set into a training set and a test set, and shuffle the training set and the test set.

[0074] S520. Read and search for a preset GPU using the pre-trained weights, and allocate the first computing power and the second computing power of the GPU.

[0075] S530. Process the shuffled training set and test set according to the first computing power and the second computing power respectively, and adjust the number of samples in real time based on the model framework to perform training and test evaluation simultaneously to obtain the target model.

[0076] In the embodiments of the present application, it can be understood that the first computing power may account for 90% of the GPU computing power, and the second computing power accounts for 10% of the GPU computing power. The training and evaluation processes can be carried out simultaneously, and the training of the model is adjusted in a timely manner based on the corresponding loss value of the loss function.

[0077] In some embodiments, the present application provides a dynamic refractive power detection system 600 based on a wearable module, as Figure 3 shown. The system 600 may include the following modules:

[0078] An excitation module 610, configured to excite light through a light source component of the wearable module so that the light enters the eye of the wearer, and the first reflected light is obtained through the diffuse reflection and refraction of the light by the retina and the lens.

[0079] A first acquisition module 620, configured to acquire first video information including the first reflected light through the optical axis of the camera module of the wearable module.

[0080] A training module 630, configured to upload the first video information to a server for training to obtain a target model, and the target model is used to analyze the light reflected by the lens to obtain corresponding refractive power information.

[0081] The second acquisition module 640 is configured to acquire second video information and upload it to the server, and use the OpenCV tool to segment multiple lens images from the second video information; wherein, the second video information includes second reflected light rays that have undergone diffuse reflection and refraction through the retina and the lens.

[0082] The analysis module 650 is configured to analyze the lens images based on the target model to output diopter change trend information; wherein, the diopter trend information is used to characterize the myopia development information of the wearer for myopia prevention and control.

[0083] According to an embodiment of the present application, any multiple of the excitation module 610, the first acquisition module 620, the training module 630, the second acquisition module 640, and the analysis module 650 may be combined and implemented in one module, or any one of them may be split into multiple modules. Alternatively, at least part of the functions of one or more of these modules may be combined with at least part of the functions of other modules and implemented in one module.

[0084] In some embodiments, the training module 630 may specifically be configured to:

[0085] Extract the valid frames of the first video information as the initial sample set;

[0086] Extract target images corresponding to the lens from each image information in the initial sample set to obtain a target sample set;

[0087] Based on a preset semantic segmentation base model, add convolutional layers, pooling layers, and upsampling layers to build a model framework;

[0088] Based on the model framework, train the model separately for each wearer in a multi-concurrent manner for the target sample set to determine the target model to adapt to the lens characteristics of multiple wearers.

[0089] Figure 3 Each module in the system shown has the function of implementing each step in the foregoing method for dynamically detecting diopter based on a wearable module and can achieve its corresponding technical effects. For the sake of brevity, it will not be described in detail here.

[0090] In some embodiments, the present application provides an electronic device, and the structural schematic diagram of the electronic device is as Figure 4 shown.

[0091] The electronic device may include a processor 710 and a memory 720 storing computer program instructions.

[0092] Specifically, the above-mentioned processor 710 may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or may be configured as one or more integrated circuits for implementing the embodiments of the present application.

[0093] The memory 720 may include a mass memory for data or instructions. By way of example and not limitation, the memory 720 may include a hard disk drive (HDD), a floppy disk drive, a flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a universal serial bus (USB) drive, or a combination of two or more of these. In a suitable case, the memory 720 may include a removable or non-removable (or fixed) medium. In a suitable case, the memory 720 may be inside or outside the integrated gateway disaster recovery device. In a specific embodiment, the memory 720 is a non-volatile solid-state memory.

[0094] The memory 720 may include a read-only memory (ROM), a random access memory (RAM), a magnetic disk storage medium device, an optical storage medium device, a flash memory device, an electrical, optical, or other physical / tangible memory storage device. Thus, generally, the memory 720 includes one or more tangible (non-transitory) computer-readable storage media (e.g., memory devices) encoded with software including computer-executable instructions, and when the software is executed (e.g., by one or more processors), it can perform the operations described in any of the above-described dynamic diopter detection methods based on wearable modules.

[0095] The processor 710 reads and executes the computer program instructions stored in the memory 720 to implement any of the above-described dynamic diopter detection methods based on wearable modules.

[0096] In one example, the electronic device may further include a communication interface 730 and a bus 700. Among them, as Figure 4 shown, the processor 710, the memory 720, and the communication interface 730 are connected through the bus 700 and complete communication with each other.

[0097] The communication interface 730 is mainly used to implement communication between various modules, devices, units, and / or devices in the embodiments of the present application.

[0098] The bus 700 includes hardware, software, or both, and couples the components of the online data flow metering device to each other. By way of example and not limitation, the bus may include an Accelerated Graphics Port (AGP) or other graphics bus, an Enhanced Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), a HyperTransport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an InfiniBand interconnect, a Low Pin Count (LPC) bus, a memory bus, a MicroChannel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local (VLB) bus, or other suitable buses or a combination of two or more of these. Where appropriate, the bus 700 may include one or more buses. Although the embodiments of the present application describe and illustrate specific buses, the present application contemplates any suitable bus or interconnect.

[0099] In addition, in combination with the dynamic diopter detection method based on the wearable module in the above embodiments, an embodiment of the present application can provide a computer storage medium to implement. Computer program instructions are stored on the computer storage medium; when the computer program instructions are executed by a processor, any one of the dynamic diopter detection methods based on the wearable module in the above embodiments is implemented.

[0100] It should be clear that the present application is not limited to the specific configurations and processes described above and illustrated in the figures. For the sake of brevity, detailed descriptions of known methods are omitted here. In the above embodiments, several specific steps are described and illustrated as examples. However, the method process of the present application is not limited to the specific steps described and illustrated, and those skilled in the art can make various changes, modifications, and additions, or change the order between steps after understanding the spirit of the present application.

[0101] The functional blocks shown in the above block diagrams can be implemented as hardware, software, firmware, or a combination thereof. When implemented in hardware, it can be, for example, an electronic circuit, an Application Specific Integrated Circuit (ASIC), appropriate firmware, a plug-in, a function card, and so on. When implemented in software, the elements of the present application are programs or code segments used to perform the required tasks. The program or code segment can be stored in a machine-readable medium or transmitted via a data signal carried in a carrier wave on a transmission medium or a communication link. A "machine-readable medium" can include any medium that can store or transmit information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROMs, flash memories, erasable ROMs (EROMs), floppy disks, CD-ROMs, optical disks, hard disks, fiber optic media, radio frequency (RF) links, and so on. The code segment can be downloaded via a computer network such as the Internet, an intranet, and so on.

[0102] It should also be noted that in the exemplary embodiments mentioned in this application, some methods or systems are described based on a series of steps or devices. However, this application is not limited to the order of the above steps. That is to say, the steps can be executed in the order mentioned in the embodiments, or different from the order in the embodiments, or several steps can be executed simultaneously.

[0103] As mentioned above, various aspects of the present disclosure have been described with reference to the flowcharts and / or block diagrams of methods, apparatuses (systems), and computer program products according to embodiments of the present disclosure. It should be understood that each block in the flowchart and / or block diagram, and the combinations of blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device to generate a machine, such that the instructions executed by the processor of the computer or other programmable data processing device enable the implementation of the functions / actions specified in one or more blocks of the flowchart and / or block diagram. Such a processor can be, but is not limited to, a general-purpose processor, a special-purpose processor, a special application processor, or a field-programmable logic circuit. It is also understood that each block in the block diagram and / or flowchart, and the combinations of blocks in the block diagram and / or flowchart, can also be implemented by dedicated hardware that performs the specified functions or actions, or by a combination of dedicated hardware and computer instructions.

[0104] In summary, compared with the prior art, the present application has the following beneficial effects:

[0105] 1. The present application performs dynamic refractive power detection based on a wearable module. When the wearer uses the wearable module, light is emitted by a light source member and enters the eyeball, and after passing through the retina and the lens, a first reflected light is obtained. By collecting the corresponding first video information through a camera module, the refractive power information of the eye can be characterized. The present application uploads the first video information to a server for personalized training to obtain a target model. After obtaining the target model corresponding to the wearer, the present application continues to collect second video information and segment the lens image. By analyzing the lens image with the target model, the refractive power change trend information can be obtained, so as to obtain the myopia situation of the wearer.

[0106] 2. The wearable module of the present application is flexible in operation and use, not limited to a specific environment. The usage scenarios can be installed on glasses, desks, bedside, walls, telecentric screens, reading and writing desks, tablet computers, and laptop computers. The wearable module exists independently in an external form, can be carried around, or can be fixed. By collecting the first video information through the wearable module to train the model to match the wearer, accurate refractive power detection of the wearer can be performed, and then the myopia development information of the wearer can be understood for myopia prevention and control.

[0107] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements on some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A dynamic diopter detection method based on a wearable module, characterized in that Including: A light source component of the wearable module excites light to make the light enter the wearer's eyeball, and the first reflected light is obtained through the diffuse reflection and refraction of the retina and the lens; Collect first video information including the first reflected light through the optical axis of the camera module of the wearable module; Upload the first video information to the server for training to obtain a target model, and the target model is used to analyze the light reflected by the lens to obtain corresponding diopter information; Collect second video information and upload it to the server, and use the OpenCV tool to segment multiple lens images from the second video information; wherein, the second video information includes second reflected light through the diffuse reflection and refraction of the retina and the lens; Analyze the lens images based on the target model to output diopter change trend information; wherein, the diopter trend information is used to characterize the myopia development information of the wearer for myopia prevention and control.

2. The dynamic diopter detection method based on a wearable module according to claim 1, wherein The wearable module includes two of the light source components and two of the camera modules, one of the light source components and one of the camera modules form a first acquisition group, and the other light source component and the other camera module form a second acquisition group; The first video information and the second video information are jointly collected by the first acquisition group and the second acquisition group; during the rotation of the wearer's eyeball, at least part of the lens is in the excitation light path regions corresponding to the first acquisition group and the second acquisition group.

3. The dynamic refractive power detection method based on a wearable module according to claim 1, wherein Before analyzing the lens images based on the target model to output diopter change trend information, the dynamic diopter detection method based on the wearable module further includes: Judge whether the performance of the user's terminal device meets a preset computing power standard, and the preset computing power standard is used to calibrate the computing power performance threshold for running the target model for data analysis; In the case that the terminal device does not meet the preset computing power standard, determine to analyze the multiple lens images on the server; In the case that the terminal device meets the preset computing power standard, determine whether to analyze the multiple lens images on the terminal device.

4. The dynamic diopter detection method based on a wearable module according to claim 3, characterized in that In the case of determining to analyze the multiple lens images on the server, after analyzing the lens images based on the target model to output diopter change trend information, the dynamic diopter detection method based on the wearable module further includes: Save the dynamic diopter change trend information as target diopter information in a target format; Send the target diopter information to the terminal device determined by the wearer; Wherein, the target format includes at least one of a document format, an image format, and a video format, and the terminal device includes at least one of a mobile phone, a tablet, and a computer.

5. The dynamic diopter detection method based on a wearable module according to claim 3, wherein In the case of determining to analyze the multiple lens images on the terminal device, analyzing the lens images based on the target model to output diopter change trend information includes: Send the multiple lens images and the target model to the terminal device; After installing the target model on the terminal device, use a low-pass filter in the frequency domain to filter and denoise the multiple lens images through fast Fourier transform. Based on the target model, perform image analysis and visualization processing to obtain diopter change trend information.

6. The dynamic diopter detection method based on a wearable module according to claim 1, characterized in that The uploading the first video information to a server for training to obtain a target model includes: Extract the valid frames of the first video information as the initial sample set. Extract the target images corresponding to the lens from each image information in the initial sample set to obtain a target sample set. Based on a preset semantic segmentation basic model, add a convolutional layer, a pooling layer, and an upsampling layer to build a model framework. Based on the model framework, train the model separately for each wearer in a multi-concurrent manner for the target sample set, and determine the target model to adapt to the lens characteristics of multiple wearers.

7. The dynamic diopter detection method based on a wearable module according to claim 6, wherein The based on the model framework, training the model separately for each wearer in a multi-concurrent manner for the target sample set, and determining the target model to adapt to the lens characteristics of multiple wearers includes: Divide the target sample set into a training set and a test set, and shuffle the training set and the test set. Use pre-trained weights to read and search for a preset GPU, and allocate the first computing power and the second computing power of the GPU. According to the first computing power and the second computing power, process the shuffled training set and test set respectively, and adjust the number of samples in real time based on the model framework to synchronously perform training and test evaluation to obtain the target model.

8. A dynamic diopter detection system based on a wearable module, characterized in that, Includes: An excitation module for exciting light through a light source component of the wearable module so that the light enters the wearer's eyeball, and the light is diffusely reflected and refracted by the retina and the lens to obtain a first reflected light. A first acquisition module for acquiring first video information containing the first reflected light through the optical axis of the camera module of the wearable module. A training module for uploading the first video information to a server for training to obtain a target model, where the target model is used to analyze the light reflected by the lens to obtain corresponding diopter information. A second acquisition module for acquiring second video information and uploading it to the server, and using the OpenCV tool to segment multiple lens images from the second video information; wherein, the second video information includes a second reflected light diffusely reflected and refracted by the retina and the lens. An analysis module for analyzing the lens images based on the target model to output diopter change trend information; wherein, the diopter trend information is used to characterize the myopia development information of the wearer for myopia prevention and control.

9. An electronic device, characterized in that, Includes: A processor, a memory, and a program stored on the memory and executable on the processor, where the program, when executed by the processor, implements the dynamic diopter detection method based on the wearable module according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, A program or instruction is stored on the computer-readable storage medium, and when the program or instruction is executed by a processor, the dynamic diopter detection method based on the wearable module according to any one of claims 1 to 7 is implemented.

Images