Strabismus detection method, device and computer readable storage medium

Abstract

The present invention discloses a strabismus detection method, device and computer-readable storage medium. The strabismus detection method comprises: obtaining a first eye image of a target user's eye, performing binocular orbital image recognition on the first eye image, obtaining the white of the eye position and the first pupil position of the target user's eye; generating the second pupil position of the target user's eye based on the white of the eye position; comparing the first pupil position and the second pupil position, generating the deviation between the first pupil position and the second pupil position, and outputting the deviation as a reference for determining whether the target user's eye has strabismus. The detection process of the user's eye strabismus is greatly facilitated, and the error of manual visual inspection is reduced, without relying too much on the doctor's experience.

Description

Technical Field

[0001] The present invention relates to the field of eye detection, and in particular to a strabismus detection method, device and computer-readable storage medium. Background Art

[0002] Strabismus is usually caused by the eye muscles being too tight or relaxed for a long time, resulting in one eye being straight while the other eye is deviated inward or outward, so that when one eye is looking, the other eye's visual axis deviates from parallel abnormal eye position. Exotropia and exotropia are the two most common types of strabismus.

[0003] In medical ophthalmology, most patients with strabismus are young and cannot undergo surgery to repair strabismus in time, or the strabismus is not serious and is not convenient for surgical treatment, which causes the patients to be unable to see normally and affects their normal life. Moreover, at this stage, even if you go to an optician or hospital, the pupil distance is mostly measured manually with a ruler, and the judgment of strabismus is mainly based on visual observation. On the one hand, it requires a high level of experience from the doctor, and on the other hand, there is a situation where mild strabismus cannot be diagnosed in time, missing the best treatment period. Therefore, the current strabismus detection method is inconvenient for users.

[0004] Therefore, there is an urgent need for a method that can conveniently and quickly detect the user's eye state.

[0005] The above contents are only used to assist in understanding the technical solution of the present invention and do not constitute an admission that the above contents are prior art. Summary of the invention

[0006] The main purpose of the present invention is to provide a strabismus detection method, device and computer-readable storage medium, aiming to solve the technical problem that the current strabismus detection method is inconvenient to users.

[0007] To achieve the above object, the present invention provides a strabismus detection method, which comprises the following steps:

[0008] Acquire a first eye image of a target user's eye, perform binocular orbital image recognition on the first eye image, and acquire a white of the eye position and a first pupil position of the target user's eye;

[0009] Generating a second pupil position of the target user's eye based on the white of the eye position;

[0010] The first pupil position and the second pupil position are compared to generate a deviation between the first pupil position and the second pupil position, and the deviation is output as a reference for determining whether the target user has strabismus.

[0011] Furthermore, the white of the eye position includes an outer position of the white of the left eye, an inner position of the white of the left eye, an outer position of the white of the right eye, and an inner position of the white of the right eye. The step of generating the second pupil position of the target user's eye based on the white of the eye position includes:

[0012] A second left eye pupil position and a second right eye pupil position are generated based on the left eye white outer position, the left eye white inner position, the right eye white outer position and the right eye white inner position, wherein the second pupil position includes the second left eye pupil position and the second right eye pupil position.

[0013] Further, the step of generating a second left eye pupil position and a second right eye pupil position based on the left eye white outer position, the left eye white inner position, the right eye white outer position and the right eye white inner position comprises:

[0014] Generate a reference distance based on the left eye white outer position, the left eye white inner position, the right eye white outer position, the right eye white inner position and a preset calculation formula;

[0015] The preset calculation formula includes:

[0016]

[0017] Wherein, N is the reference distance, D is the distance between the outer side of the left eye white and the outer side of the right eye white, and C is the distance between the inner side of the left eye white and the inner side of the right eye white;

[0018] The position where the inner part of the left white of the left eye points to the outer part of the left white of the left eye and is the distance from the inner part of the left white of the left eye as the second left eye pupil position, and the position where the inner part of the right white of the right eye points to the outer part of the right white of the right eye and is the distance from the inner part of the right white of the right eye as the second right eye pupil position.

[0019] Further, the first pupil position includes a first left eye pupil position and a first right eye pupil position, and the step of comparing the first pupil position and the second pupil position to generate a deviation between the first pupil position and the second pupil position includes:

[0020] Comparing the first left eye pupil position with the second left eye pupil position to generate a left eye pupil deviation between the first left eye pupil position and the second left eye pupil position;

[0021] Comparing the first right eye pupil position with the second right eye pupil position to generate a right eye pupil deviation between the first right eye pupil position and the second right eye pupil position;

[0022] The deviation degree includes the left eye pupil deviation degree and the right eye pupil deviation degree.

[0023] Furthermore, before the step of acquiring the first eye image of the target user's eye, the method includes:

[0024] Outputting a preset target image on a display device located in front of a target user so that the target user looks forward;

[0025] When it is detected that the target user is in a state of looking forward, the image of the target user's eyes is captured by two detection modules to obtain the first eye image.

[0026] Furthermore, before the step of outputting the preset target image, the detection method includes:

[0027] Acquiring a second eye image of the target user's eye;

[0028] generating adjustment data of the distance between the two detection modules based on the second eye image;

[0029] The distance between the two detection modules is adjusted based on the adjustment data so that the positions of the two detection modules match the eyes of the target user.

[0030] Furthermore, after the step of outputting the deviation, the method further includes:

[0031] Acquiring training content and training intensity from a preset training content library according to the deviation;

[0032] A training image is output based on the training content and the training intensity to train the eyes of the target user.

[0033] In addition, to achieve the above-mentioned purpose, the present invention also provides a strabismus detection device, the strabismus detection device comprising: two detection modules and a processing module, the two detection modules are symmetrically arranged on the left and right; the detection module comprises a display device, a reflection element and a camera element; the processing module is communicatively connected with the camera element; a preset angle is formed between the plane where the display device is located and the plane where the reflection element is located, and the camera element is located on one side of the reflection surface of the reflection element, wherein the reflection surface is a side of the reflection element away from the display device; the light incident direction of the camera element is toward the reflection surface, and the camera element is used to receive the light reflected by the reflection surface; the angle between the front light incident direction of the camera element and the front light emitting direction of the display device is twice the preset angle;

[0034] The processing module includes: a memory, a processor, and a strabismus detection program stored in the memory and executable on the processor. When the strabismus detection program is executed by the processor, the steps of the strabismus detection method described above are implemented.

[0035] Furthermore, the detection module also includes an optical lens group and an eye box; the plane where the optical lens group is located is parallel to the plane where the display device is located, the optical lens group, the reflective element and the display device are arranged in sequence, the eye box is located on the side of the optical lens group away from the display device and there is a preset distance between the eye box and the optical lens group, and the front light emitting direction of the display device is projected into the eye box.

[0036] In addition, to achieve the above-mentioned purpose, the present invention further provides a computer-readable storage medium, on which a strabismus detection program is stored, and when the strabismus detection program is executed by a processor, the steps of the strabismus detection method as described above are implemented.

[0037] A strabismus detection method, device and computer-readable storage medium proposed in an embodiment of the present invention obtain a first eye image of a target user's eye, perform binocular orbital image recognition on the first eye image, obtain the eye white position and the first pupil position of the target user's eye; generate the second pupil position of the target user's eye based on the eye white position; compare the first pupil position and the second pupil position, generate the deviation between the first pupil position and the second pupil position, and output the deviation as a reference for determining whether the target user's eye has strabismus. By performing image recognition on the target user's eye, the eye white position and the first pupil position of the user's eye are obtained, and the eye white position will be used to generate the second pupil position, which is the user's standard pupil map. The first pupil position is compared with the second pupil position, so as to accurately obtain the deviation between the first pupil position and the second pupil position. The deviation can be used as a basis for determining whether the user's eye has strabismus. The detection process of user eye strabismus is greatly facilitated, and the error of manual visual inspection is reduced, without relying too much on the doctor's experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 It is a structural schematic diagram of a strabismus detection device in a strabismus detection method of the present invention;

[0039] Figure 2 It is a schematic diagram of partial details of the strabismus detection device of the present invention;

[0040] Figure 3 It is a structural schematic diagram of the operating environment of the processing module in the strabismus detection device of the present invention;

[0041] Figure 4Schematic diagram of the process of the first embodiment of the strabismus detection method of the present invention;

[0042] Figure 5 1 is a flow chart of a second embodiment of a strabismus detection method of the present invention;

[0043] Figure 6 Schematic diagram of the process of the third embodiment of the strabismus detection method of the present invention;

[0044] Figure 7 Schematic diagram of the eyes of a target user in the strabismus detection method of the present invention;

[0045] Figure 8 A schematic diagram of the eyes of a target user for image recognition in the strabismus detection method of the present invention;

[0046] Fig. 9 It is a quantified eye map of the target user in the strabismus detection method of the present invention;

[0047] Fig.10 This is a comparison diagram of the eyes of the target user in the strabismus detection method of the present invention.

[0048] Description of the accompanying drawings of the embodiments:

[0049] Label name Label name 1000 Strabismus detection device 10 Detection module 20 Display Devices 30 Reflective element 31 Reflective surface 40 Camera Components 50 Optical lens 60 Eye box 70 Light source emitting element 80 Processing module 90 Connection mechanism 91 Adjustment detection module 100 External display module 110 Voice prompt module

[0050] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0051] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0052] The main solution of the embodiment of the present invention is: obtaining a first eye image of the target user's eye, performing binocular orbital image recognition on the first eye image, and obtaining the white of the eye position and the first pupil position of the target user's eye; generating the second pupil position of the target user's eye based on the white of the eye position; comparing the first pupil position and the second pupil position, generating a deviation between the first pupil position and the second pupil position, and outputting the deviation as a reference for determining whether the target user's eye has strabismus.

[0053] Current strabismus detection methods are inconvenient for users.

[0054] The present invention provides a solution, by performing image recognition on the eyes of the target user, the white of the eye position and the first pupil position of the user's eyes are obtained, the white of the eye position will be used to generate the second pupil position, the second pupil position is the user's standard pupil map, the first pupil position is compared with the second pupil position, so as to accurately obtain the deviation between the first pupil position and the second pupil position. The deviation can be used as a basis for judging whether the user's eyes have strabismus. The detection process of the user's eye strabismus is greatly facilitated, and the error of manual visual inspection is reduced, without relying too much on the doctor's experience.

[0055] Reference Figure 1 , Figure 1 It is a structural schematic diagram of a strabismus detection device in the strabismus detection method of the present invention.

[0056] like Figure 1 As shown, a strabismus detection device 1000 comprises: two detection modules 10, the two detection modules 10 are symmetrically arranged on the left and right; the detection module 10 comprises a display device 20, a reflective element 30 and an imaging element 40; a preset angle is formed between a plane where the display device 20 is located and a plane where the reflective element 30 is located, and the imaging element 40 is located on one side of a reflective surface of the reflective element 30, wherein the reflective surface is a side of the reflective element 30 away from the display device 20; a light incident direction of the imaging element 40 is toward the reflective surface 31, and the imaging element 40 is used to receive light reflected by the reflective surface 31; an angle between a front light incident direction of the imaging element 40 and a front light emitting direction of the display device 20 is twice the preset angle.

[0057] In this embodiment, the strabismus detection device 1000 further includes an auxiliary support member (shown in the figure), and the auxiliary support member can support the two detection modules 10 and the components in the detection modules 10 to maintain the same Figure 1 When the strabismus detection device 1000 is used to detect the eye condition of the user, the user looks at the two symmetrically arranged detection modules 10. Since the reflective element 30 only reflects invisible light (it has high transmittance to visible light), the user can see the picture displayed on the display device 20, and the display device 20 can attract the user's eyes to look forward by displaying the picture. When the invisible light in nature shines on the user's eyes, the invisible light is diffusely reflected by the eyes and is emitted to the surroundings. Since there is a preset angle (which is 1 / 40mm) between the reflective element 30 and the camera element 40, the user can see the picture displayed on the display device 20. Figure 1The preset angle can be set by a technician according to the relative position relationship of each component, and is not specifically limited here. After a portion of the invisible light is reflected by the reflective surface 31 of the reflective element 30, it will be reflected to the imaging element 40, wherein the imaging element 40 is located on one side of the reflective surface 31 of the reflective element 30. The incident light direction of the imaging element 40 is toward the reflective surface 31, so that the imaging element 40 can receive the invisible light reflected by the reflective surface 31 of the reflective element 30. Further, referring to Figure 2 , the angle between the front light incident direction of the camera element 40 and the front light incident direction of the display device 20 is 2α, and the invisible light reflected from the front of the user's eye will be emitted again through the reflective element 30 and vertically incident on the camera element 40, so that the camera elements 40 in the two detection modules 10 generate a front image of the user's eye according to the received invisible light. It can be understood that the front image of the user's eye generated in this embodiment is generated when the user's eyes are looking at the display device 20, which can eliminate the influence of the user's own distraction on the examination when the doctor manually checks, and based on the front image of the user's eye, it can be convenient to judge whether the user has strabismus symptoms.

[0058] Furthermore, the detection module 10 also includes an optical lens group 50 and an eye box 60; the plane where the optical lens group 50 is located is parallel to the plane where the display device 20 is located, the optical lens group 50, the reflective element 30 and the display device 20 are arranged in sequence, the eye box 60 is located on the side of the optical lens group 50 away from the display device and there is a preset distance between the eye box 60 and the optical lens group 50, and the front light emitting direction of the display device 20 is projected into the eye box 60.

[0059] In this embodiment, the detection module 10 also includes an optical lens group 50 and an eye box 60, the plane of the optical lens group 50 is parallel to the plane of the display device 20, and the reflective element 30 is located between the optical lens group 50 and the display device 20. The reflective element 30 and the optical lens group 50 can magnify the image displayed on the display device 20 and project it on the side of the display device 20 away from the human eye. Usually, the projection distance of the optical lens group 50 will be greater than 2m, so as to reproduce the scene of the user's normal eye use. The interval between the eye box 60 and the optical lens group 50 is usually greater than 10mm. Due to differences between different human eyes, the specific interval distance can also be adjusted by a technician or a user. The eye box 60 can limit the user's eye position, that is, the user can watch the picture on the display device 20 and collect the frontal image of the eye at the position of the eye box 60, so that the collected frontal image of the eye can be more in line with the design requirements during collection, and the reliability of the frontal image of the eye can be increased.

[0060] Furthermore, the detection module 10 also includes a light emitting element 70 ; the light emitting element 70 is located on a side of the optical lens assembly 50 away from the reflective element, wherein the light emitting element 70 is used to emit invisible light into the eye box 60 .

[0061] In this embodiment, the detection module 10 further includes a light emitting element 70. The number of the light emitting element 70 can be one or more. The light emitting element 70 is located on a side of the optical lens assembly 50 away from the reflective element 30. The light emitting element 70 can emit invisible light to the eye, thereby making up for the lack of invisible light in nature and avoiding affecting the collection of the front image of the user's eye. Moreover, the invisible light generated by the light emitting element 70 does not affect the user's viewing of the display device 20.

[0062] Furthermore, the light emitting element 70 is an infrared light emitting element, and the infrared light emitting element is used to emit infrared light to the user's eyes in the eye box.

[0063] In this embodiment, the light source emitting element 70 may be an infrared light source emitting element, such as an infrared LED lamp, which emits infrared light toward the user's eyes to enhance the imaging effect of the camera element 40 .

[0064] Furthermore, the reflective element 30 is an infrared reflective element, and the infrared reflective element is used to reflect the infrared light diffusely reflected by the user's eyes.

[0065] In this embodiment, the reflective element 30 may be an infrared reflective element, which has high reflectivity to infrared light, thereby increasing the amount of infrared light diffusely reflected by the user's eyes entering the imaging element 40 .

[0066] Furthermore, the strabismus detection device 1000 further includes a processing module 80 ; the processing module 80 is communicatively connected to the imaging element 40 ; and the processing module 80 is communicatively connected to the display device 20 .

[0067] In the present embodiment, the visual detection device 1000 further includes a processing module 80, which is an electronic device with data processing function, such as a computer or a mobile phone. The processing module 80 can be connected to the imaging element 40 and the display device 20 by wireless or wired communication. The processing module 80 will obtain the left eye image and the right eye image of the user collected by the two imaging elements 40, so as to form a complete frontal eye image of the user with the left eye image and the right eye image for output, so as to facilitate the judgment of the eye condition of the user. The processing module 80 will also input preset picture data to the display device 20 so that the display device 20 displays a preset picture.

[0068] Furthermore, the strabismus detection device 1000 further includes a connecting mechanism 90 ; the two detection modules 10 symmetrically arranged on the left and right are connected via the connecting mechanism 90 , and the connecting mechanism 90 is used to adjust the distance between the two detection modules 10 .

[0069] In this embodiment, the strabismus detection device 1000 further includes a connecting mechanism 90, which connects the two detection modules 10, and the connecting mechanism 90 can adjust the distance between the two detection modules 10, so that the strabismus detection device 1000 conforms to the actual pupil distance of the user.

[0070] Furthermore, the connection mechanism 90 further includes an adjustment detection module 91 , and the adjustment detection module 91 is used to detect the adjustment distance between the two detection modules 10 .

[0071] In this embodiment, the strabismus detection device 1000 also includes an adjustment detection module 91, which can detect the adjusted distance between the two detection modules 10, thereby obtaining the user's actual pupil distance, and the user's pupil distance can be used as one of the bases for judging the user's eye condition.

[0072] Furthermore, the strabismus detection device 1000 also includes a voice prompt module 110 ; the voice prompt module 110 is communicatively connected to the processing module 80 .

[0073] In this embodiment, the strabismus detection device 1000 further includes a voice prompt module 110 , which can receive instructions from the processing module 80 to prompt the user to look at the image screen in the display device 20 .

[0074] Furthermore, the strabismus detection device 1000 also includes an external display module 100 ; the external display module 100 is communicatively connected to the display device 20 ; or / and, the external display module 100 is communicatively connected to the processing module 80 .

[0075] In this embodiment, the strabismus detection device 1000 also includes an external display module 100. The external display module 100 can be a display screen, and the external display module 100 can be connected to the display device 20 or / and connected to the processing module 80 via a wireless network or a data cable. Specifically, the external display module 100 can receive the screen data of the display device 20 to display the same screen image as the display device 20, so that other people can watch the screen currently seen by the detected user. The external display module 100 can also receive the user's front eye image sent by the processing module 80, so as to facilitate the display of the front eye image to the outside world and facilitate the determination of the user's eye condition.

[0076] like Figure 3 As shown, Figure 3It is a structural schematic diagram of the operating environment of the processing module in the strabismus detection device of the present invention.

[0077] The processing module in the embodiment of the present invention may be a PC, or an electronic terminal device such as a smart phone, a tablet computer, or a portable computer that has data receiving, data processing, and data sending functions.

[0078] like Figure 1 As shown, the processing module may include: a processor 1001, such as a CPU, a network interface 1004, a user interface 1003, a memory 1005, and a communication bus 1002. Among them, the communication bus 1002 is used to realize the connection and communication between these components. The user interface 1003 may include a display screen (Display), an input unit such as a keyboard (Keyboard), and the optional user interface 1003 may also include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a WI-FI interface). The memory 1005 may be a high-speed RAM memory, or it may be a stable memory (non-volatile memory), such as a disk memory. The memory 1005 may also be a storage device independent of the aforementioned processor 1001.

[0079] Optionally, the processing module may also include a camera, an RF (Radio Frequency) circuit, a sensor, an audio circuit, a WiFi module, and the like. Among them, sensors include light sensors, motion sensors, and other sensors. Specifically, the light sensor may include an ambient light sensor and a proximity sensor, wherein the ambient light sensor may adjust the brightness of the display screen according to the brightness of the ambient light, and the proximity sensor may turn off the display screen and / or backlight when the processing module is moved to the ear. As a type of motion sensor, the gravity acceleration sensor can detect the magnitude of acceleration in each direction (generally three axes), and can detect the magnitude and direction of gravity when stationary. It can be used for applications that identify the posture of the mobile terminal (such as horizontal and vertical screen switching, related games, magnetometer posture calibration), vibration recognition related functions (such as pedometer, tapping), etc.; of course, the processing module can also be configured with other sensors such as gyroscopes, barometers, hygrometers, thermometers, infrared sensors, etc., which will not be repeated here.

[0080] Those skilled in the art will understand that Figure 1 The processing module structure shown in the figure does not constitute a limitation on the processing module, and may include more or less components than shown in the figure, or combine certain components, or arrange the components differently.

[0081] like Figure 3As shown, the memory 1005 as a computer storage medium may include an operating system, a network communication module, a user interface module, and a strabismus detection program.

[0082] exist Figure 3 In the processing module shown, the network interface 1004 is mainly used to connect to the backend server and perform data communication with the backend server; the user interface 1003 is mainly used to connect to the client (user end) and perform data communication with the client; and the processor 1001 can be used to call the strabismus detection program stored in the memory 1005 and perform the following operations:

[0083] Acquire a first eye image of a target user's eye, perform binocular orbital image recognition on the first eye image, and acquire a white of the eye position and a first pupil position of the target user's eye;

[0084] Generating a second pupil position of the target user's eye based on the white of the eye position;

[0085] The first pupil position and the second pupil position are compared to generate a deviation between the first pupil position and the second pupil position, and the deviation is output as a reference for determining whether the target user has strabismus.

[0086] Furthermore, the processor 1001 may call the strabismus detection program stored in the memory 1005, and further perform the following operations:

[0087] The white of the eye position includes the outer position of the white of the left eye, the inner position of the white of the left eye, the outer position of the white of the right eye, and the inner position of the white of the right eye. The step of generating the second pupil position of the target user's eye based on the white of the eye position includes:

[0088] A second left eye pupil position and a second right eye pupil position are generated based on the left eye white outer position, the left eye white inner position, the right eye white outer position and the right eye white inner position, wherein the second pupil position includes the second left eye pupil position and the second right eye pupil position.

[0089] Furthermore, the processor 1001 may call the strabismus detection program stored in the memory 1005, and further perform the following operations:

[0090] The step of generating a second left eye pupil position and a second right eye pupil position based on the left eye white outer position, the left eye white inner position, the right eye white outer position and the right eye white inner position comprises:

[0091] Generate a reference distance based on the left eye white outer position, the left eye white inner position, the right eye white outer position, the right eye white inner position and a preset calculation formula;

[0092] The preset calculation formula includes:

[0093]

[0094] Wherein, N is the reference distance, D is the distance between the outer side of the left eye white and the outer side of the right eye white, and C is the distance between the inner side of the left eye white and the inner side of the right eye white;

[0095] The position where the inner part of the left white of the left eye points to the outer part of the left white of the left eye and is the distance from the inner part of the left white of the left eye as the second left eye pupil position, and the position where the inner part of the right white of the right eye points to the outer part of the right white of the right eye and is the distance from the inner part of the right white of the right eye as the second right eye pupil position.

[0096] Furthermore, the processor 1001 may call the strabismus detection program stored in the memory 1005, and further perform the following operations:

[0097] The first pupil position includes a first left eye pupil position and a first right eye pupil position, and the step of comparing the first pupil position and the second pupil position to generate a deviation between the first pupil position and the second pupil position includes:

[0098] Comparing the first left eye pupil position with the second left eye pupil position to generate a left eye pupil deviation between the first left eye pupil position and the second left eye pupil position;

[0099] Comparing the first right eye pupil position with the second right eye pupil position to generate a right eye pupil deviation between the first right eye pupil position and the second right eye pupil position;

[0100] The deviation degree includes the left eye pupil deviation degree and the right eye pupil deviation degree.

[0101] Furthermore, the processor 1001 may call the strabismus detection program stored in the memory 1005, and further perform the following operations:

[0102] Before the step of acquiring a first eye image of the target user's eye, the method includes:

[0103] Outputting a preset target image on a display device located in front of a target user so that the target user looks forward;

[0104] When it is detected that the target user is in a state of looking forward, the image of the target user's eyes is captured by two detection modules to obtain the first eye image.

[0105] Furthermore, the processor 1001 may call the strabismus detection program stored in the memory 1005, and further perform the following operations:

[0106] Before the step of outputting the preset target image, the detection method includes:

[0107] Acquiring a second eye image of the target user's eye;

[0108] generating adjustment data of the distance between the two detection modules based on the second eye image;

[0109] The distance between the two detection modules is adjusted based on the adjustment data so that the positions of the two detection modules match the eyes of the target user.

[0110] Furthermore, the processor 1001 may call the strabismus detection program stored in the memory 1005, and further perform the following operations:

[0111] After the step of outputting the deviation, the method further comprises:

[0112] Acquiring training content and training intensity from a preset training content library according to the deviation;

[0113] A training image is output based on the training content and the training intensity to train the eyes of the target user.

[0114] Reference Figure 4 , a first embodiment of the strabismus detection method of the present invention, the strabismus detection method comprises:

[0115] Step S10, obtaining a first eye image of the target user's eyes, performing binocular orbital image recognition on the first eye image, and obtaining the white of the eye position and the first pupil position of the target user's eyes;

[0116] In this embodiment, the implementation subject is a strabismus detection device (or a processing module in the strabismus detection device), and the strabismus detection device can collect a first eye image of the target user (the first eye image can be a color image or an infrared image) through the camera element of its own detection module, and perform binocular orbital image recognition on the first eye image, thereby generating the white of the eye position and the first pupil position of the target user's eye. It can be understood that there are relatively mature image recognition technologies, and it is only necessary to mark the features to be recognized in the image samples, and put the marked images into the image recognition algorithm for training, so as to obtain an image recognition algorithm with the function of recognizing specific features, which will not be repeated here.

[0117] Step S20, generating a second pupil position of the target user's eye based on the white of the eye position;

[0118] Furthermore, the white of the eye position includes an outer position of the left white of the eye, an inner position of the left white of the eye, an outer position of the right white of the eye and an inner position of the right white of the eye, and a second left eye pupil position and a second right eye pupil position are generated based on the outer position of the left eye of the eye, the inner position of the left eye of the eye, the outer position of the right eye of the eye and the inner position of the right eye of the eye, wherein the second pupil position includes the second left eye pupil position and the second right eye pupil position.

[0119] Specifically, the white of the eye position includes the outer position of the left white of the eye, the inner position of the left white of the eye, the outer position of the right white of the eye, and the inner position of the right white of the eye. Figure 7 The target user's eye schematic diagram marks the right eye white, right eye outer canthus, right eye inner canthus, left eye length and right eye length. The outer position of the right eye white is the position where the right eye white meets the right eye outer canthus (the outer position of the right eye white can also be the position of the right eye outer canthus, and the two positions are not much different, so the technician can set it according to the actual situation), and the inner position of the right eye white is the edge position of the right eye white close to the inner canthus. Since the left and right eyes are symmetrical, the outer position of the left eye white and the inner position of the left eye white can be determined by the same logic, which will not be repeated here. The outer position of the left eye white, the inner position of the left eye white, the outer position of the right eye white and the inner position of the right eye white of the target user's eyes are identified by image recognition technology, and the second left eye pupil position and the second right eye pupil position are generated. It can be understood that the generated second left eye pupil position and the second right eye pupil position will be used as the reference position of the pupil of the target user's eye.

[0120] Further, a reference distance is generated based on the outer position of the white of the left eye, the inner position of the white of the left eye, the outer position of the white of the right eye, the inner position of the white of the right eye and a preset calculation formula;

[0121] The preset calculation formula includes:

[0122]

[0123] Wherein, N is the reference distance, D is the distance between the outer side of the left eye white and the outer side of the right eye white, and C is the distance between the inner side of the left eye white and the inner side of the right eye white;

[0124] The position where the inner part of the left white of the left eye points to the outer part of the left white of the left eye and is the distance from the inner part of the left white of the left eye as the second left eye pupil position, and the position where the inner part of the right white of the right eye points to the outer part of the right white of the right eye and is the distance from the inner part of the right white of the right eye as the second right eye pupil position.

[0125] Specifically, refer to Figure 8 , Figure 8Schematic diagram of the target user's eyes for image recognition. D is the distance between the outer position of the left eye white and the outer position of the right eye white, C is the distance between the inner position of the left eye white and the inner position of the right eye white, (DC) / 4=N, N is the reference distance. In the direction from the inner position of the right eye white to the outer position of the right eye white, the position N away from the inner position of the right eye white is the second right eye pupil position. Similarly, the position from the inner position of the left eye white to the outer position of the left eye white at the reference distance from the inner position of the left eye white is the second left eye pupil position. It can be understood that the generated second left eye pupil position is actually the midpoint position of the inner position of the left eye white and the outer position of the left eye white. Similarly, the generated second right eye pupil position is also the midpoint position of the inner position of the right eye white and the outer position of the right eye white. The second left eye pupil position and the second right eye pupil position will also be used as reference positions for detecting strabismus.

[0126] Step S30, comparing the first pupil position and the second pupil position, generating a deviation between the first pupil position and the second pupil position, and outputting the deviation as a reference for determining whether the target user has strabismus.

[0127] Furthermore, the first pupil position includes a first left eye pupil position and a first right eye pupil position, and the first left eye pupil position and the second left eye pupil position are compared to generate a left eye pupil deviation between the first left eye pupil position and the second left eye pupil position; the first right eye pupil position and the second right eye pupil position are compared to generate a right eye pupil deviation between the first right eye pupil position and the second right eye pupil position; the deviation includes the left eye pupil deviation and the right eye pupil deviation.

[0128] Specifically, the first pupil position is the actual pupil position of the user identified from the user's eye image through image recognition technology. It should be noted that the first pupil position and the white of the eye position are both identified based on the same eye image and the same recognition algorithm. Therefore, the relative position between the first pupil position and the white of the eye position can reflect the actual condition of the user's eyes. The first pupil position includes a first left eye pupil position and a first right eye pupil position. Correspondingly, the first left eye pupil position is compared with the second left eye pupil position, and the right eye pupil position is compared with the second right eye pupil position. Take the user's right eye as an example for explanation. Fig. 9 As shown in FIG. 1 , based on the quantization map generated by the inner position of the right eye white, the outer position of the right eye white and the second right eye pupil position, the second right eye pupil position is located at the center position (zero point position) of the quantization map. The first pupil position is placed in the quantization map generated with the second right eye pupil position as the center for comparison, as shown in FIG. Fig.10In the comparison diagram shown, the first right eye pupil position in the quantization diagram is -20, that is, the generated right eye pupil deviation is -20. -20 is output as a basis for judging whether the target right eye has strabismus. Similarly, the left eye pupil deviation can also be generated in the above steps, which will not be repeated here.

[0129] It can be understood that in this embodiment, the first eye image of the target user's eye is obtained, and binocular orbital image recognition is performed on the first eye image to obtain the eye white position and the first pupil position of the target user's eye; the second pupil position of the target user's eye is generated based on the eye white position; the first pupil position and the second pupil position are compared to generate the deviation between the first pupil position and the second pupil position, and the deviation is output as a reference for determining whether the target user's eye has strabismus. In this embodiment, by performing image recognition on the target user's eye, the eye white position and the first pupil position of the user's eye are obtained, and the eye white position will be used to generate the second pupil position, the second pupil position is the user's standard pupil map, and the first pupil position is compared with the second pupil position, so as to accurately obtain the deviation between the first pupil position and the second pupil position. The deviation can be used as a basis for determining whether the user's eye has strabismus. It greatly facilitates the detection process of the user's eye strabismus, and reduces the error of manual visual inspection, without relying too much on the doctor's experience.

[0130] Reference Figure 5 Based on the first embodiment of the strabismus detection method of the present invention, a second embodiment of the strabismus detection method of the present invention is proposed:

[0131] Before step S10, the strabismus detection method includes:

[0132] Step S01, collecting a second eye image of the target user's eye;

[0133] In this embodiment, the second eye image of the target user is captured by two symmetrical left-right detection modules. Specifically, the user wears two symmetrical detection modules, and the detection modules will capture the user's eyes to capture the second eye image.

[0134] Step S02, generating adjustment data of the distance between the two detection modules based on the second eye image;

[0135] Specifically, the position of the white of the eye is obtained from the second eye image (the user's pupil position may not be obtained) by also using image recognition technology, and the position of the white of the eye includes the outer position of the white of the left eye, the inner position of the white of the left eye, the outer position of the white of the right eye, and the inner position of the white of the right eye. Based on the fact that the position of the white of the eye includes the outer position of the white of the left eye, the inner position of the white of the left eye, the outer position of the white of the right eye, and the inner position of the white of the right eye, the distance D between the outer position of the white of the left eye and the outer position of the white of the right eye and the distance C between the inner position of the white of the left eye and the inner position of the white of the right eye can be obtained. Based on D and C, a reference distance N can be obtained (refer to the first embodiment). The reference pupil distance of the user's eyes can be generated by C and N, wherein the generation formula of the reference pupil distance is: reference pupil distance=C+2N. The reference pupil distance is used as the adjustment data.

[0136] Step S03, adjusting the distance between the two detection modules based on the adjustment data so that the positions of the two detection modules match the eyes of the target user;

[0137] Specifically, the distance between the detection modules is adjusted based on the reference pupil distance. Each detection module includes a display device. The distance between the two display devices is adjusted by adjusting the distance between the detection modules, so that the distance between the two display devices matches the reference pupil distance of the target user's eyes, so that the user has a better viewing effect when looking at the display device.

[0138] Step S04, outputting a preset target image on a display device located in front of the target user, so that the target user looks forward;

[0139] Specifically, after adjusting the detection module, the preset target image will be output on the two display devices so that the user can look forward. Usually, an optical lens group can be placed in front of the display device, that is, the user views the preset target image on the display device through the optical lens group, and the optical lens group can present the preset target image on the display device at a position greater than two meters in front of the user's eyes, thereby restoring the scene when the user looks forward.

[0140] Step S05, when it is detected that the target user is in a state of looking forward, the image of the target user's eyes is captured by two detection modules to obtain the first eye image.

[0141] Specifically, the left sight line of the user's left eyeball and the right sight line of the right eyeball are respectively identified through eye tracking, and the intersection of the left sight line and the right sight line is used as the user's gaze point. When the gaze point overlaps with the preset target image position presented by the display device, it is determined that the user is in a state of looking forward. The image of the target user's eyes is collected through two detection modules to obtain a first eye image.

[0142] It is understandable that the first eye image is an image of the user looking forward. Therefore, the user eye-related detection data generated based on the first eye image is more reliable.

[0143] Reference Figure 6 Based on the first embodiment of the strabismus detection method of the present invention, a second embodiment of the strabismus detection method of the present invention is proposed:

[0144] After step S30, the strabismus detection method includes:

[0145] Step S301, acquiring training content and training intensity from a preset training content library according to the deviation;

[0146] Specifically, the training content library will contain training image data of different types and stages. The training content and training intensity are obtained from the training content library based on the deviation. For example, the deviation is -20, where the minus sign (-) is the deviation direction, and 20 is the degree of deviation. The training content of the negative deviation direction type is matched from the training content library, and the training content of the stage corresponding to 20 (degree of deviation) is matched from the training content of the negative deviation direction type. In addition, the training intensity (such as the number of training sessions, etc.) corresponding to 20 is also matched in the training content library.

[0147] Step S302: outputting a training image based on the training content and the training intensity to train the eyes of the target user.

[0148] Specifically, the training image is played on the display device in the detection module based on the training content and training intensity matched above. For example, the matched training content is played on the display device a set number of times. The user can correct the user's strabismus by watching the screen of the display device.

[0149] In addition, an embodiment of the present invention further provides a strabismus detection device, the strabismus detection device comprising: two detection modules and a processing module, the two detection modules are symmetrically arranged on the left and right; the detection module comprises a display device, a reflective element and a camera element; the processing module is communicatively connected with the camera element; a plane where the display device is located and a plane where the reflective element is located form a preset angle, the camera element is located on one side of the reflective surface of the reflective element, wherein the reflective surface is a side of the reflective element away from the display device; the light incident direction of the camera element is toward the reflective surface, and the camera element is used to receive the light reflected by the reflective surface; the angle between the front light incident direction of the camera element and the front light emitting direction of the display device is twice the preset angle;

[0150] The processing module includes: a memory, a processor, and a strabismus detection program stored in the memory and executable on the processor. When the strabismus detection program is executed by the processor, the steps of the strabismus detection method described above are implemented.

[0151] The specific implementation of the strabismus detection device of the present invention is basically the same as the embodiments of the anti-falling method of the strabismus detection device described above, and will not be described in detail here.

[0152] In addition, an embodiment of the present invention further provides a computer-readable storage medium, on which a strabismus detection program is stored. When the strabismus detection program is executed by a processor, the steps of the strabismus detection method described above are implemented.

[0153] The specific implementation of the medium of the present invention is basically the same as the above-mentioned strabismus detection method embodiments, and will not be described in detail here.

[0154] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or system including the element.

[0155] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0156] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product is stored in a storage medium (such as ROM / RAM, disk, CD) as described above, including a number of instructions for a terminal device (which can be a mobile phone, computer, server, VR device, or network device, etc.) to execute the methods described in each embodiment of the present invention.

[0157] The above are only preferred embodiments of the present invention, and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A strabismus detection method implemented by a computer program, characterized in that: The strabismus detection method implemented by a computer program comprises the following steps: Acquire a first eye image of a target user's eyes, perform binocular orbital image recognition on the first eye image, and acquire a white of the eye position and a first pupil position of the target user's eyes, wherein the first pupil position is an actual pupil position of the user; Generating a second pupil position of the target user's eye based on the white of the eye position; Comparing the first pupil position and the second pupil position, generating a deviation between the first pupil position and the second pupil position, and outputting the deviation as a reference for determining whether the target user has strabismus; The white of the eye position includes the outer white of the left eye, the inner white of the left eye, the outer white of the right eye, and the inner white of the right eye. The step of generating the second pupil position of the target user's eye based on the white of the eye position includes: Generate a second left eye pupil position and a second right eye pupil position based on the left eye white outer position, the left eye white inner position, the right eye white outer position and the right eye white inner position, wherein the second pupil position includes the second left eye pupil position and the second right eye pupil position; The step of generating a second left eye pupil position and a second right eye pupil position based on the left eye white outer position, the left eye white inner position, the right eye white outer position and the right eye white inner position comprises: Generate a reference distance based on the left eye white outer position, the left eye white inner position, the right eye white outer position, the right eye white inner position and a preset calculation formula; The preset calculation formula includes: Wherein, N is the reference distance, D is the distance between the outer side of the left eye white and the outer side of the right eye white, and C is the distance between the inner side of the left eye white and the inner side of the right eye white; The position of the inner part of the left eye white pointing to the outer part of the left eye white and the distance from the inner part of the left eye white is taken as the second left eye pupil position, and the position of the inner part of the right eye white pointing to the outer part of the right eye white and the distance from the inner part of the right eye white is taken as the second right eye pupil position; Before the step of acquiring the first eye image of the target user's eye, the method includes: Outputting a preset target image on a display device located in front of a target user so that the target user looks forward, wherein the left sight line of the left eyeball of the user and the right sight line of the right eyeball are respectively identified through eye tracking, and the intersection of the left sight line and the right sight line is used as the user's gaze point, and when the gaze point overlaps with the position of the preset target image presented by the display device, it is determined that the user is in a state of looking forward; When it is detected that the target user is in a state of looking forward, the image of the target user's eyes is captured by two detection modules to obtain the first eye image.

2. The strabismus detection method implemented by a computer program according to claim 1, characterized in that: The first pupil position includes a first left eye pupil position and a first right eye pupil position, and the step of comparing the first pupil position and the second pupil position to generate a deviation between the first pupil position and the second pupil position includes: Comparing the first left eye pupil position with the second left eye pupil position to generate a left eye pupil deviation between the first left eye pupil position and the second left eye pupil position; Comparing the first right eye pupil position with the second right eye pupil position to generate a right eye pupil deviation between the first right eye pupil position and the second right eye pupil position; The deviation degree includes the left eye pupil deviation degree and the right eye pupil deviation degree.

3. The strabismus detection method implemented by a computer program according to claim 1, characterized in that: Before the step of outputting the preset target image, the detection method includes: Acquiring a second eye image of the target user's eye; Generate adjustment data of the distance between the two detection modules based on the second eye image; The distance between the two detection modules is adjusted based on the adjustment data so that the positions of the two detection modules match the eyes of the target user.

4. The strabismus detection method implemented by a computer program according to claim 1, characterized in that: After the step of outputting the deviation, the method further comprises: Acquiring training content and training intensity from a preset training content library according to the deviation; A training image is output based on the training content and the training intensity to train the eyes of the target user.

5. A strabismus detection device, characterized in that: The strabismus detection device comprises: two detection modules and a processing module, the two detection modules are symmetrically arranged on the left and right; the detection module comprises a display device, a reflective element and a camera element; the processing module is in communication connection with the camera element; a preset angle is formed between the plane where the display device is located and the plane where the reflective element is located, and the camera element is located on one side of the reflective surface of the reflective element, wherein the reflective surface is a side of the reflective element away from the display device; the light incident direction of the camera element is toward the reflective surface, and the camera element is used to receive the light reflected by the reflective surface; the angle between the front light incident direction of the camera element and the front light emitting direction of the display device is twice the preset angle; The processing module includes: a memory, a processor, and a strabismus detection program stored in the memory and executable on the processor. When the strabismus detection program is executed by the processor, the steps of the strabismus detection method implemented by a computer program as described in any one of claims 1 to 4 are implemented.

6. The strabismus detection device according to claim 5, characterized in that: The detection module also includes an optical lens group and an eye box; the plane where the optical lens group is located is parallel to the plane where the display device is located, the optical lens group, the reflective element and the display device are arranged in sequence, the eye box is located on the side of the optical lens group away from the display device and there is a preset distance between the eye box and the optical lens group, and the front light emitting direction of the display device is projected into the eye box.

7. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a strabismus detection program, and when the strabismus detection program is executed by a processor, the steps of the strabismus detection method implemented by a computer program according to any one of claims 1 to 4 are implemented.

Images