Strabismus detection method, device and system based on display equipment, equipment and medium

Abstract

The invention discloses a strabismus detection method, device, system and equipment based on display equipment and a medium, and the method comprises the steps: calibrating a detection position of a user based on a reference marker worn by the user, and determining that the user is located at a target detection position; in the detection process, in response to a detection control instruction, the display device is controlled to display a target image corresponding to the uncovered eye, and the target image is matched with the covering unit to form a covering effect on the covered eye and display a target sighting mark visible to the uncovered eye; controlling an image acquisition device to acquire a change condition of converting the uncovered eye from the covered state to the uncovered state; and based on the change condition that the uncovered eye is converted from the covered state to the uncovered state, determining the deflection data corresponding to the uncovered eye, thereby providing an implementation condition which can be widely applied for strabismus screening, reducing the difficulty of strabismus detection, and improving the accuracy and reliability of strabismus detection under the guidance of a non-physician.

Description

Technical Field

[0001] The present disclosure generally relates to the field of image processing technology, and in particular to a method, device, system, equipment and medium for detecting strabismus based on a display device. Background Art

[0002] In addition to myopia, strabismus is also a vision health problem that cannot be ignored. Strabismus not only affects appearance, but may also cause abnormal binocular vision function, such as amblyopia, visual fatigue, and stereoscopic vision. Early strabismus screening can not only help to timely detect and correct eye position problems, but also avoid the long-term effects of strabismus on vision development, such as amblyopia and other problems. Especially in the prevention and control of myopia, strabismus screening plays an important auxiliary role. Through strabismus screening, doctors can more comprehensively evaluate the eye health of users, especially teenagers, and develop more appropriate treatment or intervention measures to help teenagers maintain good vision health.

[0003] The existing strabismus screening methods include cover examination, strabismus angle measurement, corneal fluorescence method, corneal light reflection method based on ophthalmometer, etc. However, these methods need to be examined by professional doctors in hospitals, and at a time when professional doctors are in short supply, they cannot meet the needs of monitoring and screening of strabismus in adolescents. Summary of the invention

[0004] In view of the above-mentioned defects or deficiencies in the prior art, it is desired to provide a strabismus detection method, apparatus, system, device and medium based on a display device, so as to provide widely applicable implementation conditions for strabismus screening, reduce the difficulty of strabismus detection, and improve the accuracy and reliability of strabismus detection under non-physician guidance.

[0005] In a first aspect, an embodiment of the present application provides a method for detecting squint based on a display device, comprising:

[0006] Calibrate the detection position of the user based on a reference marker worn by the user to determine that the user is in a target detection position;

[0007] During the detection process, in response to the detection control instruction, the display device is controlled to display the target image corresponding to the uncovered eye, the target image cooperates with the covering unit to form a covering effect on the covered eye and displays the target sight mark visible to the uncovered eye; the image acquisition device is controlled to acquire the change of the uncovered eye from the covered state to the uncovered state;

[0008] Based on the change of the uncovered eye from the covered state to the uncovered state, the deflection data corresponding to the uncovered eye is determined.

[0009] In some embodiments, it also includes:

[0010] During the detection process, detection control instructions corresponding to the two eyes are generated in sequence at preset time intervals, so that the two eyes are detected as the uncovered eyes in sequence; and

[0011] Before determining the skew data corresponding to the uncovered eye based on the change, it is determined whether the detection times of any of the uncovered eyes reaches a preset detection times, and based on the change of the preset detection times, the skew data corresponding to the uncovered eye is determined.

[0012] In some embodiments, before the detection, the method further comprises:

[0013] For any of the covered eyes, the target image is adjusted by receiving an adjustment instruction input by the user, so that the target image cooperates with the covering unit to form a covering effect on the covered eye.

[0014] In some embodiments, the process of sequentially generating detection control instructions corresponding to both eyes respectively further includes:

[0015] An intermediate detection control instruction for removing the covered eye is generated.

[0016] In some embodiments, it also includes:

[0017] For any of the covered eyes, receiving an adjustment instruction input by the user to adjust the target image so that the target image cooperates with the covering unit to form a covering effect on the covered eye; and

[0018] Based on the target image corresponding to the two eyes, a binocularly visible target image corresponding to the intermediate detection control instruction is generated.

[0019] In some embodiments, the controlling image acquisition device to acquire the change of the uncovered eye from the covered state to the uncovered state includes:

[0020] Controlling the image acquisition device to acquire a first eye position of the uncovered eye in the covered state; wherein, when the covering unit is a semi-transparent covering, the first eye position is the position of the uncovered eye under the covering unit, and when the covering unit is an opaque covering, the first eye position is the instantaneous position of the uncovered eye when it is uncovered;

[0021] as well as

[0022] Control the image acquisition device to acquire a second eye position after the uncovered eye is in an uncovered state after the uncovered eye is uncovered;

[0023] The first eye position and the second eye position are iris positions of the uncovered eye, and the first eye position and the second eye position are iris positions corrected based on the head position of the user during the detection process.

[0024] In some embodiments, it also includes:

[0025] During the detection process, determining that the user's head is abnormal based on the change of the reference marker, and issuing an alarm for the abnormal behavior of the user's head;

[0026] as well as

[0027] Performing pixel-distance calibration on the captured image of the image capture device based on the size of the reference marker to determine a pixel-actual distance mapping relationship between pixels in the captured image and actual distances;

[0028] Determine a pixel distance between the first eye position and the second eye position in the acquired image;

[0029] Based on the pixel distance and the pixel-actual distance mapping relationship, the actual displacement of the user's iris is determined.

[0030] In some embodiments, the first eye position and the second eye position are the iris center positions of the uncovered eye.

[0031] In a second aspect, an embodiment of the present application provides a squint detection device based on a display device, comprising:

[0032] A calibration module, used to calibrate the detection position of the user based on a reference marker worn by the user, and determine that the user is in a target detection position;

[0033] The detection module is used to control the display device to display the target image corresponding to the uncovered eye in response to the detection control instruction during the detection process, wherein the target image cooperates with the covering unit to form a covering effect on the covered eye and displays the target sight mark visible to the uncovered eye; and control the image acquisition device to acquire the change of the uncovered eye from the covered state to the uncovered state;

[0034] The analysis module is used to determine the deflection data corresponding to the uncovered eye based on the change of the uncovered eye from the covered state to the uncovered state.

[0035] In a third aspect, an embodiment of the present application provides a squint detection system based on a display device, comprising:

[0036] A reference marker unit for correcting the detection position of the user;

[0037] A covering unit, used to cooperate with the target image displayed by the display device to form a covering effect on the covered eye during the detection process;

[0038] A display device, configured to display a target image corresponding to the uncovered eye in response to a detection control instruction, wherein the target image cooperates with the covering unit to form a covering effect on the covered eye and displays a target sight mark visible to the uncovered eye;

[0039] An image acquisition device, for acquiring, in response to the detection control instruction, a change in the uncovered eye from a covered state to an uncovered state;

[0040] The server is used to control the display device to display the target image in response to the detection control instruction, control the image acquisition device to acquire changes in the uncovered eye, and determine the skew data corresponding to the uncovered eye based on the changes in the uncovered eye from a covered state to an uncovered state.

[0041] In a fourth aspect, an embodiment of the present application provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method described in the embodiment of the present application when executing the program.

[0042] In a fifth aspect, an embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described in the embodiment of the present application.

[0043] In a sixth aspect, an embodiment of the present application provides a computer program product, including a computer program, characterized in that when the computer program is executed by a processor, it implements the method described in the embodiment of the present application.

[0044] The strabismus detection method, device, system, equipment and medium based on the display device provided in the embodiments of the present application first calibrate the target detection position of the user by referring to the marker to ensure the reliability of the strabismus detection; then, during the detection process, the display device is controlled to display the target image to realize the covering of the covered eye and the vision detection of the uncovered eye, thereby ensuring the effective implementation of the strabismus detection; finally, the deflection data is determined by using the change in the state of the covered eye collected by the image acquisition device, thereby effectively achieving the purpose of strabismus detection without the participation of professional medical personnel, and at the same time, the analysis results of the data collected by the image acquisition device are used to improve the accuracy of the deflection data analysis, thereby providing widely applicable implementation conditions for strabismus screening, reducing the difficulty of strabismus detection, and improving the accuracy and reliability of strabismus detection under non-physician guidance.

[0045] Additional aspects and advantages of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Other features, objects and advantages of the present application will become more apparent by reading the detailed description of non-limiting embodiments made with reference to the following drawings:

[0047] Figure 1 The following is an architecture diagram of a squint detection system based on a display device provided in an embodiment of the present application;

[0048] Figure 2 A schematic diagram of a process of a squint detection method based on a display device provided in an embodiment of the present application is shown;

[0049] Figure 3 The control flow of the alternating cover test provided by an embodiment of the present application is shown;

[0050] Figure 4 The control flow of the cover-uncover test provided by an embodiment of the present application is shown;

[0051] Figure 5 A schematic diagram of the structure of a squint detection device based on a display device provided in one embodiment of the present application is shown;

[0052] Figure 6 A schematic diagram of the structure of a computer system of an electronic device or server suitable for implementing an embodiment of the present application is shown. DETAILED DESCRIPTION

[0053] The present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the relevant invention, rather than to limit the invention. It is also necessary to explain that, for ease of description, only the parts related to the invention are shown in the accompanying drawings.

[0054] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0055] For the specific implementation environment of the squint detection method based on display device proposed in this application, please refer to Figure 1 . Figure 1 The diagram shows an architecture of a squint detection system based on a display device provided in an embodiment of the present application.

[0056] like Figure 1 As shown, the display device-based strabismus detection system includes: a reference marker unit 101, a covering unit 102, a display device 103, an image acquisition device 104 and a server 106.

[0057] The reference marker unit 101 is a reference calibration object worn by the user during the strabismus detection process, and is used by the server 106 to determine the detection distance between the user and the display device 103 and the abnormal head behavior of the user during the detection process based on the reference calibration object. The reference calibration object can be a wearable object. In the embodiment of the present application, the reference calibration object is set on a Figure 1 The clamping member or the eyeglass frame shown in the figure. The clamping member can be used to provide a wearable reference calibration object for users wearing glasses, and the eyeglass frame can provide a wearable reference calibration object for users who do not wear glasses.

[0058] The covering unit 102 is a covering, which is used to provide the user with a visual effect that one eye cannot see during the test. In some embodiments, the covering unit 102 can be an opaque covering, such as a wooden board, cardboard, etc. In the embodiment of the present application, the covering unit 102 is preferably a colored lens coated with a narrow transparent filter film, which is used to cooperate with the target image provided by the display device to form a covering effect on the covered eye, so that the image acquisition device can collect the changes of the covered eye through the lens, thereby improving the reliability and accuracy of the deflection data.

[0059] Optionally, the reference marker unit 101 and the covering unit 102 may be integrally provided, that is, the reference calibration object module 101 may be provided on the covering object module 102, and the covering object module 102 may also be provided on the reference calibration object module 101, for example Figure 1 The reference marker is set at the bridge of the eyeglass frame; the reference marker unit 101 and the covering unit 102 can also be set separately, for example Figure 1 The middle clamping member and the independent lens are respectively the reference marker unit 101 and the covering unit 102. Therefore, the present application does not make any specific limitation to this.

[0060] The display device 103 is used to display the target image according to the control instruction of the server 106. The display device 103 can be a TV, an independent display, a desktop computer, a laptop computer, a smart phone, a tablet computer, a smart watch, etc., but is not limited thereto.

[0061] The image acquisition device 104 is used to acquire a user's facial image to determine the distance and direction between the user and the display device 103, and to acquire an eye image of the user to determine the deflection data corresponding to the user's uncovered eye. The image acquisition device 104 is usually a camera provided on the display device 103, or may be an independently provided image acquisition device 104, which is not specifically limited in this application.

[0062] The server 105 is connected to the display device 103 and the image acquisition device 104 respectively, and is used to control the display device to display the target image in response to the detection control instruction, control the image acquisition device to collect the changes of the uncovered eye, and determine the deflection data corresponding to the uncovered eye based on the changes.

[0063] Server 105 can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers. It can also be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, as well as big data and artificial intelligence platforms.

[0064] The display device 103 and the image acquisition device 104 are directly or indirectly connected to the server 105 by wired or wireless communication. Optionally, the wireless network or wired network uses standard communication technology and / or protocol. The network is usually the Internet, and can also be any network, including but not limited to a local area network (LAN), a metropolitan area network (MAN), a wide area network (WAN), a mobile, wired or wireless network, a private network or any combination of a virtual private network.

[0065] In a preferred embodiment, the strabismus detection system based on the display device may also include, for example, a voice device and a control instruction input device, etc. The voice device is used to output detection control instructions, that is, to remind the user to switch the covered eye, the detection status and other information by voice, and the control instruction input device is used to input adjustment instructions for the target image and the target sight mark, that is, to receive the user's adjustment instructions for the target image and the target sight mark through the control instruction input device, including but not limited to adjustment instructions for brightness, color, transparency, etc.

[0066] It should also be understood that the control instruction input device can also be integrated into the voice device, that is, the voice device can receive the user's voice signal in addition to broadcasting voice to the outside. The server 105 obtains the user's input adjustment instructions for the target image and target visual mark by performing semantic recognition on the voice signal, and then adjusts the target image and target visual mark based on the adjustment instructions.

[0067] It should be understood that those skilled in the art may use other methods to implement the above functions, such as replacing the voice device with letters, using a remote control as a control command input device, etc., and this application does not make specific limitations.

[0068] The squint detection method based on a display device proposed in the present application can be implemented by a squint detection device based on a display device, and the squint detection device based on a display device can be installed on a terminal device or a server.

[0069] In order to further illustrate the technical solution provided by the embodiment of the present application, this is described in detail below in conjunction with the accompanying drawings and specific implementation methods. Although the embodiment of the present application provides the method operation instruction steps shown in the following embodiments or drawings, more or less operation instruction steps may be included in the method based on routine or no creative labor. In the steps where there is no necessary causal relationship logically, the execution order of these steps is not limited to the execution order provided by the embodiment of the present application. The method may be executed in the order of the method shown in the embodiment or drawings or in parallel during the actual processing process or when the device is executed.

[0070] Please refer to Figure 2 , Figure 2 FIG. 2 is a flow chart of a method for detecting a squint based on a display device provided in an embodiment of the present application. Figure 2 As shown, the method includes:

[0071] Step 201 , calibrating the detection position of the user based on the reference mark worn by the user to determine whether the user is in the target detection position.

[0072] It should be noted that in the current medical field, strabismus detection needs to be performed at a specific detection distance, such as close detection and long-distance detection, where close detection is 33 cm or 40 cm, and long-distance detection is 5 m and above. In the embodiment of the present application, strabismus detection is implemented using a display device, such as a home TV, etc. Therefore, due to environmental factors, 33 cm close detection is preferred.

[0073] Specifically, the server stores the size S of the reference marker and the calibration record L (L = 33 cm) collected by the image acquisition device when the reference marker is at the target detection position. When the user wears the reference marker near the detection position, the server can control the image acquisition device to collect the real-time size S' of the reference marker. Then the real-time detection distance L' of the user is determined by calculation, that is:

[0074]

[0075] If L'=L, it means that the user is in the target detection position and strabismus detection can be performed. If L'≠L, it means that the user is not in the target detection position. At this time, it is necessary to further issue a position adjustment reminder to the user based on the difference between L' and L, so that the user can adjust to the target detection position for strabismus detection.

[0076] Step 202, during the detection process, in response to the detection control instruction, the display device is controlled to display the target image corresponding to the uncovered eye, the target image cooperates with the covering unit to form a covering effect on the covered eye and displays the target sight mark visible to the uncovered eye; the image acquisition device is controlled to acquire the change of the uncovered eye from the covered state to the uncovered state.

[0077] It should be noted that strabismus detection is to detect the position deviation of the eyes in the non-seeing state and the seeing state. In the embodiment of the present application, the covering of the covered eye is achieved through the cooperation between the display device and the covering unit, so as to achieve the effect of removing the covered eye when the alternating covering is performed later, so that the covered eye is converted from the covered state to the uncovered state, that is, the covered eye is converted from the non-seeing state to the seeing state, so that the image acquisition device can capture the eye movement during the conversion of the covered eye from the covered state to the uncovered state, that is, whether the covered eye has a change process from the non-seeing position to the seeing position.

[0078] Step 203, based on the change of the uncovered eye from the covered state to the uncovered state, determine the deflection data corresponding to the uncovered eye.

[0079] That is to say, the server further calculates and obtains the deflection data corresponding to the unmasked eye through a preset analysis algorithm according to the changes collected by the image acquisition device, including but not limited to the presence or absence of deflection, the direction and type of deflection, etc.

[0080] For example, the deflection data can be obtained by analyzing the eye position change data according to the eye position change when the covered eye is converted from the covered state to the uncovered state. Preferably, the eye position change can be the change in the iris position of the uncovered eye of the user.

[0081] In a specific embodiment, the image acquisition device is controlled to acquire a first eye position of the uncovered eye in a covered state, and the image acquisition device is controlled to acquire a second eye position of the uncovered eye after the uncovered eye is in an uncovered state through an uncovering operation.

[0082] The first eye position and the second eye position are the iris positions of the uncovered eye.

[0083] It should be noted that the covering unit can be a translucent covering or an opaque covering. When the covering unit is a translucent covering, the first eye position is the position of the uncovered eye under the covering unit, that is, the image acquisition device acquires the first eye position of the uncovered eye through the covering unit; when the covering unit is an opaque covering, the first eye position is the instantaneous position of the uncovered eye when it is uncovered, that is, the instantaneous position of the uncovered eye exposed from under the covering unit.

[0084] Among them, the covering unit can be a colored translucent covering, a frosted translucent covering, etc., and this application does not make specific limitations.

[0085] It should be understood that the first eye position and the second eye position may be iris positions corrected based on the head position of the user during the detection process.

[0086] Preferably, the first eye position and the second eye position are the iris center positions of the uncovered eye.

[0087] Therefore, the strabismus detection method based on the display device provided in the embodiment of the present application first calibrates the target detection position of the user by referring to the marker to ensure the reliability of the strabismus detection; then, during the detection process, the display device is controlled to display the target image to realize the covering of the covered eye and the vision detection of the uncovered eye, thereby ensuring the effective implementation of the strabismus detection; finally, the deviation data is determined by using the change in the state of the covered eye collected by the image acquisition device, thereby effectively achieving the purpose of strabismus detection without the participation of professional medical personnel, and at the same time, the accuracy of the deviation data analysis is improved through the analysis results of the data collected by the image acquisition device, thereby providing widely applicable implementation conditions for strabismus screening, reducing the difficulty of strabismus detection, and improving the accuracy and reliability of strabismus detection under non-physician guidance.

[0088] In a feasible embodiment, during the detection process, detection instructions corresponding to both eyes are generated in sequence at preset time intervals, so that both eyes are detected as uncovered eyes in sequence.

[0089] Furthermore, before determining the skew data corresponding to the uncovered eye based on the change, it is determined whether the detection times of any uncovered eye reaches the detection times, and based on the change of the preset detection times, the skew data corresponding to the uncovered eye is determined.

[0090] It should be noted that strabismus includes various types such as manifest strabismus, latent strabismus, alternating strabismus and constant strabismus. Some types of strabismus are occasional or intermittent. Therefore, it is necessary to perform multiple tests on both eyes separately to achieve the effect of stimulating detection of occasional and intermittent strabismus, thereby further improving the reliability of strabismus detection, and trying to comprehensively screen the possibility of strabismus of users in one test to improve the accuracy of strabismus detection.

[0091] Based on this, the present application further proposes that it is necessary to determine when the number of detections of the uncovered eye reaches a preset number of detections before the deflection data of the uncovered eye can be analyzed based on changes in the preset number of detections.

[0092] In a specific embodiment, the preset time interval is 2s. Figure 3As shown, the detection cycle is: after the user reaches the target detection position, the user is given a "both eyes look at the sight mark" instruction based on the detection control instruction to remind the user to enter the detection state, and then a "please cover your right eye" detection control instruction is given to the user, and the display device is controlled to display the target image corresponding to the left eye, that is, the left eye can see the target sight mark and the right eye cannot see the sight mark under the cooperation of the covering unit and the target image. It should also be noted that at this time, if the user adopts Figure 1 The reference marking unit and the cover unit are separated in the middle. The user can use the cover unit to cover the right eye to achieve the matching effect of the cover unit and the target image. Figure 1 The server counts for 2 seconds, then generates a detection control instruction for the right eye, that is, sends a detection control instruction "Please cover your left eye" to the user, and controls the display device to display the target image corresponding to the right eye, that is, the right eye can see the target sight mark and the left eye cannot see the sight mark under the cooperation of the covering unit and the target image. Similar to covering the right eye, at this time, if the user uses Figure 1 The reference marking unit and the cover unit are separated in the middle. The user can use the cover unit to cover the left eye to achieve the matching effect of the cover unit and the target image. Figure 1 The server counts for 2 seconds and then generates a detection control instruction for the left eye, that is, sends a detection control instruction "Please cover your left eye" to the user, and so on, until the detection of the left eye and the right eye reaches the preset detection times.

[0093] It should be understood that in the above process, the image acquisition device can continuously acquire changes in the user's eyes, and can also detect changes in the covered eye during the transition from the covered state to the uncovered state according to the detection control instruction. Preferably, in order to prevent the different delays in the user's response to the detection control instruction from causing unreliable data acquisition, the embodiment of the present application uses the image acquisition device to continuously acquire changes in the user's eyes during the entire detection process, so as to improve the accuracy and reliability of the final strabismus detection.

[0094] It should be noted that in order to improve the effective monitoring of complex strabismus such as alternating strabismus detection, the embodiment of the present application regards the conversion of any covered eye from an uncovered state to a covered state and then to an uncovered state as a detection cycle of the eye, that is, "cover the right eye-cover the left eye-cover the right eye" is a detection cycle corresponding to the right eye, recorded as one detection of the right eye. Similarly, "cover the left eye-cover the right eye-cover the left eye" is a detection cycle corresponding to the left eye, recorded as one detection of the left eye. When the left eye and the right eye reach the preset number of detections respectively, the deflection data of the left eye and the right eye can be analyzed based on the collected changes.

[0095] It should be noted that the above-mentioned left-right alternating cover inspection method can usually only determine whether the user has an upright eye, manifest strabismus or latent strabismus, and the direction of strabismus, but cannot distinguish latent strabismus or manifest strabismus, alternating strabismus or monocular constant strabismus. Based on this, the present application further proposes a detection strategy that cooperates with the "cover-uncover" method.

[0096] Specifically, between the detection control instructions corresponding to the two eyes generated in sequence, an intermediate detection control instruction for uncovering the covered eye is further generated.

[0097] For example, Figure 4 As shown, the order of the detection control instructions generated by the server is: gaze at the sight mark with both eyes - cover the right eye - uncover the right eye - cover the left eye - uncover the left eye - cover the right eye - uncover the right eye... That is to say, between the alternating covering of the right eye and the left eye, there is a process of uncovering the currently covered eye but not covering the uncovered eye. In this process, both the left and right eyes are in a state of being unobstructed and able to see the sight mark.

[0098] Therefore, the embodiments of the present application can provide users with two strabismus detection strategies: alternating covering and covering-uncovering, provide support for reliable, comprehensive and accurate strabismus screening, and meet a wide range of strabismus detection needs.

[0099] On the other hand, the present application utilizes a covering unit and a display device to cover the user's eyes, but since the visual ability of each user's eyes is different, it is necessary to perform corresponding covering calibration for each user before detection.

[0100] Specifically, for any covered eye, an adjustment instruction input by a user is received to adjust the target image so that the target image cooperates with the covering unit to form a covering effect on the covered eye.

[0101] For example, Figure 1 Take the case where the reference marker and the covering unit are separated as an example. When the separated covering unit is used to cover the left eye, the preset target image is displayed to the user. Then the user adjusts the color and transparency of the display device until the left eye is in a state where it cannot see anything, that is, the visual mark on the display device cannot be seen through the covering. Then the covering unit is used to cover the right eye, and the color and transparency of the display device are also adjusted until the right eye is in a state where it cannot see anything. Then switch to covering the left eye for correction and adjustment, and then switch to covering the right eye for correction and adjustment, until the covering process switches between the left and right eyes, and both can achieve a state where the covered eye cannot see anything.

[0102] by Figure 1Taking the example of the reference marker and the covering unit being integrated as one, when it is necessary to cover the left eye, the preset target image corresponding to the right eye is displayed to the user, and then the user adjusts the color and transparency of the display device until the left eye is in a state where it cannot see objects and the right eye can see the target sight mark, that is, the left eye cannot see the sight mark on the display device through the covering but the right eye can see it. When it is necessary to cover the right eye, the preset target image corresponding to the left eye is displayed to the user, and then the user adjusts the color and transparency of the display device until the right eye is in a state where it cannot see objects and the left eye can see the target sight mark, that is, the right eye cannot see the sight mark on the display device through the covering but the left eye can see it. Then switch to covering the left eye for correction adjustment, and then switch to covering the right eye for correction adjustment, until the covering process switches between the left and right eyes, and both can achieve a state where the covered eye is in a state where it cannot see objects and the uncovered eye can see the sight mark.

[0103] In a specific embodiment, Figure 1 Taking the central reference marker and the covering unit as an example, when the colors of the covering lenses are red and green respectively, after adjustment, the target image corresponding to the red covering lens is a central visual mark with a yellow background and a red background, with a transparency between 30% and 40%, and the target image corresponding to the green lens is a central visual mark with a green background and a red background, with a transparency between 30% and 40%.

[0104] It should be noted that, during the cover-uncover test, in order to ensure that in the intermediate detection control instruction stage, a binocular visible target image corresponding to the intermediate detection control instruction is further generated based on the target image corresponding to both eyes.

[0105] That is, during the cover-uncover test, when the covered eye is uncovered, the server controls the display device to adjust the target image corresponding to the uncovered eye to a binocularly visible target image, which can be obtained by fusing the two target images obtained in the above-mentioned monocular cover process. Figure 1 Taking the case where the central reference marker is integrated with the covering unit as an example, the binocularly visible target image is a target image with a central sight mark and a black background and a red background. When the display device displays the sight mark, both eyes in the uncovered state can see the central sight mark.

[0106] In a feasible embodiment, the present application can also set a corresponding account for each screening user, so that the corresponding target image can be obtained according to the account information query when the user is screened, so that the user does not need to debug the target image every time when performing strabismus screening, thereby greatly saving the user's time for strabismus screening, improving the efficiency of strabismus screening, and enhancing the user experience.

[0107] Furthermore, the present application further uses an image acquisition device to detect reference markers during the detection process to detect and warn the user's head movement during the detection process, thereby effectively avoiding problems such as eye movement information errors caused by the user's head rotation, thereby improving the accuracy of strabismus detection. Among them, head movement includes but is not limited to head misalignment, distance not meeting test requirements, etc.

[0108] Preferably, during the detection process, the displacement of the user's head movement can also be analyzed, and based on the displacement of the user's head when the uncovered eye is converted from a covered state to an uncovered state, the first eye position and the second eye position collected in the process can be positionally corrected, i.e., the actual iris displacement of the user's uncovered eye during the process of converting from a covered state to an uncovered state can be determined.

[0109] In a specific embodiment, pixel-distance calibration is performed on the captured image of the image capture device based on a reference marker to determine a pixel-actual distance mapping relationship between pixels in the captured image and actual distances; the pixel distance between the first eye position and the second eye position in the captured image is determined, and based on the pixel distance and the pixel-actual distance mapping relationship, the actual displacement of the user's iris is determined.

[0110] It should be understood that the reference marker is a known object set in advance, which may be Figure 1 The white block shown may also be in other forms, which is not specifically limited in this application.

[0111] Since various attribute information of the reference marker is known, such as length, width, area, etc., after the image acquisition device acquires the captured image, the distance of the user's detection position corresponding to one pixel in the captured image can be obtained by, for example, ratio calculation, etc. For example, if it is known that the length of the reference marker is 5 cm and occupies 5 pixels in the horizontal direction in the image acquisition device, it can be determined that one pixel in the captured image corresponds to 1 cm of the user's detection position.

[0112] At this time, after obtaining the first eye position and the second eye position, the pixel distance between the first eye position and the second eye position in the captured image can be calculated, and then the actual displacement of the user's iris can be determined based on the pixel distance and the pixel-actual distance mapping relationship. For example, after obtaining the first eye position and the second eye position of the user, the pixel distance between the first eye position and the second eye position in the captured image is calculated to be 1 pixel, and the actual displacement of the user's iris can be determined to be 1 cm.

[0113] Therefore, the present application can use reference markers to directly calculate and obtain the actual displacement of the user's iris, providing a more accurate analysis value for subsequent deviation analysis compared to the relative displacement of the iris, and effectively avoiding the problem that the relative displacement of the iris cannot effectively distinguish between latent strabismus or manifest strabismus, alternating strabismus or monocular constant strabismus, etc.

[0114] In some embodiments, head abnormality also includes the user blinking during the detection process. Specifically, since the user's eyeballs are blocked during blinking, the image acquisition device cannot effectively capture the changes in the user's eyeball state, so when the user blinks, an alarm is also issued to the user, reminding the user to repeat the previous action to complete the complete eyeball rotation change information.

[0115] In a feasible embodiment, the server of the present application determines the deflection data of the user's eyes by analyzing the displacement trajectory of the user's iris in the image captured by the image acquisition device. Specifically, after the image acquisition device determines that the user is in the target detection position using the reference marker, the user's eye image is acquired to obtain the iris position of the user when the user is in the target detection position, and then the rotation of the user's eyes during the detection process, that is, the change, is determined by the iris tracking technology, and the deflection data of the user's eyes is analyzed.

[0116] It should be noted that although the operations of the method of the present invention are described in a particular order in the drawings, this does not require or imply that the operations must be performed in this particular order or that all illustrated operations must be performed to achieve desired results.

[0117] Figure 5 A schematic structural diagram of a squint detection device based on a display device provided in one embodiment of the present application is shown.

[0118] like Figure 5 As shown, the squint detection device 10 based on the display device includes:

[0119] A calibration module 11, configured to calibrate the detection position of the user based on a reference marker worn by the user, and determine that the user is in a target detection position;

[0120] The detection module 12 is used to control the display device to display the target image corresponding to the uncovered eye in response to the detection control instruction during the detection process, wherein the target image cooperates with the covering unit to form a covering effect on the covered eye and displays the target sight mark visible to the uncovered eye; and control the image acquisition device to acquire the change of the uncovered eye from the covered state to the uncovered state;

[0121] The analysis module 13 is used to determine the deflection data corresponding to the uncovered eye based on the change condition.

[0122] In some embodiments, the detection module 12 is further configured to:

[0123] During the detection process, detection control instructions corresponding to the two eyes are generated in sequence at preset time intervals, so that the two eyes are detected as the uncovered eyes in sequence;

[0124] The analysis module 13 is further used for:

[0125] Before determining the skew data corresponding to the uncovered eye based on the change, it is determined whether the detection times of any of the uncovered eyes reaches a preset detection times, and based on the change of the preset detection times, the skew data corresponding to the uncovered eye is determined.

[0126] In some embodiments, the detection module 12 is further configured to:

[0127] For any of the covered eyes, the target image is adjusted by receiving an adjustment instruction input by the user, so that the target image cooperates with the covering unit to form a covering effect on the covered eye.

[0128] In some embodiments, the detection module 12 is further configured to:

[0129] In the process of sequentially generating the detection control instructions corresponding to the two eyes respectively, an intermediate detection control instruction for uncovering the covered eye is generated.

[0130] In some embodiments, the detection module 12 is further configured to:

[0131] For any of the covered eyes, receiving an adjustment instruction input by the user to adjust the target image so that the target image cooperates with the covering unit to form a covering effect on the covered eye; and

[0132] Based on the target image corresponding to the two eyes, a binocularly visible target image corresponding to the intermediate detection control instruction is generated.

[0133] In some embodiments, the detection module 12 is further configured to:

[0134] Controlling the image acquisition device to acquire a first eye position of the uncovered eye in the covered state; wherein, when the covering unit is a semi-transparent covering, the first eye position is the position of the uncovered eye under the covering unit, and when the covering unit is an opaque covering, the first eye position is the instantaneous position of the uncovered eye when it is uncovered;

[0135] as well as

[0136] Control the image acquisition device to acquire a second eye position after the uncovered eye is in an uncovered state after the uncovered eye is uncovered;

[0137] The first eye position and the second eye position are iris positions of the uncovered eye, and the first eye position and the second eye position are iris positions corrected based on the head position of the user during the detection process.

[0138] In some embodiments, the detection module 12 is further configured to:

[0139] During the detection process, determining that the user's head is abnormal based on the change of the reference marker, and issuing an alarm for the abnormal behavior of the user's head;

[0140] as well as

[0141] Performing pixel-distance calibration on the captured image of the image capture device based on the reference marker to determine a pixel-actual distance mapping relationship between pixels in the captured image and actual distances;

[0142] Determine a pixel distance between the first eye position and the second eye position in the acquired image;

[0143] Based on the pixel distance and the pixel-actual distance mapping relationship, the actual displacement of the user's iris is determined.

[0144] It should be understood that the units or modules described in the squint detection device 10 of the display device are similar to those described in the reference Figure 2 The various steps in the described method correspond to each other. Therefore, the operations and features described above for the method are also applicable to the squint detection device 10 based on the display device and the units contained therein, and will not be repeated here. The squint detection device 10 based on the display device can be pre-implemented in the browser or other security applications of the electronic device, and can also be loaded into the browser or its security application of the electronic device by downloading or the like. The corresponding units in the squint detection device 10 based on the display device can cooperate with the units in the electronic device to implement the solution of the embodiment of the present application.

[0145] For the several modules or units mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. On the contrary, the features and functions of one module or unit described above can be further divided into multiple modules or units to be embodied.

[0146] Reference below Figure 6 , Figure 6 A schematic diagram of the structure of a computer system of an electronic device or server suitable for implementing an embodiment of the present application is shown.

[0147] like Figure 6As shown, the computer system includes a central processing unit (CPU) 601, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 602 or the program loaded from the storage part 608 to the random access memory (RAM) 603. In the RAM 603, various programs and data required for the operation instructions of the system are also stored. The CPU 601, the ROM 602 and the RAM 603 are connected to each other through a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0148] The following components are connected to the I / O interface 605: an input section 606 including a keyboard, a mouse, etc.; an output section 607 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 608 including a hard disk, etc.; and a communication section 609 including a network interface card such as a LAN card, a modem, etc. The communication section 609 performs communication processing via a network such as the Internet. A drive 610 is also connected to the I / O interface 605 as needed. A removable medium 611, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 610 as needed, so that a computer program read therefrom is installed into the storage section 608 as needed.

[0149] In particular, according to an embodiment of the present application, the above reference flow chart Figure 2 The described process can be implemented as a computer software program. For example, an embodiment of the present application includes a computer program product, which includes a computer program carried on a computer readable medium, and the computer program includes a program code for executing the method shown in the flow chart. In such an embodiment, the computer program includes a program code for executing the method shown in the flow chart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 609, and / or installed from the removable medium 611. When the computer program is executed by the central processing unit (CPU) 601, the above-mentioned functions defined in the system of the present application are executed.

[0150] It should be noted that the computer-readable medium shown in the present application may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In the present application, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium such as a computer-readable storage medium that can send, propagate or transmit a program for use by or in conjunction with an instruction execution system, device or device. The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0151] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, functions and operating instructions of the system, method and computer program product according to various embodiments of the present application. In this regard, each box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the aforementioned module, program segment or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the box can also occur in a sequence different from that marked in the accompanying drawings. For example, the boxes represented by two connections can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operating instruction, or can be implemented with a combination of dedicated hardware and computer instructions.

[0152] The units or modules involved in the embodiments described in the present application may be implemented by software or by hardware. The units or modules described may also be arranged in a processor. For example, they may be described as: a processor includes a calibration module, a detection module, and an analysis module. Among them, the names of these units or modules do not constitute a limitation on the units or modules themselves under certain circumstances. For example, the calibration module may also be described as "calibrating the detection position of the user based on the reference markers worn by the user to determine that the user is in the target detection position".

[0153] As another aspect, the present application further provides a computer-readable storage medium, which may be included in the electronic device described in the above embodiment, or may exist independently without being assembled into the electronic device. The above computer-readable storage medium stores one or more programs, and when the above programs are used by one or more processors to execute the squint detection method based on the display device described in the present application.

[0154] The above description is only a preferred embodiment of the present application and an explanation of the technical principles used. Those skilled in the art should understand that the scope of disclosure involved in the present application is not limited to the technical solution formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the aforementioned disclosed concept. For example, the above features are replaced with the technical features with similar functions disclosed in this application (but not limited to) by each other to form a technical solution.

Claims

1. A squint detection method based on a display device, characterized in that: include: Calibrate the detection position of the user based on a reference marker worn by the user to determine that the user is in a target detection position; During the detection process, in response to the detection control instruction, the display device is controlled to display the target image corresponding to the uncovered eye, the target image cooperates with the covering unit to form a covering effect on the covered eye and displays the target sight mark visible to the uncovered eye; the image acquisition device is controlled to acquire the change of the uncovered eye from the covered state to the uncovered state; Based on the change of the uncovered eye from the covered state to the uncovered state, the deflection data corresponding to the uncovered eye is determined.

2. The method for detecting squint based on a display device according to claim 1, characterized in that: Also includes: During the detection process, detection control instructions corresponding to the two eyes are generated in sequence at preset time intervals, so that the two eyes are detected as the uncovered eyes in sequence; as well as Before determining the skew data corresponding to the uncovered eye based on the change, it is determined whether the detection times of any of the uncovered eyes reaches a preset detection times, and based on the change of the preset detection times, the skew data corresponding to the uncovered eye is determined.

3. The method for detecting squint based on a display device according to claim 2, characterized in that: Before testing, it also includes: For any of the covered eyes, the target image is adjusted by receiving an adjustment instruction input by the user, so that the target image cooperates with the covering unit to form a covering effect on the covered eye.

4. The method for detecting strabismus based on a display device according to claim 2, characterized in that: The process of sequentially generating the detection control instructions corresponding to the two eyes respectively further includes: An intermediate detection control instruction for removing the covered eye is generated.

5. The method for detecting strabismus based on a display device according to claim 4, characterized in that: Also includes: For any of the covered eyes, receiving the adjustment instruction input by the user to adjust the target image so that the target image cooperates with the covering unit to form a covering effect on the covered eye; as well as Based on the target image corresponding to the two eyes, a binocularly visible target image corresponding to the intermediate detection control instruction is generated.

6. The method for detecting squint based on a display device according to any one of claims 1 to 5, characterized in that: The controlling image acquisition device to acquire the change of the uncovered eye from the covered state to the uncovered state includes: Controlling the image acquisition device to acquire a first eye position of the uncovered eye in the covered state; wherein, when the covering unit is a semi-transparent covering, the first eye position is the position of the uncovered eye under the covering unit, and when the covering unit is an opaque covering, the first eye position is the instantaneous position of the uncovered eye when it is uncovered; as well as Control the image acquisition device to acquire a second eye position after the uncovered eye is in an uncovered state after the uncovered eye is uncovered; The first eye position and the second eye position are iris positions of the uncovered eye, and the first eye position and the second eye position are iris positions corrected based on the head position of the user during the detection process.

7. The method for detecting strabismus based on a display device according to claim 6, characterized in that: Also includes: During the detection process, determining that the user's head is abnormal based on the change of the reference marker, and issuing an alarm for the abnormal behavior of the user's head; as well as Performing pixel-distance calibration on the captured image of the image capture device based on the reference marker to determine a pixel-actual distance mapping relationship between pixels in the captured image and actual distances; Determine a pixel distance between the first eye position and the second eye position in the acquired image; Based on the pixel distance and the pixel-actual distance mapping relationship, the actual displacement of the user's iris is determined.

8. The method for detecting strabismus based on a display device according to claim 6, characterized in that: The first eye position and the second eye position are iris center positions of the uncovered eye.

9. A squint detection device based on a display device, characterized in that: include: A calibration module, used to calibrate the detection position of the user based on a reference marker worn by the user, and determine that the user is in a target detection position; The detection module is used to control the display device to display the target image corresponding to the uncovered eye in response to the detection control instruction during the detection process, wherein the target image cooperates with the covering unit to form a covering effect on the covered eye and displays the target sight mark visible to the uncovered eye; and control the image acquisition device to acquire the change of the uncovered eye from the covered state to the uncovered state; The analysis module is used to determine the deflection data corresponding to the uncovered eye based on the change of the uncovered eye from the covered state to the uncovered state.

10. A squint detection system based on a display device, characterized in that: include: A reference marker unit for correcting the detection position of the user; A covering unit, used to cooperate with the target image displayed by the display device to form a covering effect on the covered eye during the detection process; A display device, configured to display a target image corresponding to the uncovered eye in response to a detection control instruction, wherein the target image cooperates with the covering unit to form a covering effect on the covered eye and displays a target sight mark visible to the uncovered eye; An image acquisition device, for acquiring, in response to the detection control instruction, a change in the uncovered eye from a covered state to an uncovered state; The server is used to control the display device to display the target image in response to the detection control instruction, control the image acquisition device to acquire changes in the uncovered eye, and determine the skew data corresponding to the uncovered eye based on the changes in the uncovered eye from a covered state to an uncovered state.

Images