Strabismus detection method, apparatus and system based on display device, device, and medium

Abstract

The present application discloses a strabismus detection method, apparatus and system based on a display device, a device, and a medium. The method comprises: calibrating a detection position of a user on the basis of a reference marker worn by the user, and determining that the user is at a target detection position; in the detection process, in response to a detection control instruction, controlling a display device to display a target image corresponding to an uncovered eye, wherein the target image works in conjunction with a covering unit to achieve a covering effect on the covered eye and display a target optotype visible to the uncovered eye, and controlling an image collection apparatus to collect the change of the uncovered eye being converted from a covered state to an uncovered state; and on the basis of the change of the uncovered eye being converted from the covered state to the uncovered state, determining deviation data corresponding to the uncovered eye. Widely applicable implementation conditions are provided for strabismus screening, so that the difficulty of strabismus detection is reduced, thereby improving the accuracy and reliability of strabismus detection under the guidance of a non-physician.

Description

Methods, devices, systems, equipment, and media for strabismus detection based on display devices.

[0001] Cross-references to related applications

[0002] This application claims priority to Chinese Patent Application No. 202510051394.1, filed on January 13, 2025, entitled "Strabismus Detection Method, Device, System, Equipment and Medium Based on Display Device". Technical Field

[0003] This disclosure generally relates to the field of image processing technology, and specifically to a strabismus detection method, apparatus, system, device, and medium based on a display device. Background Technology

[0004] Vision health problems among teenagers in China and Southeast Asian countries are becoming increasingly serious, especially due to the rapid rise in myopia rates, which is impacting the quality of life for many. Academic pressure, widespread use of electronic devices, and reduced outdoor activities are all significant contributing factors to declining vision. Besides myopia, strabismus is another vision health issue that cannot be ignored. Strabismus not only affects appearance but can also lead to binocular vision abnormalities such as amblyopia, eye strain, and impaired stereoscopic vision. Early strabismus screening can help detect and correct eye alignment problems in a timely manner and prevent long-term effects of strabismus on visual development, such as amblyopia. Strabismus screening plays a particularly important supporting role in the prevention and control of myopia. Through strabismus screening, doctors can more comprehensively assess the eye health of users, especially teenagers, and develop more appropriate treatment or intervention measures to help teenagers maintain good vision health.

[0005] Existing strabismus screening methods include occlusion testing, strabismus angle measurement, corneal fluorescein assay, and corneal reflex assay based on an optometer. However, these methods all require examination by a professional physician in a hospital. Given the current shortage of qualified physicians, these methods cannot meet the needs of strabismus monitoring and screening in adolescents. Summary of the Invention

[0006] In view of the above-mentioned defects or deficiencies in the prior art, it is desirable to provide a strabismus detection method, device, system, equipment and medium based on a display device, so as to provide widely applicable implementation conditions for strabismus screening, reduce the difficulty of strabismus detection, and improve the accuracy and reliability of strabismus detection without the guidance of a physician.

[0007] In a first aspect, embodiments of this application provide a strabismus detection method based on a display device, including:

[0008] The user's detection position is calibrated based on the reference marker worn by the user to determine that the user is at the target detection position;

[0009] During the detection process, in response to the detection control command, the display device is controlled to display the target image corresponding to the unmasked eye. The target image and the masking unit work together to form a masking effect on the masked eye and display the target visual object visible to the unmasked eye. The image acquisition device is controlled to acquire the changes in the unmasked eye from the masked state to the unmasked state.

[0010] Based on the changes in the uncovered eye from the covered state to the uncovered state, the skew data corresponding to the uncovered eye is determined.

[0011] In some embodiments, it also includes:

[0012] During the detection process, detection control commands corresponding to each eye are generated sequentially at preset time intervals, so that each eye is sequentially used as the unoccluded eye for detection; and

[0013] Before determining the skew data corresponding to the unmasked eye based on the changes, it is determined whether the number of detections for any unmasked eye has reached a preset number of detections, and the skew data corresponding to the unmasked eye is determined based on the changes in the preset number of detections.

[0014] In some embodiments, prior to detection, the method further includes:

[0015] For any of the occluded eyes, the target image is adjusted by receiving the adjustment command input by the user, so that the target image and the occlusion unit cooperate to form an occlusion effect on the occluded eye.

[0016] In some embodiments, the process of sequentially generating detection control commands corresponding to each eye further includes:

[0017] Generate intermediate detection control instructions for removing the occlusion from the occluded eye.

[0018] In some embodiments, it also includes:

[0019] For any of the occluded eyes, the target image is adjusted according to the adjustment command input by the user, so that the target image cooperates with the occlusion unit to form an occlusion effect on the occluded eye; and

[0020] Based on the target images corresponding to the two eyes, a binocular visual target image corresponding to the intermediate detection control command is generated.

[0021] In some embodiments, the control image acquisition device acquires the changes in the uncovered eye from an occluded state to an unoccluded state, including:

[0022] The image acquisition device is controlled to acquire the first eye position of the unmasked eye in the masked state; wherein, when the masking unit is a semi-transparent masking object, the first eye position is the position of the unmasked eye under the masking unit, and when the masking unit is an opaque masking object, the first eye position is the instantaneous position of the unmasked eye when the masking is removed.

[0023] as well as

[0024] The image acquisition device is controlled to acquire the second eye position after the uncovered eye has been removed and is now in an uncovered state;

[0025] Wherein, the first eye position and the second eye position are the iris positions of the uncovered eye, and the first eye position and the second eye position are the iris positions corrected based on the user's head position during the detection process.

[0026] In some embodiments, it also includes:

[0027] During the detection process, the user's head abnormality is determined based on the changes in the reference marker, and an alarm is issued for the user's abnormal head behavior;

[0028] as well as

[0029] Based on the size of the reference marker, the image acquired by the image acquisition device is pixel-distance calibrated to determine the pixel-actual distance mapping relationship between pixels in the acquired image and actual distances.

[0030] Determine the pixel distance between the first eye position and the second eye position in the acquired image;

[0031] Based on the pixel distance and the pixel-actual distance mapping relationship, the actual displacement of the user's iris is determined.

[0032] In some embodiments, the first eye position and the second eye position are the iris center positions of the uncovered eye.

[0033] Secondly, embodiments of this application provide a strabismus detection device based on a display device, comprising:

[0034] The calibration module is used to calibrate the user's detection position based on a reference marker worn by the user, and to determine that the user is at the target detection position;

[0035] The detection module is used to control the display device to display the target image corresponding to the unmasked eye in response to the detection control command during the detection process. The target image and the masking unit work together to form a masking effect on the masked eye and display the target visual object visible to the unmasked eye. The module also controls the image acquisition device to acquire the changes in the unmasked eye from the masked state to the unmasked state.

[0036] The analysis module is used to determine the skew data corresponding to the uncovered eye based on the changes in the uncovered eye from the occluded state to the unoccluded state.

[0037] Thirdly, embodiments of this application provide a strabismus detection system based on a display device, comprising:

[0038] Reference marker units are used to correct the user's detection position;

[0039] The masking unit is used to cooperate with the target image displayed by the display device during the detection process to form a masking effect on the masked eye;

[0040] A display device is configured to respond to a detection control command to display a target image corresponding to the unmasked eye, wherein the target image and the masking unit cooperate to form a masking effect on the masked eye and to display a target visual object visible to the unmasked eye;

[0041] Image acquisition device, used to acquire the changes in the occluded eye from the occluded state to the unoccluded state in response to the detection control command;

[0042] The server is configured to respond to the detection control command by controlling the display device to display the target image, controlling the image acquisition device to acquire the changes in the unmasked eye, and determining the skew data corresponding to the unmasked eye based on the changes in the unmasked eye from the masked state to the unmasked state.

[0043] Fourthly, embodiments of this application provide an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the method described in the embodiments of this application.

[0044] Fifthly, embodiments of this application provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described in embodiments of this application.

[0045] In a sixth aspect, embodiments of this application provide a computer program product, including a computer program, characterized in that, when the computer program is executed by a processor, it implements the method described in embodiments of this application.

[0046] The strabismus detection method, apparatus, system, device, and medium based on a display device provided in this application first calibrates the target detection position for the user using reference markers to ensure the reliability of strabismus detection. Then, during the detection process, the target image is displayed by controlling the display device to achieve occlusion of the occluded eye and visual detection of the unoccluded eye, ensuring the effective implementation of strabismus detection. Finally, the deviation data is determined by using the changes in the state of the occluded eye collected by the image acquisition device. This effectively achieves the goal of strabismus detection without the participation of professional medical personnel. At the same time, the analysis results of the data collected by the image acquisition device improve the accuracy of deviation data analysis, providing widely applicable implementation conditions for strabismus screening, reducing the difficulty of strabismus detection, and improving the accuracy and reliability of strabismus detection without medical guidance.

[0047] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0048] Other features, objects, and advantages of this application will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings:

[0049] Figure 1 shows the architecture diagram of the strabismus detection system based on a display device provided in an embodiment of this application;

[0050] Figure 2 shows a flowchart of a strabismus detection method based on a display device provided in an embodiment of this application;

[0051] Figure 3 illustrates the control flow of an alternating occlusion test provided in an embodiment of this application;

[0052] Figure 4 illustrates the control flow of a masking-unmasking test provided in an embodiment of this application;

[0053] Figure 5 shows a schematic diagram of the strabismus detection device based on a display device provided in an embodiment of this application;

[0054] Figure 6 shows a schematic diagram of the structure of a computer system suitable for implementing an electronic device or server according to the embodiments of this application. Detailed Implementation

[0055] The present application will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and not intended to limit it. Furthermore, it should be noted that, for ease of description, only the parts relevant to the invention are shown in the accompanying drawings.

[0056] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0057] The specific implementation environment of the strabismus detection method based on a display device proposed in this application is shown in Figure 1. Figure 1 shows the architecture diagram of the strabismus detection system based on a display device provided in the embodiment of this application.

[0058] As shown in Figure 1, the strabismus detection system based on the display device includes: a reference marker unit 101, a covering unit 102, a display device 103, an image acquisition device 104, and a server 106.

[0059] The reference marker unit 101 is a reference calibration object worn by the user during strabismus detection. It is used by the server 106 to determine the detection distance between the user and the display device 103, as well as any abnormal head behavior of the user during the detection process. The reference calibration object can be a wearable object. In this embodiment, the reference calibration object is disposed on a clamp or eyeglass frame as shown in Figure 1. The clamp can provide a wearable reference calibration object for users who wear glasses, while the eyeglass frame can provide a wearable reference calibration object for users who do not wear glasses.

[0060] The masking unit 102 is a masking object used to provide the user with a visual effect that is invisible to one eye during testing. In some embodiments, the masking unit 102 can be an opaque masking object, such as a wooden board or cardboard. In this embodiment, the masking unit 102 is preferably a colored lens coated with a narrow-permeability filter film, used to cooperate with the target image provided by the display device to form a masking effect on the masked eye, facilitating the image acquisition device to collect changes in the masked eye through the lens, and improving the reliability and accuracy of the skew data.

[0061] Optionally, the reference marker unit 101 and the covering unit 102 can be integrated, that is, the reference calibration module 101 can be disposed on the covering module 102, and the covering module 102 can also be disposed on the reference calibration module 101, for example, in Figure 1, the reference marker is disposed on the bridge of the eyeglass frame; the reference marker unit 101 and the covering unit 102 can also be disposed separately, for example, in Figure 1, the clamping member and the independent lens are respectively the reference marker unit 101 and the covering unit 102. Therefore, this application does not specifically limit this.

[0062] Display device 103 is used to display target images according to control instructions from server 106. Display device 103 may be a television, stand-alone monitor, desktop computer, laptop computer, smartphone, tablet computer, smartwatch, etc., but is not limited to these.

[0063] The image acquisition device 104 is used to acquire images of the user's face to determine the distance and direction between the user and the display device 103, and to acquire images of the user's eyes to determine the skew data corresponding to the user removing the eye covering. The image acquisition device 104 is typically a camera mounted on the display device 103, but it can also be a stand-alone image acquisition device; this application does not impose any specific limitations.

[0064] Server 105 is connected to display device 103 and image acquisition device 104 respectively, and is used to respond to detection control commands to control display device to display target image, control image acquisition device to acquire changes in the unmasked eye, and determine the skew data corresponding to the unmasked eye based on the changes.

[0065] Server 105 can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms.

[0066] The display device 103 and image acquisition device 104 are directly or indirectly connected to the server 105 via wired or wireless communication. Optionally, the aforementioned wireless or wired network uses standard communication technologies and / or protocols. The network is typically the Internet, but can also be any network, including but not limited to any combination of Local Area Network (LAN), Metropolitan Area Network (MAN), Wide Area Network (WAN), mobile, wired or wireless network, private network, or virtual private network.

[0067] In a preferred embodiment, the strabismus detection system based on a display device may further include, for example, a voice device and a control command input device. The voice device outputs detection control commands, i.e., it reminds the user via voice to switch the occluded eye, the detection status, and other information. The control command input device receives adjustment commands from the user for the target image and target visual object, including but not limited to adjustments to brightness, color, and transparency.

[0068] It should also be understood that the control command input device can also be integrated into the voice device. That is, in addition to broadcasting voice, the voice device can also receive the user's voice signal. The server 105 obtains the user's adjustment command for the target image and target view by performing semantic recognition on the voice signal, and then adjusts the target image and target view based on the adjustment command.

[0069] It should be understood that those skilled in the art can use other methods to achieve the above functions, such as using letters instead of a voice device, or using a remote control as a control command input device, etc., and this application does not make any specific limitations.

[0070] The strabismus detection method based on a display device proposed in this application can be implemented by a strabismus detection device based on a display device, which can be installed on a terminal device or a server.

[0071] To further illustrate the technical solutions provided in the embodiments of this application, a detailed description is provided below in conjunction with the accompanying drawings and specific implementation methods. Although the embodiments of this application provide method operation instruction steps as shown in the following embodiments or drawings, the method may include more or fewer operation instruction steps based on conventional or non-creative effort. In steps where there is no logically necessary causal relationship, the execution order of these steps is not limited to the execution order provided in the embodiments of this application. In actual processing or when the device executes the method, it may be executed sequentially or in parallel according to the method shown in the embodiments or drawings.

[0072] Please refer to Figure 2, which shows a flowchart of a strabismus detection method based on a display device according to an embodiment of this application. As shown in Figure 2, the method includes:

[0073] Step 201: Based on the reference object worn by the user, calibrate the user's detection position to determine that the user is at the target detection position.

[0074] It should be noted that in the current medical field, strabismus detection needs to be performed at specific detection distances, such as near detection and far detection. Near detection is 33cm or 40cm, and far detection is 5m or more. In this embodiment, strabismus detection is performed using a display device, such as a home television. Therefore, due to environmental factors, a near detection distance of 33cm is preferred.

[0075] Specifically, the server internally stores the size S of the reference marker and the calibration record L (L = 33cm) acquired by the image acquisition device when the reference marker is at the target detection position. When the user wears the reference marker near the detection position, the server can control the image acquisition device to acquire the real-time size S' of the reference marker. Then, the real-time detection distance L' of the user is calculated and determined, i.e.:

[0076] If L' = L, it means the user is in the target detection position and strabismus detection can be performed. If L' = L, it means the user is not in the target detection position. In this case, it is necessary to further remind the user to adjust their position based on the difference between L' and L, so that the user can adjust to the target detection position for strabismus detection.

[0077] Step 202: During the detection process, in response to the detection control command, the display device is controlled to display the target image corresponding to the unmasked eye. The target image and the masking unit work together to form the masking effect on the masked eye and display the target visual object visible to the unmasked eye. The image acquisition device is controlled to acquire the changes in the unmasked eye from the masked state to the unmasked state.

[0078] It should be noted that strabismus detection detects the positional deviation of the eye in a non-visual state and a visual state. In this embodiment, the cooperation between the display device and the occlusion unit achieves the occlusion of the occluded eye, so that the occluded eye can be unoccluded during subsequent alternating occlusion, causing the occluded eye to change from an occluded state to an unoccluded state, or from a non-visual state to a visual state. This allows the image acquisition device to capture the eye movement during the transition from the occluded state to the unoccluded state, i.e., to capture whether the occluded eye has changed from a non-visual position to a visual position.

[0079] Step 203: Based on the changes in the unmasked eye from the occluded state to the unmasked state, determine the skew data corresponding to the unmasked eye.

[0080] In other words, the server further calculates and obtains the deviation data corresponding to the uncovered eye based on the changes collected by the image acquisition device and through a preset analysis algorithm, including but not limited to whether there is deviation, the direction and type of deviation, etc.

[0081] For example, deviation data can be obtained by analyzing the changes in eye position when the covered eye transitions from a covered state to an uncovered state. Preferably, the changes in eye position can be the changes in the position of the iris of the user's uncovered eye.

[0082] In one specific embodiment, the image acquisition device is controlled to acquire the first eye position of the unmasked eye in the masked state, and the image acquisition device is controlled to acquire the second eye position of the unmasked eye after the unmasking operation forms an unmasked state.

[0083] The first eye position and the second eye position refer to the iris positions of the uncovered eye.

[0084] It should be noted that the masking unit can be a semi-transparent mask or an opaque mask. When the masking unit is a semi-transparent mask, the first eye position is the position of the unmasked eye under the masking unit, that is, the image acquisition device acquires the first eye position of the unmasked eye through the masking unit. When the masking unit is an opaque mask, the first eye position is the instantaneous position of the unmasked eye when it is unmasked, that is, the instantaneous position of the unmasked eye when it is exposed from under the masking unit.

[0085] The covering unit can be a colored semi-transparent covering, a frosted semi-transparent covering, etc., and this application does not make specific limitations.

[0086] It should be understood that the first and second eye positions can be the iris positions corrected based on the user's head position during the detection process.

[0087] Preferably, the first and second eye positions are the center positions of the iris of the uncovered eye.

[0088] Therefore, the strabismus detection method based on a display device provided in this application first calibrates the target detection position of the user using reference markers to ensure the reliability of strabismus detection; then, during the detection process, the target image is displayed by controlling the display device to achieve the occlusion of the occluded eye and the visual detection of the unoccluded eye, ensuring the effective implementation of strabismus detection; finally, the deviation data is determined by using the changes in the state of the occluded eye collected by the image acquisition device, effectively achieving the goal of strabismus detection without the participation of professional medical personnel. At the same time, the analysis results of the data collected by the image acquisition device improve the accuracy of deviation data analysis, providing widely applicable implementation conditions for strabismus screening, reducing the difficulty of strabismus detection, and improving the accuracy and reliability of strabismus detection without physician guidance.

[0089] In one feasible embodiment, during the detection process, detection instructions corresponding to each eye are generated sequentially at preset time intervals, so that each eye is used as the un-covered eye for detection.

[0090] Furthermore, before determining the skew data corresponding to the unmasked eye based on the changes, it is determined whether the number of detections for any unmasked eye has reached the required number of detections, and the skew data corresponding to the unmasked eye is determined based on the changes in the preset number of detections.

[0091] It should be noted that strabismus includes various types such as manifest strabismus, latent strabismus, alternating strabismus, and constant strabismus. Some types of strabismus are sporadic or intermittent. Therefore, it is necessary to perform multiple tests on both eyes separately to achieve the effect of detecting sporadic or intermittent strabismus, thereby further improving the reliability of strabismus detection and comprehensively screening the user's strabismus possibility in a single test, thus improving the accuracy of strabismus detection.

[0092] Based on this, this application further proposes that it is necessary to determine when the number of detections of the unmasked eye reaches a preset number of detections before the deviation data of the unmasked eye can be analyzed based on the changes in the preset number of detections.

[0093] In one specific embodiment, the preset time interval is 2 seconds. As shown in Figure 3, the detection loop is as follows: after the user reaches the target detection position, a "both eyes fixate on the target" instruction is issued to the user based on the detection control command to remind the user to enter the detection state. Then, a "please cover the right eye" detection control command is issued to the user, and the display device is controlled to display the target image corresponding to the left eye. That is, the left eye can see the target, and the right eye cannot see the target due to the cooperation of the covering unit and the target image. It should also be noted that if the user uses the separate reference mark unit and covering unit in Figure 1, the user will cover the right eye by using the covering unit to achieve the effect of the covering unit and the target image. If the user uses the integrated reference mark unit and covering unit in Figure 1, no other operation is required from the user. The server times for 2 seconds, and then generates a detection control command for the right eye, that is, a "please cover the left eye" detection control command is issued to the user, and the display device is controlled to display the target image corresponding to the right eye. That is, the right eye can see the target, and the left eye cannot see the target due to the cooperation of the covering unit and the target image. Similar to covering the right eye, if the user uses the separate reference marker unit and covering unit as shown in Figure 1, the user will automatically cover the left eye using the covering unit to achieve the effect of the covering unit and the target image working together. If the user uses the integrated reference marker unit and covering unit as shown in Figure 1, no further operation is required. The server times for 2 seconds and then generates the detection control command for the left eye, i.e., sends the detection control command "Please cover the left eye" to the user. This process continues until the detection of both the left and right eyes reaches the preset number of detections.

[0094] It should be understood that during the above process, the image acquisition device can continuously collect changes in the user's eyes, and can also detect changes in the occluded eye during the transition from an occluded state to an uncovered state according to detection control commands. Preferably, in order to prevent unreliable data acquisition due to different delays in the user's response to detection control commands, the embodiments of this application employ an image acquisition device that continuously collects changes in the user's eyes throughout the detection process, thereby improving the accuracy and reliability of the final strabismus detection.

[0095] It should be noted that, in order to improve the effective monitoring of complex strabismus such as alternating strabismus detection, this embodiment of the application defines a detection cycle for any occluded eye as the transition from an uncovered state to an uncovered state and back to an uncovered state. That is, "occluding the right eye - occluding the left eye - occluding the right eye" is one detection cycle for the right eye, recorded as one detection for the right eye. Similarly, "occluding the left eye - occluding the right eye - occluding the left eye" is one detection cycle for the left eye, recorded as one detection for the left eye. When the left and right eyes have reached the preset number of detections, the deviation data of the left and right eyes can be analyzed based on the collected changes.

[0096] It should be noted that the above-mentioned alternating left and right occlusion test method can usually only determine whether the user has orthorhombic eyes, manifest or latent strabismus, and the direction of strabismus, but cannot distinguish between latent and manifest strabismus, alternating strabismus, or monocular constant strabismus. Based on this, this application further proposes a detection strategy that works in conjunction with the "occlusion-unocclusion" method.

[0097] Specifically, between the detection control instructions generated sequentially for each eye, an intermediate detection control instruction for removing the occlusion of the occluded eye is further generated.

[0098] For example, as shown in Figure 4, the sequence of detection control instructions generated by the server is as follows: both eyes fixate on the target - cover the right eye - uncover the right eye - cover the left eye - uncover the left eye - cover the right eye - uncover the right eye... That is, between the alternating covering of the right and left eyes, there is a process of uncovering the currently covered eye but not covering the uncovered eye. During this process, both eyes are unobstructed and can see the target.

[0099] Therefore, the embodiments of this application can provide users with two strabismus detection strategies: alternating occlusion and occlusion-unocclusion, providing support for reliable, comprehensive, and accurate strabismus screening and meeting a wide range of strabismus detection needs.

[0100] On the other hand, this application uses a covering unit and a display device to cover the user's eye. However, since each user's visual ability is different, it is necessary to perform corresponding covering calibration for each user before testing.

[0101] Specifically, for any occluded eye, the target image is adjusted by receiving adjustment instructions input by the user, so that the target image and the occlusion unit cooperate to form an occlusion effect on the occluded eye.

[0102] For example, taking the separate reference marker and covering unit in Figure 1 as an example, when the left eye is covered using the separate covering unit, a preset target image is displayed to the user. The user then adjusts the color and transparency of the display device until the left eye is unable to see anything, that is, the target on the display device cannot be seen through the covering. Then, the right eye is covered using the covering unit, and the color and transparency of the display device are adjusted until the right eye is unable to see anything. Then, the process switches to covering the left eye for correction and adjustment, and then switches to covering the right eye for correction and adjustment, until the covering process switches between the left and right eyes, and the covered eye is unable to see anything.

[0103] Taking the integrated reference marker and covering unit in Figure 1 as an example, when the left eye needs to be covered, a preset target image corresponding to the right eye is displayed to the user. The user then adjusts the color and transparency of the display device until the left eye is unable to see and the right eye can see the target. That is, the left eye cannot see the target on the display device through the covering, but the right eye can. When the right eye needs to be covered, a preset target image corresponding to the left eye is displayed to the user. The user then adjusts the color and transparency of the display device until the right eye is unable to see and the left eye can see the target. That is, the right eye cannot see the target on the display device through the covering, but the left eye can. Then, the process switches to covering the left eye for correction and adjustment, and then switches to covering the right eye for correction and adjustment, until the covering process switches between the left and right eyes, achieving a state where the covered eye is unable to see and the uncovered eye can see the target.

[0104] In one specific embodiment, taking the reference marker and the masking unit as an example in Figure 1, when the colors of the masking lenses are red and green respectively, after adjustment, the target image corresponding to the red masking lens has a yellow center target and a red background with a transparency between 30% and 40%, and the target image corresponding to the green lens has a green center target and a red background with a transparency between 30% and 40%.

[0105] It should be noted that, during the occlusion-de-occlusion test, in order to ensure that during the intermediate detection control command stage, a binocular visual target image corresponding to the intermediate detection control command is further generated based on the target images corresponding to both eyes.

[0106] In other words, during the occlusion-de-occlusion test, in the de-occlusion phase of the occluded eye, the server controls the display device to adjust the target image corresponding to the de-occluded eye into a binocularly visible target image. This target image can be obtained by fusing the two target images obtained in the aforementioned monocular occlusion process. For example, taking the reference marker and occlusion unit integrated in Figure 1 as an example, the binocularly visible target image is a target image with a black background and a red center target. When the display device displays this target, both eyes in the de-occlusion phase can see the center target.

[0107] In a feasible embodiment, this application can also set up a corresponding account for each screening user, so that the corresponding target image can be retrieved according to the account information when the user is screening. This eliminates the need for users to adjust the target image every time they are screening for strabismus, thereby greatly saving users' time in strabismus screening, improving the efficiency of strabismus screening, and enhancing the user experience.

[0108] Furthermore, this application utilizes an image acquisition device to detect reference markers during the detection process, thereby enabling the detection and alerting of head movements during the user's detection process. This effectively avoids problems such as errors in eye movement information caused by head rotation, thus improving the accuracy of strabismus detection. Head movements include, but are not limited to, head misalignment or distance not meeting test requirements.

[0109] Preferably, during the detection process, the displacement of the user's head movement can also be analyzed, and based on the displacement of the user's head during the transition from the self-covered state to the uncovered state of the uncovered eye, the position of the first eye and the second eye position collected during this process can be corrected, thereby determining the actual iris displacement during the transition from the self-covered state to the uncovered state of the user's uncovered eye.

[0110] In one specific embodiment, the image acquired by the image acquisition device is calibrated with pixels and distances based on reference markers to determine the pixel-to-actual-distance mapping relationship between pixels in the acquired image and actual distances; the pixel distance between the first eye position and the second eye position in the acquired image is determined; and the actual displacement of the user's iris is determined based on the pixel distance and the pixel-to-actual-distance mapping relationship.

[0111] It should be understood that the reference marker is a pre-set known object, which can be a white block as shown in Figure 1, or other shapes. This application does not make any specific limitations.

[0112] Since the various attributes of the reference marker are known, such as length, width, and area, after the image acquisition device obtains the captured image, the distance to the user's detection position corresponding to one pixel in the captured image can be obtained through methods such as proportional calculation. For example, if the length of the reference marker is known to be 5cm and it occupies 5 pixels horizontally in the image acquisition device, then it can be determined that one pixel in the captured image corresponds to 1cm of the user's detection position.

[0113] At this point, after acquiring the first and second eye positions, the pixel distance between them in the captured image can be calculated. Then, based on the pixel distance and the pixel-to-actual-distance mapping, the actual displacement of the user's iris can be determined. For example, if the pixel distance between the first and second eye positions in the captured image is calculated to be 1 pixel, then the actual displacement of the user's iris can be determined to be 1 cm.

[0114] Therefore, this application can directly calculate the actual displacement of the user's iris using reference markers, providing a more accurate analytical value for subsequent skew analysis compared to the relative displacement of the iris, effectively avoiding the problem that the relative displacement of the iris cannot effectively distinguish between latent and manifest skew, alternating skew or monocular constant skew.

[0115] In some embodiments, head abnormalities also include behaviors such as blinking during the detection process. Specifically, because the blinking process obscures the eyeball, the image acquisition device cannot effectively capture changes in the user's eyeball state. Therefore, an alarm is also issued to the user when the blinking occurs, reminding the user to repeat the previous action to complete the information on the complete eyeball movement changes.

[0116] In one feasible embodiment, the server of this application determines the user's eye deviation data by analyzing the user's iris displacement trajectory in the image acquired by the image acquisition device. Specifically, after the image acquisition device determines that the user is at the target detection position using reference markers, it acquires an image of the user's eyes to obtain the iris position of the user at the target detection position. Then, through iris tracking technology, it determines the rotation or change of the user's eyes during the detection process, and thereby analyzes the user's eye deviation data.

[0117] It should be noted that although the operation of the method of the present invention is described in a specific order in the accompanying drawings, this does not require or imply that the operations must be performed in that specific order, or that all the operations shown must be performed in order to achieve the desired result.

[0118] Figure 5 shows a schematic diagram of the structure of a strabismus detection device based on a display device provided in an embodiment of this application.

[0119] As shown in Figure 5, the strabismus detection device 10 based on the display device includes:

[0120] The calibration module 11 is used to calibrate the user's detection position based on the reference marker worn by the user, and determine that the user is at the target detection position;

[0121] Detection module 12 is used to control the display device to display the target image corresponding to the unmasked eye in response to detection control commands during the detection process. The target image and the masking unit cooperate to form a masking effect on the masked eye and display the target visual object visible to the unmasked eye. It also controls the image acquisition device to acquire the changes in the unmasked eye from the masked state to the unmasked state.

[0122] Analysis module 13 is used to determine the skew data corresponding to the uncovered eye based on the changes.

[0123] In some embodiments, the detection module 12 is further configured to:

[0124] During the detection process, detection control commands corresponding to each eye are generated sequentially according to a preset time interval, so that the eyes are sequentially used as the uncovered eyes for detection.

[0125] Analysis module 13 is also used for:

[0126] Before determining the skew data corresponding to the unmasked eye based on the changes, it is determined whether the number of detections for any unmasked eye has reached a preset number of detections, and the skew data corresponding to the unmasked eye is determined based on the changes in the preset number of detections.

[0127] In some embodiments, the detection module 12 is further configured to:

[0128] For any of the occluded eyes, the target image is adjusted by receiving the adjustment command input by the user, so that the target image and the occlusion unit cooperate to form an occlusion effect on the occluded eye.

[0129] In some embodiments, the detection module 12 is further configured to:

[0130] During the sequential generation of detection control instructions corresponding to each eye, an intermediate detection control instruction for removing the occlusion of the occluded eye is generated.

[0131] In some embodiments, the detection module 12 is further configured to:

[0132] For any of the occluded eyes, the target image is adjusted according to the adjustment command input by the user, so that the target image cooperates with the occlusion unit to form an occlusion effect on the occluded eye; and

[0133] Based on the target images corresponding to the two eyes, a binocular visual target image corresponding to the intermediate detection control command is generated.

[0134] In some embodiments, the detection module 12 is further configured to:

[0135] The image acquisition device is controlled to acquire the first eye position of the unmasked eye in the masked state; wherein, when the masking unit is a semi-transparent masking object, the first eye position is the position of the unmasked eye under the masking unit, and when the masking unit is an opaque masking object, the first eye position is the instantaneous position of the unmasked eye when the masking is removed.

[0136] as well as

[0137] The image acquisition device is controlled to acquire the second eye position after the uncovered eye has been removed and is now in an uncovered state;

[0138] Wherein, the first eye position and the second eye position are the iris positions of the uncovered eye, and the first eye position and the second eye position are the iris positions corrected based on the user's head position during the detection process.

[0139] In some embodiments, the detection module 12 is further configured to:

[0140] During the detection process, the user's head abnormality is determined based on the changes in the reference marker, and an alarm is issued for the user's abnormal head behavior;

[0141] as well as

[0142] Based on the reference markers, pixel-distance calibration is performed on the images acquired by the image acquisition device to determine the pixel-actual distance mapping relationship between pixels in the acquired images and actual distances.

[0143] Determine the pixel distance between the first eye position and the second eye position in the acquired image;

[0144] Based on the pixel distance and the pixel-actual distance mapping relationship, the actual displacement of the user's iris is determined.

[0145] It should be understood that the units or modules described in the strabismus detection device 10 based on the display device correspond to the various steps in the method described with reference to FIG2. Therefore, the operations and features described above for the method are also applicable to the strabismus detection device 10 based on the display device and the units contained therein, and will not be repeated here. The strabismus detection device 10 based on the display device can be pre-implemented in the browser or other security applications of an electronic device, or can be loaded into the browser or other security applications of an electronic device by downloading or other means. The corresponding units in the strabismus detection device 10 based on the display device can cooperate with units in the electronic device to implement the solution of the embodiments of this application.

[0146] The division of modules or units mentioned in the detailed description above is not mandatory. In fact, according to the embodiments of this disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by multiple modules or units.

[0147] Referring now to FIG6, FIG6 illustrates a schematic diagram of a computer system suitable for implementing an electronic device or server according to embodiments of the present application.

[0148] As shown in Figure 6, the computer system includes a central processing unit (CPU) 601, which can perform various appropriate actions and processes based on programs stored in read-only memory (ROM) 602 or programs loaded from storage section 608 into random access memory (RAM) 603. RAM 603 also stores various programs and data required for the system's operating instructions. CPU 601, ROM 602, and RAM 603 are interconnected via bus 604. Input / output (I / O) interface 605 is also connected to bus 604.

[0149] The following components are connected to I / O interface 605: an input section 606 including a keyboard, mouse, etc.; an output section 607 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 608 including a hard disk, etc.; and a communication section 609 including a network interface card such as a LAN card, modem, etc. The communication section 609 performs communication processing via a network such as the Internet. A drive 610 is also connected to I / O interface 605 as needed. A removable medium 611, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on drive 610 as needed so that computer programs read from it can be installed into storage section 608 as needed.

[0150] Specifically, according to embodiments of this application, the processes described above with reference to flowchart FIG2 can be implemented as computer software programs. For example, embodiments of this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowchart. In such an embodiment, the computer program contains program code for performing the methods shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network via communication section 609, and / or installed from removable medium 611. When the computer program is executed by central processing unit (CPU) 601, it performs the functions defined above in the system of this application.

[0151] It should be noted that the computer-readable medium shown in this application can be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this application, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this application, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media can also be any computer-readable medium other than computer-readable storage media, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0152] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operational instructions of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two connected blocks may actually be executed substantially in parallel, or they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified functions or operational instructions, or using a combination of dedicated hardware and computer instructions.

[0153] The units or modules described in the embodiments of this application can be implemented in software or hardware. The described units or modules can also be housed in a processor; for example, a processor may be described as including a calibration module, a detection module, and an analysis module. The names of these units or modules do not necessarily limit the specific unit or module itself. For example, a calibration module may also be described as "calibrating the user's detection position based on a reference marker worn by the user to determine that the user is at a target detection position."

[0154] In another aspect, this application also provides a computer-readable storage medium, which may be included in the electronic device described in the above embodiments, or may exist independently and not assembled into the electronic device. The computer-readable storage medium stores one or more programs that, when used by one or more processors, execute the strabismus detection method based on a display device described in this application.

[0155] The above description is merely a preferred embodiment of this application and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of disclosure in this application is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the foregoing disclosed concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features with similar functions disclosed in this application.

Claims

1. A strabismus detection method based on a display device, characterized in that, include: The user's detection position is calibrated based on the reference marker worn by the user to determine that the user is at the target detection position; During the detection process, in response to the detection control command, the display device is controlled to display the target image corresponding to the unmasked eye. The target image and the masking unit work together to form a masking effect on the masked eye and display the target visual object visible to the unmasked eye. The image acquisition device is controlled to acquire the changes in the unmasked eye from the masked state to the unmasked state. Based on the changes in the uncovered eye from the covered state to the uncovered state, the skew data corresponding to the uncovered eye is determined.

2. The strabismus detection method based on a display device according to claim 1, characterized in that, Also includes: During the detection process, detection control commands corresponding to each eye are generated sequentially according to a preset time interval, so that the eyes are sequentially used as the uncovered eyes for detection. as well as Before determining the skew data corresponding to the unmasked eye based on the changes, it is determined whether the number of detections for any unmasked eye has reached a preset number of detections, and the skew data corresponding to the unmasked eye is determined based on the changes in the preset number of detections.

3. The strabismus detection method based on a display device according to claim 2, characterized in that, Before the testing, it also includes: For any of the occluded eyes, the target image is adjusted by receiving the adjustment command input by the user, so that the target image and the occlusion unit cooperate to form an occlusion effect on the occluded eye.

4. The strabismus detection method based on a display device according to claim 2, characterized in that, The process of sequentially generating detection control commands corresponding to each eye also includes: Generate intermediate detection control instructions for removing the occlusion from the occluded eye.

5. The strabismus detection method based on a display device according to claim 4, characterized in that, Also includes: For any of the covered eyes, the target image is adjusted by receiving the adjustment command input by the user, so that the target image and the covering unit cooperate to form a covering effect on the covered eye; as well as Based on the target images corresponding to the two eyes, a binocular visual target image corresponding to the intermediate detection control command is generated.

6. The strabismus detection method based on a display device according to any one of claims 1-5, characterized in that, The controlled image acquisition device acquires the changes in the uncovered eye from the occluded state to the unoccluded state, including: The image acquisition device is controlled to acquire the first eye position of the unmasked eye in the masked state; wherein, when the masking unit is a semi-transparent masking object, the first eye position is the position of the unmasked eye under the masking unit, and when the masking unit is an opaque masking object, the first eye position is the instantaneous position of the unmasked eye when the masking is removed. as well as The image acquisition device is controlled to acquire the second eye position after the uncovered eye has been removed and is now in an uncovered state; Wherein, the first eye position and the second eye position are the iris positions of the uncovered eye, and the first eye position and the second eye position are the iris positions corrected based on the user's head position during the detection process.

7. The strabismus detection method based on a display device according to claim 6, characterized in that, Also includes: During the detection process, the user's head abnormality is determined based on the changes in the reference marker, and an alarm is issued for the user's abnormal head behavior; as well as Based on the reference markers, pixel-distance calibration is performed on the images acquired by the image acquisition device to determine the pixel-actual distance mapping relationship between pixels in the acquired images and actual distances. Determine the pixel distance between the first eye position and the second eye position in the acquired image; Based on the pixel distance and the pixel-actual distance mapping relationship, the actual displacement of the user's iris is determined.

8. The strabismus detection method based on a display device according to claim 6, characterized in that, The first eye position and the second eye position are the center positions of the iris of the uncovered eye.

9. A strabismus detection device based on a display device, characterized in that, include: The calibration module is used to calibrate the user's detection position based on a reference marker worn by the user, and to determine that the user is at the target detection position; The detection module is used to control the display device to display the target image corresponding to the unmasked eye in response to the detection control command during the detection process. The target image and the masking unit work together to form a masking effect on the masked eye and display the target visual object visible to the unmasked eye. The module also controls the image acquisition device to acquire the changes in the unmasked eye from the masked state to the unmasked state. The analysis module is used to determine the skew data corresponding to the uncovered eye based on the changes in the uncovered eye from the occluded state to the unoccluded state.

10. A strabismus detection system based on a display device, characterized in that, include: Reference marker units are used to correct the user's detection position; The masking unit is used to cooperate with the target image displayed by the display device during the detection process to form a masking effect on the masked eye; A display device is configured to respond to a detection control command to display a target image corresponding to the unmasked eye, wherein the target image and the masking unit cooperate to form a masking effect on the masked eye and to display a target visual object visible to the unmasked eye; Image acquisition device, used to acquire the changes in the occluded eye from the occluded state to the unoccluded state in response to the detection control command; The server is configured to respond to the detection control command by controlling the display device to display the target image, controlling the image acquisition device to acquire the changes in the unmasked eye, and determining the skew data corresponding to the unmasked eye based on the changes in the unmasked eye from the masked state to the unmasked state.

Images