Strabismus detection method, strabismus detection device and wearable equipment

Abstract

The invention relates to a strabismus detection method, a strabismus detection device and wearable equipment, and the method comprises the steps: obtaining the eye movement fluctuation information of each eye of a detection object based on a first detection rule; when the eye movement fluctuation information of each eye of the detection object is lower than a first threshold value, first eye movement information of each eye of the detection object is obtained when strabismus detection is carried out based on a first detection rule; wherein the eye movement fluctuation information and the first eye movement information are associated with changes between eye movement direction vector mean values of the eyes in various eye movement platform periods when strabismus detection is carried out based on a first detection rule; and when the first eye movement information of any eye is greater than a second threshold value, determining that the detected object has eye position deflection. The objective and unified eye position deflection measurement process is realized, and the strabismus detection efficiency is improved.

Description

Technical Field

[0001] The present application relates to the field of eye tracking technology, and in particular to a strabismus detection method, a strabismus detection device and a wearable device. Background Art

[0002] Strabismus refers to the inability of both eyes to focus on the same target at the same time. When one eye is looking at the target, the other eye's line of sight will deviate, causing asymmetry in the positions of the two eyes. For children in their developmental period, strabismus is one of the risk factors for amblyopia. In addition, strabismus may also affect the development of visual function, motor function and fine hand-brain-eye coordination, causing significant negative impacts on the patient's physical and mental health. Qualitative and quantitative measurement of strabismus can assess whether the subject has strabismus, as well as the type and degree of strabismus, so as to formulate a targeted treatment plan.

[0003] At present, strabismus detection is often performed and judged subjectively by doctors in clinical practice based on strabismus measurement equipment, such as synoptophores, prisms, penlights, Maddox rods, etc. This process relies on the doctor's measurement techniques, experience and equipment accuracy, and lacks an objective and unified measurement process, resulting in low strabismus detection efficiency.

[0004] With regard to the problem of low efficiency of strabismus detection in related technologies, no effective solution has been proposed so far. Summary of the invention

[0005] In this embodiment, a strabismus detection method, a strabismus detection device and a wearable device are provided to solve the problem of low strabismus detection efficiency in the related art.

[0006] In a first aspect, a strabismus detection method is provided in this embodiment, including: Based on the first detection rule, obtaining eye movement fluctuation information of each eye of the detection object; When the eye movement fluctuation information of each eye of the detection object is lower than a first threshold, obtaining first eye movement information of each eye of the detection object when performing strabismus detection based on the first detection rule; The eye movement fluctuation information and the first eye movement information are associated with changes between mean values ​​of eye movement direction vectors of the eyes in each eye movement platform period when strabismus detection is performed based on the first detection rule; When the first eye movement information of any eye is greater than a second threshold, it is determined that the detected object has eye deviation; wherein: determining the eye movement platform phase includes: Collecting an eye movement direction vector of an eye, and generating an eye movement waveform diagram based on the eye movement direction vector; An effective eye movement waveform in the eye movement waveform is determined, and an eye movement platform period of the eye is determined according to an extreme point in the effective eye movement waveform.

[0007] In some of the embodiments, it also includes: Based on a second detection rule, second eye movement information of at least one eye of the detection object is obtained, and based on the second eye movement information, the type of eye deviation of the detection object is determined; the second eye movement information is associated with the changes between the mean values ​​of the eye movement direction vectors of the eyes in each eye movement platform period when strabismus type detection is performed based on the second detection rule.

[0008] In some embodiments, the second detection rule is a cover-uncover detection rule; and obtaining second eye movement information of at least one eye of the detection object, and determining the eye deviation type of the detection object based on the second eye movement information, includes: covering the first eye of the detection object; Based on the second eye movement information of the second eye, determining whether the eye deviation type of the detection object is manifest strabismus, wherein the second eye movement information of the second eye is associated with the change between the mean value of the eye movement direction vector of the second eye in the eye movement platform period before and after covering; If not, uncover the first eye, and determine whether the eye deviation type of the detection object is latent strabismus based on the second eye movement information of the first eye, wherein the second eye movement information of the first eye is associated with the change between the mean values ​​of the eye movement direction vectors of the first eye in the eye movement platform period before and after uncovering; If it is not possible to determine whether the eye deviation type of the detection object is latent strabismus based on the second eye movement information of the first eye, covering the second eye of the detection object; Based on the second eye movement information of the first eye, determining whether the eye deviation type of the detection object is manifest strabismus, wherein the second eye movement information of the first eye is associated with the change between the mean values ​​of the eye movement direction vector of the first eye in the eye movement platform period before and after covering; If after covering the second eye of the detection object, it is impossible to determine whether the eye deviation type of the detection object is manifest strabismus based on the second eye movement information of the first eye, the second eye is uncovered, and based on the second eye movement information of the second eye, it is determined whether the eye deviation type of the detection object is latent strabismus, wherein the second eye movement information of the second eye is associated with the change between the mean values ​​of the eye movement direction vectors of the second eye in the eye movement platform period before and after uncovering.

[0009] In some embodiments, the first detection rule is an alternating cover rule, and obtaining eye movement fluctuation information of one eye of the detection object based on the first detection rule includes: The eye movement fluctuation information of an eye is obtained based on the difference between the average values ​​of the eye movement direction vectors of an eye that is alternately covered M times in each eye movement plateau period; wherein M is greater than or equal to 2.

[0010] In some embodiments, the first detection rule is an alternating cover rule, and when performing strabismus detection based on the first detection rule, obtaining first eye movement information of one eye of the detection object includes: The absolute value of the average of the differences between the average values ​​of the eye movement direction vectors of an eye that is alternately covered M times in each eye movement plateau period is the first eye movement information of the eye; wherein M is greater than or equal to 2.

[0011] In some embodiments, determining a valid eye movement waveform in the eye movement waveform includes: The eye movement waveform is filtered to remove invalid eye movement signals in the eye movement waveform to obtain the valid eye movement waveform.

[0012] In some embodiments, the method further comprises: If the detected subject has eye deviation; covering the first eye of the detection object; Adjust the display position of the target based on the movement direction and stride; Uncovering the first eye and covering the second eye of the detection object; Determine whether the third eye movement information of the second eye is less than or equal to a third threshold; if so, determine the strabismus angle based on the current display position of the target object, the pupil distance of the detection object, and the distance from the eyes of the detection object to the target object; the third eye movement information of the second eye is associated with the change between the mean values ​​of the eye movement direction vectors of the second eye in the eye movement platform period before and after the second eye is covered; if not, repeat the above steps until the third eye movement information of the second eye is less than or equal to the third threshold to determine the strabismus angle.

[0013] In some embodiments thereof: the stride includes an initial stride and an adaptive stride; wherein the adaptive stride is associated with the third eye movement information corresponding to the second eye before and after each covering of the second eye; The moving direction includes an initial moving direction and a target moving direction, and the target moving direction is associated with a change between the mean values ​​of the eye movement direction vectors of the second eye in the eye movement platform period before and after each covering of the second eye.

[0014] In a second aspect, a strabismus detection device is provided in this embodiment, including: a fluctuation information acquisition module, a first eye movement information acquisition module and a detection module; wherein: The fluctuation information acquisition module is used to acquire eye movement fluctuation information of each eye of the detection object based on the first detection rule; The first eye movement information acquisition module is used to acquire first eye movement information of each eye of the detection object when strabismus detection is performed based on the first detection rule when the eye movement fluctuation information of each eye of the detection object is lower than a first threshold; wherein the eye movement fluctuation information and the first eye movement information are associated with changes between mean values ​​of eye movement direction vectors of the eyes in each eye movement platform period when strabismus detection is performed based on the first detection rule; The detection module is used to determine that the detected object has eye deviation when the first eye movement information of any eye is greater than a second threshold; wherein: determining the eye movement platform phase includes: Collecting an eye movement direction vector of an eye, and generating an eye movement waveform diagram based on the eye movement direction vector; An effective eye movement waveform in the eye movement waveform is determined, and an eye movement platform period of the eye is determined according to an extreme point in the effective eye movement waveform.

[0015] In a third aspect, a wearable device is provided in this embodiment, including a display component; The display component is used to display the target object according to the received display control signal; the display control signal is generated according to the eye movement direction vector and the eye deviation detection rule when the detection object observes the target object.

[0016] In some of the embodiments, the wearable device further includes a data acquisition component; wherein: The data acquisition component is used to acquire the eye movement direction vector when the detection subject observes the target object.

[0017] In some of the embodiments, the wearable device further comprises a strabismus detection processor; the strabismus detection processor is connected to the display component and the data acquisition component; The strabismus detection processor is used to receive the eye movement direction vector collected by the data collection component and send the display control signal to the display component; and send the strabismus detection result to the display component.

[0018] In some of the embodiments, the wearable device is head-mounted.

[0019] Compared with the related art, the present embodiment provides a strabismus detection method, a strabismus detection device and a wearable device. The strabismus detection method, based on the first detection rule, obtains the eye movement fluctuation information of each eye of the detection object; when the eye movement fluctuation information of each eye of the detection object is lower than the first threshold, the first eye movement information of each eye of the detection object is obtained when the strabismus detection is performed based on the first detection rule; wherein the eye movement fluctuation information and the first eye movement information are associated with the change between the mean values ​​of the eye movement direction vectors of the eyes in each eye movement platform period when the strabismus detection is performed based on the first detection rule; when the first eye movement information of any eye is greater than the second threshold, it is determined that the detection object has eye deviation; wherein: determining the eye movement platform period of the eye includes: collecting the eye movement direction vector of the eye, generating an eye movement waveform diagram based on the eye movement direction vector; determining the effective eye movement waveform diagram in the eye movement waveform diagram, and determining the eye movement platform period of the eye according to the extreme point in the effective eye movement waveform diagram. The above processes can all be performed by electronic devices with data computing capabilities, realizing qualitative analysis of eye deviation. Compared with related technologies, this application eliminates the reliance on the professional's own experience, making the test results more objective and accurate, and the test process more efficient.

[0020] Details of one or more embodiments of the present application are set forth in the following drawings and description to make other features, objects, and advantages of the present application more readily apparent. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings: Figure 1 is a hardware structure block diagram of a terminal of the strabismus detection method according to an embodiment of the present application; Figure 2 is a flow chart of a strabismus detection method according to an embodiment of the present application; Figure 3 is a schematic diagram of an adaptive filtering embodiment of the present application; Figure 4 is a schematic diagram of squint angle calculation in an embodiment of the present application; Figure 5 is a flow chart of a strabismus detection method of some embodiments of the present application; Figure 6 is a structural block diagram of a strabismus detection device according to an embodiment of the present application; Figure 7 It is a schematic diagram of the structure of a wearable device according to an embodiment of the present application. DETAILED DESCRIPTION

[0022] In order to more clearly understand the purpose, technical solutions and advantages of the present application, the present application is described and illustrated below in conjunction with the accompanying drawings and embodiments.

[0023] Unless otherwise defined, the technical terms or scientific terms involved in this application shall have the general meaning understood by people with general skills in the technical field to which this application belongs. The words "one", "a", "the", "these" and the like in this application do not indicate a quantitative limitation, and they may be singular or plural. The terms "include", "comprise", "have" and any variants thereof involved in this application are intended to cover non-exclusive inclusions; for example, a process, method and system, product or device comprising a series of steps or modules (units) is not limited to the listed steps or modules (units), but may include unlisted steps or modules (units), or may include other steps or modules (units) inherent to these processes, methods, products or devices. The words "connect", "connected", "coupled" and the like involved in this application are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. The "multiple" involved in this application refers to two or more. "And / or" describes the association relationship of associated objects, indicating that there may be three relationships, for example, "A and / or B" may mean: A exists alone, A and B exist at the same time, and B exists alone. Generally, the character " / " indicates that the objects associated with each other are in an "or" relationship. The terms "first", "second", "third", etc. in this application are only used to distinguish similar objects and do not represent a specific ordering of the objects.

[0024] The method embodiment provided in this embodiment can be executed in a terminal, a computer or a similar computing device. For example, running on a terminal, Figure 1 FIG. 1 is a hardware structure diagram of a terminal of the strabismus detection method of this embodiment. Figure 1 As shown, the terminal may include one or more ( Figure 1 Only one is shown in the figure) a processor 102 and a memory 104 for storing data, wherein the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA. The above terminal may also include a transmission device 106 and an input and output device 108 for communication functions. It can be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the above terminal. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations shown.

[0025] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to the strabismus detection method in this embodiment. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, to implement the above method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories may be connected to the terminal via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0026] The transmission device 106 is used to receive or send data via a network. The above network includes a wireless network provided by the communication provider of the terminal. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (Radio Frequency, referred to as RF) module, which is used to communicate with the Internet wirelessly.

[0027] In this embodiment, a strabismus detection method is provided. Figure 2 is a flow chart of the strabismus detection method of this embodiment, such as Figure 2 As shown, the process includes the following steps: Step S210: acquiring eye movement fluctuation information of each eye of the detection object based on the first detection rule.

[0028] First of all, it should be noted that all the steps of the strabismus detection method provided in this embodiment can be implemented by a processor with data computing capabilities in a wearable device provided with a display component, or by a terminal or server associated with the display component and independent of the display component. The detection object is an object that needs to be detected for eye deviation. During the detection process, display control can be performed on the display component worn by the detection object to form a cover and uncover of the detection object's field of view. Generally speaking, the split screen (left screen and right screen) display is realized by the display component, that is, a target object is displayed in the middle of the left screen and the right screen. When the left and right screens are turned on at the same time, based on the display of the left and right screens, the target object is presented in front of the binocular field of view. Taking the processor in the wearable device running the strabismus detection method as an example, if the wearable device is VR or AR glasses, when the strabismus detection method is executed, the detection object wears VR glasses, and sees the left and right screens in the virtual space through the VR glasses, as well as the presented target object. The target object can be: a simple shape, clear structure, and easy-to-watch graphics such as a sight mark, a geometric figure, or a light spot. A data acquisition component integrated in the wearable device or associated with the wearable device, such as an eye tracker, can be used to collect the eye movement direction vector of the detection subject when looking at the target object.

[0029] The first detection rule may be specifically set according to a qualitative strabismus detection method, such as controlling the display of a display component that the detection object is looking at based on the first detection rule, or controlling the display presented in a virtual space based on the first detection rule.

[0030] In this embodiment, the first detection rule may be an alternating cover detection rule set based on the alternating cover detection method, or other cover rules that can play a qualitative strabismus detection function like the alternating cover detection method. Exemplarily, the first detection rule may be a display control rule of a display component that can alternately cover the visual field of the left and right eyes of the detection object. This step is based on the first detection rule, controls the display component to display the target object, and on this basis, collects the eye movement direction vector when the detection object looks at the target object.

[0031] The eye movement direction vector may be a difference between the eye directions at two moments, and the eye movement direction at any moment may be specifically represented by a quaternion, which can effectively reduce the storage of the device and improve the computing performance. A quaternion is a simple hypercomplex number, which can generally be represented as q=(ω, x, y, z), where ω is the real part and x, y, and z are the imaginary parts.

[0032] After obtaining the quaternion representation of the eye movement direction vector, the coordinate system can be converted to obtain the Euler angle containing three free quantities (yaw, pitch, and roll). In this embodiment, only the horizontal eye position deviation can be considered, so only the rotation of the x-axis (roll angle) can be considered. At this point, the eye movement direction vector that can be used for eye movement information statistics can be obtained.

[0033] In most cases, people with strabismus have coordinated eyes, but there are also a few uncoordinated strabismus types, such as restrictive strabismus and paralytic strabismus. Therefore, for coordinated strabismus, strabismus can be judged based on only one eye. In this step, in order to take uncoordinated strabismus into account and improve the sensitivity of screening, the eye movement direction vectors of both eyes of the test subject are collected.

[0034] The above-mentioned eye movement platform periods refer to: the eye movement platform period when the left eye is covered (the right eye is not covered); the eye movement platform period when the right eye is covered (the left eye is not covered).

[0035] Taking the first detection rule of alternating cover detection as an example, the fluctuation value of the left and right eyes of the detected object when alternating cover is performed once is defined as follows: Assume that the fluctuation value of the right eye is , the fluctuation value of the left eye is .at this time, R 1. R 2. L 1. L 2 Specifically: When the right eye is not covered (for example, the right screen of the head-mounted display is turned on and the left screen is turned off): when the right eye is in the eye movement platform period, the average value of the right eye eye movement direction vector R 1; when the left eye is in the eye movement platform period, the average value of the left eye eye movement direction vector L 1.

[0036] When the right eye is covered (for example, the right screen of the head-mounted display is turned off and the left screen is turned on): when the right eye is in the eye movement platform period, the average value of the right eye eye movement direction vector R 2. The average value of the left eye movement direction vector when the left eye is in the eye movement platform period L 2.

[0037] The process of obtaining the eye movement fluctuation information of each eye of the test object under the first detection rule according to the changes between the mean values ​​of the eye movement direction vectors of the two eyes of the test object in the eye movement platform period can be specifically as follows: the variance of the above fluctuation values ​​of the left eye is calculated multiple times , calculate the variance of the above fluctuation values ​​of the right eye multiple times , in order to obtain the eye movement fluctuation of each eye of the test object, that is, the variance is the eye movement fluctuation information of the left eye, variance It is the eye movement fluctuation information of the right eye.

[0038] In addition, in addition to obtaining eye movement fluctuation information by statistically analyzing the fluctuation values ​​based on the variance, the eye movement fluctuation information of each eye of the detection object can also be obtained by calculating the autocorrelation function, information entropy, etc. of the above fluctuation values.

[0039] Therefore, in step S210, under the first detection rule, the eye movement direction vectors of each eye of the detection object can be collected multiple times to obtain the eye movement fluctuation information of each eye of the detection object. Specifically, the difference between the mean values ​​of the eye movement direction vectors of any eye collected during the eye movement platform period before and after each cover can be used as the fluctuation value of one alternating cover; then, the fluctuation values ​​corresponding to multiple alternating covers are statistically calculated to measure the stable state (for example, to calculate the variance), thereby obtaining the eye movement fluctuation information to determine whether the eye movement of the detection object has reached a stable state.

[0040] Step S220, when the eye movement fluctuation information of each eye of the detection object is lower than the first threshold, obtaining the first eye movement information of each eye of the detection object when strabismus detection is performed based on the first detection rule; wherein the eye movement fluctuation information and the first eye movement information are associated with the changes between the mean values ​​of the eye movement direction vectors of the eyes in each eye movement platform period when strabismus detection is performed based on the first detection rule.

[0041] Step S230: When the first eye movement information of any eye is greater than a second threshold, it is determined that the detected object has eye deviation.

[0042] In this step, for the accuracy of strabismus detection, it is possible to determine whether there is eye deviation (i.e. strabismus) after recording that the eye movement fluctuations of both eyes of the detected object are stable. In the case where the eye movement fluctuation information in step S210 is variance, the first threshold value may be a variance threshold value. The variance of the fluctuation values ​​of the left eye M times may be calculated. , and calculate the variance of the fluctuation value of the right eye M times ,exist and When the eye movement fluctuations of the detection object are all less than the first threshold, it is confirmed that the eye movement fluctuations of the detection object have reached a stable state. Next, the first eye movement information of the detection object is determined. Continuing to use the head-mounted display component to detect the detection object, and taking the first detection rule as the alternating cover detection rule as an example, the process of determining the first eye movement information is described: First, in the initial state, both the left and right screens of the head mounted display device are turned on, and the target object is located in the middle of the two screens. For one alternation: When the left screen is turned off and the right screen is turned on, the average value of the eye movement direction vector of the right eye during its eye movement platform period is obtained. R1, and the average value of the eye movement direction vector of the left eye during the eye movement plateau period L 1.

[0043] in, R 1 is obtained by summing the horizontal dimension of the eye movement direction vector of the right eye during the eye movement platform period and dividing it by N get.( , T is the duration of the eye movement plateau period, t f The time required to record the eye movement direction vector once, that is, in this embodiment, the frame rate is taken according to a certain frame rate. N For example , which means taking 90 points continuously within one second).

[0044] L 1 is calculated in the same way as R The calculation method of 1 is similar and will not be repeated here.

[0045] When the right screen is turned off and the left screen is turned on, the average value of the eye movement direction vector of the left eye during its eye movement platform period is obtained. L 2, and the average value of the eye movement direction vector of the right eye during its eye movement platform cycle R 2.

[0046] Calculate separately , , and the calculated difference is used as the change between the mean values ​​of the eye movement direction vectors of the left and right eyes during the eye movement platform period during an alternating cover.

[0047] After performing M alternating covers, for the left eye: M ( ) values, and for the right eye, we get M ( ) values. Calculate M ( ) Or M ( ) ,exist and When both are less than the first threshold, the eye movement situation is judged.

[0048] Calculate M ( ) L a (This value is used to measure the eye movement of the left eye, and its absolute value is used as the first eye movement information of the left eye), or calculate M ( ) R a (This value is used to measure the eye movement of the right eye, and its absolute value is used as the first eye movement information of the right eye). L aThe absolute value of R a If the absolute value of is greater than the second threshold, it is confirmed that the detected object has eye deviation. L a or R a The positive and negative signs of the test object are used to determine whether the strabismus is esotropia or exotropia. Generally speaking: for the left eye, L a If it is negative, it is outward slanting. L a For the right eye, R a If it is negative, it is internal oblique. R a It is positive exotropia. It should be noted that the positive and negative in this embodiment are relative concepts and need to be determined according to the definition of the eye movement vector output by the eye tracker. In this embodiment, for the left eye, the value of the eye movement direction vector changes from large to small from the nasal side to the temporal side; for the right eye, the value of the eye movement direction vector changes from small to large from the nasal side to the temporal side.

[0049] The above-mentioned eye movement direction vector can be specifically realized based on eye tracking technology. The eye movement direction vector can be acquired by using an eye tracking detector. Among them, based on the principle of the above-mentioned eye tracking technology, a data acquisition component of an eye movement direction vector can be implemented in combination with different components to collect the eye movement direction vector. It is also possible to directly use a display device that has an eye tracking detector integrated in it, such as a virtual reality (VR) head-mounted display device. When executing the strabismus detection method of this embodiment based on a VR head-mounted display device, the eye movement direction vector can be directly acquired based on the application programming interface (API) provided by the display device itself. In this way, a qualitative analysis of the eye deviation of the detection object can be realized based on an electronic device with data calculation and processing capabilities such as a computer, based on the collected eye movement direction vector.

[0050] In the related art, strabismus detection and judgment are usually performed manually and subjectively by professionals such as doctors based on the strabismus measurement process. That is, a fixed, unified and standardized strabismus detection workflow has not yet been formed in clinical practice. Currently, commonly used strabismus measurement methods include qualitative detection by the cover-uncover method and the alternating cover method, as well as the corneal reflection method and the prism corneal reflection method. The existing strabismus measurement methods in the related art currently lack objective means and rely on the experience of professional doctors and the accuracy of professional detection equipment, resulting in low efficiency in strabismus detection.

[0051] This embodiment provides a set of objective and standardized eye deviation detection workflows through the above steps S210 to S230, wherein the detection results of eye deviation are obtained from the data collection and processing of the eye movement direction vector to the final data analysis, and the entire process can be performed by an electronic device with data computing capabilities, thereby achieving a qualitative analysis of eye deviation. Compared with related technologies, the reliance on the experience of professionals is eliminated, making the detection results more objective and accurate, and the detection process more efficient.

[0052] Therefore, in the above steps S210 to S230, based on the first detection rule, eye movement fluctuation information of each eye of the detection object is obtained; when the eye movement fluctuation information of each eye of the detection object is lower than the first threshold, the first eye movement information of each eye of the detection object is obtained when strabismus detection is performed based on the first detection rule; wherein the eye movement fluctuation information and the first eye movement information are associated with the change between the mean values ​​of the eye movement direction vectors of the eyes in each eye movement platform period when strabismus detection is performed based on the first detection rule; when the first eye movement information of any eye is greater than the second threshold, it is determined that the detection object has eye deviation. It realizes an objective and unified strabismus detection process and improves the efficiency of strabismus detection.

[0053] In one embodiment, the strabismus detection method may further include: Based on the second detection rule, second eye movement information of at least one eye of the detection object is obtained, and based on the second eye movement information, the type of eye deviation of the detection object is determined; the second eye movement information is associated with the changes between the mean values ​​of the eye movement direction vectors of the eyes in each eye movement platform period when strabismus type detection is performed based on the second detection rule.

[0054] The second detection rule may be a cover-uncover detection rule for identifying the type of eye deviation of the detection object when it is determined that the detection object has eye deviation. Similarly, the second detection rule is also a display control rule that acts on the display component to form a cover-uncover for the detection object's visual field. Specifically, the second detection rule may be a detection rule set based on a cover-uncover detection method (a rule for covering and uncovering a single eye of at least one eye of the detection object), or other covering rules that can also realize the identification of the type of eye deviation.

[0055] In order to achieve independent covering of the binocular vision of the detection object, the binocular vision of the detection object can be separated. In this embodiment, the method of achieving binocular vision separation includes but is not limited to: split screen display, providing polarized glasses for the detection object, and providing red and green glasses for the detection object. In particular, the split screen display is that the display component includes two independent display screens, corresponding to the left eye and the right eye of the detection object respectively. The two display screens can display the same or different images. The two display screens are turned off and on independently of each other. The detection object wears polarized glasses to achieve binocular vision separation, specifically, the polarization directions of the polarized lenses corresponding to the left and right eyes are different, and the display component is used to provide only polarized light in the polarization direction corresponding to the left eye, or polarized light in the polarization direction corresponding to the right eye, to achieve monocular vision separation. The binocular vision separation is achieved by making the detection object wear red and green glasses, specifically, one lens is set as a red lens and the other lens is set as a green lens. When the target object on the display component is red, the eye vision corresponding to the red lens is equivalent to being covered. Similarly, when the target object on the display component is green, the eye vision corresponding to the green lens is equivalent to being covered. Those skilled in the art will appreciate that different methods may be set to separate the binocular fields of view according to the requirements of actual application scenarios, so as to achieve independent covering of the binocular fields of view according to different covering rules.

[0056] The above-mentioned eye deviation type can be used to characterize the ability of fusion control. Manifest strabismus cannot be controlled by the binocular fusion mechanism, while latent strabismus can maintain binocular single vision under the control of fusion reflex without eye deviation. However, when brain fusion is blocked or out of control, this potential eye deviation will appear. Therefore, based on the difference between manifest strabismus and latent strabismus, the above-mentioned second detection rule can be set to realize the display control of the display component, and then the data collection and processing of the eye movement direction vector can be realized under the display control to obtain the second eye movement information, and the eye deviation type of the detection object can be determined according to the second eye movement information.

[0057] In this embodiment, the type of eye deviation is identified, and the qualitative detection of eye deviation can be achieved in a standardized and unified process, thereby improving the accuracy and efficiency of eye deviation detection.

[0058] In one embodiment, the second detection rule is a cover-uncover detection rule; obtaining second eye movement information of at least one eye of the detection object, and determining the eye deviation type of the detection object based on the second eye movement information, includes: Cover the first eye of the detection object; based on the second eye movement information of the second eye, determine whether the type of eye deviation of the detection object is explicit strabismus, wherein the second eye movement information of the second eye is associated with the change between the mean value of the eye movement direction vector of the second eye in the eye movement platform period before and after covering; if not, uncover the first eye, and based on the second eye movement information of the first eye, determine whether the type of eye deviation of the detection object is latent strabismus, wherein the second eye movement information of the first eye is associated with the change between the mean value of the eye movement direction vector of the first eye in the eye movement platform period before and after uncovering; if it is impossible to determine whether the type of eye deviation of the detection object is latent strabismus based on the second eye movement information of the first eye, then cover the detection object. the second eye of the object; based on the second eye movement information of the first eye, judging whether the eye deviation type of the detection object is manifest strabismus, wherein the second eye movement information of the first eye is associated with the change between the mean values ​​of the eye movement direction vector of the first eye in the eye movement plateau period before and after covering; if after covering the second eye of the detection object, based on the second eye movement information of the first eye, it is impossible to determine whether the eye deviation type of the detection object is manifest strabismus, the second eye is uncovered, and based on the second eye movement information of the second eye, judging whether the eye deviation type of the detection object is latent strabismus, wherein the second eye movement information of the second eye is associated with the change between the mean values ​​of the eye movement direction vector of the second eye in the eye movement plateau period before and after uncovering.

[0059] Next, taking the use of a head mounted display device to display the detection object in split screen and using the cover and uncover rule as the second detection rule as an example, the process of determining the type of eye deviation is described: In the initial state, both the left and right screens are turned on, and the target object is located in the middle of the two screens respectively.

[0060] Calculate the average value of the right eye eye movement direction vector when the right eye is in the eye movement plateau period R 3. Turn off the left screen and calculate the average value of the right eye eye movement direction vector when the right eye is in the eye movement platform phase R 4 (During the detection, either the left eye or the right eye can be covered first. Here, the example of covering the left eye first is used for explanation).

[0061] calculate The absolute value of The absolute value of is the second eye movement information of the current right eye. If the absolute value of is greater than the preset threshold, it is manifest strabismus; Otherwise, turn on the left screen and calculate the average value of the left eye eye movement direction vector when the left eye is in the eye movement platform period L 4; Calculate the average value of the left eye eye movement direction vector when the left eye is in the eye movement platform period before the left screen is turned on (that is, when the left screen is turned off) L 3; calculate The absolute value of The absolute value of is the second eye movement information of the current left eye. like If the absolute value of is greater than the preset threshold, it is latent strabismus; like If the absolute value of is less than or equal to the preset threshold, the right screen is turned off and the average value of the left eye eye movement direction vector is calculated when the left eye is in the eye movement platform period. L 5; Before closing the right screen (that is, when the left screen in the previous step is turned on), the average value of the left eye eye movement direction vector when the left eye is in the eye movement platform period is L 4; calculate The absolute value of The absolute value of is the second eye movement information of the current left eye. If the absolute value of is greater than the preset threshold, it is manifest strabismus; like If the absolute value of is less than or equal to the preset threshold, the right screen is turned on and the average value of the right eye movement direction vector is calculated when the right eye is in the eye movement platform period. R 6; Calculate the average value of the right eye eye movement direction vector when the right eye is in the eye movement platform period before the right screen is turned on (that is, when the right screen is turned off) R 5; calculate The absolute value of The absolute value of is the second eye movement information of the current right eye. If the absolute value of is greater than the preset threshold, it is latent strabismus.

[0062] In particular, if the calculated If the value is less than or equal to the preset threshold, the suspected strabismus can be reported and a prompt message is generated to recommend follow-up with a strabismus and amblyopia specialist. At this time, if the subject does not cooperate well or there are very few special strabismuses, such as intermittent strabismus with excellent control, further confirmation by a physician is required.

[0063] In this embodiment, according to the second detection rule, when it is determined that the detection object has eye deviation, the type of eye deviation is further identified, thereby providing more accurate eye deviation detection.

[0064] In one embodiment, the first detection rule is an alternating cover rule. Based on the first detection rule, obtaining eye movement fluctuation information of one eye of the detection object includes: Based on the difference between the average values ​​of the eye movement direction vectors of an eye covered M times in each eye movement platform period, the eye movement fluctuation information of the eye is obtained; wherein M is greater than or equal to 2. In practical applications, M can be 3. Specifically, the process of obtaining the eye movement fluctuation information of each eye after M times of alternating cover can refer to the above-mentioned process of obtaining multiple fluctuation values, and calculating the variance of the fluctuation values ​​of the left eye for multiple times respectively. And the variance of the fluctuation value of the right eye multiple times The process will not be described in detail here.

[0065] In one embodiment, the first detection rule is an alternating cover rule. When performing strabismus detection based on the first detection rule, first eye movement information of one eye of the detection object includes: The absolute value of the average difference between the average values ​​of the eye movement direction vectors of an eye covered M times in each eye movement platform period is the first eye movement information of the eye; where M is greater than or equal to 2. In practical applications, M can be 3. The specific process of obtaining the first eye movement information of an eye after M times of alternating cover can refer to the above method for obtaining the average value. L a , R a The process will not be described in detail here.

[0066] Furthermore, in one embodiment, determining the eye movement plateau phase may include: The eye movement direction vector of the eye is collected, and an eye movement waveform is generated based on the eye movement direction vector; invalid eye movement signals in the eye movement waveform are removed to obtain a valid eye movement waveform; extreme points in the valid eye movement waveform are determined; and the eye movement platform period of the eye is determined based on the extreme points in the valid eye movement waveform.

[0067] In this embodiment, specifically: In the process of generating an eye movement waveform graph, the collected eye movement direction vector may be preprocessed first, and the preprocessing may include denoising and coordinate conversion. That is, the collected eye movement direction vector is denoised, and then the coordinate system of the denoised eye movement direction vector is converted to generate an eye movement waveform graph with the eye movement direction vector of the converted coordinate system. At this time, the horizontal coordinate of the eye movement waveform graph may be time or frame number; the vertical coordinate may be the Euler angle of the eye movement direction vector obtained after the coordinate conversion of the eye movement direction vector. It should be noted that in this embodiment, the eye movement direction vector converted to the Euler angle is used as the vertical axis of the eye movement waveform graph. In actual applications, the eye movement direction vector may be converted to the required coordinate system as needed to generate an eye movement waveform graph.

[0068] Next, invalid eye movement signals in the eye movement waveform are removed to obtain a valid eye movement waveform; wherein the invalid eye movement signals may include: eye movement direction vectors collected due to eye closure or nystagmus of the user. Specifically, the invalid eye movement signals may be removed by filtering the eye movement waveform. The above-mentioned determination of the extreme point of the valid eye movement waveform; the extreme point refers to the peak or trough of the valid eye movement waveform.

[0069] Finally, the eye movement platform period is determined based on the extreme points of the effective eye movement waveform. As an implementable method, the peak or trough of the effective eye movement waveform can be found first. After the peak or trough is determined, the flat curve composed of points near the peak or the points near the trough is confirmed as the curve where the eye movement platform period is located. The points near the peak or trough need to satisfy that the standard deviation of the eye movement direction vectors corresponding to these points is less than the preset value. That is, the peak or trough of the signal is judged by feature extraction of the eye movement signal, and then the corresponding platform period is determined according to the peak or trough. It should be noted that the way to determine the eye movement platform period can be diverse. For example, when determining the points near the peak or trough, it can also be determined based on probability expectation. Therefore, the above-mentioned way to determine the eye movement platform period should not be used as a limitation to the technical solution of the present invention.

[0070] In this embodiment, the eye fluctuation information and the eye movement information are determined based on the changes between the mean values ​​of the eye movement direction vectors in the eye movement plateau period, which can improve the accuracy of strabismus detection to a certain extent.

[0071] Furthermore, when removing invalid eye movement signals from the eye movement waveform (denoising the eye movement waveform), considering that eye movement waveforms vary greatly from person to person, it is difficult to use a fixed filter parameter to denoise eye movement waveforms of different people. Therefore, in order to achieve high robustness filtering of the eye movement waveform, this embodiment can filter the eye movement waveform by adaptive filtering to remove invalid eye movement signals from the eye movement waveform.

[0072] Specifically, during the filtering process, the filter parameters obtained at the previous moment can be used to achieve the filtering result after filtering, so as to adjust the filter parameters used for filtering at the current moment, so that the filtered eye movement waveform signal is adapted to the statistical characteristics of the noise position or the statistical characteristics that change over time, so as to achieve the purpose of optimal filtering of the eye movement waveform.

[0073] More specifically, Figure 3 FIG. 1 is a schematic diagram of an adaptive filtering of this embodiment. Figure 3 As shown, the eye movement waveform signal corresponding to the eye movement waveform diagram x ( n) as the input signal, and then input into the digital filter with adjustable parameters to generate the output signal after filtering with the current filter parameters. y ( n ). The output signal y ( n ) and the reference signal d ( n ) to obtain the error signal e( n ):

[0074]

[0075] The filter parameters are dynamically adjusted based on the adaptive algorithm until the error signal The root mean square of is the smallest, and the adjustment of the filter parameters can be expressed as:

[0076] in, x ( n ) is the input eye movement waveform signal, w ( n ) is the current filter parameter, y ( n ) is the output signal, d ( n ) is the reference signal, e( n ) is the error signal, w ( n+ 1) is the filter parameter for the next iteration, μ is the learning factor.

[0077] In this embodiment, according to the needs of the actual application scenario, time domain filtering or frequency domain filtering of the eye movement waveform can be selected to be combined with the above-mentioned adaptive filtering to achieve more flexible and efficient eye movement information extraction in a dynamic and complex signal environment. Among them, time domain filtering can be implemented based on moving average filtering, median filtering, etc. Taking moving average filtering as an example, the output feedback of the adaptive filter is used to adjust the window size or coefficient of the moving average filter. The moving average filter aims to smooth the time series data by calculating the sliding average of several data points that are continuous in time, so as to achieve the purpose of reducing random fluctuations. The specific working principle is: for the data point at the target moment, the data in the sample window of the preset time length at the target moment is taken to calculate the average value as the output value of the data point at the target moment. When the sample window moves with the time dimension, the average value is calculated as the output value of the data point at the new target moment according to the data falling into the sample window after the movement, until the smoothing of all data points in the time series is completed, and the final smoothed time series data is obtained. The mathematical expression of the moving average filter can be:

[0078] in, is the eye movement waveform signal input to the moving average filter, N is the sample window length of the moving average filter, which can determine the length of the data sequence used for averaging processing; n is the target time, and the output signal of the moving average filter at the target time is ; m The first m data points.

[0079] Among them, the moving average filter also includes: unweighted moving average filter and weighted moving average filter. Among them, the unweighted moving average filter assigns the same weight to all data in the sample window for averaging; the weighted moving average filter assigns different weights to different data points in the sample window, and makes the sum of the weights 1. In particular, the exponentially weighted moving average filter assigns higher weights to data points within the preset range of the eye movement waveform data points that currently need to be smoothed, and assigns lower weights to data points outside the preset range but also falling within the sample window, and the weight decreases exponentially with the degree of deviation from the data point that currently needs to be balanced over time.

[0080] In this embodiment, by adopting moving average filtering, it is possible to reduce measurement errors and noise components in the eye movement waveform diagram, thereby improving the accuracy of the final eye deviation detection.

[0081] An adaptive filter is designed in the frequency domain, and the gain of the frequency domain filter is adjusted to adapt to the spectral characteristics of the signal. The filter parameters are adjusted in the frequency domain using an adaptive algorithm. Exemplarily, frequency domain filtering can be implemented based on Fourier frequency domain filtering. The moving average filter implements smoothing filtering by convolving the data in the time domain. Time domain convolution is equivalent to frequency domain multiplication. Different from the above-mentioned moving average filter that performs convolution in the time domain, the purpose of Fourier frequency domain filtering is to obtain the eye movement waveform signal of a certain specific frequency band required by frequency domain filtering for different frequency characteristics of different eye movement waveform signals in frequency, such as obtaining the frequency interval of the eye movement waveform signal in the platform period. When using Fourier frequency domain filtering, a low-pass filter or a band-pass filter can be selected to implement frequency domain filtering according to the needs of the actual application scenario.

[0082] Among them, the low-pass filter only allows low-frequency signals to pass through and blocks high-frequency signals, so that by designing a low-pass filter, high-frequency noise with a frequency higher than a preset frequency in the eye movement waveform signal can be removed. Here, the noise formed by electronic noise or rapid tremor can be removed. The bandpass filter can allow eye movement waveform signals within a specific frequency range to pass. Therefore, low-frequency components with a frequency lower than a preset first frequency and high-frequency components with a frequency higher than a preset second frequency can be removed at the same time. For example, slow drift signals and rapid tremor signals in the eye movement waveform signal are filtered out based on a bandpass filter. Those skilled in the art should be aware that a low-pass filter or a bandpass filter can be selected according to the needs of the actual application scenario.

[0083] Next, we will take the Fourier frequency domain filtering with a low-pass filter as an example. x ( t ), the corresponding Fourier transform for:

[0084] in, f Represents the frequency of the eye movement waveform signal, in Hertz, in the Fourier transform formula t Represents the time dimension of the eye movement waveform signal.

[0085] Ideally, the frequency response of a low-pass filter is for:

[0086] in, is the cutoff frequency, which is determined according to the distribution of the expected signal and the eye movement noise signal in the frequency domain. The frequency response of the filter Multiply them to get the filtered signal The frequency representation is:

[0087] Afterwards, Perform inverse Fourier transform to obtain the filtered time domain output signal .because exist When is 0, the above Fourier inverse transform integral processing can be simplified to:

[0088] In particular, in practical application scenarios, the functional impulse response of an ideal low-pass filter will result in an infinite time delay. For this, approximate methods can be used, such as using a suitable window function to design an actual frequency domain filter, and an adaptive filter can adjust the frequency domain filter parameters in real time.

[0089] Based on this, Fourier frequency domain filtering can be used to filter out signals outside the desired frequency domain, retaining the eye movement waveform signal that effectively realizes eye deviation detection, thereby improving the accuracy of subsequent eye movement deviation detection. In addition, based on adaptive filtering, it is also possible to achieve adaptive digital filtering processing in a more robust manner for the differences in eye movement waveforms of different detection objects, thereby improving the accuracy of the filtering results and thus improving the accuracy of strabismus detection.

[0090] In addition to the above-mentioned adaptive filtering, the denoising of the eye movement waveform graph may also be achieved based on other filtering algorithms, such as the least mean square algorithm, the Kalman filter, etc. This embodiment does not limit the specific filtering method.

[0091] In one embodiment, a strabismus detection method is provided. After determining that the detected object has eye deviation by the above steps, the strabismus angle can be further determined. The strabismus detection method further includes the following steps: If the detection object has eye deviation; cover the first eye of the detection object; adjust the display position of the target object based on the moving direction and stride; uncover the first eye and cover the second eye of the detection object; determine whether the third eye movement information of the second eye is less than or equal to a third threshold; if so, determine the strabismus angle based on the current display position of the target object, the pupil distance of the detection object and the distance from both eyes to the target object; the third eye movement information of the second eye is associated with the change between the mean of the eye movement direction vector of the second eye in the eye movement platform period before and after the second eye is covered; if not, repeat the above steps until the third eye movement information of the second eye is less than or equal to the third threshold to determine the strabismus angle.

[0092] In this embodiment, the above-mentioned stride includes an initial stride and an adaptive stride; wherein the adaptive stride is associated with the third eye movement information corresponding to the second eye before and after each covering of the second eye; the moving direction includes an initial moving direction and a target moving direction, and the target moving direction is associated with the change between the mean values ​​of the eye movement direction vectors of the second eye in the eye movement platform period before and after each covering of the second eye.

[0093] It can be understood that, as in the aforementioned steps for determining whether there is strabismus, when detecting the strabismus angle, the display component is still used for split-screen display, that is, a target object is displayed in the middle of the left screen and the right screen. When the left and right screens are turned on at the same time, based on the display of the left and right screens, the target object will appear in front of the field of vision of both eyes. When detecting the strabismus angle, the healthy eye of the detection object can be covered as the first eye. In addition, the detection of the strabismus angle is usually performed alternately under the default state that the detection object's eyes are open, and the alternating covering can correspond to the closing of the left and right screens of the display device. For example: covering the right eye corresponds to closing the right screen, and covering the left eye corresponds to closing the left screen.

[0094] The following takes the left eye as the first eye as an example to illustrate the determination of the squint angle in this embodiment.

[0095] Cover the left eye of the object to be detected, that is, turn off the left screen and turn on the right screen. At this time, the target object on the right screen appears in the center of the right screen.

[0096] Calculate the average value of the right eye eye movement direction vector during the eye movement platform period when the left screen is turned off and the right screen is turned on .

[0097] Before opening the left screen, the position of the target on the left screen needs to be determined. In this embodiment, when the target on the left screen is moved for the first time, the initial moving direction and initial stride can be determined as follows: As for the initial moving direction, it can be determined according to whether the detected object is esotropy or exotropy. As can be seen from the above, whether the left eye is esotropy or exotropy is determined by L a The sign of the right eye determines whether it is esotropia or exotropia. R a For esotropia, the target object usually moves inward, that is, close to the nasal direction, and for exotropia, the target object usually moves outward, that is, close to the temporal direction.

[0098] For the initial stride It can be obtained by the following formula:

[0099] in, is the distance from the eye to the screen, PD is the pupil distance, is the angle of convergence (which can be calculated based on the pupil distance PD and The calculation is as follows:

[0100] Additionally, is the eye movement angle, which can be: the mean of the difference between the average values ​​of the eye movement direction vectors of the left eye covered M times in each eye movement platform period L a The corresponding angle (such as Euler angle) or the average value of the difference between the average values ​​of the eye movement direction vectors of the right eye covered M times in each eye movement platform period R a For this embodiment, since the left eye is used as the first eye for explanation, the angle corresponding to the left eye is for R a The corresponding angle.

[0101] In other embodiments, the initial stride may also be determined according to actual needs.

[0102] After knowing the initial stride and the initial moving direction, the target object located in the center of the left screen is adjusted to the corresponding position (display position) according to the initial stride and the initial moving direction. This position is the current display position of the target object when the left screen is turned on.

[0103] Turn on the left screen and turn off the right screen (at this time, the target object on the left screen has moved to the corresponding position). Calculate the average value of the right eye eye movement direction vector during the eye movement platform period after the right screen is turned off. .

[0104] calculate is less than or equal to a third threshold; if so, the display position of the target object on the left screen is determined according to the initial stride and the initial moving direction, and the strabismus angle is determined based on the current display position of the target object, the pupil distance of the detection object, and the distance from the eyes of the detection object to the target object.

[0105] Figure 4 is a schematic diagram of the squint angle calculation of the embodiment of the present application, see Figure 4 , Figure 4 The O in the figure refers to the center of the display screen. For the left screen, it is the center of the left screen, and for the right screen, it is the center of the right screen. The current display position of the target object refers to the position of the target object relative to the center point O. Squint angle Depend on and Specifically: squint angle It can be obtained by the following formula:

[0106] in, PD is the pupil distance, is the distance from the eye to the target, dt is the distance between the current position of the target and the center point O (that is, the absolute value of the current position coordinates of the target).

[0107] for For virtual space, the distance from the eye to the target is the distance from the eye to the virtual screen, because the target is displayed on the display screen.

[0108] like If the absolute value of (the third eye movement information of the right eye) is greater than the third threshold, the position of the target object on the left screen can be adjusted according to the adaptive stride and the target moving direction to ensure that the target object has moved to the corresponding position when the left screen is turned on next time.

[0109] In this embodiment, the adaptive stride can be output by PID, and the size of the adaptive stride is associated with the third eye movement information corresponding to the right eye before and after covering. In other embodiments, after the target object is initially moved by the initial stride and the initial moving direction, a fixed stride can be used to move the target object subsequently, and the size of the fixed stride can be determined according to actual experience.

[0110] The target moving direction can be based on In this embodiment, When it is positive, the target moves toward the temporal direction (away from the center point O). When it is negative, the target moves toward the nose (closer to the center point O).

[0111] because The absolute value of is greater than the third threshold, so the above alternating masking is repeated once more, that is: Turn off the left screen and turn on the right screen; calculate the average value of the right eye eye movement direction vector during the eye movement platform period after the left screen is turned off and the right screen is turned on .

[0112] Turn on the left screen, turn off the right screen, and calculate the average value of the right eye eye movement direction vector during the eye movement platform period after the left screen is turned on . (Before the left screen is turned on, the target object has been moved to the new display position based on the last display position, according to the adaptive stride and target movement direction).

[0113] calculate Whether the absolute value of (the third eye movement information of the second eye during this alternating covering process) is less than or equal to the third threshold, if so, the strabismus angle is determined according to the current display position of the target display object, the pupil distance of the detection object, and the distance from the eyes of the detection object to the target object (see the aforementioned strabismus angle calculation formula).

[0114] like If the absolute value of is still greater than the third threshold, the position to which the target object will move in the next alternating covering process is determined according to the adaptive stride and the target moving direction. Similarly, the alternation of "turning off the left screen and turning on the right screen; turning on the left screen and turning off the right screen" is continued until: The absolute value of is less than or equal to the third threshold (here n is greater than or equal to 1).

[0115] in, For the n When the left screen is turned on and the right screen is turned off during the alternating cover, the average value of the right eye eye movement vector during the eye movement platform period; The n When the left screen is turned off and the right screen is turned on during the alternating cover, the average value of the right eye eye movement direction vector during the eye movement platform period.

[0116] It should be noted that the above is an example of detecting the squint angle by alternately covering the left and right eyes in the initial state where both the left and right screens are turned on (which can be understood as the state where both eyes of the detection object are open). When performing alternate covering, the squint angle can also be calculated in the order of "close the right screen, turn on the left screen, turn on the right screen, and close the left screen". Therefore, the order of alternating covering should not be used as a limitation on the technical solution of the present invention. In addition, in practical applications, the squint angle detection method of this embodiment can also be used to test the squint angle multiple times, and the squint angles obtained by multiple measurements are statistically averaged, and the average value is used as the final squint angle.

[0117] In this solution, the inventors consider that the measurement of the strabismus angle can be abstracted as a gradual process from a certain initial point to a target point. The initial point can correspond to the third eye movement information when the detected object has eye deviation. The target point can be the third eye movement information when the eye returns to the normal position. or To determine. The measurement process of the strabismus angle can be: a gradual approach from the initial position (eye movement is at the strabismus position) to the target position (eye movement returns to the normal position). Based on this, the inventor proposed that a PID controller can be used to dynamically and adaptively adjust the stride of the target object, so that in the process of multiple alternating covers, the best time to stop the eye movement under alternating covers is found (that is, when the third eye movement information corresponding to the current alternating cover is less than or equal to the third threshold). The PID controller can quickly and accurately dynamically and adaptively adjust the stride of the target object's movement, and then the position of the target object in the process of strabismus angle detection can be adjusted according to the adaptive stride. When the third eye movement information is less than or equal to the third threshold during a certain transaction cover process, the strabismus angle of the detected object can be determined.

[0118] In this embodiment, the adaptive stride can be calculated by the following formula:

[0119] in, is the adaptive stride, , is the difference between the average value of the eye movement direction vector of the second eye in the eye movement plateau phase at the current moment and the average value of the eye movement direction vector of the second eye in the eye movement plateau phase at the initial moment; wherein the initial moment refers to the moment when the healthy eye of the test subject is covered for the first time during the strabismus angle test and the other eye is not covered and is in the upright position; Based on the above or To express; is the proportionality coefficient, is the integration coefficient, is the differential coefficient. t Refers to each iteration process, t control cycle.

[0120] During the squint angle detection process, if the left screen is turned off and the right screen is turned on, or the left screen is turned on and the right screen is turned off, the following is applied:

[0121] in, To conduct the n When the right eye is covered during the alternating cover (left screen is on, right screen is off), the average value of the right eye eye movement direction vector during the eye movement platform period is, for example, the corresponding value when the right eye is covered during the first alternating cover mentioned above. , the corresponding time when the right eye is covered during the second alternating cover .

[0122] During the squint angle detection process, if the right screen is turned off and the left screen is turned on, or the right screen is turned on and the left screen is turned off, the following is applied:

[0123] in, When the right screen is turned off for the first time and the left screen is turned on, the average value of the left eye eye movement direction vector during the eye movement platform period. To conduct the n When the left eye is covered during the alternating cover (right screen is on and left screen is off), the average value of the left eye eye movement direction vector during the eye movement platform period.

[0124] In this embodiment, , , In practical applications, the detection accuracy and detection time of the squint angle can be adjusted according to actual needs. , , A selection is made to dynamically output an adaptive stride, and then the position of the target object is adjusted based on the adaptive stride.

[0125] The following is a brief description of each step of the above adaptive stride calculation: The first link is the proportional link P (Proportion), and the component force generated by this link is: , Its size and error Directly proportional.

[0126] The second link is the differential link D (Differential), the component force generated by this link is: , and The change speed of the eye movement information is related to the speed of change. Adding the component force generated by the differential link on the basis of the first link will produce a damping effect, so that the change of the eye movement direction vector will always be subject to a "resistance". Therefore, in the process of approaching the target position, the amplitude of the left and right swings will gradually decrease, and finally converge to a position where the third eye movement information is less than or equal to the third threshold. Differential coefficient It can affect the size of this "resistance". In practical applications, the differential coefficient Increase it to speed up the convergence process.

[0127] The third link is the integral link I (Integral), and the component forces generated by this link are: , The component force generated by the integral link is proportional to The integral of When it persists, the integral force component will gradually increase in an attempt to eliminate the error. Adding the integral force component enables PID to accurately achieve accurate control in the presence of constant force interference. For example, under the premise of external interference such as hardware error, the error caused by external interference can be more accurately eliminated.

[0128] At this point, by using PID to output an adaptive step to continuously adjust the display position of the target object, when the third eye movement information is less than or equal to the third threshold, the strabismus angle can be determined based on the current display position of the target object, the pupil distance of the detection object, and the distance from the eyes of the detection object to the target object.

[0129] Figure 5 1 is a flow chart of a method for detecting eye deviation in some embodiments, and the method for detecting eye deviation can be applied to a head-mounted display device with split-screen display, and the head-mounted display device includes a display component. Figure 5 As shown, the eye deviation detection method comprises the following steps: Step S501, in the initial display state of the display component, the left screen and the right screen are alternately opened and closed, and the eye movement direction vector of each eye of the detection object is recorded. The initial display state of the display component may be: the target object is located in front of the detection object at a preset distance, and the left screen corresponding to the left eye of the detection object and the right screen corresponding to the right eye are both turned on.

[0130] Step S502, after the left and right screens are closed alternately for a preset number of times, it is determined whether the eyes of the detection object have reached a preset eye movement stable state; if so, step S503 is executed; otherwise, step S501 is returned to be executed. Specifically, it can be based on the eye movement fluctuation information of each eye of the detection object. Among them, the left screen and the right screen are closed alternately in a cycle, and the difference of the mean of the eye movement direction vector in the platform period corresponding to the opening and closing of the left screen or the right screen in each cycle is collected, and then the variance is calculated for the difference of the mean of the eye movement direction vector in multiple cycles (for example, the number of cycles is not less than the preset number M) (that is, the fluctuation value of the above multiple times), and when the variance is greater than or equal to the first threshold, the cycle is continued until the variance is less than the first threshold, and it is confirmed that the eye movement stable state is reached. After obtaining the above eye movement direction vector, the eye movement waveform of any eye can be obtained according to the eye movement direction vector, and the eye movement waveform is adaptively filtered and feature recognized, and then the difference of the mean of the eye movement direction vector corresponding to the opening and closing of the screen in the platform period is obtained. The specific data processing process can refer to the above embodiment, which will not be repeated.

[0131] Step S503, whether the first eye movement information of each eye of the detection object is greater than the second threshold; if so, execute steps S504 and S516 respectively, and it is confirmed that the detection object has eye deviation; otherwise, end the judgment process and confirm that the detection object does not have eye deviation. The process of calculating the first eye movement information can refer to the calculation of the above embodiment. , process.

[0132] Step S504: When both the left and right screens are turned on, based on the display of the left and right screens, the target object is placed in front of the field of view of the detection object, and the average value of the right eye movement direction vector when the right eye is in the eye movement platform phase is obtained. , turn off the left screen, and get the average value of the right eye eye movement direction vector when the right eye is in the eye movement platform period ; Among them, the left and right eyes and the left and right screens can be swapped.

[0133] Step S505, calculate The absolute value of is obtained to obtain the second eye movement information of the right eye. When the right screen is closed in step S504, the second eye movement information of the left eye is obtained by counting the eye movement direction vector of the left eye.

[0134] Step S506, determining whether the second eye movement information of step S505 is greater than a second threshold; if so, the determination process is terminated, and it can be determined that the detected object has manifest strabismus; otherwise, executing step S507.

[0135] Step S507, turning on the left screen; when the right screen is turned off in step S504, the right screen is turned on here.

[0136] Step S508: Obtain the second eye movement information of the left eye according to the eye movement direction vector of the left eye before and after the left screen is turned on. The specific calculation process can be obtained by referring to the above embodiment. When the right screen is closed in step S504, the second eye movement information of the right eye is obtained by statistics based on the eye movement direction vector of the right eye.

[0137] Step S509, determining whether the second eye movement information of step S508 is greater than a second threshold; if so, the determination process is terminated, and it can be determined that the detected object has latent strabismus; otherwise, executing step S510.

[0138] Step S510, close the right screen, and obtain the eye movement direction vector of the left eye when the left eye is in the eye movement platform phase; wherein, when step S504 is to close the right screen, here it is to close the left screen and obtain the eye movement direction vector of the right eye.

[0139] Step S511, according to the eye movement direction vector of the left eye in the eye movement platform period before and after closing the right screen, the second eye movement information of the left eye is obtained by statistics. The specific calculation process can refer to the calculation of the above embodiment When the right screen is closed in step S504, the second eye movement information of the right eye is obtained by statistics based on the eye movement direction vector of the right eye.

[0140] Step S512, determining whether the second eye movement information of step S511 is greater than a second threshold; if so, ending the determination process, and at this time it can be determined that the detected object has manifest strabismus; otherwise, executing step S513.

[0141] Step S513, opening the right screen; when the right screen is closed in step S504, the left screen is opened here.

[0142] Step S514, according to the eye movement direction vector of the right eye in the eye movement platform period before and after the right screen is turned on, the second eye movement information of the right eye is obtained by statistics. The specific calculation process can be obtained by referring to the above embodiment. When the right screen is closed in step S504, the second eye movement information of the left eye is obtained by statistics based on the eye movement direction vector of the left eye.

[0143] Step S515, determining whether the second eye movement information of step S514 is greater than the second threshold; if so, the determination process is terminated, and it can be determined that the detected object has latent strabismus; otherwise, returning to step S501 to retest the eye deviation.

[0144] Step S516, taking the alternating closing of the left and right screens as a loop iteration, adjusting the stride and moving direction of the target object on the display component according to the third eye movement information of the second eye in the previous iteration round. The healthy eye of the left and right eyes can be used as the first eye, and the other eye can be used as the second eye, and the first eye and the second eye can be alternately covered according to the opening and closing of the left and right screens. The specific alternating method and the stride adjustment method can refer to the above embodiments, which will not be repeated here. Among them, during the initial iteration, the initial stride can be set according to the first eye movement information of step S503, the interpupillary distance between the two eyes, and the distance from the eyes of the detection object to the target object, and the initial moving direction can be determined according to the situation of esotropia or exotropia.

[0145] Step S517, after controlling the target object to move according to the above-mentioned stride and moving direction, the left and right screens are alternately closed.

[0146] Step S518, obtaining the eye movement direction vector of the second eye, and determining the third eye movement information of the second eye. The process of calculating the third eye movement information of the second eye can refer to the calculation in the above embodiment. process.

[0147] Step S519, determining whether the third eye movement information of the second eye in the current iteration round is less than or equal to the third threshold; if so, executing step S520; otherwise, returning to executing step S516 in the next iteration round.

[0148] Step S520, determining the squint angle. Specifically, the squint angle may be determined by referring to the squint angle calculation formula in the above embodiment.

[0149] It should be noted that the steps shown in the above process or the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a different order than here, for example, steps S516 to S520 and steps S504 to S515.

[0150] Above-mentioned steps S501 to S520 realize the qualitative and quantitative detection process of fully automatic eye deviation based on split screen display and eye tracking.Wherein, by tracking the eye movement of the detection object, the screen opening and closing of the split screen display and the position change of the target object are controlled to determine whether the detection object has eye deviation, esotropia or exotropia, explicit strabismus or latent strabismus.If there is eye deviation, the quantitative measurement result of the deviation angle is provided.By controlling the position of the target object on the screen of the left and right eyes and the opening and closing of the screen based on the eye movement principle of eye deviation, the eye movement situation is monitored, and the detection object is automatically subjected to eye deviation detection.In the data processing process of the specific detection of eye deviation angle, the eye movement direction vector of the left and right eye split screen display is obtained and recorded in real time, the left and right eye movement waveforms are generated by digital filtering of the eye movement direction vector, and the eye movement features are extracted by the waveform to determine whether a new round of split screen display or termination of inspection is needed. Specifically, this embodiment introduces a step-by-step algorithm, that is, the initial movement stride and movement direction are set by the average eye movement information of the left and right eyes collected in the previous step. When the target object moves, the movement stride and movement direction of the next iteration are dynamically adjusted according to the average eye movement information recorded by the split-screen display in the new iteration, so as to quickly reach convergence and output the eye deviation angle.

[0151] In this embodiment, a strabismus detection device is also provided, which is used to implement the above-mentioned embodiments and preferred implementation modes, and the descriptions that have been made will not be repeated. The terms "module", "unit", "subunit" and the like used below can implement a combination of software and / or hardware of predetermined functions. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0152] Figure 6 is a structural block diagram of the strabismus detection device 60 of this embodiment, as shown in Figure 6 As shown, the strabismus detection device 60 includes: a fluctuation information acquisition module 62, a first eye movement information acquisition module 64 and a detection module 66; wherein: The fluctuation information acquisition module 62 is used to obtain the eye movement fluctuation information of each eye of the detection object based on the first detection rule; the first eye movement information acquisition module 64 is used to obtain the first eye movement information of each eye of the detection object when strabismus detection is performed based on the first detection rule when the eye movement fluctuation information of each eye of the detection object is lower than the first threshold; wherein the eye movement fluctuation information and the first eye movement information are associated with the changes between the mean values ​​of the eye movement direction vectors of the eyes in each eye movement platform period when strabismus detection is performed based on the first detection rule; the detection module 66 is used to determine that the detection object has eye deviation when the first eye movement information of any eye is greater than the second threshold.

[0153] It should be noted that the above modules can be functional modules or program modules, and can be implemented by software or hardware. For modules implemented by hardware, the above modules can be located in the same processor; or the above modules can be located in different processors in any combination.

[0154] It should be noted that the specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementation modes, and will not be repeated in this embodiment.

[0155] In this embodiment, a wearable device is also provided. Figure 7 is a schematic diagram of the structure of the wearable device 70 of this embodiment, as shown in Figure 7 As shown, the wearable device 70 may include a display component 72; the display component 72 is used to display the target object according to the received display control signal; the display control signal is generated according to the eye movement direction vector of the detection object when observing the target object and the eye deviation detection rule. It can be understood that the strabismus detection result can be displayed on the same display component, or on a terminal display associated with the wearable device, such as a computer, a mobile phone, or another display of the wearable device.

[0156] It should be noted that the specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementation modes, and will not be repeated in this embodiment.

[0157] In particular, in one embodiment, the wearable device 70 may further include a data acquisition component 74, which is used to acquire the eye movement direction vector of the detection subject when observing the target object. The data acquisition component 74 may be an eye tracking detector.

[0158] In addition, in one embodiment, the wearable device 70 further includes a strabismus detection processor 76, which is communicatively connected to the display component 72 and the data acquisition component 74; the strabismus detection processor 76 is used to receive the eye movement direction vector collected by the data acquisition component 74, and send a display control signal to the display component 72; and, send the calculated eye deviation angle to the display component 72.

[0159] In some of the embodiments, the wearable device 70 may be head-mounted.

[0160] In addition, the settings of various thresholds, expected signals, and the duration of an alternating mask involved in the above embodiments can all be obtained based on clinical experience or actual measured data obtained from multiple experiments.

[0161] It should be understood that the specific embodiments described herein are only used to explain the application, rather than to limit it. Based on the embodiments provided in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the protection scope of this application.

[0162] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0163] Obviously, the drawings are only some examples or embodiments of the present application. For ordinary technicians in the field, the present application can also be applied to other similar situations based on these drawings without creative work. In addition, it is understandable that although the work done in this development process may be complicated and lengthy, for ordinary technicians in the field, certain changes in design, manufacturing or production based on the technical content disclosed in this application are only conventional technical means and should not be regarded as insufficient content disclosed in this application.

[0164] The term "embodiment" in this application refers to a specific feature, structure or characteristic described in conjunction with the embodiment that can be included in at least one embodiment of the present application. The appearance of this phrase in various locations in the specification does not necessarily mean the same embodiment, nor does it mean that it is mutually exclusive with other embodiments and is independent or optional. It is clearly or implicitly understood by those of ordinary skill in the art that the embodiments described in this application can be combined with other embodiments without conflict.

[0165] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of patent protection. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the scope of protection of the present application. Therefore, the scope of protection of the present application shall be subject to the attached claims.

Claims

1. A strabismus detection method, characterized in that: include: Based on the first detection rule, obtaining eye movement fluctuation information of each eye of the detection object; When the eye movement fluctuation information of each eye of the detection object is lower than a first threshold, obtaining first eye movement information of each eye of the detection object when performing strabismus detection based on the first detection rule; The eye movement fluctuation information and the first eye movement information are associated with changes between mean values ​​of eye movement direction vectors of the eyes in each eye movement platform period when strabismus detection is performed based on the first detection rule; When the first eye movement information of any eye is greater than a second threshold, it is determined that the detected object has eye deviation; wherein: determining the eye movement platform phase includes: Collecting an eye movement direction vector of an eye, and generating an eye movement waveform diagram based on the eye movement direction vector; An effective eye movement waveform in the eye movement waveform is determined, and an eye movement platform period of the eye is determined according to an extreme point in the effective eye movement waveform.

2. The strabismus detection method according to claim 1, characterized in that: Also includes: Based on the second detection rule, obtaining second eye movement information of at least one eye of the detection object, and determining the eye deviation type of the detection object based on the second eye movement information; The second eye movement information is associated with changes between mean values ​​of eye movement direction vectors of the eye in each eye movement platform period when strabismus type detection is performed based on the second detection rule.

3. The strabismus detection method according to claim 2, characterized in that: The second detection rule is a cover-uncover detection rule; the step of obtaining second eye movement information of at least one eye of the detection object and determining the eye deviation type of the detection object based on the second eye movement information includes: covering the first eye of the detection object; Based on the second eye movement information of the second eye, determining whether the eye deviation type of the detection object is manifest strabismus, wherein the second eye movement information of the second eye is associated with the change between the mean value of the eye movement direction vector of the second eye in the eye movement platform period before and after covering; If not, uncover the first eye, and determine whether the eye deviation type of the detection object is latent strabismus based on the second eye movement information of the first eye, wherein the second eye movement information of the first eye is associated with the change between the mean values ​​of the eye movement direction vectors of the first eye in the eye movement platform period before and after uncovering; If it is not possible to determine whether the eye deviation type of the detection object is latent strabismus based on the second eye movement information of the first eye, covering the second eye of the detection object; Based on the second eye movement information of the first eye, determining whether the eye deviation type of the detection object is manifest strabismus, wherein the second eye movement information of the first eye is associated with the change between the mean values ​​of the eye movement direction vector of the first eye in the eye movement platform period before and after covering; If after covering the second eye of the detection object, it is impossible to determine whether the eye deviation type of the detection object is manifest strabismus based on the second eye movement information of the first eye, the second eye is uncovered, and based on the second eye movement information of the second eye, it is determined whether the eye deviation type of the detection object is latent strabismus, wherein the second eye movement information of the second eye is associated with the change between the mean values ​​of the eye movement direction vectors of the second eye in the eye movement platform period before and after uncovering.

4. The strabismus detection method according to claim 1, characterized in that: The first detection rule is an alternating cover rule. Based on the first detection rule, obtaining eye movement fluctuation information of one eye of the detection object includes: The eye movement fluctuation information of an eye is obtained based on the difference between the average values ​​of the eye movement direction vectors of an eye that is alternately covered M times in each eye movement plateau period; wherein M is greater than or equal to 2.

5. The strabismus detection method according to claim 1, characterized in that: The first detection rule is an alternating cover rule. When strabismus detection is performed based on the first detection rule, first eye movement information of one eye of the detection object includes: The absolute value of the average of the differences between the average values ​​of the eye movement direction vectors of an eye that is alternately covered M times in each eye movement plateau period is the first eye movement information of the eye; wherein M is greater than or equal to 2.

6. The strabismus detection method according to claim 1, characterized in that: Determining a valid eye movement waveform in the eye movement waveform includes: The eye movement waveform is filtered to remove invalid eye movement signals in the eye movement waveform to obtain the valid eye movement waveform.

7. The strabismus detection method according to any one of claims 1 to 6, characterized in that: The method further comprises: If the detected subject has eye deviation; covering the first eye of the detection object; Adjust the display position of the target based on the movement direction and stride; Uncovering the first eye and covering the second eye of the detection object; Determine whether the third eye movement information of the second eye is less than or equal to a third threshold; if so, determine the strabismus angle based on the current display position of the target object, the pupil distance of the detection object, and the distance from the eyes of the detection object to the target object; the third eye movement information of the second eye is associated with the change between the mean values ​​of the eye movement direction vectors of the second eye in the eye movement platform period before and after the second eye is covered; if not, repeat the above steps until the third eye movement information of the second eye is less than or equal to the third threshold to determine the strabismus angle.

8. The strabismus detection method according to claim 7, characterized in that: The stride includes an initial stride and an adaptive stride; wherein the adaptive stride is associated with the third eye movement information corresponding to the second eye before and after each covering of the second eye; The moving direction includes an initial moving direction and a target moving direction, and the target moving direction is associated with a change between the mean values ​​of the eye movement direction vectors of the second eye in the eye movement platform period before and after each covering of the second eye.

9. A strabismus detection device, characterized in that: include: A wave information acquisition module, a first eye movement information acquisition module and a detection module; wherein: The fluctuation information acquisition module is used to acquire eye movement fluctuation information of each eye of the detection object based on the first detection rule; The first eye movement information acquisition module is used to acquire first eye movement information of each eye of the detection object when strabismus detection is performed based on the first detection rule when the eye movement fluctuation information of each eye of the detection object is lower than a first threshold; wherein the eye movement fluctuation information and the first eye movement information are associated with changes between mean values ​​of eye movement direction vectors of the eyes in each eye movement platform period when strabismus detection is performed based on the first detection rule; The detection module is used to determine that the detected object has eye deviation when the first eye movement information of any eye is greater than a second threshold; wherein: determining the eye movement platform phase includes: Collecting an eye movement direction vector of an eye, and generating an eye movement waveform diagram based on the eye movement direction vector; An effective eye movement waveform in the eye movement waveform is determined, and an eye movement platform period of the eye is determined according to an extreme point in the effective eye movement waveform.

10. A wearable device, characterized in that: The strabismus detection method according to any one of claims 1 to 8 is applied, wherein the wearable device comprises a display component; The display component is used to display the target object according to the received display control signal; the display control signal is generated according to the eye movement direction vector and the eye deviation detection rule when the detection object observes the target object.

Images