Head-mounted device and strabismus detection system

Abstract

The present disclosure relates to a head-mounted device and a strabismus detection system, wherein the head-mounted device comprises: a display module configured to sequentially display a set of visual markers in a plurality of set display regions; an image acquisition module configured to acquire a set of eye images corresponding to each eyeball, the eye images being images of a user wearing the head-mounted device when viewing the set of visual markers displayed in the corresponding display region; a gaze position vector determination module configured to determine a corresponding actual gaze position vector based on each eye image; an offset and angle determination module configured to obtain a target point gaze vector corresponding to each eyeball, determine a corresponding eyeball offset and a first angle based on each actual gaze position vector and the corresponding target point gaze vector; and a strabismus determination module configured to determine that the corresponding eyeball has strabismus when viewing the set of visual markers displayed in the corresponding display region, if the corresponding eyeball offset is greater than a first preset threshold and / or the first angle is greater than a first angle threshold.

Description

Technical Field

[0001] This disclosure relates to strabismus detection technology, and more specifically, to a head-mounted device and strabismus detection system. Background Technology

[0002] Strabismus is a common eye disease, especially in children. Its main symptom is that the visual axes of both eyes are not parallel, making it impossible to simultaneously focus on the same target. Strabismus is not always obvious. Besides the readily visible, constant strong strabismus, there are many other types that are difficult to detect, such as: intermittent strabismus: which only appears occasionally when the child is tired, distracted, or looking at distant objects, and is easily missed by parents during brief observations; microscopic strabismus: the deviation angle is very small (less than 10 prism diopters), and the abnormal eye position is almost invisible, but it severely affects binocular vision and requires professional examination to detect; accommodative esotropia: usually occurs in young children (2-3 years old) with moderate to high hyperopia, caused by excessive use of accommodation. Initially, it may be intermittent and easily overlooked, thus missing the optimal intervention time.

[0003] Timely strabismus screening and diagnosis are crucial for preventing serious consequences such as amblyopia and loss of stereoscopic vision. Currently, commonly used clinical methods for strabismus examination include corneal reflex test, cover test, prism-cover test, and synoptophore test. However, traditional strabismus examination is a subjective examination that heavily relies on doctor-patient cooperation, making it particularly difficult for children in several ways: the examination process is complex, requiring a combination of alternating cover tests, corneal reflex test, prism test, and synoptophore test; the process is lengthy, the environment is unfamiliar, and it can easily cause fear and resistance in children; it depends on the child's cooperation, requiring the child to continuously focus on a target (visual object) and accurately report the image seen; young children have short attention spans and poor cooperation, often leading to incomplete examinations or inaccurate results; and the medical terminology is difficult to understand, as the examination results are mostly professional assessment reports given by doctors, which are difficult for parents to understand and for them to directly perceive their child's condition. Summary of the Invention

[0004] One object of the present invention is to provide a new technical solution for head-mounted devices.

[0005] According to a first aspect of the present invention, a head-mounted device is provided, comprising: The display module is used to sequentially display designated visual markers in multiple designated display areas; The image acquisition module is used to acquire eye images corresponding to each eyeball. The eye images are images of a user wearing the head-mounted device viewing a set visual marker displayed in the corresponding display area. The gaze position vector determination module is used to determine the corresponding actual gaze position vector based on each eye image; The offset and angle determination module is used to obtain the target point gaze vector corresponding to each eyeball, and determine the corresponding eyeball offset and first angle based on each actual gaze position vector and the corresponding target point gaze vector. The target point gaze vector is the gaze position vector of the eyeball without strabismus viewing the set visual mark displayed in the corresponding display area, and the first angle is the angle between each actual gaze position vector and the corresponding target point gaze vector. The strabismus determination module is used to determine that the corresponding eyeball is strabismus when viewing the set visual mark displayed in the corresponding display area if the corresponding eyeball offset is greater than a first preset threshold and / or the first included angle is greater than a first angle threshold.

[0006] Optionally, the gaze position vector determination module is used to determine the corresponding quaternion information based on each eye image. Each quaternion information is the three-dimensional rotation state of the eyeball in the corresponding eye image relative to a set position. The set position is the position of the eyeball without strabismus when viewing the set visual mark displayed in the display area directly in front of it. Obtain a set gaze position vector, wherein the set gaze position vector is the gaze position vector in a virtual scene when the non-squinting eye looks at a set visual marker displayed in the display area directly in front of it; Based on each quaternion and the set gaze position vector, the corresponding actual gaze position vector in the virtual scene is determined.

[0007] Optionally, the offset and angle determination module is used to determine the length of each actual gaze position vector and the length of the corresponding target point gaze vector; The corresponding eye offset is determined based on the length of each actual gaze position vector and the length of the corresponding target point gaze vector.

[0008] Optionally, if it is determined that the user is strabismic when viewing the set visual mark displayed in the corresponding display area, the offset and angle determination module is used to determine the first offset obtained by projecting each actual gaze position vector along the set direction, and to determine the second offset obtained by projecting the gaze vector of the corresponding target point along the set direction. The set direction is the direction of the line of sight when the eye without strabismic gazes at the set visual mark displayed in the display area directly in front of it. A first horizontal offset and a first vertical offset are determined based on a first offset, and a second horizontal offset and a second vertical offset are determined based on a second offset. Based on each actual gaze position vector and the corresponding target point gaze vector, the horizontal offset difference is determined according to the corresponding first horizontal offset and the corresponding second horizontal offset, and the vertical offset difference is determined according to the corresponding first vertical offset and the corresponding second vertical offset, wherein... The strabismus determination module is used to determine that the corresponding eyeball has horizontal strabismus when viewing the set visual mark displayed in the corresponding display area if the difference in horizontal offset is greater than a second preset threshold, and to determine that the corresponding eyeball has vertical strabismus when viewing the set visual mark displayed in the corresponding display area if the difference in vertical offset is greater than a third preset threshold.

[0009] Optionally, if it is determined that the user is strabismus when viewing the set visual marker displayed in the corresponding display area, the offset and angle determination module is used to obtain the actual gaze position vector corresponding to the left eye and the actual gaze position vector corresponding to the right eye. Based on the actual gaze position vector corresponding to the left eyeball and the actual gaze position vector corresponding to the right eyeball, the offset and second angle between the two eyeballs are determined, wherein the second angle is the angle between the first offset corresponding to the left eyeball and the first offset corresponding to the right eyeball; The strabismus determination module is used to determine that the left and right eyes have non-common strabismus when viewing the set visual markers displayed in the corresponding display area if the offset between the two eyeballs is greater than a fourth preset threshold and / or the second included angle is greater than a second angle threshold.

[0010] Optionally, the plurality of set display areas are nine-grid display areas, and each grid display area displays set visual markers in a set order.

[0011] Optionally, the head-mounted device further includes a transmitting module for sending user identification information and corresponding strabismus result information to the server, wherein, The strabismus result information includes the strabismus result when the user views the set visual markers displayed in the corresponding display area.

[0012] According to a second aspect of the present invention, a strabismus detection system is provided, comprising a head-mounted device as described in any of the first aspects and a server, wherein, The head-mounted device is used to send user identification information and corresponding strabismus result information to the server, wherein the strabismus result information includes the strabismus result when the user views the set visual markers displayed in the corresponding display area; The server is used to store the received user identification information and corresponding strabismus result information.

[0013] Optionally, the system further includes a terminal device, wherein, The terminal device is used to send a visual report retrieval request to the server, wherein the retrieval request carries user identification information; The server is used to obtain the corresponding strabismus result information according to the acquisition request, generate a visualization report according to the user identification information and the corresponding strabismus result information, and send the visualization report to the corresponding terminal device; The terminal device is used to display the received visual report.

[0014] Optionally, the strabismus result information also includes a first offset obtained by projecting each actual gaze position vector along a set direction, where the set direction is the gaze direction of an eye without strabismus when viewing a set visual marker displayed in a display area directly in front of it. The server is used to generate left-eye and right-eye images corresponding to each display area based on a first offset obtained by projecting each actual gaze position vector along a set direction. The left-eye image includes a first marker, which is the center point of the left eye when viewing the set visual marker displayed in the display area directly in front of the user without strabismus. The right-eye image includes a second marker, which is the center point of the right eye when viewing the set visual marker displayed in the display area directly in front of the user without strabismus. A visualization report is generated based on user identification information, corresponding strabismus result information, and left-eye and right-eye images corresponding to each display area.

[0015] The head-mounted device disclosed herein acquires eye images corresponding to each eyeball by sequentially displaying set visual markers in multiple designated display areas. The eye images are the eye images corresponding to the user wearing the head-mounted device when viewing the set visual markers displayed in each display area. Based on each eye image, the corresponding actual gaze position vector is determined, and the target point gaze vector corresponding to each display area is obtained. Based on each actual gaze position vector and the corresponding target point gaze vector, the corresponding eyeball offset is determined. This can solve the pain points of traditional medical strabismus detection, such as high professional requirements, complex operation, and unfriendly interface. By guiding the user to gaze at set visual markers in different directions in a virtual scene to complete data collection, the tedious examination is transformed into an immersive game experience, which can greatly attract the attention of users, especially children, and make them actively cooperate in a relaxed state, ensuring the smooth progress of screening and the validity of data. It simplifies the complex examination process, eliminates the need for professional physicians to operate throughout the process, significantly reduces the screening threshold and time cost, and makes it possible to conduct large-scale, low-cost preliminary screening in communities, schools, and homes, greatly improving the universality and accessibility of strabismus screening.

[0016] The features and advantages of the embodiments of this specification will become clear from the following detailed description of exemplary embodiments with reference to the accompanying drawings. Attached Figure Description

[0017] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments of this specification and, together with their description, serve to explain the principles of these embodiments.

[0018] Figure 1 This is a schematic block diagram of a head-mounted device according to an embodiment of the present invention.

[0019] Figure 2 This is a schematic diagram of the structure of a head-mounted device according to an embodiment of the present invention.

[0020] Figure 3 This is a schematic diagram of a strabismus detection system according to an embodiment of the present invention.

[0021] Figure 4 This is a schematic diagram of a strabismus detection system according to another embodiment of the present invention.

[0022] Figure 5 This is a partial schematic diagram of a visualization report according to an embodiment of the present invention.

[0023] Figure 6 This is a schematic diagram of a strabismus detection process according to an embodiment of the present invention. Detailed Implementation

[0024] Various exemplary embodiments of this specification will now be described in detail with reference to the accompanying drawings.

[0025] The following description of at least one exemplary embodiment is merely illustrative and is in no way intended to limit the embodiments of this specification or their application or use.

[0026] It should be noted that similar labels and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be discussed further in subsequent figures.

[0027] To address the aforementioned technical issues, this disclosure provides a head-mounted device that sequentially displays predefined visual markers in multiple designated display areas, acquiring eye images corresponding to each eyeball. These eye images are those of the user viewing the predefined visual markers displayed in each area while wearing the head-mounted device. Based on each eye image, a corresponding actual gaze position vector is determined, and a target point gaze vector corresponding to each display area is obtained. The corresponding eyeball offset is determined based on each actual gaze position vector and the corresponding target point gaze vector. This addresses the pain points of traditional medical strabismus detection, such as high professional requirements, complex operation, and unfriendly interface. By guiding the user to gaze at predefined visual markers in different directions within a virtual scene to complete data collection, the tedious examination is transformed into an immersive gaming experience, greatly attracting the attention of users, especially children, and encouraging their active cooperation in a relaxed state. This ensures the smooth progress of the screening and the validity of the data, simplifying the complex examination process. It eliminates the need for a professional physician to operate the entire process, significantly reducing the screening threshold and time costs. This makes large-scale, low-cost preliminary screening possible in communities, schools, and homes, greatly improving the universality and accessibility of strabismus screening.

[0028] In one embodiment of the present invention, a head-mounted device is provided. Figure 1 As shown, the head-mounted device includes a display module 110, an image acquisition module 120, a gaze position vector determination module 130, an offset and angle determination module 140, and a strabismus determination module 150.

[0029] The display module 110 is used to sequentially display set visual markers in multiple set display areas.

[0030] In some embodiments, multiple display areas are nine-grid display areas, and each grid display area displays a set visual marker in a set order.

[0031] Define visual markers as visual stimuli to guide the user's gaze, such as a glowing dot or a glowing pattern.

[0032] according to Figure 2 As shown, the display area of ​​the head-mounted device's display module is divided into a nine-grid display area. Figure 2 The numbers shown indicate the order in which the visual markers are displayed in each grid area, that is, in ascending order.

[0033] From a physiological perspective, the layout of the nine-grid display area simulates the nine key directions the eyeball can turn in under the coordinated action of the six extraocular muscles (medial rectus, lateral rectus, superior rectus, inferior rectus, superior oblique, and inferior oblique muscles in each eye). By having the user sequentially look at the set visual markers displayed in these nine areas, the function of each muscle can be systematically assessed as normal, overactive (excessive), or underactive (insufficient).

[0034] By setting up a nine-grid display area, multiple discrete fixation points can be provided, allowing for a systematic examination of the user's eye position in different fixation directions, providing rich, comprehensive, and detailed data for the subsequent determination of strabismus results.

[0035] The image acquisition module 120 is used to acquire eye images corresponding to each eyeball. The eye images are the eye images corresponding to the set visual markers displayed in the corresponding display area when a user wearing a head-mounted device views the eye images.

[0036] The user wearing the headset keeps their head still. The user's eyes move in accordance with changes in the display area where the set visual markers appear.

[0037] The image acquisition module is a camera built into the head-mounted device. There can be one or two cameras.

[0038] With a built-in camera in the head-mounted device, the camera can capture images of the left eyeball alone to obtain an image of the eye corresponding to the left eyeball, and it can also capture images of the right eyeball alone to obtain an image of the eye corresponding to the right eyeball. Alternatively, the camera can capture images of both the left and right eyeballs simultaneously to obtain an image of the eye that includes both the left and right eyeballs.

[0039] With two cameras built into the head-mounted device, one camera is used to capture the left eyeball to obtain the corresponding eye image, and the other camera is used to capture the right eyeball to obtain the corresponding eye image.

[0040] The gaze position vector determination module 130 is used to determine the corresponding actual gaze position vector based on each eye image.

[0041] In some embodiments, the gaze position vector determination module is used to determine corresponding quaternion information based on each eye image, where each quaternion information is the three-dimensional rotation state of the eyeball relative to a set position in the corresponding eye image, and the set position is the position of the eyeball without strabismus when viewing a set visual marker displayed in a display area directly in front of it; obtain a set gaze position vector, wherein the set gaze position vector is the gaze position vector in a virtual scene when the eyeball without strabismus views the set visual marker displayed in a display area directly in front of it; and determine the corresponding actual gaze position vector in the virtual scene based on each quaternion information and the set gaze position vector.

[0042] Each quaternion is represented by (x, y, z, w).

[0043] In some embodiments, corresponding quaternion information is determined based on each eye image. The type of the quaternion information is a custom type of the head-mounted device. The type of the quaternion information needs to be converted to a standard type, i.e., a type defined by the Unity engine, in order to prepare for subsequent data processing of the head-mounted device.

[0044] Based on the following calculation formula, the type of quaternion information is converted.

[0045] Here, `eyePose.orientation` represents the quaternion information before conversion, and its type is a custom quaternion information type determined based on the type of the head-mounted device. `quaternion` represents the quaternion information after conversion, and its type is a Unity engine-defined type (`UnityEngine.Quaternion`). `ToQuat()` is the function corresponding to the conversion between the two types. This conversion facilitates subsequent calculations using Unity. Specifically, public UnityEngine.Quaternion ToQuat(){ return new UnityEngine.Quaternion() { x = x, y = y, z = z, w = w}; } The gaze position vector is set as a vector within a 3D coordinate system of the virtual scene. This 3D coordinate system is established based on the user's viewpoint. The positive X-axis points directly to the user's right, the positive Y-axis points directly above the user, and the positive Z-axis points directly in front of the user. The gaze position vector is a pre-stored vector that can be directly obtained.

[0046] Based on the following calculation formula, the actual gaze position vector corresponding to each quaternion and the set gaze position vector is determined in the virtual scene.

[0047] Where V is the vector representing the actual gaze position in the virtual scene. For a quaternion, To set the gaze position vector.

[0048] The offset and angle determination module 140 is used to obtain the target point gaze vector corresponding to each eyeball, and determine the corresponding eyeball offset and first angle based on each actual gaze position vector and the corresponding target point gaze vector. The target point gaze vector is the gaze position vector of the eyeball without strabismus viewing the set visual mark displayed in the corresponding display area, and the first angle is the angle between each actual gaze position vector and the corresponding target point gaze vector.

[0049] Based on the set visual markers displayed in each display area, the target point gaze vector includes the target point gaze vector corresponding to the left eye and the target point gaze vector corresponding to the right eye.

[0050] In some embodiments, the offset and angle determination module 140 is used to determine the length of each actual gaze position vector and the length of the corresponding target point gaze vector; and to determine the corresponding eyeball offset based on the length of each actual gaze position vector and the length of the corresponding target point gaze vector.

[0051] The length of each actual gaze position vector and the length of the corresponding target point gaze vector are both the magnitude of the vector. For details, please refer to the calculation process of the vector magnitude.

[0052] In some embodiments, the offset and angle determination module 140 is used to determine a first offset obtained by projecting each actual gaze position vector along a set direction, and to determine a second offset obtained by projecting the corresponding target point gaze vector along a set direction. The set direction is the direction of gaze when an eye without strabismus views a set visual mark displayed in the display area directly in front of it. Based on each actual gaze position vector and the corresponding target point gaze vector, the difference between the corresponding first offset and the corresponding second offset is determined as the corresponding eyeball offset.

[0053] Along a predetermined direction, a first offset is obtained by projecting each actual gaze position vector, and a second offset is obtained by projecting the corresponding target point gaze vector. The first offset and the second offset are two different vectors. Based on the length values ​​of these two different vectors, the length difference is determined as the corresponding eyeball offset.

[0054] The strabismus determination module 150 is used to determine that the corresponding eyeball is strabismus when viewing the set visual mark displayed in the corresponding display area if the corresponding eyeball offset is greater than a first preset threshold and / or the first included angle is greater than a first angle threshold.

[0055] If the corresponding eyeball offset is less than or equal to a first preset threshold and the first included angle is less than or equal to a first angle threshold, it is determined that the user does not have strabismus when viewing the set visual markers displayed in the corresponding display area.

[0056] This allows us to obtain the strabismus determination results when setting visual markers for each display area, making the strabismus determination results more comprehensive and detailed.

[0057] In some embodiments, when it is determined that the user is squinting while viewing the designated visual marker displayed in the corresponding display area, the offset and angle determination module is used to determine a first offset obtained by projecting each actual gaze position vector along a set direction, and to determine a second offset obtained by projecting the corresponding target point gaze vector along the set direction, wherein the set direction is the direction of the line of sight when the eye without squinting views the designated visual marker displayed in the display area directly in front of it; a first horizontal offset and a first vertical offset are determined based on the first offset, and a second horizontal offset and a second vertical offset are determined based on the second offset; based on each actual gaze position vector and the corresponding target point gaze vector, a horizontal offset difference is determined based on the corresponding first horizontal offset and the corresponding second horizontal offset, and a vertical offset difference is determined based on the corresponding first vertical offset and the corresponding second vertical offset.

[0058] In this embodiment, the strabismus determination module is used to determine that the corresponding eyeball has horizontal strabismus when viewing the set visual mark displayed in the corresponding display area if the horizontal offset difference is greater than a second preset threshold, and to determine that the corresponding eyeball has vertical strabismus when viewing the set visual mark displayed in the corresponding display area if the vertical offset difference is greater than a third preset threshold.

[0059] The first horizontal offset is obtained based on the X-axis decomposition provided in the above embodiment. The first vertical offset is obtained based on the Y-axis decomposition provided in the above embodiment. The second horizontal offset is obtained based on the X-axis decomposition provided in the above embodiment. The second vertical offset is obtained based on the Y-axis decomposition provided in the above embodiment.

[0060] This allows for further refinement of the strabismus determination results when setting visual markers for each display area, enabling the strabismus determination results to be specifically identified as strabismus in which direction.

[0061] In some embodiments, when it is determined that the user is strabismus while viewing the set visual marker displayed in the corresponding display area, the offset and angle determination module is used to obtain the actual gaze position vector corresponding to the left eye and the actual gaze position vector corresponding to the right eye; based on the actual gaze position vector corresponding to the left eye and the actual gaze position vector corresponding to the right eye, the offset and the second angle between the two eyeballs are determined, wherein the second angle is the angle between the first offset corresponding to the left eye and the first offset corresponding to the right eye.

[0062] The strabismus determination module is used to determine that the left and right eyes have non-common strabismus when viewing the set visual markers displayed in the corresponding display area if the offset between the two eyeballs is greater than a fourth preset threshold and / or the second included angle is greater than a second angle threshold.

[0063] Specifically, the offset and angle determination module 140 is used to determine the length of the actual gaze position vector corresponding to the left eye and the length of the actual gaze position vector corresponding to the right eye; and to determine the corresponding eye offset based on the length of each actual gaze position vector and the length of the corresponding target point gaze vector.

[0064] The lengths of the actual gaze position vectors corresponding to the left and right eyes are both the magnitudes of the vectors. For details, please refer to the calculation process of the vector magnitude.

[0065] In comitant strabismus, the degree of strabismus is basically the same in the left and right eyes. In non-comitant strabismus, the degree of strabismus differs greatly between the left and right eyes.

[0066] This embodiment can further determine the specific type of strabismus, such as comitant strabismus or non-comitant strabismus.

[0067] One embodiment of the present invention provides a head-mounted device. According to... Figure 2 As shown, the head-mounted device includes a memory 220 and a processor 210. The memory 220 stores a computer program that controls the processor 210 to perform the following method according to any of the above embodiments: sequentially displaying a set visual mark in multiple set display areas; acquiring an eye image corresponding to each eyeball, the eye image being an image of a user wearing the head-mounted device viewing the set visual mark displayed in the corresponding display area; determining a corresponding actual gaze position vector based on each eye image; obtaining a target point gaze vector corresponding to each eyeball; determining a corresponding eyeball offset and a first angle based on each actual gaze position vector and the corresponding target point gaze vector, wherein the target point gaze vector is the gaze position vector of an eye without strabismus viewing the set visual mark displayed in the corresponding display area, and the first angle is the angle between each actual gaze position vector and the corresponding target point gaze vector; determining that the corresponding eyeball has strabismus when viewing the set visual mark displayed in the corresponding display area if the corresponding eyeball offset is greater than a first preset threshold and / or the first angle is greater than a first angle threshold.

[0068] The processor 210 is used to execute computer instructions, which can be written using instruction sets of architectures such as x86, Arm, RISC, MIPS, and SSE. The memory 420 includes, for example, ROM (Read-Only Memory), RAM (Random Access Memory), and non-volatile memory such as a hard disk, etc., and is not limited thereto.

[0069] Headset devices can be VR glasses.

[0070] One embodiment of the present invention provides a strabismus detection system, based on Figure 3 As shown, the strabismus detection system includes a head-mounted device and a server as provided in any of the above embodiments.

[0071] The head-mounted device is used to send user identification information and corresponding strabismus result information to the server. The strabismus result information includes the strabismus result of the user when viewing the set visual markers displayed in the corresponding display area. The server is used to generate a visualization report based on the user identification information and the corresponding strabismus result information.

[0072] In some embodiments, according to Figure 4 As shown, the system also includes terminal equipment.

[0073] The terminal device receives a visualization report retrieval request from the server, which carries user identification information. The server retrieves the corresponding strabismus result information based on the request, generates a visualization report using the user identification information and the corresponding strabismus result information, and sends the visualization report to the corresponding terminal device. The terminal device displays the received visualization report.

[0074] In some embodiments, the strabismus result information further includes a first offset obtained by projecting each actual gaze position vector along a set direction, where the set direction is the direction of the line of sight when the eye without strabismus views the set visual mark displayed in the display area directly in front of it. The server is used to generate a left-eye image and a right-eye image corresponding to each display area based on the first offset obtained by projecting each actual gaze position vector along the set direction. The left-eye image includes a first mark, which is the center point of the left eye when the left eye without strabismus views the set visual mark displayed in the display area directly in front of it. The right-eye image includes a second mark, which is the center point of the right eye when the right eye without strabismus views the set visual mark displayed in the display area directly in front of it. A visualization report is generated based on the user identification information, the corresponding strabismus result information, and the left-eye image and the right-eye image corresponding to each display area.

[0075] When the visualization report includes user identification information, corresponding strabismus results, and left and right eye images for each display area, the terminal device displays it. See details below. Figure 5The images shown are the left and right eye images corresponding to each display area. The deviation of both eyes in each gaze direction is presented graphically, making the type, direction, and degree of strabismus immediately clear. This intuitive format greatly facilitates users or professionals in quickly understanding the condition, enabling remote diagnosis and long-term follow-up, significantly improving diagnostic efficiency and patient management capabilities.

[0076] The detection process of the oblique side detection system provided in this embodiment of the invention will be described below with a specific example.

[0077] according to Figure 6 As shown, the strabismus detection process includes steps S601 to S611.

[0078] Step S601: The head-mounted device sequentially displays the set visual markers in multiple set display areas; Step S602: The head-mounted device acquires eye images corresponding to each eyeball. The eye images are images of a user wearing the head-mounted device viewing a set visual marker displayed in the corresponding display area. Step S603: The head-mounted device determines the corresponding actual gaze position vector based on each eye image; Step S604: The head-mounted device acquires the target point gaze vector corresponding to each eyeball, and determines the corresponding eyeball offset and first angle based on each actual gaze position vector and the corresponding target point gaze vector. The target point gaze vector is the gaze position vector of the eyeball without strabismus viewing the set visual marker displayed in the corresponding display area, and the first angle is the angle between each actual gaze position vector and the corresponding target point gaze vector; In step S605, the head-mounted device determines the strabismus result of the corresponding eyeball when viewing the set visual marker displayed in the corresponding display area based on the corresponding eyeball offset and the first angle.

[0079] In step S606, the head-mounted device sends the user identification information and corresponding strabismus result information to the server. The strabismus result information includes the user's strabismus result when viewing the designated visual marker displayed in the corresponding display area, and the first offset obtained by projecting each actual gaze position vector along a designated direction. The designated direction is the direction of gaze when an eye without strabismus views the designated visual marker displayed in the display area directly in front of it.

[0080] Step S607: The server stores the received user identification information and corresponding strabismus result information.

[0081] In step S608, the terminal device sends a visual report retrieval request to the server, and the retrieval request carries user identification information.

[0082] In step S609, the server obtains the corresponding strabismus result information based on the request, and generates a visualization report based on the user identification information and the corresponding strabismus result information.

[0083] In step S610, the server sends the visualization report to the corresponding terminal device.

[0084] Step S611: The terminal device displays the received visual report.

[0085] The various embodiments in this specification are described in a progressive manner. The same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on describing the differences from other embodiments.

[0086] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0087] Embodiments of this specification may be systems, methods, and / or computer program products. A computer program product may include a computer-readable storage medium having computer instructions stored thereon for causing a processor to implement various aspects of the embodiments of this specification.

[0088] Computer-readable storage media can be tangible devices capable of holding and storing computer instructions for use by computer instruction execution devices. Computer-readable storage media can be, for example—but not limited to—electrical storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital multifunction disc (DVD), memory sticks, floppy disks, mechanical encoding devices, such as punch cards or recessed protrusions storing computer instructions thereon, and any suitable combination of the foregoing. The computer-readable storage media used herein are not to be construed as transient signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses through fiber optic cables), or electrical signals transmitted through wires.

[0089] The computer instructions described herein can be downloaded from computer-readable storage media to various computing / processing devices, or downloaded via a network, such as the Internet, local area network, wide area network, and / or wireless network, to an external computer or external storage device. The network may include copper cables, fiber optic cables, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives computer instructions from the network and forwards them to computer-readable storage media within the respective computing / processing device.

[0090] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this specification. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of computer instructions, which contains one or more executable computer instructions for implementing a specified logical function. In some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions. It will be known to those skilled in the art that implementation in hardware, implementation in software, and implementation using a combination of software and hardware are equivalent.

[0091] Various embodiments of this specification have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or improvement of the technology in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. A head-mounted device, characterized in that, include: The display module is used to sequentially display designated visual markers in multiple designated display areas; The image acquisition module is used to acquire eye images corresponding to each eyeball. The eye images are images of a user wearing the head-mounted device viewing a set visual marker displayed in the corresponding display area. The gaze position vector determination module is used to determine the corresponding actual gaze position vector based on each eye image; The offset and angle determination module is used to obtain the target point gaze vector corresponding to each eyeball, and determine the corresponding eyeball offset and first angle based on each actual gaze position vector and the corresponding target point gaze vector. The target point gaze vector is the gaze position vector of the eyeball without strabismus viewing the set visual mark displayed in the corresponding display area, and the first angle is the angle between each actual gaze position vector and the corresponding target point gaze vector. The strabismus determination module is used to determine that the corresponding eyeball is strabismus when viewing the set visual mark displayed in the corresponding display area if the corresponding eyeball offset is greater than a first preset threshold and / or the first included angle is greater than a first angle threshold.

2. The head-mounted device according to claim 1, characterized in that, The gaze position vector determination module is used to determine the corresponding quaternion information based on each eye image. Each quaternion information is the three-dimensional rotation state of the eyeball in the corresponding eye image relative to a set position. The set position is the position of the eyeball without strabismus when viewing the set visual mark displayed in the display area directly in front of it. Obtain a set gaze position vector, wherein the set gaze position vector is the gaze position vector in a virtual scene when the non-squinting eye looks at a set visual marker displayed in the display area directly in front of it; Based on each quaternion and the set gaze position vector, the corresponding actual gaze position vector in the virtual scene is determined.

3. The head-mounted device according to claim 1, characterized in that, The offset and angle determination module is used to determine the length of each actual gaze position vector and the length of the corresponding target point gaze vector. The corresponding eye offset is determined based on the length of each actual gaze position vector and the length of the corresponding target point gaze vector.

4. The head-mounted device according to claim 1, characterized in that, If it is determined that the user is squinting when viewing the set visual mark displayed in the corresponding display area, the offset and angle determination module is used to determine the first offset obtained by projecting each actual gaze position vector along the set direction, and to determine the second offset obtained by projecting the gaze vector of the corresponding target point along the set direction. The set direction is the direction of the line of sight when the eye without squinting views the set visual mark displayed in the display area directly in front of it. A first horizontal offset and a first vertical offset are determined based on a first offset, and a second horizontal offset and a second vertical offset are determined based on a second offset. Based on each actual gaze position vector and the corresponding target point gaze vector, the horizontal offset difference is determined according to the corresponding first horizontal offset and the corresponding second horizontal offset, and the vertical offset difference is determined according to the corresponding first vertical offset and the corresponding second vertical offset, wherein... The strabismus determination module is used to determine that the corresponding eyeball has horizontal strabismus when viewing the set visual mark displayed in the corresponding display area if the difference in horizontal offset is greater than a second preset threshold, and to determine that the corresponding eyeball has vertical strabismus when viewing the set visual mark displayed in the corresponding display area if the difference in vertical offset is greater than a third preset threshold.

5. The head-mounted device according to claim 3, characterized in that, When it is determined that the user is strabismus while viewing the set visual markers displayed in the corresponding display area, the offset and angle determination module is used to obtain the actual gaze position vector corresponding to the left eyeball and the actual gaze position vector corresponding to the right eyeball. Based on the actual gaze position vector corresponding to the left eyeball and the actual gaze position vector corresponding to the right eyeball, the offset and second angle between the two eyeballs are determined, wherein the second angle is the angle between the first offset corresponding to the left eyeball and the first offset corresponding to the right eyeball; The strabismus determination module is used to determine that the left and right eyes have non-common strabismus when viewing the set visual markers displayed in the corresponding display area if the offset between the two eyeballs is greater than a fourth preset threshold and / or the second included angle is greater than a second angle threshold.

6. The head-mounted device according to claim 1, characterized in that, The multiple set display areas are nine-grid display areas, and each grid display area displays set visual markers in a set order.

7. The head-mounted device according to claim 1, characterized in that, The head-mounted device also includes a transmission module for sending user identification information and corresponding strabismus result information to the server. The strabismus result information includes the strabismus result when the user views the set visual markers displayed in the corresponding display area.

8. A strabismus detection system, characterized in that, Includes the head-mounted device and server as described in any one of claims 1-7, wherein, The head-mounted device is used to send user identification information and corresponding strabismus result information to the server, wherein the strabismus result information includes the strabismus result when the user views the set visual markers displayed in the corresponding display area; The server is used to store the received user identification information and corresponding strabismus result information.

9. The strabismus detection system according to claim 8, characterized in that, The system also includes terminal equipment, wherein... The terminal device is used to send a visual report retrieval request to the server, wherein the retrieval request carries user identification information; The server is used to obtain the corresponding strabismus result information according to the acquisition request, generate a visualization report according to the user identification information and the corresponding strabismus result information, and send the visualization report to the corresponding terminal device; The terminal device is used to display the received visual report.

10. The strabismus detection system according to claim 8 or 9, characterized in that, The strabismus result information also includes a first offset obtained by projecting each actual gaze position vector along a set direction, where the set direction is the line of sight when an eye without strabismus views a set visual marker displayed in a display area directly in front of it. The server is used to generate left-eye and right-eye images corresponding to each display area based on a first offset obtained by projecting each actual gaze position vector along a set direction. The left-eye image includes a first marker, which is the center point of the left eye when viewing the set visual marker displayed in the display area directly in front of the left eye without strabismus. The right-eye image includes a second marker, which is the center point of the right eye when viewing the set visual marker displayed in the display area directly in front of the right eye without strabismus. A visualization report is generated based on user identification information, corresponding strabismus results, and left and right eye images for each display area.

Images