Auxiliary diagnostic apparatus for ocular myogenic compensatory head posture based on instantaneous eye movement capture

Abstract

An auxiliary diagnostic apparatus for ocular myogenic compensatory head posture based on instantaneous eye movement capture, relating to the technical field of medical devices, and comprising a rotated video playback module, a skeletal muscle key point capture module, a capture module, and an image output module; a subject visually fixates on a video, wherein the video is rotated in a direction opposite to a compensatory head posture until reaching a horizontal position, and if a head posture at the horizontal position remains unimproved, then the video is rotated in a direction of the compensatory head posture; during rotation, the skeletal muscle key point capture module determines that captured facial and neck key points of the subject are symmetrical and that the head posture is properly aligned, and the capture module is started and automatically captures an image; the image is analyzed for ocular misalignment, and if ocular misalignment is present in the captured image, ocular myogenic compensatory head posture can be definitively determined; if the head posture does not improve significantly, one eye of the subject is covered while the other eye continues to fixate on the video. A subject visually fixates on a video, and after the video stops rotating, the subject maintains a stable head posture and fixation point until an automatic image capture process is completed. If the head posture in the captured image improves, ocular myogenic compensatory head posture can be definitively determined.

Description

A head position identification device based on instantaneous eye movement capture for oculomotor compensatory head position Technical Field

[0001] This invention relates to the field of medical device technology, and in particular to an oculomotor compensatory head position identification device based on instantaneous eye movement capture. Background Technology

[0002] Torticollis is a common symptom in childhood, with one-quarter of cases being compensatory head postures caused by strabismus. These patients adopt a compensatory head posture because visual problems such as abnormal eye alignment and nystagmus are more pronounced in a normal head position. They often compensate by tilting their head to achieve optimal vision or maintain binocular single vision, resulting in torticollis. Ophthalmologists frequently encounter patients with torticollis in their clinical practice. These patients are often very young and have difficulty cooperating with examinations, making it difficult for doctors to determine whether the cause is oculomotor-related compensatory head posture. Currently, clinical practice mainly involves observing eye alignment after correcting the patient's head posture, combined with complex oculomotor examinations to determine the cause of compensatory head posture. However, this process is complex, and young children cannot cooperate with head posture correction, making the examination susceptible to the influence of patient cooperation. Therefore, a simple, rapid, and easily cooperated-by device is needed to help identify whether compensatory head posture is caused by oculomotor factors, in order to facilitate further ophthalmological examination. Summary of the Invention

[0003] To address the shortcomings of existing technologies, this invention provides a head position assistance identification device based on instantaneous eye movement capture that compensates for ocular muscle-related compensation by tilting the gaze object. The device captures the eye position of both eyes in an instantaneous photograph after the skeletal key points capture the symmetry of the left and right facial data.

[0004] A head position identification device based on instantaneous eye movement capture for oculomotor compensatory head position includes: a rotating video playback module, a skeletal muscle key point capture module, a capture module, and a photo output module;

[0005] The rotating video playback module includes a display screen and a speaker. The display screen plays animations and rotates by a set degree at a set time using a video rotation algorithm. It first rotates in the opposite direction of the compensating head position. If the head position is still not improved when it is rotated to a horizontal position, it then rotates in the same direction as the compensating head position to improve the head position.

[0006] The software component of the display screen comprises three modules: a video rendering engine, a video playback control, and a user interaction platform. The video rendering engine works by using a graphics library and a video rotation algorithm during video playback to change vertex coordinates or use shaders to achieve a rotation effect, thus rotating the video frame. A video decoding library decodes the video stream into frames, loads each frame as a texture into a framebuffer, applies a rotation matrix, and displays the rotated image on the screen. The video playback control creates a user interface that allows users to control playback, specifically by controlling video playback and pausing. The user interaction platform allows users to interact with the video during playback and adjust the video's rotation angle.

[0007] The skeletal muscle key point capture module includes a camera and an embedded processor. The camera captures real-time images of the patient's frontal face, with a resolution of at least 1080P. The module manages and communicates the acquired data stream, transmitting the captured patient's frontal image to the embedded processor in real-time via the WebSocket protocol. The embedded processor processes the image data using sensor data and then extracts skeletal key point data using a skeletal key point detection algorithm. A skeletal muscle symmetry threshold is set; when the coordinate deviation of corresponding points on the left and right sides is within the threshold, the head position is determined to be correct.

[0008] The skeletal keypoint detection algorithm uses computer vision technology to locate anatomical landmarks in images and then constructs the topology of the human skeleton. Specifically, after inputting a frontal image of the patient, a pre-trained CNN backbone network is used to extract multi-scale features; a keypoint heatmap and a site association field are generated through a dual-branch network; keypoint matching is completed using the Hungarian algorithm; and keypoints on the face and neck are detected using existing algorithms such as OpenPose's Part Affinity Fields and AlphaPose's RMPE framework.

[0009] The sensor data processing is responsible for the preprocessing and fusion of multimodal data. For the patient's frontal image captured by the camera, white balance correction and non-uniform illumination compensation are performed first. Then, Kalman filtering is applied to eliminate head micro-motion noise. At the same time, hardware timestamp synchronization technology is used to ensure that the video stream and the inertial measurement unit data are aligned in the time domain with a time error of less than 3ms.

[0010] The data management and communication adopts real-time data stream transmission, and the data frames are encoded using Protobuf; batch data is uploaded to the cloud, and the data storage adopts a hybrid architecture: a time-series database records the motion trajectory of key points, a relational database manages patient metadata, and Redis caches real-time detection results; in terms of security, it follows the HIPAA standard, uses AES-256-GCM to encrypt key point data, and protects patient privacy through facial feature desensitization processing.

[0011] The capture module is implemented using a zoom camera, an embedded processor, and a display screen. The zoom camera captures the patient's eye movements and eye positions in real time, transmits the data to the embedded processor, and visualizes it on the display screen. Eye tracking algorithms and calibrations are then performed. The iris-corneal reflection method is used to extract the eye's fixation point position. Nine-point calibration is performed before video playback to establish a screen-eye mapping relationship. The eye tracking data is aligned with the video frame timestamps to calculate eye movement events. The WebSocket communication protocol is used to transmit the eye movement event data to the front-end webpage in real time, enabling the analysis and visualization of eye movement events.

[0012] The eye-tracking algorithm and calibration described above have been optimized for children based on the traditional iris-corneal reflex method. Specifically, it adopts a 9-point calibration guided by dynamic animation, and adds dynamic attention guidance. During eye tracking, if the gaze point deviates from the center of the screen by more than 50% for more than 5 seconds, a flashing effect is automatically superimposed in the animation to guide the gaze back. In addition, the movement speed of all dynamic elements is ≤0.6m / s, and blink compensation is designed. When the pupil occlusion rate is detected to be >90%, data acquisition is paused for 150ms, and historical data is used for interpolation to fill the gap. Head micro-motion correction is designed, which combines the nose tip coordinates of the skeletal key point data and corrects the error caused by head displacement in real time through affine transformation.

[0013] The analysis and visualization of eye movement events are as follows: eye movement events are identified by real-time processing of eye movement data streams and combining multi-dimensional parameters. Specifically, this includes: eye movement speed lasting <30° / s and duration ≥100ms is identified as a fixation event; instantaneous eye movement speed >80° / s is identified as a saccade event, and amplitude and direction vectors are calculated; when pupil occlusion rate >90% and lasts for 30-150ms, feature recognition identifies blinking; real-time comparison of the ratio of eye movement speed to target movement speed determines tracking motion events; when the saccade path return angle >120°, it is marked as an anomaly and returned as a saccade event; after capture, eye movement event data of children of the same age are analyzed and compared, and abnormal indicators in eye movement events are visualized.

[0014] The photo output module is implemented through a display screen. When the head is straight and the gaze point is stable, it automatically captures the current scene and generates a photo for visualizing the eye position under the gaze point mapping after the head is straightened. The technology involved is gaze point mapping and visualization.

[0015] When the skeletal muscle key point capture module judges head position alignment, if the head position cannot be aligned voluntarily, monocular occlusion is performed, and the head position is evaluated again under monocular fixation through the rotating video playback module.

[0016] The beneficial effects of adopting the above technical solution are as follows:

[0017] This invention provides an auxiliary identification device for oculomotor compensatory head position based on instantaneous eye movement capture. This invention provides a method to determine the cause of compensatory head position by using screen rotation to alleviate oculomotor compensatory head position. Based on the equipment requirements in the process of this method, an auxiliary identification device for oculomotor compensatory head position was invented. Compared with traditional identification methods, this device is simple to operate and can be completed by young children. Attached Figure Description

[0018] Figure 1 is a flowchart of the auxiliary identification process for ocular muscle-derived compensatory head position according to the present invention.

[0019] Figure 2 is a schematic diagram of the system structure of the present invention;

[0020] Figure 3 is a schematic diagram of the external structure of the present invention;

[0021] Figure 4 is a flowchart of the operation method of the oculomotor compensatory head position auxiliary identification device in an embodiment of the present invention;

[0022] In the diagram, 1- Rotate video playback module, 2- Skeletal muscle keypoint capture module, 3- Capture module, 4- Photo output module, 11- Video rendering engine, 12- Video rotation algorithm, 13- Video playback control, 14- User interaction platform, 15- Display screen, 16- Speaker, 17- Control playback button, 21- Skeletal keypoint detection algorithm, 22- Sensor data processing, 23- Data management and communication, 24- Camera, 25- Embedded processor, 31- Eye tracking algorithm and calibration, 32- Eye movement event analysis and visualization, 33- Zoom camera, 41- Gaze point mapping and visualization. Detailed Implementation

[0023] The specific embodiments of the present invention will be described in further detail below with reference to the accompanying drawings and examples. The following examples are for illustrative purposes only and are not intended to limit the scope of the invention.

[0024] A head position identification device based on instantaneous eye movement capture for ocular muscle-derived compensatory head position, as shown in Figure 2, includes: a rotating video playback module 1, a skeletal muscle key point capture module 2, a capture module 3, and a photo output module 4.

[0025] The rotating video playback module 1, as shown in Figure 3, includes a display screen 15 and a speaker 16, used to correct the patient's head position. The display screen 15 plays animations and rotates a set number of degrees at a set time using a video rotation algorithm 12. In this embodiment, it rotates 5 degrees per minute, first rotating in the opposite direction of the compensatory head position. If the head position is still not improved after rotating to a horizontal position, it rotates in the same direction as the compensatory head position to improve the head position. The software part of the display screen 15 includes three modules: a video rendering engine 11, a video playback control 13, and a user interaction platform 14. The video rendering engine 11 is used to: during video playback, ... The video rotation algorithm 12, implemented using a graphics library (not limited to OpenGL or DirectX), rotates the video frame by changing vertex coordinates or using shaders. A video decoding library decodes the video stream into frames, loads each frame as a texture into a framebuffer, applies a rotation matrix, and displays the rotated image on the screen. The video playback control 13 involves creating a user interface that allows users to control playback, specifically by controlling video playback and pause. The user interaction platform 14 allows users to interact during video playback and adjust the video's rotation angle.

[0026] The skeletal muscle keypoint capture module 2 includes a camera 24 and an embedded processor 25, used to determine whether the head position is correct. When the coordinate positions of the corresponding skeletal muscle nodes on the left and right sides of the face and neck are completely symmetrical, the head position is determined to be correct. Using a skeletal keypoint detection algorithm in computer vision technology, not limited to OpenPose or AlphaPose, the patient's facial and neck images are analyzed to extract the coordinates of the keypoints of the left and right skeletal muscles. The camera 24 acquires a frontal image of the patient in real time, wherein the resolution of the frontal image of the patient is not less than 1080P. The acquired frontal image of the patient is transmitted to the embedded processor 25 in real time via the WebSocket protocol. The embedded processor 25 performs sensor data processing 22 on the image, and then implements a skeletal keypoint detection algorithm 21 using Python + OpenCV / TensorFlow to extract skeletal keypoint data. A skeletal muscle symmetry threshold is set, and when the coordinate deviation of the corresponding points on the left and right sides is within the threshold, the head position is determined to be correct.

[0027] The skeletal key point detection algorithm 21 uses computer vision technology to locate anatomical landmarks in images and then constructs the topology of the human skeleton. Specifically, after inputting a frontal image of the patient, a pre-trained CNN backbone network is used to extract multi-scale features. A key point heatmap and a site association field are generated through a dual-branch network. The Hungarian algorithm is used to complete key point matching. Using the existing algorithms OpenPose's Part Affinity Fields and AlphaPose's RMPE framework, 16 key points on the face and neck are detected, with the coordinate deviation threshold controlled within 2 mm.

[0028] The sensor data processing 22 is responsible for the preprocessing and fusion of multimodal data. For the patient's frontal image captured by the camera 24, white balance correction and non-uniform illumination compensation are performed first. Then, Kalman filtering is applied to eliminate head micro-motion noise. At the same time, hardware timestamp synchronization technology is used to ensure that the video stream and the inertial measurement unit data are aligned in the time domain with a time error of less than 3ms.

[0029] The data management and communication 23 adopts real-time data stream transmission, and the data frames are encoded using Protobuf, containing key point information such as normalized coordinates and confidence scores; batch data is uploaded to the cloud, and data storage adopts a hybrid architecture: a time-series database records the motion trajectory of key points, a relational database manages patient metadata, and Redis caches real-time detection results (TTL 300 seconds); in terms of security, it follows the HIPAA standard, uses AES-256-GCM to encrypt key point data, and protects patient privacy through facial feature desensitization processing.

[0030] The capture module 3 is implemented through a zoom camera 33, an embedded processor 25, and a display screen 15. The zoom camera 33 captures the patient's eye movements and eye positions in real time, transmits the data to the embedded processor 25, visualizes it on the display screen 15, and performs eye tracking algorithms and calibration 31. The eye fixation point position is extracted using the iris-corneal reflection method, and a 9-point calibration is performed before video playback to establish a screen-eye mapping relationship. The eye tracking data is aligned with the video frame timestamps to calculate eye movement events. The eye movement event data is transmitted to the front-end webpage in real time using WebSocket to realize the analysis and visualization of eye movement events 32.

[0031] The eye-tracking algorithm and calibration 31 are optimized for children based on the traditional iris-corneal reflex method. Specifically, a 9-point calibration guided by dynamic animation (such as cartoon character movement calibration) is adopted to improve children's cooperation. At the same time, dynamic attention guidance is added. During eye tracking, when the gaze point is detected to deviate from the center of the screen by more than 50% for more than 5 seconds, a flashing star effect is automatically superimposed in the animation to guide the gaze back. In addition, the movement speed of all dynamic elements is ≤0.6m / s, which is within the range of children's smooth tracking ability. During eye tracking, eyelash occlusion is eliminated by fusing near-infrared imaging and visible light data, improving the localization of corneal reflex points. Blink compensation is designed. When the pupil occlusion rate is detected to be >90%, data acquisition is paused for 150ms, and historical data is used for interpolation to fill the gap. Head micro-movement correction is designed. Combining the nose tip coordinates of the skeletal key point data, the error caused by head displacement is corrected in real time through affine transformation.

[0032] The analysis and visualization of eye movement events 32 are as follows: eye movement events are identified by real-time processing of eye movement data streams and combining multi-dimensional parameters. Specifically, this includes: when the eye movement speed is continuously <30° / s and the duration is ≥100ms, it is determined as a fixation event; when the instantaneous eye movement speed is >80° / s, it is determined as a saccade event, and the amplitude (angle difference) and direction vector are calculated; when the pupil occlusion rate is >90% and lasts for 30-150ms, it is identified as a blink; the ratio of eye movement speed to target movement speed is compared in real time to determine tracking motion events; when the saccade path return angle is >120°, it is marked as an abnormality and returned as a saccade event; after capture, the eye movement event data of children of the same age are analyzed and compared, and abnormal indicators in the eye movement events are visualized. For example, if there is an abnormal fixation, a fixation abnormality is indicated.

[0033] The photo output module 4 is implemented through the display screen 15. When the head position is straight and the gaze point is stable, it automatically captures the current scene and generates a photo for eye position visualization 41 under the gaze point mapping after the head position is straightened; the technology involved is gaze point mapping and visualization 41.

[0034] When the skeletal muscle key point capture module 2 judges the head position, if the head position cannot be corrected on its own, it will perform monocular occlusion for 10 minutes, and then evaluate the head position again under monocular fixation through the rotating video playback module 1.

[0035] In this embodiment, based on the aforementioned oculomotor compensatory head position auxiliary identification device, an operation method for the oculomotor compensatory head position auxiliary identification device is implemented, as shown in Figure 4, including the following steps:

[0036] Step 1: Preparation phase;

[0037] The examinee sits upright in front of the display screen, adjusting the chair height so that the eyes are roughly level with the center of the screen; the distance from the patient's eyes to the screen is measured and recorded, and the camera zoom is adjusted.

[0038] Following the voice prompts, the patient focuses on nine markers on the screen, holding each marker for approximately two seconds.

[0039] Keep your head still during calibration, and move your eyes to focus on each point;

[0040] After successful calibration, the system automatically enters video rotation mode;

[0041] Step 2: Video rotation test, as shown in Figure 1;

[0042] The subject watches the video, which first rotates in the opposite direction from the compensatory head position at a rate of 5 degrees per minute until it reaches a horizontal position. If the head position does not improve in the horizontal position, the video then rotates back towards the compensatory head position until the process is complete.

[0043] During rotation, the skeletal muscle key points capture the subject, the face and neck key points are on the opposite side, the head position is dignified, the capture module is activated, and the system automatically captures photos.

[0044] By analyzing photographs, abnormal eye positioning can be identified as a compensatory head posture due to oculomotor muscle origin when abnormal eye positioning is present in the captured image.

[0045] Step 3: Cover one eye;

[0046] If the head position does not improve significantly in step 2, cover one eye of the examiner (e.g., the left eye) while the other eye continues to focus on the video.

[0047] During the occlusion period, the patient should keep the covered eye completely closed and not peek out. The default occlusion duration is 10 minutes, during which the patient can close their eyes to rest.

[0048] Ten minutes later, the subject watches the video. Once the video stops rotating, the subject keeps their head and gaze stable until the photo is automatically captured.

[0049] When the head position in the captured image is significantly improved, it can be clearly identified as oculomotor compensatory head position.

[0050] The above description is merely a preferred embodiment of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of the invention involved in the embodiments of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described inventive concept. For example, technical solutions formed by substituting the above-described features with (but not limited to) technical features with similar functions disclosed in the embodiments of this disclosure.

Claims

1. A head position identification device based on instantaneous eye movement capture for oculomotor compensatory head position, characterized in that, include: Rotation video playback module, skeletal muscle key point capture module, capture module, photo output module; The rotating video playback module includes a display screen and a speaker. The display screen plays animations and rotates by a set degree at a set time using a video rotation algorithm. It first rotates in the opposite direction of the compensating head position. If the head position is still not improved when it is rotated to a horizontal position, it then rotates in the same direction as the compensating head position to improve the head position. The skeletal muscle key point capture module includes a camera and an embedded processor. The camera captures a frontal image of the patient in real time, wherein the resolution of the frontal image of the patient is not less than 1080P. The module manages and communicates the acquired data stream and transmits the acquired frontal image of the patient to the embedded processor in real time via the WebSocket protocol. The embedded processor processes the image data from the sensor, and then extracts the key point data of the skeleton through the skeleton key point detection algorithm; a skeletal muscle symmetry threshold is set, and when the coordinate deviation of the corresponding points on the left and right sides is within the threshold, it is determined that the head position is correct. The capture module is implemented using a zoom camera, an embedded processor, and a display screen. The zoom camera captures the patient's eye movements and eye positions in real time, transmits the data to the embedded processor, and visualizes it on the display screen. Eye tracking algorithms and calibrations are then performed. The iris-corneal reflection method is used to extract the eye's fixation point position. Nine-point calibration is performed before video playback to establish a screen-eye mapping relationship. The eye tracking data is aligned with the video frame timestamps to calculate eye movement events. The WebSocket communication protocol is used to transmit the eye movement event data to the front-end webpage in real time, enabling the analysis and visualization of eye movement events. The photo output module is implemented through a display screen. When the head is straight and the gaze point is stable, it automatically captures the current scene and generates a photo for visualizing the eye position under the gaze point mapping after the head is straight. The technology involved is gaze point mapping and visualization.

2. The oculomotor-derived compensatory head position identification device based on instantaneous eye movement capture according to claim 1, characterized in that, The software component of the display screen comprises three modules: a video rendering engine, a video playback control, and a user interaction platform. The video rendering engine works by using a graphics library and a video rotation algorithm during video playback to change vertex coordinates or use shaders to achieve a rotation effect, thus rotating the video frame. A video decoding library decodes the video stream into frames, loads each frame as a texture into a framebuffer, applies a rotation matrix, and displays the rotated image on the screen. The video playback control creates a user interface that allows users to control playback, specifically by controlling video playback and pausing. The user interaction platform allows users to interact with the video during playback and adjust the video's rotation angle.

3. The oculomotor-induced compensatory head position identification device based on instantaneous eye movement capture according to claim 1, characterized in that, The skeletal key point detection algorithm uses computer vision technology to locate anatomical landmarks in images and then constructs the topology of the human skeleton; specifically, after inputting a frontal image of the patient, a pre-trained CNN backbone network is used to extract multi-scale features. A keypoint heatmap and part association field are generated through a dual-branch network; keypoint matching is performed using the Hungarian algorithm; and keypoints on the face and neck are detected using existing algorithms such as OpenPose's Part Affinity Fields and AlphaPose's RMPE framework.

4. The oculomotor-derived compensatory head position identification device based on instantaneous eye movement capture according to claim 1, characterized in that, The sensor data processing is responsible for the preprocessing and fusion of multimodal data. For the patient's frontal image captured by the camera, white balance correction and non-uniform illumination compensation are performed first. Then, Kalman filtering is applied to eliminate head micro-motion noise. At the same time, hardware timestamp synchronization technology is used to ensure that the video stream and the inertial measurement unit data are aligned in the time domain with a time error of less than 3ms.

5. The oculomotor-derived compensatory head position identification device based on instantaneous eye movement capture according to claim 1, characterized in that, The data management and communication adopts real-time data stream transmission, and the data frames are encoded using Protobuf; batch data is uploaded to the cloud, and the data storage adopts a hybrid architecture: a time-series database records the motion trajectory of key points, a relational database manages patient metadata, and Redis caches real-time detection results; in terms of security, it follows the HIPAA standard, uses AES-256-GCM to encrypt key point data, and protects patient privacy through facial feature desensitization processing.

6. The oculomotor-derived compensatory head position identification device based on instantaneous eye movement capture according to claim 1, characterized in that, The eye-tracking algorithm and calibration have been optimized for children based on the traditional iris-corneal reflex method. Specifically, a 9-point calibration guided by dynamic animation is adopted, and dynamic attention guidance is added. During eye tracking, when the gaze point is detected to be deviating from the center of the screen by more than 50% for more than 5 seconds, a flashing effect is automatically superimposed in the animation to guide the gaze back. In addition, the movement speed of all dynamic elements is ≤0.6m / s, and blink compensation is designed. When the pupil occlusion rate is detected to be >90%, data acquisition is paused for 150ms, and historical data is used for interpolation to fill the gap. Head micro-motion correction is designed. Combined with the nose tip coordinates of the skeletal key point data, the error caused by head displacement is corrected in real time through affine transformation.

7. The oculomotor-induced compensatory head position identification device based on instantaneous eye movement capture according to claim 1, characterized in that, The analysis and visualization of eye movement events are as follows: eye movement events are identified by real-time processing of eye movement data streams and combining multi-dimensional parameters. Specifically, this includes: eye movement speed lasting <30° / s and duration ≥100ms is identified as a fixation event; instantaneous eye movement speed >80° / s is identified as a saccade event, and amplitude and direction vectors are calculated; when pupil occlusion rate >90% and lasts for 30-150ms, feature recognition identifies blinking; real-time comparison of the ratio of eye movement speed to target movement speed determines tracking motion events; when the saccade path return angle >120°, it is marked as an anomaly and returned as a saccade event; after capture, eye movement event data of children of the same age are analyzed and compared, and abnormal indicators in eye movement events are visualized.

8. The oculomotor-derived compensatory head position identification device based on instantaneous eye movement capture according to claim 1, characterized in that, When the skeletal muscle key point capture module judges head position alignment, if the head position cannot be aligned voluntarily, monocular occlusion is performed, and head position is evaluated again under monocular fixation through the rotating video playback module.

Images