BPPV automatic auxiliary diagnosis method based on eye movement video

Abstract

The invention relates to the technical field of medical image processing, in particular to a BPPV automatic auxiliary diagnosis method based on an eye movement video, and the method comprises an image collection module which is used for collecting the eye movement video of a patient; the iris segmentation and positioning module is used for extracting an iris region from the eye movement video and positioning the inner and outer contours of the iris region; the three-dimensional nystagmus detection module is used for generating eye movement data in horizontal, vertical and torsional dimensions based on the iris information; and the classification diagnosis module is used for classifying BPPV types according to the three-dimensional eye movement data. According to the invention, by adopting a multi-task end-to-end iris segmentation and positioning strategy, iris segmentation mask prediction and internal and external contour parameter extraction are integrated in the same network, so that deep collaboration and mutual promotion of two tasks are realized; according to the design, the process complexity and manual design errors caused by dependence on a large number of post-processing steps in a traditional method are avoided, and accurate segmentation results and positioning parameters can be directly output from the original eye movement video image.

Description

Technical Field

[0001] This invention relates to the field of medical image processing technology, and more specifically, to an automatic assisted diagnosis method for BPPV based on eye-tracking video. Background Technology

[0002] Vertigo is a common clinical symptom, among which benign paroxysmal positional vertigo (BPPV) is one of the most common vertigo disorders, especially in the elderly, with a higher incidence in women than men. The diagnosis of BPPV mainly relies on medical history and postural provocation tests, such as the Dix-Hallpike test and the supine rolling test, by observing whether characteristic nystagmus is induced. However, these traditional methods have obvious limitations: the operation and interpretation lack standardization, they are highly dependent on the doctor's experience, and they are difficult to capture subtle or atypical nystagmus features.

[0003] In recent years, video nystagmus imaging technology has been gradually applied in clinical practice. By recording eye movements through a camera, it provides objective evidence for the diagnosis of BPPV. VOG can capture nystagmus details that are difficult to detect with the naked eye, which helps to improve the accuracy and repeatability of diagnosis. At the same time, artificial intelligence, especially deep learning technology, has shown great potential in medical image processing. Some studies have attempted to apply convolutional neural networks to eye movement video analysis to achieve automatic recognition and classification of BPPV.

[0004] Nevertheless, existing technologies still have the following problems: 1. Insufficient accuracy in iris segmentation and localization: In complex real-world scenarios, such as ptosis, eyelash obstruction, changes in lighting, or interference from cosmetics, traditional iris segmentation methods struggle to accurately extract the iris region. Furthermore, most methods only output a binary mask, lacking precise parameter estimation of the inner and outer elliptical contours of the iris, leading to the accumulation of errors in subsequent feature extraction and eye movement analysis.

[0005] 2. Lack of three-dimensional nystagmus detection capability: Existing systems are mostly limited to horizontal and vertical eye movement analysis, neglecting torsional nystagmus, which is a key indicator for BPPV subtyping diagnosis. In addition, the lack of a geometric correction mechanism for non-concentric elliptical irises leads to inaccurate generation of three-dimensional eye movement data.

[0006] 3. Cumbersome system process and low computational efficiency: Existing auxiliary diagnostic systems are usually composed of multiple independent modules. There are a lot of post-processing steps in the process, which not only introduces human error, but also makes the system run slowly and difficult to deploy in real-time clinical scenarios.

[0007] Therefore, there is a need to design a BPPV-assisted diagnostic method that can achieve high-precision iris segmentation and localization, complete three-dimensional nystagmus detection, and end-to-end automated processing capabilities, in order to improve diagnostic efficiency and accuracy, reduce reliance on expert experience, and meet the needs of practical clinical applications. Summary of the Invention

[0008] To overcome the above deficiencies, the present invention provides a solution that overcomes or at least partially solves the above technical problems.

[0009] This invention provides an automatic assisted diagnosis system for BPPV based on eye-tracking video, comprising: The image acquisition module is used to acquire videos of the patient's eye movements. The iris segmentation and localization module is used to extract the iris region from eye-tracking videos and locate its inner and outer contours. The three-dimensional nystagmus detection module is used to generate eye movement data in three dimensions: horizontal, vertical, and torsional, based on iris information. The classification and diagnosis module is used to classify BPPV types based on 3D eye-tracking data; The output module is used to output the diagnostic results.

[0010] Preferably, the iris segmentation and localization module is implemented based on an improved DETR network, including: Multi-task branching, simultaneously outputting iris segmentation mask and inner and outer elliptical contour parameters; The improved deformable attention module adjusts the sampling points according to the reference ellipse and embeds angle information. The sampling point position is determined by the following formula:

[0011] in: As a reference point, This is the offset. For rotation matrix, The angle is an ellipse.

[0012] Preferably, the multi-task branch includes: Segmentation Branch: Fuse backbone network features with Transformer encoder features to generate pixel-level embedding maps, and perform dot product with query embeddings to output segmentation masks; Positioning branch: Outputs the center coordinates, major and minor axis lengths, and rotation angles of the inner and outer ellipses of the iris.

[0013] Preferably, the three-dimensional nystagmus detection module includes: The distortion correction unit is used to transform non-concentric elliptical iris regions into concentric circles, which is achieved through an affine transformation matrix:

[0014] Where: T is the affine transformation matrix; The iris unfolding unit unfolds the corrected iris region into a rectangle with a fixed width of 360 pixels and a height determined by the difference between the inner and outer radii.

[0015] Preferably, the three-dimensional eye-tracking data generation unit includes: Horizontal or vertical velocity calculation: obtained by applying a difference to the pupil center coordinate sequence; Torsional velocity calculation: The similarity of iris patterns between consecutive frames and the torsion angle are calculated using a template matching algorithm. Determined by the following formula:

[0016] Wherein: NCC represents the normalized cross-correlation function.

[0017] Preferably, in the template matching algorithm, the reference iris pattern is extended by ±15 degrees to cover the maximum torsion range, avoiding abrupt changes at the boundary between 0° and 360°.

[0018] Preferably, the classification and diagnosis module employs a one-dimensional convolutional neural network: 1D-CNN, specifically including: Five one-dimensional convolutional layers, each followed by batch normalization and max pooling; Three fully connected layers; The output layer uses the Softmax function to output the probabilities of various BPPV types.

[0019] Preferably, the input to the 1D-CNN is a three-dimensional eye-tracking velocity sequence, with each dimension input independently and features extracted separately, and finally feature fusion is performed.

[0020] Preferably, the system further includes: The results visualization module is used to overlay and display the iris segmentation region, inner and outer elliptical contours, and diagnostic results in the eye-tracking video. The data storage module is used to store patient videos, eye movement data, and diagnostic records.

[0021] Preferably, the system supports an end-to-end automated diagnostic process, requiring no manual intervention from video input to result output, with a diagnostic time of less than 10 minutes.

[0022] The targeted solution provided by this invention has the following beneficial effects: 1. This invention adopts a multi-task end-to-end iris segmentation and localization strategy, integrating iris segmentation mask prediction and inner and outer contour parameter extraction into the same network, achieving deep collaboration and mutual promotion between the two tasks. This design avoids the process complexity and human design errors caused by relying on a large number of post-processing steps in traditional methods. It can directly output accurate segmentation results and localization parameters from the original eye-tracking video images, effectively simplifying the overall processing flow, improving the stability and recognition accuracy of the system, and laying a reliable foundation for the accurate generation of subsequent three-dimensional nystagmus data. 2. This invention introduces distortion correction and iris unfolding technology in the three-dimensional nystagmus detection process, mapping the non-concentric and non-coaxial elliptical iris region into a regular rectangular space, which significantly simplifies the implementation difficulty of the template matching algorithm. This method can effectively overcome measurement fluctuations caused by changes in viewing angle or differences in eye structure, and truly restore the iris morphological characteristics, thereby ensuring accurate extraction of eye movement data in the horizontal, vertical and torsional dimensions without the need for additional noise reduction preprocessing, thus improving data generation efficiency and overall system operating speed. 3. This invention constructs a complete automatic auxiliary diagnostic system that organically combines iris segmentation and localization, three-dimensional nystagmus detection, and disease classification models to achieve end-to-end processing from eye-tracking video input to BPPV classification result output. The system can automatically identify and label iris regions and output corresponding disease prediction results, which helps doctors make rapid and objective diagnostic decisions, reduce missed diagnoses or misdiagnoses caused by human factors, improve clinical work efficiency and diagnostic reliability, and provide feasible technical support for the accurate auxiliary diagnosis of BPPV. Attached Figure Description

[0023] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained from these drawings without creative effort.

[0024] Figure 1 This is a schematic diagram of the overall process provided by an embodiment of the present invention; Figure 2 The Detr-based end-to-end precise iris segmentation and localization network provided for embodiments of the present invention; Figure 3 Examples of sampling points provided for embodiments of the present invention (original deformable attention module in the left figure and improved deformable attention module in the right figure). Figure 4 A Transformer decoder integrating an improved deformable attention module is provided for embodiments of the present invention; Figure 5 Three-dimensional nystagmus detection provided for embodiments of the present invention; Figure 6 The design of a deep learning-based classification model is provided for embodiments of the present invention. Detailed Implementation

[0025] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0026] Example 1 Reference Figures 1-6 This invention provides a technical solution: an automatic assisted diagnosis system for BPPV based on eye-tracking video, characterized in that it includes: The image acquisition module is used to acquire videos of the patient's eye movements. The iris segmentation and localization module is used to extract the iris region from eye-tracking videos and locate its inner and outer contours. The three-dimensional nystagmus detection module is used to generate eye movement data in three dimensions: horizontal, vertical, and torsional, based on iris information. The classification and diagnosis module is used to classify BPPV types based on 3D eye-tracking data; The output module is used to output the diagnostic results.

[0027] The iris segmentation and localization module is based on an improved DETR network and includes: Multi-task branching, simultaneously outputting iris segmentation mask and inner and outer elliptical contour parameters; The improved deformable attention module adjusts the sampling points according to the reference ellipse and embeds angle information. The sampling point position is determined by the following formula:

[0028] in: As a reference point, This is the offset. For rotation matrix, The angle is an ellipse.

[0029] The multi-task branches include: Segmentation Branch: Fuse backbone network features with Transformer encoder features to generate pixel-level embedding maps, and perform dot product with query embeddings to output segmentation masks; Positioning branch: Outputs the center coordinates, major and minor axis lengths, and rotation angles of the inner and outer ellipses of the iris.

[0030] The three-dimensional oculomotor detection module includes: The distortion correction unit is used to transform non-concentric elliptical iris regions into concentric circles, which is achieved through an affine transformation matrix:

[0031] Where: T is the affine transformation matrix; The iris unfolding unit unfolds the corrected iris region into a rectangle with a fixed width of 360 pixels and a height determined by the difference between the inner and outer radii.

[0032] The 3D eye-tracking data generation unit includes: Horizontal or vertical velocity calculation: obtained by applying a difference to the pupil center coordinate sequence; Torsional velocity calculation: The similarity of iris patterns between consecutive frames and the torsion angle are calculated using a template matching algorithm. Determined by the following formula:

[0033] Wherein: NCC represents the normalized cross-correlation function.

[0034] In the template matching algorithm, the reference iris pattern is extended by ±15 degrees to cover the maximum torsion range, avoiding abrupt changes at the boundary between 0° and 360°.

[0035] The classification and diagnosis module employs a one-dimensional convolutional neural network: 1D-CNN, specifically including: Five one-dimensional convolutional layers, each followed by batch normalization and max pooling; Three fully connected layers; The output layer uses the Softmax function to output the probabilities of various BPPV types.

[0036] The input to 1D-CNN is a three-dimensional eye-tracking velocity sequence. Each dimension is input independently and features are extracted separately, and finally the features are fused.

[0037] The system also includes: The results visualization module is used to overlay and display the iris segmentation region, inner and outer elliptical contours, and diagnostic results in the eye-tracking video. The data storage module is used to store patient videos, eye movement data, and diagnostic records.

[0038] The system supports an end-to-end automated diagnostic process, requiring no manual intervention from video input to result output, with a diagnostic time of less than 10 minutes.

[0039] Example 2 In BPPV classification and diagnosis, relying solely on 3D oculomotor data from eye-tracking videos may be limited by individual differences or noise interference. To improve the robustness and generalization ability of the classification model, this embodiment proposes a classification enhancement method that integrates multimodal information, as follows: 1. Data input, including: Three-dimensional eye-tracking data, specifically horizontal, vertical, and torsional velocity sequences; Patient's basic information, specifically age and gender; The type of positional test is either Dix-Hallpike or Roll Test; 2. Feature Fusion Module: Use 1DCNN to extract spatiotemporal features of eye-tracking sequences; A fully connected layer is used to encode patient basic information and trial type into feature vectors; The two types of features are concatenated at the feature layer to form a multimodal feature vector; 3. Classifier Design: A multilayer perceptron is used as the classifier, and the input is multimodal features; The output includes the BPPV type and confidence level. The specific BPPV types are P-BPPV, L-BPPV, and ASC-BPPV. 4. Training strategies: Use the cross-entropy loss function in combination with FocalLoss to handle class imbalance; An attention mechanism is introduced to dynamically weight the importance of different modal features during the feature fusion stage.

[0040] Example 3 Existing iris segmentation models such as Detr have high accuracy, but their computational complexity is high, making it difficult to run in real time on mobile devices or embedded systems. This embodiment proposes a lightweight iris segmentation model that is suitable for mobile BPPV-assisted diagnostic devices. 1. Model structure design: Use lightweight backbone networks, such as MobileNetV3 or ShuffleNetV2, instead of traditional CNNs; The improved deformable attention module is retained, but it is deployed in the critical layer to reduce computational cost; Use depthwise separable convolution instead of standard convolution to reduce the number of parameters and computational complexity; 2. Integrated segmentation and positioning: An elliptic parameter regression head is embedded in the encoder-decoder structure to achieve end-to-end iris mask and contour parameter output; Using knowledge distillation, the original Detr model is used as the teacher model, and the lightweight model is used as the student model to improve the performance of the smaller model. 3. Mobile optimization: Use TensorFlow Lite or ONNXRuntime for model conversion and quantization; It supports offline operation, requires no cloud dependency, and ensures patient privacy and system real-time performance. 4. System Integration: Integrate lightweight models into portable eye trackers or smartphone apps; It supports real-time iris segmentation, 3D eye-tracking generation, and preliminary BPPV risk warning.

[0041] Innovation: This patented technology employs an end-to-end iris segmentation and localization network based on Detr. Through a multi-task learning mechanism, it simultaneously achieves accurate segmentation of the iris region and direct prediction of internal and external contour parameters. The network structure integrates multi-scale features from the backbone features and the Transformer encoder to generate a pixel-level embedding map. This map is then multiplied by the query embedding in the decoder to output an iris segmentation mask. Furthermore, the network can directly regress the geometric parameters of the internal and external iris ellipses without traditional post-processing steps, achieving integrated processing from the original eye-tracking image to the segmentation and localization results. This method improves the accuracy of contour localization through collaborative supervision between tasks and provides a reliable geometric foundation for the subsequent generation of 3D eye-tracking data. This technology proposes a three-dimensional nystagmus detection method. Based on iris segmentation and localization results, it transforms the non-concentric and non-coaxial elliptical iris region into a standard concentric ring through distortion correction and iris unfolding technology, and further unfolds it into a rectangular region. Within this rectangular region, the similarity between consecutive frames is calculated using a template matching algorithm, thereby extracting eye movement velocity data in the torsional dimension. The horizontal and vertical eye movement data are obtained by tracking the positional changes of the pupil center point and then performing differential processing to obtain velocity curves. This method effectively integrates geometric correction and motion tracking, realizing the recovery of three-dimensional eye movement information containing horizontal, vertical, and torsional components from two-dimensional image sequences, providing data support for the fine diagnosis of BPPV. This technology constructs a classification and diagnostic model based on a one-dimensional convolutional neural network to identify BPPV types based on three-dimensional eye movement data. The model takes eye movement velocity time series in three dimensions (horizontal, vertical, and torsional) as input, extracts features through five layers of one-dimensional convolution, batch normalization, and pooling operations, and then performs high-level feature fusion and mapping through three fully connected layers. Finally, the disease classification result is output through a Softmax layer. This structure fully utilizes the advantages of one-dimensional convolution in temporal signal processing, realizes end-to-end recognition of nystagmus patterns, significantly improves computational efficiency, and provides a feasible path for real-time clinical auxiliary diagnosis.

[0042] The above are merely preferred embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A BPPV automatic auxiliary diagnostic system based on eye-tracking video, characterized in that, include: The image acquisition module is used to acquire videos of the patient's eye movements. The iris segmentation and localization module is used to extract the iris region from eye-tracking videos and locate its inner and outer contours. The three-dimensional nystagmus detection module is used to generate eye movement data in three dimensions: horizontal, vertical, and torsional, based on iris information. The classification and diagnosis module is used to classify BPPV types based on 3D eye-tracking data; The output module is used to output the diagnostic results.

2. The automatic auxiliary diagnosis method for BPPV based on eye-tracking video according to claim 1, characterized in that: The iris segmentation and localization module is implemented based on an improved DETR network and includes: Multi-task branching, simultaneously outputting iris segmentation mask and inner and outer elliptical contour parameters; The improved deformable attention module adjusts the sampling points according to the reference ellipse and embeds angle information. The sampling point position is determined by the following formula: ； in: As a reference point, This is the offset. Let be a rotation matrix. The angle is an ellipse.

3. The automatic auxiliary diagnosis method for BPPV based on eye-tracking video according to claim 2, characterized in that: The multi-task branches include: Segmentation Branch: Fuse backbone network features with Transformer encoder features to generate pixel-level embedding maps, and perform dot product with query embeddings to output segmentation masks; Positioning branch: Outputs the center coordinates, major and minor axis lengths, and rotation angles of the inner and outer ellipses of the iris.

4. The automatic auxiliary diagnosis method for BPPV based on eye-tracking video according to claim 3, characterized in that: The three-dimensional nystagmus detection module includes: The distortion correction unit is used to transform non-concentric elliptical iris regions into concentric circles, which is achieved through an affine transformation matrix: ； Where: T is the affine transformation matrix; The iris unfolding unit unfolds the corrected iris region into a rectangle with a fixed width of 360 pixels and a height determined by the difference between the inner and outer radii.

5. The automatic auxiliary diagnosis method for BPPV based on eye-tracking video according to claim 4, characterized in that: The three-dimensional eye-tracking data generation unit includes: Horizontal or vertical velocity calculation: obtained by applying a difference to the pupil center coordinate sequence; Torsional velocity calculation: The similarity of iris patterns between consecutive frames and the torsion angle are calculated using a template matching algorithm. Determined by the following formula: ； Wherein: NCC represents the normalized cross-correlation function.

6. The automatic auxiliary diagnosis method for BPPV based on eye-tracking video according to claim 5, characterized in that: In the template matching algorithm, the reference iris pattern is extended by ±15 degrees to cover the maximum torsion range, avoiding abrupt changes at the boundary between 0° and 360°.

7. The automatic auxiliary diagnosis method for BPPV based on eye-tracking video according to claim 6, characterized in that: The classification and diagnosis module employs a one-dimensional convolutional neural network: 1D-CNN, specifically including: Five one-dimensional convolutional layers, each followed by batch normalization and max pooling; Three fully connected layers; The output layer uses the Softmax function to output the probabilities of various BPPV types.

8. The automatic auxiliary diagnosis method for BPPV based on eye-tracking video according to claim 7, characterized in that: The input to the 1D-CNN is a three-dimensional eye-tracking velocity sequence. Each dimension is input independently and features are extracted separately, and finally, the features are fused.

9. The automatic auxiliary diagnosis method for BPPV based on eye-tracking video according to claim 8, characterized in that: The system also includes: The results visualization module is used to overlay and display the iris segmentation region, inner and outer elliptical contours, and diagnostic results in the eye-tracking video. The data storage module is used to store patient videos, eye movement data, and diagnostic records.

10. The automatic auxiliary diagnosis method for BPPV based on eye-tracking video according to claim 9, characterized in that: The system supports an end-to-end automated diagnostic process, requiring no manual intervention from video input to result output, with a diagnostic time of less than 10 minutes.

Images