Otolithiasis nystagmus information acquisition and intelligent analysis method and system

Abstract

The invention provides an otolithiasis nystagmus information acquisition and intelligent analysis method and system. The system comprises a semantic segmentation module, an image processing module and a motion analysis module. The semantic segmentation module is used for collecting a continuous dynamic eyeball image set and performing eyeball structure segmentation on eyeball images based on a semantic segmentation model; the image processing module is used for reading the eyeball structure segmentation image and extracting eyeball structure feature points based on an image processing technology; the motion analysis module is used for analyzing the motion trail of the eyeball structure feature points, decomposing and extracting nystagmus signals based on a statistical method, and analyzing the strength of the nystagmus signals. The otolithiasis nystagmus information acquisition and analysis method based on deep learning and image processing is innovatively provided, the visual anti-interference capability is good, and a visual and clear nystagmus signal distribution diagram and various nystagmus signal analysis reports such as horizontal nystagmus, vertical nystagmus and rotary nystagmus are obtained through analysis; and the clinical diagnosis precision of otolithiasis can be improved.

Description

Technical Field

[0001] The present invention relates to the technical field of medical robots, and particularly relates to a method and system for collecting and intelligently analyzing otolith nystagmus information. Background Art

[0002] Benign paroxysmal positional vertigo, also known as otolithiasis, is a common disease in the clinic. It is caused by the detachment of otolith particles on the utricular macula due to the patient suffering from a head impact, staying in a reclined position for a long time, or having an inner ear disease. When the otoliths are displaced from their original positions, if they enter the lumen of the semicircular canal, the otoliths will be displaced relative to the semicircular canal under the action of gravity, thus causing endolymph flow and making the patient sensitive to changes in gravity. Therefore, when the patient gets in and out of bed, turns over, tilts the head backward, or bends down, the head position changes relative to the direction of gravity, causing the patient to suddenly experience transient symptoms such as vertigo, nausea, vomiting, hallucination, and unsteadiness of balance. Nowadays, with the increase of people's mental stress, the incidence of this disease is also increasing continuously. With the continuous acceleration of the aging process of the population, the incidence of dizziness is also increasing continuously. Therefore, it has attracted extensive attention from medical researchers at home and abroad.

[0003] Currently, the commonly used diagnostic equipment in hospitals is greatly affected by environmental interference. When there is occlusion of the pupil by eyelashes or eyelids or the surrounding light changes, the positioning of the pupil center by the instrument will deviate, resulting in incorrect analysis results. Moreover, the analysis reports generated by the diagnostic equipment are not intuitive enough. The output result is a heuristic curve, which requires doctors to undergo special training to make judgments based on the curve, and the analysis result is relatively single. When there are special nystagmus such as rotation, doctors still need to manually discriminate the original nystagmus video. In addition, the diagnosis and treatment equipment only provides the recording of nystagmus signals and lacks the intelligent analysis function of nystagmus signals, so the diagnostic results are not clear and intuitive enough.

[0004] In order to improve the diagnostic accuracy of otolithiasis in clinical practice and reduce the learning cost of doctors for medical equipment, this patent proposes a more effective method for collecting and intelligently analyzing otolith nystagmus information, aiming to assist doctors in making more intuitive and accurate judgments on the disease, facilitating doctors to perform otolith reduction for patients, shortening the patient's medical treatment time, and improving the efficiency and accuracy of diagnosis. Summary of the Invention

[0005] The purpose of the present invention is to propose a method and system for collecting and intelligently analyzing otolith nystagmus information, which can assist doctors in making more intuitive and accurate judgments on the disease and improve the efficiency and accuracy of diagnosis.

[0006] To achieve the above purpose, the technical solution of the present invention is as follows:

[0007] On the one hand, the present invention proposes a method for collecting and intelligently analyzing otolith nystagmus information, which specifically includes the following steps:

[0008] S1. Collect a set of continuous dynamic eye images, and perform eye structure segmentation on the eye images based on a semantic segmentation model;

[0009] S2. Read the eye structure segmentation images, and extract eye structure feature points based on image processing techniques; the eye structure feature points include the pupil center and iris feature points with rotational invariance;

[0010] S3. Analyze the movement trajectories of the eye structure feature points, and decompose and extract nystagmus signals based on statistical methods, and analyze the nystagmus signal intensity.

[0011] Preferably, the Ege-Unet model is used to perform eye structure segmentation on the eye images, specifically as follows:

[0012] Construct a dataset for model training: collect a dataset of eye images under different occlusion and illumination conditions, and perform eye structure marking on the eye images; preprocess the images, including resizing the images, image normalization and standardization processing;

[0013] Use the constructed dataset to train the Ege-Unet model, optimize the model through a loss function, and the Ege-Unet model is used to extract image features and generate a segmentation mask; use the trained Ege-Unet model to perform eye structure segmentation on the eye images.

[0014] Preferably, the specific content of S2 is as follows:

[0015] Read the eye structure segmentation images and perform contour extraction, including iris and pupil contour extraction;

[0016] Extract motion feature points from the edge and texture characteristics of the eye structure through an image processing algorithm: use the center of the average inscribed circle of the pupil contour as the pupil center (X0, Y0), and use the iris internal rotation invariant feature point as the iris feature point (X1, Y1).

[0017] Preferably, the SIFT feature matching algorithm is used to calculate the iris internal rotation invariant feature points.

[0018] Preferably, the extraction of the nystagmus signal is specifically as follows:

[0019] The pupil velocity (u0, v0) is obtained by calculating the differences in the horizontal and vertical coordinates of the pupil center in adjacent time-series frames. The angular velocity ω0 of the feature points is obtained by calculating the differences in the angles between the lines connecting the pupil center and the iris feature points and the horizontal line in adjacent time-series frames. A velocity sequence is constructed based on the pupil velocity (u0, v0) and the angular velocity ω0 of the feature points to obtain the time distributions of the three types of nystagmus signals: horizontal nystagmus, vertical nystagmus, and rotary nystagmus.

[0020] The horizontal nystagmus is judged based on the sign of the horizontal component u0 of the pupil velocity.

[0021] The vertical nystagmus is judged based on the sign of the vertical component v0 of the pupil velocity.

[0022] The rotary nystagmus is judged based on the sign of the angular velocity ω0 of the feature points.

[0023] In the velocity sequence, the signs of the horizontal component u0 of the pupil velocity, the vertical component v0 of the pupil velocity, and the angular velocity ω0 of the feature points represent the movement directions of nystagmus. The critical points where the velocity signs change are the velocity extreme points of nystagmus movement. The start and end points of the nystagmus signals are determined by two adjacent velocity extreme points, and all nystagmus signals are screened and labeled one by one in the time-series frames to obtain the time distributions of the three types of nystagmus signals: horizontal nystagmus, vertical nystagmus, and rotary nystagmus.

[0024] Preferably, the analysis of the nystagmus signal intensity is specifically as follows:

[0025] Based on the velocity distributions of the pupil velocity (u0, v0) and the angular velocity ω0 of the feature points within the time distribution range of the nystagmus signal, the window fast Fourier transform is applied to obtain the nystagmus velocity conversion frequencies within the moving window time series one by one, which are used to characterize the oscillation intensity of the nystagmus signal, thereby realizing the analysis of the nystagmus signal intensity.

[0026] On the other hand, the present invention also proposes an otolith nystagmus information acquisition and intelligent analysis system. The system is implemented by using any of the above otolith nystagmus information acquisition and intelligent analysis methods, and includes: a semantic segmentation module, an image processing module, and a motion analysis module.

[0027] The semantic segmentation module is used to collect a set of continuous dynamic eye images and perform eye structure segmentation on the eye images based on a semantic segmentation model.

[0028] The image processing module is used to read the eye structure segmentation image and extract eye structure feature points based on image processing techniques. The eye structure feature points include the pupil center and iris feature points with rotational invariance.

[0029] The motion analysis module is used to analyze the motion trajectories of the eye structure feature points and decompose and extract nystagmus signals based on statistical methods, and analyze the nystagmus signal intensity.

[0030] Preferably, the semantic segmentation module includes: an image acquisition module and an image segmentation module; the image acquisition module is used to acquire a set of continuous dynamic eye images; the image segmentation module is used to segment the eye structure of the eye image.

[0031] Preferably, the image processing block includes: an edge processing module and a feature point extraction module; the edge processing module is used to extract the contour of the eye structure segmentation image; the feature point extraction module is used to extract motion feature points from the edge and texture characteristics of the eye structure, including the pupil center and iris feature points.

[0032] Preferably, the motion analysis module includes: a nystagmus signal extraction module and a nystagmus intensity calculation module; the nystagmus signal extraction module is used to obtain the time distribution of the nystagmus signal; the nystagmus intensity calculation module is used to perform nystagmus signal intensity analysis.

[0033] Compared with the prior art, the present invention has the following beneficial effects:

[0034] The present application innovatively proposes to use deep learning and image processing methods for otolith nystagmus information collection and intelligent analysis, which can assist doctors in making more intuitive and accurate judgments on the disease, facilitate doctors to perform otolith reduction for patients, shorten the patient's medical treatment time, and improve the efficiency and accuracy of diagnosis. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 is a flowchart of the method of the present invention;

[0036] Figure 2 is a schematic diagram of the system principle of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0037] The following combines the attached Figure 1-2 , and specifically describes the technical solution of the present invention.

[0038] Example 1

[0039] In this embodiment, as Figure 1 shown, an otolith nystagmus information collection and intelligent analysis system includes: a semantic segmentation module, an image processing module, and a motion analysis module.

[0040] The semantic segmentation module is used to collect a set of continuous dynamic eye images and perform eye structure segmentation on the eye images based on a semantic segmentation model.

[0041] The semantic segmentation module includes: an image acquisition module and an image segmentation module; the image acquisition module is used to acquire the eye image; the image segmentation module is used to segment the eye structure image in the eye image.

[0042] In this embodiment, a neural network algorithm is used to segment the cavity contour. The specific process is as follows: label the eye structure in the eye image dataset collected under different occluded lighting conditions; then preprocess the images, including resizing the images, normalizing and standardizing the images, as the training set; construct an Ege-Unet model to extract image features and generate a segmentation mask; use the training set data to train the Ege-Unet model, optimize the model through a loss function; finally, use the trained Ege-Unet model to extract the eye structure; finally, post-process the segmentation result to obtain the eye structure segmentation image.

[0043] The image processing module is used to read the eye structure segmentation image and extract the eye structure feature points based on image processing technology.

[0044] The image processing module includes: an edge processing module and a feature point extraction module; the edge processing module is used to extract the contour of the eye structure segmentation image; the feature point extraction module is used to extract motion feature points from the edge and texture characteristics of the eye structure through an image processing algorithm, with the center of the average inscribed circle of the pupil contour as the pupil center (X0, Y0), and the rotation invariant feature points obtained by using the SIFT feature matching algorithm in the iris as the iris feature points (X1, Y1).

[0045] The motion analysis module is used to analyze the motion trajectory of the eye structure feature points, decompose and extract the nystagmus signal based on statistical methods, and analyze the nystagmus signal intensity.

[0046] The motion analysis module includes: a nystagmus signal extraction module and a nystagmus intensity calculation module;

[0047] The nystagmus signal extraction module obtains the pupil velocity (u0, v0) by calculating the difference in the horizontal and vertical coordinates of the pupil center in adjacent time series frames, obtains the angular velocity ω0 of the feature points by calculating the difference in the angle between the line connecting the pupil center and the iris feature points and the horizontal line in adjacent time series frames, and constructs a velocity sequence based on the pupil velocity (u0, v0) and the angular velocity ω0 of the feature points to obtain the time distribution of the three nystagmus signals of horizontal nystagmus, vertical nystagmus, and torsional nystagmus;

[0048] The horizontal nystagmus is judged according to the sign of the horizontal component u0 of the pupil velocity; the vertical nystagmus is judged according to the sign of the vertical component v0 of the pupil velocity; the torsional nystagmus is judged according to the sign of the angular velocity ω0 of the feature points;

[0049] The positive and negative values of the horizontal component u0 of the pupil velocity, the vertical component v0 of the pupil velocity, and the angular velocity ω0 of the feature point in the velocity sequence represent the movement direction of nystagmus. The critical point where the velocity changes from positive to negative is the velocity extreme point of the nystagmus movement. From the characteristics of the fast and slow directions of nystagmus, the velocity sequence between the two extreme points is a complete segment of fast or slow nystagmus. The number of velocity sequences within the fast nystagmus is less than that within the slow nystagmus, so as to determine the fast movement sequence. Therefore, the starting point and ending point of the nystagmus signal are determined by two adjacent velocity extreme points, and all nystagmus signals are screened and labeled one by one in the time series frame to obtain the time distribution of three types of nystagmus signals: horizontal nystagmus, vertical nystagmus, and rotational nystagmus.

[0050] Based on the velocity distribution of the pupil velocity (u0, v0) and the angular velocity ω0 of the feature point within the time distribution range of the nystagmus signal, the nystagmus intensity calculation module applies the window fast Fourier transform to obtain the nystagmus velocity conversion frequency within the moving window time series one by one, which is used to characterize the oscillation intensity of the nystagmus signal and realize the analysis of the nystagmus signal intensity.

[0051] Embodiment 2

[0052] In this embodiment, a method for collecting and intelligently analyzing otolith nystagmus information includes the following steps:

[0053] S1. Collect a set of continuous dynamic eye images, and perform eye structure segmentation on the eye images based on a semantic segmentation model. In this embodiment, the Ege-Unet model is used to perform eye structure segmentation on the eye images (the specific method refers to Embodiment 1).

[0054] S2. Read the eye structure segmentation image, and extract eye structure feature points based on image processing technology. The eye structure feature points include the pupil center and iris feature points with rotational invariance.

[0055] Read the eye structure segmentation image and perform contour extraction, including iris and pupil contour extraction.

[0056] Extract the motion feature points from the edge and texture characteristics of the eye structure. Take the center of the average inscribed circle of the pupil contour as the pupil center (X0, Y0), and take the rotation-invariant feature points obtained by using the SIFT feature matching algorithm within the iris as the iris feature points (X1, Y1).

[0057] S3. Analyze the motion trajectories of the eye structure feature points, decompose and extract the nystagmus signal based on statistical methods, and analyze the nystagmus signal intensity.

[0058] The pupil velocity (u0, v0) is obtained by calculating the difference in the horizontal and vertical coordinates of the pupil center in adjacent time-series frames, and the angular velocity ω0 of the feature point is obtained by calculating the difference in the angle between the line connecting the pupil center and the iris feature point and the horizontal line in adjacent time-series frames; the velocity extreme points where the velocity direction changes are screened from the velocity sequence, and the start point and end point of the nystagmus signal are determined based on the velocity extreme points, so as to screen all nystagmus signals in the time-series frames and label them one by one, and obtain the time distribution of the nystagmus signal;

[0059] Based on the velocity distribution of the pupil velocity (u0, v0) and the angular velocity ω0 of the feature point within the time distribution range of the nystagmus signal, the window fast Fourier transform is applied to obtain the nystagmus velocity conversion frequency within the moving window time series one by one, which is used to characterize the oscillation intensity of the nystagmus signal, and the nystagmus signal intensity analysis is realized.

[0060] The above embodiments are only descriptions of the preferred modes of the present application, and do not limit the scope of the present application. Without departing from the design spirit of the present application, various deformations and improvements made by those of ordinary skill in the art to the technical solutions of the present application shall fall within the protection scope determined by the claims of the present application.

Claims

1. An otolithiasis nystagmus information acquisition and intelligent analysis method, characterized in that Specifically, it includes the following steps: S1. Collect a set of continuous dynamic eye images, and segment the eye structure of the eye images based on a semantic segmentation model; S2. Read the segmented eye structure images, and extract the eye structure feature points based on image processing techniques; the eye structure feature points include the pupil center and iris feature points with rotational invariance; S3. Analyze the motion trajectories of the eye structure feature points, and decompose and extract the nystagmus signals based on statistical methods, and analyze the nystagmus signal intensity.

2. The otolith nystagmus information acquisition and intelligent analysis method according to claim 1, characterized in that Use the Ege-Unet model to segment the eye structure of the eye images, specifically as follows: Construct a dataset for model training: collect a dataset of eye images under different occlusion and illumination conditions, and mark the eye structure of the eye images; preprocess the images, including resizing the images, image normalization and standardization processing; Use the constructed dataset to train the Ege-Unet model, optimize the model through a loss function, and the Ege-Unet model is used to extract image features and generate a segmentation mask; use the trained Ege-Unet model to segment the eye structure of the eye images.

3. The otolith nystagmus information acquisition and intelligent analysis method according to claim 1, characterized in that, The specific content of S2 is as follows: Read the segmented eye structure images and perform contour extraction, including iris and pupil contour extraction; Extract motion feature points from the edge and texture characteristics of the eye structure through image processing algorithms: use the center of the average inscribed circle of the pupil contour as the pupil center (X0, Y0), and use the rotationally invariant feature points inside the iris as the iris feature points (X1, Y1).

4. The otolith nystagmus information acquisition and intelligent analysis method according to claim 3, characterized in that Use the SIFT feature matching algorithm to calculate the rotationally invariant feature points inside the iris.

5. A method for collecting and intelligently analyzing otolith nystagmus information according to claim 3, characterized in that, The extraction of the nystagmus signal is specifically as follows: Obtain the pupil velocity (u0, v0) by calculating the difference in the horizontal and vertical coordinates of the pupil center in adjacent time series frames, obtain the angular velocity ω0 of the feature points by calculating the difference in the angle between the line connecting the pupil center and the iris feature points and the horizontal line in adjacent time series frames, and construct a velocity sequence according to the pupil velocity (u0, v0) and the angular velocity ω0 of the feature points to obtain the time distribution of the three nystagmus signals of horizontal nystagmus, vertical nystagmus, and torsional nystagmus; The horizontal nystagmus is judged according to the positive or negative of the horizontal component u0 of the pupil velocity; The vertical nystagmus is judged according to the positive or negative of the vertical component v0 of the pupil velocity; The torsional nystagmus is judged according to the positive or negative of the angular velocity ω0 of the feature points; The positive or negative of the horizontal component u0 of the pupil velocity, the vertical component v0 of the pupil velocity, and the angular velocity ω0 of the feature points in the velocity sequence represent the motion direction of the nystagmus, and the critical point of the velocity positive or negative change is the velocity extreme point of the nystagmus motion; determine the starting point and ending point of the nystagmus signal by two adjacent velocity extreme points, and screen all nystagmus signals in the time series frame and mark them one by one to obtain the time distribution of the three nystagmus signals of horizontal nystagmus, vertical nystagmus, and torsional nystagmus.

6. The otolith nystagmus information acquisition and intelligent analysis method according to claim 5, characterized in that, The analysis of the nystagmus signal intensity is specifically as follows: Based on the velocity distribution of the pupil velocity (u0, v0) and the angular velocity ω0 of the feature points within the time distribution range of the nystagmus signal, apply the window fast Fourier transform to obtain the nystagmus velocity conversion frequency within the moving window time series one by one, which is used to characterize the oscillation intensity of the nystagmus signal and realize the analysis of the nystagmus signal intensity.

7. An otolithiasis nystagmus information acquisition and intelligent analysis system, characterized in that, The system is implemented by using the otolith nystagmus information acquisition and intelligent analysis method described in any one of claims 1-6, and includes: a semantic segmentation module, an image processing module, and a motion analysis module; The semantic segmentation module is used to collect a set of continuous dynamic eye images and perform eye structure segmentation on the eye images based on a semantic segmentation model; The image processing module is used to read the eye structure segmentation image and extract the eye structure feature points based on image processing techniques; the eye structure feature points include the pupil center and iris feature points with rotational invariance; The motion analysis module is used to analyze the motion trajectory of the eye structure feature points, decompose and extract the nystagmus signal based on statistical methods, and analyze the nystagmus signal intensity.

8. The otolith nystagmus information acquisition and intelligent analysis system according to claim 7, characterized in that, The semantic segmentation module includes: an image acquisition module and an image segmentation module; The image acquisition module is used to collect a set of continuous dynamic eye images; The image segmentation module is used to perform eye structure segmentation on the eye images.

9. The otolithiasis nystagmus information acquisition and intelligent analysis system according to claim 7, characterized in that, The image processing block includes: an edge processing module and a feature point extraction module; The edge processing module is used to extract the contour of the eye structure segmentation image; The feature point extraction module is used to extract motion feature points from the edge and texture characteristics of the eye structure, including the pupil center and iris feature points.

10. The otolithiasis nystagmus information acquisition and intelligent analysis system according to claim 7, characterized in that, The motion analysis module includes: a nystagmus signal extraction module and a nystagmus intensity calculation module; The nystagmus signal extraction module is used to obtain the time distribution of the nystagmus signal; The nystagmus intensity calculation module is used to realize the analysis of the nystagmus signal intensity.

Images