A Parkinson's disease recognition method based on the SlowFast two-stream knowledge distillation network

Abstract

The present invention discloses a Parkinson's recognition method based on a SlowFast dual-stream knowledge distillation network. First, MRI data is converted into PNG data and then into MP4 video data to generate a file list, and the training set and test set are divided. Then, the training set data is fed into the SlowFast network for feature extraction. Finally, knowledge distillation is used to generalize the network to ensure that the network has strong generalization ability. The present invention can greatly reduce the workload of medical staff, improve the work efficiency of doctors, enable rapid screening, and at the same time enable remote medical diagnosis.

Description

Technical Field

[0001] The present invention belongs to the field of biomedical electronics technology, and particularly relates to a Parkinson's recognition method based on a SlowFast two-stream knowledge distillation network. Background Art

[0002] In traditional medical diagnosis methods, doctors usually rely on years of film reading experience and habits to observe and analyze diseases from Magnetic Resonance Imaging (MRI) images obtained from medical devices. The image structure is complex and the information entropy is high. It is difficult for doctors to analyze whether the patient is ill through MRI images in a short time, and factors such as visual fatigue caused by long-term film reading are likely to cause misdiagnosis. Moreover, due to factors such as long film reading time and few radiologists, the time is relatively long, which may lead to delayed diagnosis and treatment.

[0003] In the current diagnosis and treatment plan, clinicians usually need to post-process MRI medical images on a workstation and diagnose and analyze the condition based on the results of the post-processing. This manual operation not only has a large workload and low efficiency, but also due to a large amount of human intervention, there is a certain subjective judgment in the processing results.

[0004] Brief Introduction of the Existing Technology:

[0005] (1) Patent - Three-dimensional Reconstruction Method, Device, Equipment and Storage Medium for Coronary Vessels (Chinese Patent, Application Publication No. CN 107392994 A, Application Publication Date November 24, 2017). This invention belongs to the field of computer technology and provides a three-dimensional reconstruction method, device, equipment and storage medium for coronary vessels. The method includes preprocessing coronary angiography images, extracting vascular edge contours and two-dimensional guidewires, performing inner and outer membrane segmentation on intravascular ultrasound images, translating the two-dimensional guidewires in images located in a first angiographic plane and a second angiographic plane to the same starting point, constructing a vertically intersecting surface, setting the intersection line as the three-dimensional guidewire, arranging each frame of image at equal intervals on the three-dimensional guidewire and rotating it to be perpendicular to the tangent vector at the corresponding position, rotating the image in the plane perpendicular to the tangent vector and back-projecting it onto the image, and determining the optimal orientation angle according to the back-projection and the distance from the vascular edge contour to the three-dimensional guidewire, and finally reconstructing the vascular surface, so as to simultaneously check the external morphological structure and internal cavity lesion information of the blood vessels, improving the efficiency and accuracy of three-dimensional reconstruction of coronary vessels. However, when performing three-dimensional segmentation of coronary arteries, very small blood vessels may not be traceable due to reasons such as vascular bifurcations.

[0006] (2) Paper - (Zhang Yuqian & Gu Dongyun. (2019). Review of Computer-Aided Diagnosis Methods for Parkinson's Tremor and Essential Tremor. Computer Science (07), 22-29.) This paper mentions a method of computer-aided diagnosis through multi-sensor data. Multiple sensors such as wearable accelerometers, wearable gyroscopes, and wearable surface electromyography sensors are worn on individuals suspected of having the disease, and then the individuals are asked to complete a number of prescribed actions, and the data of multiple sensors are recorded. Then the data is sent into a neural network or a support vector machine for auxiliary diagnosis. This method is relatively complex to implement. It not only brings inconvenience to the daily life of patients as they need to wear various sensors for a long time, but also due to the fact that sensors are easily affected by the surrounding environment, resulting in a large amount of noise in the collected data, so the effect of auxiliary diagnosis is not very good.

[0007] (3) Paper - (Song Ge, Wang Miao, Gao Zhongbao, Wang Wei, Chen Tong, Lou Ran & Wang Zhenfu. (2020). An Exploratory Clinical Study of an Artificial Intelligence Speech Analysis System in the Diagnosis of Parkinson's Disease. Chinese Journal of Geriatric Heart, Brain and Vascular Diseases (05), 514-519.) This paper proposes a Parkinson's auxiliary diagnosis system based on a speech system. First, 5s of audio of the Chinese vowels [a], [o], and [i] of the diseased individuals is collected, and then the following features are extracted using various existing algorithms: pitch or fundamental frequency, fundamental frequency change rate, fundamental frequency perturbation, amplitude perturbation, harmonic-to-noise ratio, etc. Then, the KNN machine learning method is used to analyze the extracted data to obtain the differences in vowel pronunciation between diseased individuals and normal individuals. However, this method has poor generalization performance. It only works on the Chinese vowel system and cannot be promoted worldwide. Moreover, it only analyzes based on three vowel sounds, lacking rigor. Other diseases may also cause symptoms such as slurred speech, monotone pitch, weakened volume, hoarseness, difficulty in speaking, incoordination in voice production, and decreased speech clarity. Summary of the Invention

[0008] In order to overcome the deficiencies of the prior art, the present invention provides a Parkinson's recognition method based on a SlowFast dual-stream knowledge distillation network. First, MRI data is converted into PNG data and then into MP4 video data to generate a file list, and the training set and test set are divided; then the training set data is sent into the SlowFast network for feature extraction; finally, knowledge distillation is used to generalize the network to ensure that the network has strong generalization ability. The present invention can greatly reduce the workload of medical staff, improve the work efficiency of doctors, enable rapid screening, and at the same time can perform remote medical diagnosis.

[0009] The technical solutions adopted by the present invention to solve its technical problems include the following steps:

[0010] Step 1: Convert the MRI data into PNG data and then into an MP4 format file, generate a file list, and divide the MP4 format files into a training set and a test set;

[0011] Step 1-1: The input original image is NII format image data, i.e., three-dimensional CT image data. Convert the NII format image data into PNG pictures, where each NII format image data can be converted into several PNG pictures;

[0012] Step 1-2: Pack the PNG pictures into an MP4 video file with a specified frame rate;

[0013] Step 1-3: Traverse all the MP4 video files to generate a file list; when generating the MP4 video files, mark whether the patient is diseased, and mark clearly whether the example corresponding to the video file is positive when generating the file list;

[0014] Step 1-4: Use a random algorithm to divide the MP4 video files into a training set and a test set;

[0015] Step 2: Feed the training set generated in Step 1 into the SlowFast network for feature extraction;

[0016] Step 2-1: Perform dense frame sampling and sparse frame sampling on the videos in each training set to obtain two different feature vectors;

[0017] Step 2-2: Respectively perform feature extraction on the two different vectors obtained in Step 2-1 through one of the ResNet3D in the joint model SlowFast composed of 2 ResNet3D to obtain the final feature vector;

[0018] Step 2-3: Calculate the loss of the SlowFast model through the cross-entropy loss function;

[0019] Step 2-4: Clear the gradients of the optimizer to prepare for the next iteration;

[0020] Step 2-5: Update the model gradients and perform the next round of iteration;

[0021] Step 2-6: Finally, obtain the trained SlowFast model;

[0022] Step 3: Generalize the SlowFast model using knowledge distillation;

[0023] Step 3-1: Segment the red nucleus and substantia nigra parts existing in the video files in the training set;

[0024] Step 3-2: Build a distillation structure using the SlowFast model. Use the network with part of the data from the substantia nigra rubra as the student network, and the network with the MP4 video data in the training set as the teacher network for distillation. After each round of distillation, use the momentum update method to update the teacher network with the student network, and finally achieve the generalization of the SlowFast model.

[0025] The beneficial effects of the present invention are as follows:

[0026] The present invention can greatly reduce the workload of medical staff, improve the work efficiency of doctors, enable rapid screening, and at the same time perform remote medical diagnosis. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 The structure diagram of the SlowFast model of the present invention.

[0028] Figure 2 The schematic diagram of knowledge distillation of the present invention.

[0029] Figure 3 The flowchart of the method of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0030] The present invention will be further described below with reference to the drawings and embodiments.

[0031] The present invention is a medical image intelligent post-processing technology for brain CT. It is commonly used in Parkinson's examinations. CPR (Parkinsons Disease) (medicine) is used to predict whether a sample is diseased based on medical images.

[0032] The present invention is based on the SlowFast two-stream knowledge distillation deep learning network for Parkinson's prediction. It uses the video data converted from MRI as metadata, uses the SlowFast algorithm for feature extraction, and uses knowledge distillation for generalization enhancement, which can ensure good accuracy and appropriate generalization of the algorithm prediction results.

[0033] A method for Parkinson's recognition based on the SlowFast two-stream knowledge distillation network includes the following steps:

[0034] Step 1: Convert the MRI data into PNG data and then into MP4 format files, generate a file list, and then divide the MP4 format files into a training set and a test set;

[0035] Step 1-1: The input original image is NII format image data, i.e., three-dimensional CT image data. Convert the NII format image data into PNG pictures, and each NII format image data can be converted into several PNG pictures;

[0036] Step 1-2: Package the PNG images into an MP4 video file with a specified frame rate;

[0037] Step 1-3: Traverse all the MP4 video files to generate a file list; when generating the MP4 video files, mark whether the patient is ill, and when generating the file list, clearly mark whether the sample corresponding to the video file is positive;

[0038] Step 1-4: Use a random algorithm to divide the MP4 video files into a training set and a test set;

[0039] Step 2: Feed the training set generated in Step 1 into the SlowFast network for feature extraction;

[0040] Step 2-1: Perform dense frame sampling and sparse frame sampling on the videos in each training set to obtain two different feature vectors;

[0041] Step 2-2: Respectively perform feature extraction on the two different vectors obtained in Step 2-1 through one of the ResNet3D in the combined model SlowFast composed of 2 ResNet3D to obtain the final feature vector;

[0042] Step 2-3: Calculate the loss of the SlowFast model through the cross-entropy loss function;

[0043] Step 2-4: Clear the gradients of the optimizer to prepare for the next iteration;

[0044] Step 2-5: Update the model gradients and perform the next round of iteration;

[0045] Step 2-6: Finally obtain the trained SlowFast model;

[0046] Step 3: Generalize the SlowFast model using knowledge distillation;

[0047] Step 3-1: Segment the red nucleus and substantia nigra parts existing in the video files in the training set;

[0048] Step 3-2: Use the SlowFast model to construct a distillation structure. Take the network using the data of the red nucleus and substantia nigra parts as the student network, and the network using the MP4 video data in the training set as the teacher network for distillation. After each round of distillation, use the momentum update method to update the teacher network with the student network, and finally achieve the generalization of the SlowFast model. Specific embodiments:

[0050] Parkinson's disease is one of the neurodegenerative diseases with a relatively high incidence clinically, and most cases occur in people over 60 years old. The clinical symptoms of Parkinson's disease patients mainly include resting tremor, bradykinesia, muscle rigidity, postural and gait disorders, etc. Non-motor symptoms such as hyposmia, insomnia, constipation, depression, and cognitive impairment may also occur, which will have a greater impact on the patient's quality of life, social function, and prognosis.

[0051] Deep learning can automatically learn the features of samples by constructing a network model and then give results. Based on deep learning, a model that can correctly predict Parkinson's disease can be obtained by training MRI data, which can help doctors screen individuals with a high probability of getting the disease to a great extent, improve the doctor's work efficiency, and increase more treatment time for patients. At the same time, through continuous learning, the software can be made more perfect and can be applied to areas with scarce medical resources to serve everyone.

[0052] By converting MRI data into videos as metadata for prediction and training, it not only ensures the integrity of Nii data but also emphasizes the temporal information of the data. Through knowledge distillation, it brings a certain degree of generalization to the model and provides a good guarantee for comprehensive evaluation of the patient's symptoms.

[0053] The first step is to traverse all the data to determine whether they are all in NII format. The NIFTI (NII) format was originally invented for neuroimaging. The Neuroimaging Informatics Technology Initiative (NIFTI) presets the NIFTI format as an alternative to the ANALYZE7.5 format. Its original application field is neuroimaging, but it is also used in other fields. The main feature of this format is that it contains two affine coordinates that can associate the index (i, j, k) of each voxel with its spatial position (x, y, z).

[0054] For a brain NII file, it is a set of slices of the brain continuously made in a certain direction. Taking it as a video means combining slices at different levels in sequence and at a fixed frame rate to form a video file.

[0055] The second step is to extract a part of the red nucleus and substantia nigra. Generally speaking, for NII files generated by the same scanning instrument, the positions of the red nucleus and substantia nigra appear roughly fixed. Therefore, by counting the distribution of the red nucleus and substantia nigra in different data files, a range can be deduced, and each video is sampled within a fixed interval to obtain video samples of the red nucleus and substantia nigra.

[0056] The third step is to send all the videos and the videos containing only the red nucleus and substantia nigra into two independent slowfast networks for training.

[0057] The fourth step is to perform reverse knowledge distillation. Knowledge distillation is a technique for pruning by using a large network as the teacher and a small network as the student model to learn the output of the large network. Reverse distillation means that the student continuously influences the teacher during the learning process. Under such continuous influence, the generalization ability of the teacher is enhanced. Use the model that takes all the data as the teacher and the model that only takes the substantia nigra and red nucleus as the student, and use the model of the substantia nigra and red nucleus to update the teacher model to improve the generalization ability.

Claims

1. A Parkinson's recognition method based on the SlowFast two-stream knowledge distillation network, characterized in that, It includes the following steps: Step 1: Convert MRI data into PNG data and then into an MP4 format file, generate a file list, and divide the MP4 format file into a training set and a test set; Step 1-1: The input original image is NII format image data, i.e., three-dimensional CT image data. Convert the NII format image data into PNG pictures, where each NII format image data can be converted into several PNG pictures; Step 1-2: Package the PNG pictures into an MP4 video file with a specified frame rate; Step 1-3: Traverse all MP4 video files to generate a file list; when generating the MP4 video file, mark whether the patient is ill, and mark clearly whether the sample corresponding to the video file is positive when generating the file list; Step 1-4: Use a random algorithm to divide the MP4 video files into a training set and a test set; Step 2: Send the training set generated in Step 1 into the SlowFast network for feature extraction; Step 2-1: Perform dense frame sampling and sparse frame sampling on the videos in each training set to obtain two different feature vectors; Step 2-2: Respectively perform feature extraction on the two different vectors obtained in Step 2-1 through one of the ResNet3D in the joint model SlowFast composed of 2 ResNet3D to obtain the final feature vector; Step 2-3: Calculate the loss of the SlowFast model through the cross-entropy loss function; Step 2-4: Clear the gradients of the optimizer to prepare for the next iteration; Step 2-5: Update the model gradients and perform the next round of iteration; Step 2-6: Finally, obtain the trained SlowFast model; Step 3: Use knowledge distillation to generalize the SlowFast model; Step 3-1: Segment the red nucleus and substantia nigra parts existing in the video files in the training set; Step 3-2: Use the SlowFast model to construct a distillation structure. Take the network using the data of the red nucleus and substantia nigra parts as the student network, and the network using the MP4 video data in the training set as the teacher network for distillation. After each round of distillation, use the momentum update method to update the teacher network with the student network, and finally achieve the generalization of the SlowFast model.

Images