A Parkinson's Hand Tremor Recognition Method Based on an Event Camera

Abstract

The present invention discloses a method for identifying Parkinson's hand tremors based on an event camera, including: collecting video data of a subject through the event camera, and using a DV platform to collect the collected video data to obtain an asynchronous event stream of a hand tremor sequence; preprocessing the asynchronous event stream by using a denoising algorithm based on optical flow velocity; extracting characteristic parameters of Parkinson's disease tremor signals from the preprocessed asynchronous event stream through discrete Fourier transform, and dividing the characteristic parameters into a training data set and a test data set; inputting the training data set into an identification network model for training, and judging whether the subject has tremors through a tremor classification method based on a support vector machine (SVM). The present invention uses a new type of sensor, the event camera, for data collection, which meets the requirements of clinical tremor detection, does not affect the patient's movements, does not cause discomfort to the patient, and can perform long-term detection.

Description

Technical Field

[0001] The present invention relates to the technical fields of medical health and computer vision, and particularly relates to a method for identifying Parkinson's hand tremors based on an event camera. Background Art

[0002] Parkinson's disease, also known as paralysis agitans, is one of the most common neurodegenerative diseases, mostly occurring in middle-aged and elderly people. Tremor is the main symptom of Parkinson's disease, which is an involuntary, rhythmic muscle contraction and relaxation, involving the swinging or twitching movement of one or more body parts. The most common sign of Parkinson's disease tremor is the slow tremor of the patient's hand at rest, which weakens with the patient's voluntary movement and disappears when the patient falls asleep. Since the early symptoms of Parkinson's disease are easily confused with the aging of the patient's body functions, it brings great difficulties to clinical early diagnosis. The diagnosis of Parkinson's tremor has always been a difficult problem in clinical practice, and correct diagnosis and medication are particularly important for the treatment and rehabilitation of patients.

[0003] The existing diagnosis methods for Parkinson's hand tremors are diversified. According to the data acquisition method, they can be divided into contact device data acquisition and visual camera data acquisition. Contact devices can easily extract the characteristic descriptions of hand tremors. According to the nature of the signals they collect, they can be further divided into bioelectric signals and non-bioelectric signals. The bioelectric signals collected by contact methods are obtained by surrounding and closely attaching electrode patches to the skin surface to collect the action potential signals of the examined muscle contractions. In addition to the mobility of the wires affecting the signal acquisition, electromagnetic interference and other factors of the wires will also affect the signal acquisition. Moreover, the bioelectric signals of the human body are very weak and have individual differences, and are extremely susceptible to the influence of sweat on the skin surface. The most representative contact device for collecting non-bioelectric signals is the wearable instrument. The wearable instrument is essentially a wired interface with a sensing unit, and all sensing units are in contact with finger or joint parts. The disadvantage is that the detection process requires the patient to wear a complex detection device, and the process is cumbersome. The inconvenience caused by its attached connection wires and complex wearing has resulted in an unfree human-computer interaction operation.

[0004] The data acquisition method based on a vision camera enables the operator to perform human-computer interaction in a more natural way and has greater flexibility, so it has received more research and attention. Usually, an ordinary camera (traditional RGB camera) is used for data acquisition. When the object changes slowly, it has little impact on imaging. However, when encountering strong light, dim light, or when the object changes very fast, it is difficult for an ordinary camera to capture clear images, and there will be overexposed, underexposed, or blurred motion pictures. The Parkinson's hand tremor data collected by an ordinary camera is based on each frame for calculation. If the picture of each frame is not clear, it is impossible to distinguish the patient's hand movements and their feature points. For the phenomenon of motion blur, if the method of increasing the frame rate is used for shooting, theoretically, if the frame rate is fast enough, the picture can be relatively static. However, in fact, after increasing the frame rate, there are higher requirements for the processing algorithm, and it is necessary to complete the calculation in a very short time, otherwise it will lag significantly behind the actual time flow rate. For overexposed or underexposed images caused by excessive light differences, some key object information will be lost, which poses a great challenge to the feature recognition algorithm. For the detection of Parkinson's tremors, there is still no non-invasive, simple and effective technical solution proposed in China. Summary of the Invention

[0005] The purpose of the present invention is to provide a method for identifying Parkinson's hand tremors based on an event camera, which extracts features by collecting video data through the event camera, realizes a non-invasive diagnosis technology for Parkinson's tremor detection, and greatly protects the privacy of patients.

[0006] To achieve the above object, the present application proposes a method for identifying Parkinson's hand tremors based on an event camera, including:

[0007] Collect video data of the subject through the event camera, and use the DV platform to collect the collected video data to obtain an asynchronous event stream of the hand tremor sequence;

[0008] Preprocess the asynchronous event stream by using a noise reduction algorithm based on the optical flow velocity;

[0009] Extract the characteristic parameters of the Parkinson's disease tremor signal from the preprocessed asynchronous event stream through discrete Fourier transform, and divide the characteristic parameters into a training data set and a test data set. The training data set includes known normal characteristic data and tremor signal characteristic data marked according to the severity level of the disease;

[0010] Input the training data set into the recognition network model for training, and judge whether the subject has tremors through a tremor classification method based on the support vector machine SVM.

[0011] Furthermore, once the hand of the subject trembles, the event camera is triggered to collect video data; the triggering condition of the event camera is:

[0012]

[0013] P represents the polarity of the hand tremor event, and L xy represents the luminance magnitude at the (x, y) position of the hand tremor occurrence point. t and t new represent two moments before and after the hand tremor event point. w represents the threshold for triggering the hand tremor event, which can be set manually; f is a truncation function; when a hand tremor occurs, L xy (t) is updated to a new value L xy (t new ) and triggers a hand tremor event with (x i , y i , t i , p i ) as the basic unit. The (x i , y i , t i , p i ) is a quadruple asynchronously output by the event camera, called the event stream; x i is the pixel x-axis coordinate where the hand tremor event occurs, y i is the pixel y-axis coordinate where the hand tremor event occurs, t i is the timestamp when the hand tremor event occurs, and p i is the hand tremor event polarity (usually +1 indicates an increase in luminance and -1 indicates a decrease in luminance); by continuously trembling, the subject continuously triggers the asynchronous event stream of the hand tremor sequence obtained by the event camera.

[0014] Furthermore, a noise reduction algorithm based on the optical flow velocity is used to preprocess the asynchronous event stream. Specifically: a method of eliminating noise by using the motion consistency in the tremor event stream, that is, in a local area, the events caused by the hand tremor motion should be able to form a consistent "motion plane" in the spatio-temporal domain, while the noise events cannot. This property is judged by observing the optical flow velocity in the hand tremor event stream, and it can naturally evaluate the motion consistency. Specifically, to obtain the optical flow velocity of a tremor event, the corresponding hand tremor event polarity p i is removed. In three-dimensional space, the hand tremor event is represented as w i =(x i , y i , t i ). Then the plane N where w i is located is described as:

[0015] c1x i +c2y i +c3t i +c4 = 0

[0016] c1, c2, c3, and c4 are all real numbers;

[0017] For events that are very close to w both on the time axis and the space axis, i a plane is fitted by least squares:

[0018]

[0019] where (1, i) is the tremor event set, and Δt is set to 1 ms; then the velocity of the event is obtained according to the plane: the motion consistency is verified by detecting the velocity v; if v is greater than 0 and less than v max , then w is retained i , otherwise w i is classified as noise and deleted from the asynchronous event stream, and each event is continuously iterated until all asynchronous events are valid values.

[0020] Furthermore, the characteristic parameters of the Parkinson's disease tremor signal are extracted from the preprocessed asynchronous event stream through discrete Fourier transform. Specifically: the tremor signal is analyzed for characteristics, and the characteristic parameters used to distinguish the severity are extracted. The parameters related to the disease severity in the tremor signal of Parkinson's disease patients include mean value, root mean square, tremor peak value, tremor peak power, main frequency, and tremor amplitude, etc. The discrete Fourier transform has the same sampling values as the original input sequence, so the discrete Fourier transform is considered the frequency domain representation of the original input sequence. And the main frequency of the tremor signal needs to be analyzed in the frequency domain. To obtain the frequency representation of the data on the x, y, and z axes, discrete Fourier transform can be used. Specifically, the finite sequence of hand tremors with equally spaced samples is converted into a sequence of the same length with equally long samples of discrete-time Fourier transform. According to the tremor main frequency ff obtained by density functional theory DFT, the tremor peak value FF of the tremor signal within a period of time can be obtained. The tremor peak value is the combined acceleration of the accelerations on the x, y, and t axes at the tremor main frequency ff; the accelerations on the x, y, and t axes are respectively:

[0021]

[0022]

[0023] a t = 1

[0024] The combined acceleration at the main frequency ff is:

[0025]

[0026] In the process of studying Parkinson's hand tremor signals, the selection of characteristic parameters is the most crucial factor in classifier design and the basis for signal classification. To reduce the dimensionality of the algorithm, parameters with a relatively high degree of correlation with the severity of Parkinson's hand tremors are preferably selected. The root mean square value of Parkinson's hand tremor signals can quantify the severity of Parkinson's hand tremors, but the main frequency has the greatest impact on the severity of tremor signals. Finally, the main frequency and the tremor peak frequency are selected as the characteristic quantities for grading and evaluating the severity of PD tremors.

[0027] Further, the training data set is input into the recognition network model for training. The tremor classification method based on the support vector machine (SVM) is used to determine whether the subject has tremors. Specifically, through the polynomial kernel function and its parameters, the input data is mapped into different high-dimensional feature spaces, and the Lagrange coefficients are solved according to the constraint conditions. The support vectors are obtained, the threshold of the classification hyperplane is solved, and the optimal classification hyperplane of the training data set is established. The test data set is put into the classifier to obtain the tremor result. Although the classifier is a hyperplane in the feature space, it may be non-linear in the original input space. Classifying in a higher-dimensional feature space will increase the generalization error of the support vector machine. The input training data set is used to adjust the margin of the classification hyperplane.

[0028] The main frequencies and tremor peaks extracted from Parkinson's tremor signals with different severities cross each other more, and are linearly inseparable. By combining the support vector machine with the kernel function, the input data is mapped into a high-dimensional feature space to achieve the purpose of good classification effect. The kernel is a measure of sample similarity. By combining the support vector machine with the kernel method, an efficient and fast execution algorithm can be obtained through flexible and easily available prior knowledge. The commonly used kernel functions are the following three: linear kernel function, polynomial kernel function, and radial basis function. The present invention adopts the polynomial kernel function, and the formula is:

[0029] K(u, v) = exp(zln(u*v + cf))

[0030] In the above formula, z refers to the polynomial order, x and y are user-defined parameters, and cf is the setting of coef0 in the kernel function; the classification decision function of the support vector machine is:

[0031] f(x) = sign(ω * x + b * )

[0032] The feature extraction part of the recognition network model uses 4 convolutional layers, with depths increasing from 64 to 512. The size of the convolutional kernel is 3x3, and the stride is 2. After that, 2 residual layers are added. By adding the residual layers, the network model can better integrate the features extracted by the convolutional layers, and at the same time reduce the risk of gradient disappearance caused by deepening the network. The classifier part applies a 3-layer fully connected network, with the number of nodes being 1024, 512, and 2 respectively. Among them, 2 is the number of final classifications; finally, a Softmax layer is added to output the probability for each possible category, and the category with the highest probability is selected as the final classification result. The activation function between adjacent layers is the ReLu activation function; the function of the fully connected layer is to decode the features and pass them to the Softmax function to perform the classification task.

[0033] The above technical solution adopted by the present invention has the following advantages compared with the prior art:

[0034] 1. A new type of sensor event camera is used for data acquisition, which meets the requirements of clinical tremor detection, does not affect the patient's movement, does not cause discomfort to the patient, and can detect for a long time.

[0035] 2. The data collected by the event camera is event data, grayscale images, and optical flow of the quadruple (x, y, t, p). Unlike traditional cameras that can collect portrait information, it can well protect the privacy of patients.

[0036] 3. This method is less affected by light, has a high dynamic range, and has no motion blur.

[0037] 4. The denoising method used is far better than other existing denoising algorithms for dynamic vision sensors, and it distinguishes whether it is a noise event through the speed obtained from the optical flow. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 It is a flowchart of a Parkinson's hand tremor recognition method based on an event camera;

[0039] Figure 2 It is a comparison diagram of the outputs of a traditional camera and an event camera;

[0040] Figure 3 It is a diagram of event points recorded by an event camera over a period of time. DETAILED DESCRIPTION OF THE INVENTION

[0041] In order to make the purpose, technical solution and advantages of the present application clearer, the present application will be further described in detail below with reference to the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application, and are not used to limit the present application, that is, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments.

[0042] Example 1

[0043] In this example, video data is collected by an event camera for feature extraction to achieve non-invasive diagnosis of Parkinson's tremors. At the same time, it avoids phenomena such as overexposure, underexposure, or motion blur that exist in the current traditional cameras during the shooting of Parkinson's hand tremors, and also greatly protects the privacy of patients. The specific implementation method is as Figure 1 shown and may include the following steps:

[0044] Step 1: Fix the event camera Celex5 or DVS356 in an indoor scene using a tripod, connect the event camera to the computer through a usb interface, and collect data through the DV platform. Start the DV software and directly connect the output of the event camera to the visualizer; select the Record configuration in the left sidebar of DV, and the current configuration will be replaced by the standard recording configuration; record all events, frames, imu, and trigger data from the event camera;

[0045] Step 2: Preprocess the original asynchronous event stream through a noise reduction algorithm based on the optical flow velocity, remove its polarity, and obtain its optical flow information.

[0046] Step 3: Use the discrete Fourier transform to extract the characteristic parameters of the Parkinson's disease tremor signal from the preprocessed asynchronous event stream. Divide the characteristic parameters into a training data set and a test data set. The training data set includes known normal characteristic data and tremor signal characteristic data labeled according to the severity level of the disease;

[0047] Step 4: Input the training data set into the recognition network model for training, and use the tremor classification method based on the support vector machine (SVM) to determine whether the subject has tremors. Select the polynomial kernel function and its parameters, map the input data into different high-dimensional feature spaces. Solve the Lagrange coefficients according to the constraint conditions. Obtain the support vectors, solve the threshold of the classification hyperplane, and establish the optimal classification hyperplane for the training data set; put the test data set into the classifier to obtain the tremor result.

[0048] The foregoing description of the specific exemplary embodiments of the present invention is for purposes of illustration and exemplification. These descriptions are not intended to limit the invention to the precise forms disclosed, and it is apparent that many changes and variations are possible in light of the above teachings. The purpose of selecting and describing the exemplary embodiments is to explain the specific principles of the present invention and its practical applications, so that those skilled in the art can implement and utilize various different exemplary embodiments of the present invention, as well as various different selections and changes. The scope of the present invention is intended to be defined by the claims and their equivalents.

Claims

1. A method for identifying Parkinson's hand tremors based on an event camera, characterized in that, Including: Collect video data of the subject through an event camera, and use a DV platform to collect the collected video data to obtain an asynchronous event stream of the hand tremor sequence; Preprocess the asynchronous event stream using a noise reduction algorithm based on the optical flow velocity. Specifically, to obtain the optical flow velocity of a tremor event, remove the corresponding polarity p of the hand tremor event, that is, represent the hand tremor event in three-dimensional space as w i Remove it, that is, the hand tremor event in three-dimensional space is represented as w i =(x i , y i , t i ), then the plane N where w i is located is described as: c1x i +c2y i +c3t i +c4 = 0 c1, c2, c3, and c4 are all real numbers; For events that are close to w both in the time axis and the space axis i Fit a plane by least squares: Among them, (1, i) is the tremor event set, and Δt is set to 1 ms; then the velocity of the event is obtained according to the plane: the motion consistency is verified by detecting the velocity v; if v is greater than 0 and less than v max , then w is retained i , otherwise w i is classified as noise and deleted from the asynchronous event stream, while continuously iterating over each event until all asynchronous events are valid values; Extract the characteristic parameters of the Parkinson's disease tremor signal from the preprocessed asynchronous event stream through discrete Fourier transform, and divide the characteristic parameters into a training data set and a test data set. The training data set includes known normal characteristic data and tremor signal characteristic data labeled according to the disease severity level; Input the training data set into the recognition network model for training, and judge whether the subject has tremors through a tremor classification method based on the support vector machine SVM.

2. The method for identifying Parkinson's hand tremors based on an event camera according to claim 1, wherein Once the hand of the subject trembles, it triggers the event camera to collect video data; the triggering condition of the event camera is: P represents the polarity of the hand tremor event, L xy represents the brightness magnitude of the hand tremor occurrence point at the (x, y) position, t and t new represent two moments before and after the hand tremor event point, and w represents the threshold for triggering the hand tremor event; f is a truncation function; when hand tremors occur, L xy (t) is updated to a new value L xy (t new ), and a hand tremor event with (x i , y i , t i , p i ) as the basic unit is triggered. The (x i , y i , t i , p i ) is a quadruple output asynchronously by the event camera, called the event stream; x i is the pixel x-axis coordinate where the hand tremor event occurs, y i is the pixel y-axis coordinate where the hand tremor event occurs, t i is the timestamp when the hand tremor event occurs, p i is the polarity of the hand tremor event; by continuously tremoring the subject, the event camera is continuously triggered to obtain the asynchronous event stream of the hand tremor sequence.

3. The method for identifying Parkinson's hand tremors based on an event camera according to claim 1, characterized in that, Extract the characteristic parameters of the Parkinson's disease tremor signal from the preprocessed asynchronous event stream through discrete Fourier transform. Specifically, convert the finite sequence of hand tremors with equally spaced samples into a sequence of the same length of equally long samples of the discrete-time Fourier transform, and obtain the main tremor frequency ff according to the density functional theory DFT. Obtain the tremor peak FF of the tremor signal within a period of time. The tremor peak is the combined acceleration of the accelerations on the x, y, and t axes at the main tremor frequency ff; the accelerations on the x, y, and t axes are respectively: a t ＝1 The combined acceleration at the main frequency ff is: Use the main tremor frequency and the tremor peak frequency as the characteristic quantities for evaluating the tremor severity level.

4. The method for recognizing Parkinson's hand tremors based on an event camera according to claim 1, wherein, Input the training data set into the recognition network model for training, and judge whether the subject has tremors through a tremor classification method based on the support vector machine SVM. Specifically, combine the support vector machine with a kernel function to map the input data into a high-dimensional feature space. The kernel function uses a polynomial kernel function, and the formula is: K(u,v)=exp(zln(u*v+cf)) In the above formula, z refers to the polynomial order, x and y are user-defined parameters, and cf is the setting of coef0 in the kernel function; the classification decision function of the support vector machine is: f(x) = sign(ω * x + b * ) The feature extraction part of the recognition network model uses 4 convolutional layers, and the depths increase from 64 to 512; the size of the convolutional kernel is 3x3, and the stride is 2; after that, 2 residual layers are added; the classifier part applies a 3-layer fully connected network, and the number of nodes is 1024, 512, and 2 respectively; 2 is the number of the final classification; finally, a Softmax layer is added to output the probability of each possible category, and the category with the highest probability is selected as the final classification result.

Images