Hand tremor frequency detection method and system based on video analysis and machine learning, and electronic equipment

Abstract

The invention provides a hand tremor frequency detection method and system based on video analysis and machine learning and electronic equipment, and the method comprises the steps: obtaining a hand video, extracting a key point track, and generating a fusion track signal through principal component analysis; extracting a motion feature vector of the fusion signal, and predicting an optimal signal processing parameter for the current tremor mode by using a pre-trained machine learning model; and performing smooth filtering and frequency domain conversion on the original trajectory by using the parameter, thereby calculating the tremor frequency. According to the invention, the technical problems of insufficient slight tremor detection precision and weak anti-noise capability caused by the adoption of fixed signal processing parameters in the existing video analysis method can be solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of medical detection, in particular to a hand tremor frequency detection method and system based on video analysis and machine learning and an electronic device. BACKGROUND

[0002] Hand tremor is a common early symptom of Parkinson's disease, essential tremor and other nervous system diseases. Tremor frequency is an important clinical indicator for differentiating disease types, assessing disease progression and judging drug efficacy.

[0003] With the development of computer vision technology, non-contact detection methods based on video analysis have gradually become a research hotspot. Existing video detection technology usually collects hand video through a camera, extracts hand motion trajectory using an optical flow method or a key point detection algorithm, and then calculates the frequency through a fast Fourier transform method. However, existing video analysis methods have serious technical bottlenecks in actual application, especially for mild or slight tremors with small amplitudes: Existing video detection methods usually use fixed signal processing parameters to process the extracted motion trajectory. However, the amplitude of hand tremor spans a wide range, and fixed processing parameters cannot accommodate different tremor patterns. For obvious tremors, a relatively loose parameter setting can be detected; but for slight tremors, if the filter window is too large, the effective tremor signal will be eliminated; if the window is too small, the camera noise cannot be filtered out, resulting in a large detection error, or even no detection.

[0004] At the same time, there are environmental noise such as light changes and camera jitter during video collection. In the case of very small tremor amplitude, the tremor signal is often submerged in noise. Existing technologies directly analyze the trajectory of a single key point, lack effective fusion of multi-key point motion information, and result in unstable frequency calculation results in low signal-to-noise ratio conditions, making it difficult to achieve the required precision for clinical diagnosis.

[0005] Therefore, how to maintain the convenience of non-contact detection while overcoming the defects of existing video analysis methods, such as sensitivity to parameters and low detection accuracy for slight tremors, is a technical problem that needs to be solved in the field. SUMMARY

[0006] In view of the deficiencies of the prior art, the present application proposes a hand tremor frequency detection method and system based on video analysis and machine learning and an electronic device to solve the technical problem of insufficient detection accuracy of existing video analysis methods for slight tremors in the prior art.

[0007] The technical scheme adopted by the present application is a hand tremor frequency detection method, system and electronic equipment based on video analysis and machine learning. In a first implementation mode, a hand tremor frequency detection method based on video analysis and machine learning is provided, characterized in that it comprises: S1: acquiring a video sequence containing a hand, extracting coordinates of a plurality of hand key points in each frame image of the video sequence, and generating original motion trajectory data; S2: performing principal component analysis fusion processing on the original motion trajectory data to generate a fusion trajectory signal; S3: performing feature extraction on the fusion trajectory signal to obtain a motion feature vector representing a tremor pattern; S4: inputting the motion feature vector into a pre-trained machine learning model to predict an optimal signal processing parameter for the current tremor pattern; S5: performing smoothing filtering and frequency domain conversion processing on the original motion trajectory data using the optimal signal processing parameter to obtain a power spectral density distribution; S6: identifying a main peak in the power spectral density distribution within a preset frequency range, and taking the frequency corresponding to the main peak as the detection result of the hand tremor frequency.

[0008] Further, the hand key points include fingertip key points of the thumb, index finger, middle finger, ring finger and little finger.

[0009] Further, in step S2, the principal component analysis fusion processing on the original motion trajectory data is performed to generate a fusion trajectory signal, comprising: calculating the coordinate difference value of the original motion trajectory data between adjacent time frames to obtain difference motion data; reshaping the difference motion data into a two-dimensional feature matrix; performing principal component analysis on the two-dimensional feature matrix to extract a first principal component as a fusion trajectory signal.

[0010] Further, in step S3, the motion feature vector extracted from the fusion trajectory signal includes time domain features and frequency domain features; The time domain features include: average absolute amplitude, standard deviation and kurtosis of the fusion trajectory signal; The frequency domain features include: signal-to-noise ratio, power spectrum peak saliency and spectral entropy of the fusion trajectory signal in the target frequency band.

[0011] Further, in step S4, the optimal signal processing parameter includes a smoothing filter window length and a power spectrum analysis segment length. The machine learning model is a random forest regression model, and the machine learning model comprises a first prediction model and a second prediction model, which are respectively used for predicting a smoothing window length and a power spectrum analysis segment length.

[0012] Preferably, the training process of the machine learning model comprises: A sample data set containing different tremor amplitude levels is constructed, and the sample data set contains a plurality of groups of sample original motion trajectory data. For each group of sample original motion trajectory data, the parameter space is traversed, the frequency detection error under different parameter combinations is calculated, and the parameter combination with the minimum error is selected as the sample optimal parameter label. Steps S2 to S3 are performed on each group of sample original motion trajectory data to obtain a sample motion feature vector. The sample motion feature vector is used as input, and the sample optimal parameter label is used as output to train the machine learning model.

[0013] Further, in step S5, the original motion trajectory data is filtered and processed in the frequency domain using the optimal signal processing parameters to obtain a power spectrum density distribution, comprising: The smoothing filter window length in the optimal signal processing parameters is used to perform smoothing filtering and detrending processing on the original motion trajectory data using a Savitzky-Golay filter to obtain preprocessed trajectory data. The power spectrum analysis segment length in the optimal signal processing parameters is used to perform power spectrum density estimation on the preprocessed trajectory data using the Welch method to obtain the power spectrum density distribution.

[0014] Preferably, the preset frequency range is 3Hz to 12Hz.

[0015] In combination with the first implementation manner, in the second implementation manner, 9. A hand tremor frequency detection system based on video analysis and machine learning, comprising: A trajectory extraction module is configured to obtain a video sequence containing a hand, extract coordinates of a plurality of hand key points in each frame of the video sequence, and generate original motion trajectory data. A trajectory fusion module is configured to perform principal component analysis fusion processing on the original motion trajectory data to generate a fused trajectory signal. A feature extraction module is configured to perform feature extraction on the fused trajectory signal to obtain a motion feature vector representing a tremor pattern. A parameter prediction module is configured to input the motion feature vector into a pre-trained machine learning model to predict optimal signal processing parameters for the current tremor pattern. The frequency calculation module is configured to perform smoothing filtering and frequency domain conversion processing on the original motion trajectory data by using the optimal signal processing parameter, to obtain a power spectral density distribution, and to identify a main peak in the power spectral density distribution within a preset frequency range, and to take a frequency corresponding to the main peak as a detection result of the hand tremor frequency.

[0016] In a third implementation manner, in combination with the first implementation manner, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory, and the processor implements the method in the first implementation manner when executing the program.

[0017] From the above technical solutions, the present application has the following beneficial technical effects: 1. By extracting the time-frequency domain features of the motion trajectory, the optimal signal processing parameter for the current tremor mode is predicted by using the pre-trained machine learning model. This mechanism enables the system to accurately detect tremor signals of different intensities, solving the problem that the signal is erased or the noise is too large to be detected due to fixed parameters in the prior art in the detection of slight tremor, and improving the detection accuracy.

[0018] 2. The motion trajectories of multiple hand key points are fused by principal component analysis technology, and the first principal component representing the main motion mode is extracted as the analysis object. PCA fusion can effectively extract the common mode signal of multiple finger motions, while suppressing random noise generated by individual key points due to changes in illumination, camera jitter, or random finger motion. This improves the signal-to-noise ratio of the original signal, ensuring that even weak tremor signals can be accurately extracted and analyzed in a non-contact video shooting environment. BRIEF DESCRIPTION OF DRAWINGS

[0019] In order to more clearly illustrate the specific embodiments of the present application or the technical solutions in the prior art, the drawings needed in the specific embodiments or prior art description will be briefly introduced below. In all the drawings, similar elements or parts are generally identified by similar reference numerals. In the drawings, the elements or parts are not necessarily drawn according to the actual proportions.

[0020] Figure 1 The method flowchart of the embodiment 1 of the present application; Figure 2 The system structure diagram of the embodiment 2 of the present application; REFERENCE NUMERALS 21 - hand tremor frequency detection system based on video analysis and machine learning, 22 - trajectory extraction module, 23 - trajectory fusion module, 24 - feature extraction module, 25 - parameter prediction module, 26 - frequency calculation module. DETAILED DESCRIPTION

[0021] The embodiments of the technical solutions of the present application will be described in detail below with reference to the drawings. The following embodiments are only used to more clearly illustrate the technical solutions of the present application, and therefore only serve as examples, but cannot limit the protection scope of the present application.

[0022] It should be noted that, unless otherwise specified, the technical terms or scientific terms used in the present application should be understood as the usual meaning understood by the skilled person in the field to which the present application belongs.

[0023] Embodiment 1 The present embodiment provides a hand tremor frequency detection method based on video analysis and machine learning, and the working principle of embodiment 1 is described in detail as follows: The method flowchart of the present embodiment is shown in Figure 1 The steps include: S1: Obtain a video sequence containing a hand, extract the coordinates of a plurality of hand key points in each frame image of the video sequence, and generate original motion trajectory data; S2: Perform principal component analysis fusion processing on the original motion trajectory data to generate a fusion trajectory signal; S3: Extract features from the fusion trajectory signal to obtain a motion feature vector representing the tremor pattern; S4: Input the motion feature vector into a pre-trained machine learning model to predict the optimal signal processing parameters for the current tremor pattern; S5: Use the optimal signal processing parameters to perform smoothing filtering and frequency domain conversion processing on the original motion trajectory data to obtain a power spectral density distribution; S6: Identify the main peak in the power spectral density distribution within a predetermined frequency range, and take the frequency corresponding to the main peak as the detection result of the hand tremor frequency.

[0024] In the present embodiment, further, the hand key points include the fingertip key points of the thumb, index finger, middle finger, ring finger and little finger.

[0025] In the present embodiment, further, in step S2, the principal component analysis fusion processing on the original motion trajectory data to generate a fusion trajectory signal includes: Calculate the coordinate difference value between adjacent time frames of the original motion trajectory data to obtain difference motion data.

[0026] Reshape the difference motion data into a two-dimensional feature matrix.

[0027] Perform principal component analysis on the two-dimensional feature matrix to extract the first principal component as the fusion trajectory signal.

[0028] In the present embodiment, further, in step S3, the motion feature vector extracted from the fusion trajectory signal includes time domain features and frequency domain features.

[0029] Temporal characteristics include: mean absolute amplitude, standard deviation, and kurtosis of the fused trajectory signal; Frequency domain characteristics include: signal-to-noise ratio, power spectrum peak significance, and spectral entropy of the fused trajectory signal in the target frequency band.

[0030] The formulas for calculating time-domain and frequency-domain features include: Average absolute amplitude: in, This represents the average absolute amplitude, and N represents the signal length. This represents the displacement value of the fused trajectory signal in the i-th frame; Standard deviation: in, The standard deviation is represented by N, and the signal length is represented by N. This represents the value of the fused trajectory signal at point i. Indicates the signal mean; Kuroshi: in, This represents kurtosis, and N represents the signal length. This represents the value of the fused trajectory signal at point i. Indicates the signal mean. This represents the standard deviation of the fused trajectory signal; Signal-to-noise ratio of the target frequency band: in, Indicates the signal-to-noise ratio of the target frequency band. This indicates the upper frequency limit of the preset vibration band. This indicates the lower limit frequency of the preset vibration band. Represents the power spectral density of the fused trajectory signal; Significance of power spectrum peaks: Where P represents the significance of the power spectrum peak. This represents the power spectral density value at the peak of the main peak. This indicates the lowest topographic contour line elevation corresponding to the main peak; Spectral entropy: in, Represents spectral entropy. This represents the normalized power spectral density value, satisfying... , .

[0031] In this embodiment, further, in step S4, the optimal signal processing parameters include a smoothing filter window length and a power spectrum analysis segment length. The machine learning model is a random forest regression model, and the machine learning model includes a first prediction model and a second prediction model, which are respectively used to predict the smoothing window length and the power spectrum analysis segment length.

[0032] In this embodiment, further, the training process of the machine learning model includes: A sample data set containing different tremor amplitude levels is constructed, and the sample data set contains multiple groups of sample original motion trajectory data.

[0033] For each group of sample original motion trajectory data, the preset parameter space is traversed, the frequency detection error under different parameter combinations is calculated, and the parameter combination with the minimum error is selected as the sample optimal parameter label.

[0034] Steps S2 to S3 are performed on each group of sample original motion trajectory data to obtain a sample motion feature vector.

[0035] The sample motion feature vector is used as input, and the sample optimal parameter label is used as output to train the machine learning model.

[0036] In this embodiment, further, in step S5, the original motion trajectory data is smoothed and frequency domain converted using the optimal signal processing parameters to obtain a power spectrum density distribution, including: The smoothing filter window length in the optimal signal processing parameters is used to perform smoothing filtering and detrending processing on the original motion trajectory data using a Savitzky-Golay filter to obtain preprocessed trajectory data.

[0037] The power spectrum analysis segment length in the optimal signal processing parameters is used to perform power spectrum density estimation on the preprocessed trajectory data using a Welch method to obtain a power spectrum density distribution.

[0038] In this embodiment, further, the preset frequency range is 3Hz to 12Hz.

[0039] In this embodiment, the time-frequency domain features of the motion trajectory are extracted, and a pre-trained machine learning model is used to predict the optimal signal processing parameters for the current tremor pattern. This mechanism enables the system to accurately detect tremor signals of different intensities, solves the problem of signal being erased or excessive noise in the prior art due to fixed parameters in the detection of slight tremor, and improves the detection accuracy; the motion trajectories of multiple hand key points are fused by principal component analysis technology, and the first principal component representing the main motion pattern is extracted as the analysis object. PCA fusion can effectively extract the common mode signal of multiple finger motions, while suppressing random noise generated by individual key points due to changes in illumination, camera jitter or random finger movement. This improves the signal-to-noise ratio of the original signal and ensures that even weak tremor signals can be accurately extracted and analyzed in a non-contact video shooting environment.

[0040] Embodiment 2 In combination with Embodiment 1, in Embodiment 2, a hand tremor frequency detection system 21 based on video analysis and machine learning is provided, as shown in the system structure diagram Figure 2 , which includes: A trajectory extraction module 22 is configured to obtain a video sequence containing a hand, extract the coordinates of multiple hand key points in each frame image of the video sequence, and generate original motion trajectory data.

[0041] A trajectory fusion module 23 is configured to perform principal component analysis fusion processing on the original motion trajectory data to generate a fused trajectory signal.

[0042] A feature extraction module 24 is configured to perform feature extraction on the fused trajectory signal to obtain a motion feature vector representing a tremor pattern.

[0043] A parameter prediction module 25 is configured to input the motion feature vector into a pre-trained machine learning model to predict the optimal signal processing parameters for the current tremor pattern.

[0044] A frequency calculation module 26 is configured to perform smoothing filtering and frequency domain conversion processing on the original motion trajectory data using the optimal signal processing parameters to obtain a power spectral density distribution, and identify a main peak in the power spectral density distribution within a predetermined frequency range, and take the frequency corresponding to the main peak as the detection result of the hand tremor frequency.

[0045] It should be pointed out finally that the above embodiments are only used to illustrate the technical solutions of the present application, but not to limit the same; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that the technical solutions recorded in the foregoing embodiments can still be modified, or some or all of the technical features can be replaced equivalently; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present application, and they should be covered in the scope of the claims and the specification of the present application.

Claims

1. A method for tremor frequency detection of a hand based on video analysis and machine learning, characterized in that, The method comprises the following steps: S1: acquiring a video sequence containing a hand, extracting coordinates of a plurality of hand key points in each frame image of the video sequence, and generating original motion trajectory data; S2: performing principal component analysis fusion processing on the original motion trajectory data to generate a fusion trajectory signal; S3: extracting a motion feature vector representing a tremor pattern from the fusion trajectory signal; S4: inputting the motion feature vector into a pre-trained machine learning model to predict the optimal signal processing parameters for the current tremor pattern; S5: using the optimal signal processing parameters to perform smoothing filtering and frequency domain conversion processing on the original motion trajectory data to obtain a power spectral density distribution; S6: identifying the main peak in the power spectral density distribution within a predetermined frequency range, and taking the frequency corresponding to the main peak as the detection result of the hand tremor frequency. 2.The hand tremor frequency detection method based on video analysis and machine learning according to claim 1, wherein, The hand key points include the key points of the fingertips of the thumb, index finger, middle finger, ring finger, and little finger. 3.The video analysis and machine learning based hand tremor frequency detection method of claim 1, wherein, In step S2, the original motion trajectory data is processed by principal component analysis fusion to generate a fusion trajectory signal, which comprises the following steps: Calculate the coordinate difference between adjacent time frames of the original motion trajectory data to obtain differential motion data; Reshape the differential motion data into a two-dimensional feature matrix; Perform principal component analysis on the two-dimensional feature matrix to extract the first principal component as the fusion trajectory signal. 4.The video analysis and machine learning based hand tremor frequency detection method of claim 1, wherein, In step S3, the motion feature vector extracted from the fusion trajectory signal includes time domain features and frequency domain features; The time domain features include: the average absolute amplitude, standard deviation, and kurtosis of the fusion trajectory signal; The frequency domain features include: the signal-to-noise ratio, power spectrum peak significance, and spectral entropy of the fusion trajectory signal in the target frequency band.

5. The method of claim 1, wherein the method is based on video analysis and machine learning. In step S4, the optimal signal processing parameters include the smoothing filter window length and the power spectrum analysis segment length; The machine learning model is a random forest regression model, and the machine learning model includes a first prediction model and a second prediction model for predicting the smoothing window length and the power spectrum analysis segment length, respectively.

6. The method of claim 5, wherein the method is based on video analysis and machine learning. The training process of the machine learning model comprises: Construct a sample data set containing different tremor amplitude levels, which contains multiple groups of sample original motion trajectory data; For each group of sample original motion trajectory data, traverse the preset parameter space, calculate the frequency detection error under different parameter combinations, and select the parameter combination with the smallest error as the sample optimal parameter label; Perform steps S2 to S3 on each group of sample original motion trajectory data to obtain a sample motion feature vector; Use the sample motion feature vector as input and the sample optimal parameter label as output to train the machine learning model.

7. The method of claim 1, wherein the method is based on video analysis and machine learning. In step S5, the optimal signal processing parameters are used to perform smoothing filtering and frequency domain conversion processing on the original motion trajectory data to obtain a power spectral density distribution, which comprises the following steps: Use the smoothing filter window length in the optimal signal processing parameters to perform smoothing filtering and detrending processing on the original motion trajectory data using a Savitzky-Golay filter to obtain preprocessed trajectory data; The power spectrum analysis segment length in the optimal signal processing parameter is used to perform power spectrum density estimation on the preprocessed trajectory data by using a Welch method, to obtain the power spectrum density distribution.

8. The method for hand tremor frequency detection based on video analytics and machine learning according to claim 1, wherein, The preset frequency range is 3 Hz to 12 Hz.

9. A hand tremor frequency detection system based on video analysis and machine learning, characterized in that, The method comprises the following steps: a trajectory extraction module configured to obtain a video sequence containing a hand, extract coordinates of a plurality of hand key points in each frame image of the video sequence, and generate original motion trajectory data; a trajectory fusion module configured to perform principal component analysis fusion processing on the original motion trajectory data, to generate a fused trajectory signal; a feature extraction module configured to perform feature extraction on the fused trajectory signal, to obtain a motion feature vector representing a tremor pattern; a parameter prediction module configured to input the motion feature vector into a pre-trained machine learning model, to predict an optimal signal processing parameter for the current tremor pattern; a frequency calculation module configured to perform smoothing filtering and frequency domain conversion processing on the original motion trajectory data by using the optimal signal processing parameter, to obtain a power spectrum density distribution, and identify a main peak in the power spectrum density distribution within a preset frequency range, and take a frequency corresponding to the main peak as a detection result of a hand tremor frequency.

10. An electronic device comprising a memory, a processor, and a computer program stored on the memory, wherein the computer program, when executed by the processor, is arranged to perform the method of any one of claims 1 to 9. The processor implements the method in any one of claims 1 to 8 when executing the program.

Images