Construction method and system of intelligent recognition model for Parkinson's disease

Abstract

The invention relates to the technical field of image recognition, in particular to a method and a system for constructing an intelligent recognition model for Parkinson's disease. The method comprises the following steps: acquiring a facial image video of a Parkinson's disease patient; analyzing the face image video frame by frame, extracting each frame of image in the face image video, and generating a face frame-by-frame image; identifying facial skin features of the facial frame-by-frame image; performing texture wrinkle detection on the facial skin features to generate skin texture wrinkle data; performing wrinkle lamination degree measurement and calculation on the skin texture wrinkle data to obtain a skin wrinkle lamination degree; according to the Parkinson's disease recognition method and system, the facial skin muscle features of the Parkinson's disease patient are recognized through the data processing technology and the image recognition technology, and the voice information of the patient and the presented facial skin muscle features are subjected to correlation mapping to construct the Parkinson's disease intelligent recognition model, so that the Parkinson's disease recognition accuracy is improved, and the Parkinson's disease recognition efficiency is improved. And the identification time is shortened.

Description

Technical Field

[0001] The present invention relates to the technical field of image recognition, and in particular to a method and system for constructing an intelligent recognition model for Parkinson's disease. Background Art

[0002] Parkinson's disease patients mainly show characteristics such as resting tremor, muscle rigidity, bradykinesia, postural instability and abnormal gait. Early studies mainly used traditional machine learning algorithms, such as decision trees, support vector machines, Bayesian classifiers, etc., to analyze and classify gait data and speech data of Parkinson's disease patients. These methods showed certain recognition capabilities under limited data volume, but were limited by the complexity of feature extraction and the generalization ability of the model. With the development of medical imaging technology, researchers began to use technologies such as magnetic resonance imaging (MRI) to build recognition models by analyzing changes in brain structure of Parkinson's disease patients. In recent years, research has gradually turned to the fusion of multimodal data, including clinical data, imaging data, genetic data, etc., to more comprehensively identify Parkinson's disease and its subtypes. For example, the deep phenotypic progression embedding (DPPE) model combined with long short-term memory (LSTM) units was used to analyze the longitudinal clinical records of Parkinson's disease patients and identify different disease progression patterns. However, existing Parkinson's disease recognition technology fails to recognize the facial skin and muscle features of patients, and it is difficult to associate and map the patient's voice information with the facial skin and muscle features presented, resulting in low accuracy in Parkinson's disease recognition and long recognition time. Summary of the invention

[0003] Based on this, it is necessary to provide a method and system for constructing an intelligent recognition model for Parkinson's disease to solve at least one of the above technical problems.

[0004] To achieve the above object, a method for constructing an intelligent recognition model for Parkinson's disease is provided, the method comprising the following steps: Step S1: collecting a facial image video of a Parkinson's disease patient; analyzing the facial image video frame by frame, and extracting each frame of the facial image video to generate a facial frame-by-frame image; Step S2: identifying facial skin features of facial frame-by-frame images; performing texture wrinkle detection on facial skin features to generate skin texture wrinkle data; performing wrinkle layering calculation on the skin texture wrinkle data to obtain skin wrinkle layering; determining skin texture smoothness based on the skin wrinkle layering; merging skin wrinkle layering and skin texture smoothness as texture indices to obtain facial texture index data; Step S3: collecting speech data of Parkinson's disease patients and identifying rhythmic feature data of the speech data; determining facial expression loss features of facial texture index data according to the rhythmic feature data, and quantifying facial stiffness degree data based on the facial expression loss features; Step S4: Perform Parkinson's disease status assessment based on the facial texture index data and the facial stiffness degree data to obtain Parkinson's disease assessment data; and construct a Parkinson's disease intelligent recognition model based on the Parkinson's disease assessment data.

[0005] The present invention can fully capture the dynamic changes of the patient's facial expression by collecting facial image videos of Parkinson's patients and analyzing and extracting facial frame-by-frame images frame by frame, so as to provide rich and continuous image data for subsequent analysis, and ensure the integrity and accuracy of facial feature extraction; identify the facial skin features of the facial frame-by-frame images and perform texture wrinkle detection to generate skin texture wrinkle data, and then measure the skin wrinkle layering and determine the skin texture smoothness, and finally merge to obtain facial texture index data. This process can quantify the subtle changes of facial skin, provide objective skin feature indicators for the early identification of Parkinson's disease, and help to more accurately reflect the facial muscle state of the patient; collect the speech data of Parkinson's patients and identify their rhythm feature data, based on which determine the facial expression missing features and quantify the facial stiffness data. Combining the speech rhythm features with the facial texture index data, the facial expression state of the patient can be comprehensively evaluated from multiple dimensions, make up for the limitations of single modality data, and improve the accuracy of the evaluation of the facial stiffness of Parkinson's disease; Parkinson's disease state evaluation is performed based on the facial texture index data and the facial stiffness data, and the Parkinson's disease evaluation data is obtained, and the intelligent recognition model of Parkinson's disease is constructed accordingly. The model integrates multimodal data features and can achieve accurate identification and evaluation of Parkinson's disease. Therefore, the present invention uses data processing technology and image recognition technology to identify the facial skin muscle features of Parkinson's disease patients, and associates and maps the patient's voice information with the presented facial skin muscle features to build a Parkinson's disease intelligent recognition model, thereby improving the accuracy of Parkinson's disease recognition and shortening the recognition time.

[0006] Preferably, step S1 comprises the following steps: Step S11: using a camera to collect data at a frame rate of 30 frames per second, the resolution of the camera is set to be no less than 1920×1080 pixels, the collection time is no less than 3 minutes each time, and the distance between the camera and the patient's face is maintained between 50-70 cm; Step S12: using multi-angle acquisition, including front, left and right sides, to acquire at least 1 minute of video from each angle to obtain a facial image video of the Parkinson's disease patient; Step S13: extracting each frame of the facial image video at a rate of 30 frames per second, and graying each frame to obtain grayscale image format data; Step S14: unify the image resolution of the grayscale image format data and save it as a frame-by-frame facial image.

[0007] The present invention uses a frame rate of 30 frames per second and a resolution of not less than 1920×1080 pixels for video acquisition, ensuring the clarity and fluency of facial image videos, and can fully record the subtle dynamic changes of facial expressions of Parkinson's patients. At the same time, the acquisition time of not less than 3 minutes each time and the distance between the camera and the patient's face of 50-70 cm ensure the stability and consistency of data acquisition, providing high-quality original data for subsequent analysis; adopting a multi-angle acquisition method of the front, left side and right side, and collecting at least 1 minute of video respectively, can fully cover various areas of the face of Parkinson's patients, avoiding information omissions caused by a single viewing angle. This multi-angle acquisition strategy can capture the differences in facial expressions at different angles, providing more comprehensive facial image video data for subsequent analysis; extract each frame of the facial image video at a rate of 30 frames per second, and grayscale processing is performed to obtain grayscale image format data. Grayscale processing can remove the interference of color information, highlight facial texture and structural features, reduce data complexity, and improve the efficiency and accuracy of subsequent processing; the grayscale image format data is unified in image resolution and saved as facial frame-by-frame images, ensuring data consistency and standardization. The unified image resolution facilitates subsequent processing and analysis, and the standardized facial frame-by-frame images provide high-quality, unified format input data for the construction of intelligent recognition models, laying the foundation for the accuracy and reliability of the model.

[0008] Preferably, the step of identifying facial skin features of facial frame-by-frame images in step S2 includes: Identify facial regions in face frame-by-frame images; Extract skin texture features of the facial area, identify the texture direction and texture density of the skin texture features, and generate skin texture data; Perform skin shape recognition on the facial area according to texture direction and texture density to generate skin shape data; Extract skin wrinkle features in the facial area, identify the length, depth and distribution direction of skin wrinkle features, and obtain skin wrinkle data; Extract skin sagging features in the facial area, identify the sagging amount of the skin sagging features, and obtain skin sagging data; Skin texture data, skin shape data, skin wrinkle data and skin sagging data are used to generate facial skin features.

[0009] The present invention can ensure the pertinence and accuracy of subsequent feature extraction by accurately identifying the facial area in the facial frame-by-frame image, avoid interference from non-facial areas, and provide a clear analysis object for the intelligent identification of Parkinson's disease; the extraction of skin texture features, especially the identification of texture direction and density, can quantify subtle changes in facial skin. This provides an important objective indicator for the early identification of Parkinson's disease, because the stiffness of facial muscles in Parkinson's patients may cause abnormal skin texture; skin shape recognition based on texture direction and density can further reveal the movement state of facial muscles and changes in facial contours. This multi-dimensional analysis helps to more comprehensively evaluate the facial features of Parkinson's patients and provide rich data support for intelligent recognition models; the extraction of skin wrinkle features, including the length, depth and distribution direction of wrinkles, can reflect the activity of facial muscles and changes in skin elasticity. Parkinson's patients often show facial stiffness and reduced expression, and changes in wrinkle features can be used to quantify this pathological state; the extraction of skin sagging features, especially the identification of sagging, can reflect changes in facial muscle tension. The relaxation and drooping of facial muscles in Parkinson's patients is one of their typical symptoms. The quantification of this feature provides important pathological information for the intelligent recognition model. Integrating multiple skin feature data to generate comprehensive facial skin features can provide multi-dimensional and comprehensive input data for the intelligent recognition model of Parkinson's disease.

[0010] Preferably, the step S2 of performing texture wrinkle detection on facial skin features includes: Detect the texture lines of facial skin features and record the texture line spacing. When the texture line spacing is less than 1 mm, it is marked as wrinkle spacing data; Detect the texture depth of facial skin features. When the texture depth exceeds 0.5 mm, it is marked as wrinkle depth data. Detect the texture continuity of facial skin features, record the interruption position of texture continuity, and mark it as wrinkle interruption data when the interruption position exceeds twice; Detect the skin texture density of facial skin features and record the number of texture lines per unit area. When the texture density exceeds 10 per square centimeter, it is marked as wrinkle density data.

[0011] The present invention can quantify the slight changes of facial skin by accurately detecting the spacing of texture lines and marking the wrinkle spacing data. Parkinson's patients often have abnormally reduced skin texture spacing due to facial muscle stiffness. This data provides important pathological features for the intelligent recognition model, which helps to improve the model's recognition ability of Parkinson's disease; the detection of texture depth can reflect the microstructural changes on the skin surface. The facial skin of Parkinson's patients often has wrinkles with increased depth due to muscle stiffness. The marked wrinkle depth data provides the model with a quantitative skin change indicator, which further enriches the input features of the model; the interruption of texture continuity reflects the irregularity of skin wrinkles. The facial skin wrinkles of Parkinson's patients are often interrupted due to muscle movement disorders. The marked wrinkle interruption data can capture this irregularity, provide the model with more comprehensive skin feature information, and help improve the accuracy of recognition; the detection of texture density can quantify the number of texture lines per unit area. The texture density of the facial skin of Parkinson's patients may be significantly increased due to muscle stiffness. The marked wrinkle density data provides the model with quantitative skin texture change information, which further enhances the model's recognition ability of Parkinson's disease.

[0012] Preferably, the step S2 of calculating the wrinkle layering of the skin texture wrinkle data includes: Quantifying the pixel grayscale values ​​of skin texture wrinkle data, dividing the pixel grayscale values ​​into depth levels, and obtaining texture wrinkle level data; Determine the wrinkle direction of the texture wrinkle grade data and measure the change angle of the wrinkle direction. When the change angle is less than 30 degrees, it is marked as a normal wrinkle area; when the change angle exceeds 30 degrees, it is marked as a complex wrinkle area; The number of wrinkle layers in normal wrinkle areas is identified and recorded as the number of normal wrinkle layers; the number of wrinkle layers in complex wrinkle areas is identified and recorded as the number of complex wrinkle layers; The stacking ratio of normal wrinkle layers and complex wrinkle layers is calculated to obtain the skin wrinkle stacking degree.

[0013] The present invention can convert the subtle changes of skin texture into quantifiable texture wrinkle grade data by quantifying the pixel gray value of skin texture wrinkle data and dividing the depth grade. This quantification method provides an objective skin texture depth index for the intelligent identification of Parkinson's disease, which helps to distinguish the skin texture changes in normal and pathological states; by judging the wrinkle direction and measuring the change angle, it can distinguish the normal wrinkle area from the complex wrinkle area. The wrinkle direction changes of the facial skin of Parkinson's patients may be more complex. This distinction method can more accurately identify the pathological characteristics and provide more distinguishing input data for the model; the wrinkle layer number identification of different wrinkle areas can further refine the characteristic analysis of skin texture. The distinction between the normal wrinkle layer number and the complex wrinkle layer number can reflect the layer changes of the facial skin of Parkinson's patients and provide richer texture feature information for the intelligent identification model; the skin wrinkle layer degree is obtained by calculating the layer stacking ratio, which can comprehensively reflect the complexity and layer changes of skin wrinkles. This indicator can provide more representative skin texture features for the intelligent identification of Parkinson's disease, which helps to improve the accuracy and reliability of the model for Parkinson's disease.

[0014] Preferably, determining the skin texture smoothness based on the skin wrinkle stacking degree in step S2 includes: Mark the wrinkle contours of skin wrinkles and calculate the length of wrinkle contours; According to the length of wrinkle contour lines, the overlapping area of ​​skin wrinkles is segmented to obtain the wrinkle overlapping segmentation area; Identify the wrinkle layer ripple features of the wrinkle stack segmentation area, and determine the ripple bulge degree according to the wrinkle layer ripple features; The regional skin texture ridge mapping is performed based on the ripple ridge degree, and the skin texture smoothness is calculated using the regional skin texture ridge.

[0015] The present invention can accurately quantify the geometric features of skin wrinkles by marking wrinkle contours and calculating their lengths. This process provides basic data for subsequent wrinkle analysis, and can effectively distinguish normal skin texture from abnormal wrinkles caused by Parkinson's disease; layered region segmentation based on wrinkle contour length can divide complex skin texture regions into sub-regions with different features. This segmentation method can analyze the changes in skin texture more carefully, and provide more distinguishing feature data for the intelligent identification of Parkinson's disease; by identifying the ripple features of the wrinkle layer and determining the degree of ripple bulge, the microstructural changes of skin wrinkles can be further quantified. The determination of the degree of ripple bulge provides richer texture features for the intelligent identification model, which helps to improve the accuracy of the model for Parkinson's disease; mapping the degree of ripple bulge and measuring the smoothness of skin texture can comprehensively reflect the overall state of skin texture. The measurement of skin texture smoothness provides a key quantitative indicator for the intelligent identification model of Parkinson's disease, and more comprehensively evaluates the facial skin state of patients.

[0016] Preferably, the step S3 of collecting speech data of Parkinson's disease patients and identifying prosodic feature data of the speech data includes: The microphone sampling rate was set to 44.1-54.1kHz, and the microphone was used to continuously collect speech data from Parkinson's patients; The speech data is segmented, with each segment being 5 seconds long and 2 seconds apart, to obtain speech segmentation data; Calculate the fundamental frequency jitter of the speech segment data, and count the average value, standard deviation and variation range of the fundamental frequency jitter to generate fundamental frequency jitter data; Calculating the intonation intensity of the speech segment data, and calculating the average value of the intonation intensity to generate intonation intensity data; Calculate the speech duration of the speech segment data, and count the variation range of the speech duration to generate speech duration data; The fundamental frequency jitter data, intonation intensity data and speech duration data are used as prosodic feature data.

[0017] The present invention can ensure the acquisition of high-quality and high-resolution voice signals by setting a higher sampling rate (44.1-54.1kHz), so as to fully capture the subtle changes in the voice of Parkinson's patients. This high-quality voice data acquisition provides a reliable basis for subsequent feature extraction and model construction; segmentation of voice data can divide continuous voice signals into short-term voice segments with statistical significance. This segmentation method can effectively capture transient features and rhythmic changes in voice signals, while reducing the complexity of voice signals, facilitating subsequent feature extraction and analysis; fundamental frequency jitter is one of the common pathological features in the voice of Parkinson's patients, which is manifested as instability and slight fluctuations of fundamental frequency. By calculating the mean, standard deviation and range of fundamental frequency jitter, this pathological change in voice signals can be quantified, providing key rhythmic feature data for intelligent recognition models; intonation intensity reflects the energy and loudness changes of voice signals. Parkinson's patients often show characteristics such as single intonation and no obvious changes in sound intensity. By calculating the average value of intonation intensity, the patient's voice intensity characteristics can be effectively quantified, further enriching the model's input data; the range of changes in speech duration can reflect the changes in rhythm and fluency of Parkinson's patients when they pronounce words. The extraction of this feature can capture pathological phenomena such as pauses and dragging sounds in the patient's voice, providing the model with important information about speech rhythm; integrating the above three feature data into rhythmic feature data can comprehensively reflect the pathological changes in the voice of Parkinson's patients. This multi-dimensional feature extraction method provides rich input for the intelligent recognition model, which helps to improve the model's recognition accuracy and robustness for Parkinson's disease.

[0018] Preferably, the step S3 of determining the facial expression loss feature of the facial texture index data according to the prosody feature data, and quantifying the facial stiffness degree data based on the facial expression loss feature comprises: Detecting facial movements of the patient in the pronunciation state according to the rhythmic feature data, and recording movement amplitude data of the patient's facial movements; The movement amplitude data is divided into amplitude levels to obtain the facial movement amplitude level; Perform facial expression mapping on the prosody feature data based on the level of facial movement amplitude to obtain facial expression features; Marking the texture change missing of the facial texture index data according to the facial expression characteristics to obtain the texture change missing data; and determining the facial detail missing data according to the texture change missing data; Recognize facial expression presentation features for facial detail missing data to generate facial expression presentation data; determine facial expression missing features based on the facial expression presentation data; Quantify the missing features of facial expressions to generate missing feature quantification data; The missing feature quantification data is mapped to the degree of facial stiffness to generate facial stiffness data.

[0019] The present invention can capture the facial muscle activity of Parkinson's patients during pronunciation by analyzing the association between prosodic feature data and facial movements. The recording of facial movement amplitude data provides quantitative muscle movement information for subsequent analysis, which helps to reveal the intrinsic connection between facial muscle stiffness caused by Parkinson's disease and speech disorders; the movement amplitude data is graded to convert complex facial movement changes into quantifiable indicators. This grading method provides standardized input for subsequent feature mapping and analysis, further improving the model's ability to recognize Parkinson's facial features; through the mapping of facial movement amplitude levels and prosodic feature data, the pathological information in speech can be combined with facial expression changes. This multimodal feature fusion method can more comprehensively reflect the pathological state of Parkinson's patients and provide richer feature information for intelligent recognition models; marking texture changes based on facial expression features can accurately identify the lack of facial texture changes caused by Parkinson's disease. This process further refines the analysis of facial features, provides the model with quantitative data on facial detail changes, and helps to improve the model's recognition accuracy for Parkinson's disease; further analysis of facial detail missing data can identify abnormal features presented by facial expressions of Parkinson's patients. The determination of facial expression loss features provides the model with key pathological indicators, which helps to more accurately assess the degree of facial stiffness in patients; by quantifying facial expression loss features and mapping them to facial stiffness, complex facial pathological changes can be converted into quantifiable indicators. Facial stiffness data provides important input for the intelligent recognition model of Parkinson's disease, which helps to improve the recognition performance and reliability of the model.

[0020] Preferably, step S4 comprises the following steps: Step S41: performing wrinkle quantitative analysis on the facial texture index data to obtain Parkinson's disease wrinkle quantitative data; performing stiffness shape analysis on the facial stiffness degree data to obtain Parkinson's disease stiffness shape data; Step S42: performing Parkinson's disease state assessment according to the Parkinson's disease wrinkle quantification data and the Parkinson's disease stiffness shape data to obtain Parkinson's disease assessment data; Step S43: constructing a Parkinson's disease intelligent recognition model according to the Parkinson's disease assessment data to obtain an intelligent recognition pre-model; and training the intelligent recognition pre-model based on the Parkinson's disease assessment data to obtain an intelligent recognition training model; Step S44: performing model cross-validation evaluation on the intelligent recognition training model to obtain training model evaluation data; adjusting model parameters of the intelligent recognition training model using the training model evaluation data to obtain a Parkinson's disease intelligent recognition model.

[0021] The present invention performs wrinkle quantitative analysis on facial texture index data, and can convert the texture changes of facial skin into quantifiable Parkinson's wrinkle quantitative data. At the same time, the facial stiffness data is subjected to stiff shape analysis to obtain Parkinson's stiff shape data, and the extraction of these data provides objective quantitative indicators for the pathological characteristics of Parkinson's disease, and provides a basis for further state evaluation; Parkinson's disease state evaluation based on Parkinson's disease wrinkle quantitative data and Parkinson's disease stiff shape data can comprehensively reflect the pathological characteristics of patients. This multi-dimensional evaluation method can more comprehensively reflect the severity of the disease of Parkinson's disease patients and provide accurate input for the construction of intelligent recognition models; a Parkinson's disease intelligent recognition pre-model is constructed according to Parkinson's disease evaluation data, and the pre-model is trained based on the evaluation data to obtain an intelligent recognition training model. Through model training, the model can learn the pathological characteristic pattern of Parkinson's disease, thereby improving the model's recognition ability for Parkinson's disease. In addition, data-driven learning in the training process can effectively improve the generalization ability of the model; model cross-validation evaluation is performed on the intelligent recognition training model, and the performance and stability of the model can be comprehensively evaluated. The training model evaluation data obtained through cross-validation can provide a scientific basis for adjusting model parameters. Adjusting the model parameters based on the evaluation data can further optimize the model performance and ultimately obtain a Parkinson's disease intelligent recognition model with high accuracy and strong generalization ability.

[0022] The present invention also provides a system for constructing an intelligent recognition model for Parkinson's disease, which is used in the above-mentioned method for constructing an intelligent recognition model for Parkinson's disease. The system for constructing an intelligent recognition model for Parkinson's disease includes: The facial image video acquisition module is used to acquire facial image videos of Parkinson's disease patients; analyze the facial image videos frame by frame, and extract each frame of the facial image videos to generate facial frame-by-frame images; The facial texture feature recognition module is used to recognize facial skin features of facial frame-by-frame images; perform texture wrinkle detection on facial skin features to generate skin texture wrinkle data; perform wrinkle layering calculation on skin texture wrinkle data to obtain skin wrinkle layering; determine skin texture smoothness based on skin wrinkle layering; combine skin wrinkle layering and skin texture smoothness as texture indices to obtain facial texture index data; A facial stiffness quantification module is used to collect speech data of Parkinson's disease patients and identify rhythmic feature data of the speech data; determine facial expression loss features of facial texture index data according to the rhythmic feature data, and quantify facial stiffness data based on the facial expression loss features; The Parkinson's disease intelligent recognition model building module is used to evaluate the Parkinson's disease state based on facial texture index data and facial stiffness data to obtain Parkinson's disease evaluation data; and to build a Parkinson's disease intelligent recognition model based on the Parkinson's disease evaluation data.

[0023] The present invention collects facial image videos of Parkinson's patients and generates facial frame-by-frame images, providing high-quality original data for subsequent analysis, and ensuring the integrity and accuracy of facial feature extraction; identifies skin texture features of facial frame-by-frame images, generates skin texture fold data, wrinkle stacking and texture smoothness, and merges them into facial texture index data, to achieve quantitative analysis of facial skin features of Parkinson's patients; collects voice data and identifies rhythmic features, determines facial expression loss features based on facial texture index data, quantifies facial stiffness data, and achieves multimodal fusion analysis from voice and facial features; evaluates the state of Parkinson's disease based on facial texture index data and facial stiffness data, builds an intelligent recognition model, and achieves accurate state evaluation of Parkinson's disease. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 A schematic diagram of the steps of a method for constructing an intelligent recognition model for Parkinson's disease; Figure 2 for Figure 1 Detailed implementation steps of step S1 in FIG. Figure 3 for Figure 1 Detailed implementation steps of step S4 in FIG. The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0025] The technical method of the present invention is described clearly and completely below in conjunction with the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by technicians in this field without creative work are within the scope of protection of the present invention.

[0026] In addition, the accompanying drawings are only schematic illustrations of the present invention and are not necessarily drawn to scale. The same reference numerals in the figures represent the same or similar parts, and their repeated description will be omitted. Some of the block diagrams shown in the accompanying drawings are functional entities and do not necessarily correspond to physically or logically independent entities. The functional entities can be implemented in software form, or implemented in one or more hardware modules or integrated circuits, or implemented in different networks and / or processor methods and / or microcontroller methods.

[0027] It should be understood that, although the terms "first", "second", etc. may be used herein to describe various units, these units should not be limited by these terms. These terms are used only to distinguish one unit from another unit. For example, without departing from the scope of the exemplary embodiments, the first unit may be referred to as the second unit, and similarly the second unit may be referred to as the first unit. The term "and / or" used herein includes any and all combinations of one or more of the listed associated items.

[0028] To achieve this, please refer to Figures 1 to 3 , a method and system for constructing an intelligent recognition model for Parkinson's disease, the method comprising the following steps: Step S1: collecting a facial image video of a Parkinson's disease patient; analyzing the facial image video frame by frame, and extracting each frame of the facial image video to generate a facial frame-by-frame image; Step S2: identifying facial skin features of facial frame-by-frame images; performing texture wrinkle detection on facial skin features to generate skin texture wrinkle data; performing wrinkle layering calculation on the skin texture wrinkle data to obtain skin wrinkle layering; determining skin texture smoothness based on the skin wrinkle layering; merging skin wrinkle layering and skin texture smoothness as texture indices to obtain facial texture index data; Step S3: collecting speech data of Parkinson's disease patients and identifying rhythmic feature data of the speech data; determining facial expression loss features of facial texture index data according to the rhythmic feature data, and quantifying facial stiffness degree data based on the facial expression loss features; Step S4: Perform Parkinson's disease status assessment based on the facial texture index data and the facial stiffness degree data to obtain Parkinson's disease assessment data; and construct a Parkinson's disease intelligent recognition model based on the Parkinson's disease assessment data.

[0029] The present invention can fully capture the dynamic changes of the patient's facial expression by collecting facial image videos of Parkinson's patients and analyzing and extracting facial frame-by-frame images frame by frame, so as to provide rich and continuous image data for subsequent analysis, and ensure the integrity and accuracy of facial feature extraction; identify the facial skin features of the facial frame-by-frame images and perform texture wrinkle detection to generate skin texture wrinkle data, and then measure the skin wrinkle layering and determine the skin texture smoothness, and finally merge to obtain facial texture index data. This process can quantify the subtle changes of facial skin, provide objective skin feature indicators for the early identification of Parkinson's disease, and help to more accurately reflect the facial muscle state of the patient; collect the speech data of Parkinson's patients and identify their rhythm feature data, based on which determine the facial expression missing features and quantify the facial stiffness data. Combining the speech rhythm features with the facial texture index data, the facial expression state of the patient can be comprehensively evaluated from multiple dimensions, make up for the limitations of single modality data, and improve the accuracy of the evaluation of the facial stiffness of Parkinson's disease; Parkinson's disease state evaluation is performed based on the facial texture index data and the facial stiffness data, and the Parkinson's disease evaluation data is obtained, and the intelligent recognition model of Parkinson's disease is constructed accordingly. The model integrates multimodal data features and can achieve accurate identification and evaluation of Parkinson's disease. Therefore, the present invention uses data processing technology and image recognition technology to identify the facial skin muscle features of Parkinson's disease patients, and associates and maps the patient's voice information with the presented facial skin muscle features to build a Parkinson's disease intelligent recognition model, thereby improving the accuracy of Parkinson's disease recognition and shortening the recognition time.

[0030] In the embodiment of the present invention, reference Figure 1 FIG. 1 is a schematic diagram of a method for constructing an intelligent recognition model for Parkinson's disease according to the present invention. In this example, the method for constructing an intelligent recognition model for Parkinson's disease includes the following steps: Step S1: collecting a facial image video of a Parkinson's disease patient; analyzing the facial image video frame by frame, and extracting each frame of the facial image video to generate a facial frame-by-frame image; In an embodiment of the present invention, a high-resolution camera (with a resolution of not less than 1920×1080 pixels) is used to capture a facial video of a patient at a frame rate of 30 frames per second. The camera should be fixed in front of the patient to ensure that the facial image in the video is clear and unobstructed, and to ensure uniform illumination to avoid image quality being affected by light changes; the collected video data is imported into a video processing system, and the video is pre-processed, including removing noise, adjusting brightness and contrast, to enhance image quality. In addition, the video is cropped to remove irrelevant background information and retain only the facial area; the video processing software is used to extract images frame by frame in the order of the video frame rate. The specific operation is: set frame-by-frame extraction parameters, extract each frame of the video in sequence, and store it as an independent image file. During the extraction process, ensure that the resolution and color depth of the image remain consistent to ensure the accuracy of subsequent processing; after the extraction is completed, all images are arranged in chronological order to form a facial frame-by-frame image sequence. Each image file should contain timestamp information so that it can accurately correspond to a specific time point in the video during subsequent analysis. At the same time, the extracted images are uniformly formatted to ensure that the image size is consistent, which is convenient for subsequent feature extraction and analysis.

[0031] Step S2: identifying facial skin features of facial frame-by-frame images; performing texture wrinkle detection on facial skin features to generate skin texture wrinkle data; performing wrinkle layering calculation on the skin texture wrinkle data to obtain skin wrinkle layering; determining skin texture smoothness based on the skin wrinkle layering; merging skin wrinkle layering and skin texture smoothness as texture indices to obtain facial texture index data; In the embodiment of the present invention, the facial image is grayed frame by frame, and the image is converted from RGB color format to gray format to highlight texture information and reduce the amount of data. Subsequently, the gray image is smoothed using Gaussian filtering, and the filter kernel size is set to 5×5 and the standard deviation is 1.5 to remove noise in the image and ensure the accuracy of subsequent processing; the smoothed gray image is processed using the Canny edge detection algorithm to extract the wrinkle edge in the skin texture. In the Canny algorithm, the low threshold is set to 50 and the high threshold is set to 150, and the wrinkle edge information is extracted by double threshold detection. The detection result is a binary image, in which the wrinkle edge is represented by white pixels and the non-edge area is represented by black pixels. At the same time, combined with the facial key point detection technology, the wrinkle concentration areas such as the forehead, eye area and mouth corners are located, and these areas are analyzed in detail to improve the accuracy of wrinkle detection; for the detected wrinkle area, the gray standard deviation is calculated to quantify the density and depth of the wrinkles. The specific operation is: first, the gray value distribution in the wrinkle area is counted, and the average value and standard deviation of the gray value in the area are calculated. The larger the standard deviation, the higher the degree of wrinkle stacking, that is, the denser or deeper the wrinkles; based on the wrinkle stacking data, the smoothness of the skin texture is further evaluated. The smoothness of the skin texture is quantified by calculating the grayscale variance of the wrinkle area. The smaller the grayscale variance, the smoother the skin texture; conversely, the larger the variance, the rougher the skin texture. This index reflects the overall smoothness of the skin texture; the wrinkle stacking and skin texture smoothness are combined to generate facial texture index data. The specific operation is: multiply the wrinkle stacking data by the weight coefficient 0.6, multiply the skin texture smoothness data by the weight coefficient 0.4, and then add the two together to obtain the final facial texture index.

[0032] Step S3: collecting speech data of Parkinson's disease patients and identifying rhythmic feature data of the speech data; determining facial expression loss features of facial texture index data according to the rhythmic feature data, and quantifying facial stiffness degree data based on the facial expression loss features; In the embodiment of the present invention, first, voice data of Parkinson's patients are collected, and the patients are required to complete voice tasks such as sustained vowels (such as / a / , / i / , / u / ), repeated syllables (such as / pa / , / ta / ) and situational dialogues (such as reading a specified sentence). The voice collection device uses a high-quality microphone to ensure a quiet recording environment to reduce the interference of background noise. The collected voice data is preprocessed, including noise reduction, normalization and other operations to improve the quality and consistency of the voice signal; then, the collected voice data is subjected to prosodic feature recognition. The prosodic features in the voice are extracted using an acoustic analysis tool, including parameters such as intonation, sound intensity, and sound length. The prosodic features of the voice are quantified by calculating indicators such as the fluctuation range of the intonation, the stability of the sound intensity, and the rate of change of the sound length. For example, Parkinson's patients usually show the characteristics of a single tone, small changes in sound intensity, and large changes in sound length. Subsequently, the facial texture index data is analyzed according to the prosodic feature data to determine the facial expression loss feature. The specific operation is: to correlate the speech prosody features with the facial texture indicators, and identify the features of stiff or missing facial expressions caused by abnormal speech prosody. For example, when the speech prosody features show a single tone and large changes in the length of the sound, it may correspond to stiff facial expressions or lack of natural dynamic changes. Finally, the facial stiffness data is quantified based on the facial expression loss features. By setting a threshold, the facial expression loss features are mapped to the facial stiffness. For example, when the facial expression loss features exceed the preset threshold, it is judged that the facial stiffness is high; otherwise, the stiffness is low. The facial stiffness data is expressed in numerical form and used for subsequent Parkinson's disease recognition analysis.

[0033] Step S4: Perform Parkinson's disease status assessment based on the facial texture index data and the facial stiffness degree data to obtain Parkinson's disease assessment data; and construct a Parkinson's disease intelligent recognition model based on the Parkinson's disease assessment data.

[0034] In the embodiment of the present invention, first, the Parkinson's disease state is evaluated based on the facial texture index data and the facial stiffness data obtained in step S3. The facial texture index data and the facial stiffness data are fused and processed, and the Parkinson's disease evaluation data is generated by weighted summation. The specific parameters are set as follows: the weight of the facial texture index data is 0.5, and the weight of the facial stiffness data is 0.5. Comprehensive evaluation data is obtained by weighted summation, which is used to quantify the patient's Parkinson's disease state. Subsequently, a Parkinson's disease intelligent recognition model is constructed based on the Parkinson's disease evaluation data. The collected multimodal data (including voice data and facial texture data) are preprocessed to extract key features. The voice data extracts rhythm features, and the facial texture data extracts texture and stiffness features. These features are normalized to ensure data consistency. Then, the processed data is divided into a training set and a test set, wherein 70% of the data is used as a training set and 30% of the data is used as a test set. The training set is used for model training, and a support vector machine (SVM) algorithm is used as a classifier. In the SVM model, the penalty parameter C is set to 9.6057 and the kernel function parameter g is set to 0.5261. During the training process, the 10-fold cross-validation method is used to evaluate the performance of the model to ensure the generalization ability of the model. Finally, the trained model is verified using the test set to evaluate the model's accuracy, sensitivity, and F1 score. The accuracy and sensitivity of the model both reached a high level, and the F1 score was used to comprehensively evaluate the performance of the model. In this way, an intelligent recognition model for Parkinson's disease based on multimodal data was constructed.

[0035] As an example of the present invention, refer to Figure 2 As shown, in this example, step S1 includes: Step S11: using a camera to collect data at a frame rate of 30 frames per second, the resolution of the camera is set to be no less than 1920×1080 pixels, the collection time is no less than 3 minutes each time, and the distance between the camera and the patient's face is maintained between 50-70 cm; Step S12: using multi-angle acquisition, including front, left and right sides, to acquire at least 1 minute of video from each angle to obtain a facial image video of the Parkinson's disease patient; Step S13: extracting each frame of the facial image video at a rate of 30 frames per second, and graying each frame to obtain grayscale image format data; Step S14: unify the image resolution of the grayscale image format data and save it as a frame-by-frame facial image.

[0036] In an embodiment of the present invention, first, a high-definition camera is used for video acquisition, and the frame rate of the camera is set to 30 frames per second and the resolution is 1920×1080 pixels to ensure that the acquired image is clear and rich in details. The acquisition time is not less than 3 minutes each time, and the distance between the camera and the patient's face is maintained between 50-70 cm to ensure the integrity and consistency of the facial image. During the acquisition process, the camera is fixed on a tripod to avoid image blur due to device shaking. Then, a multi-angle acquisition method is adopted, including the front, left side and right side. At least 1 minute of video is collected at each angle to ensure that the patient's facial features are fully captured from different perspectives. During the acquisition process, the patient maintains a natural expression, avoids large movements or changes in expression, and reduces interference factors. Subsequently, the acquired facial image video is extracted frame by frame. At a rate of 30 frames per second, each frame of the video is extracted in turn, and each frame is grayed. Grayscale processing is achieved by converting color images into grayscale images. The specific method is to convert each pixel value of the RGB image into a grayscale value. The grayscale value is calculated as follows: grayscale value = 0.299 × R + 0.587 × G + 0.114 × B, where R, G, and B represent the pixel values ​​of the red, green, and blue channels, respectively. Finally, the grayscale image format data is processed uniformly, and the image resolution is adjusted to be consistent with the acquisition resolution, that is, 1920 × 1080 pixels. When adjusting the resolution, a bilinear interpolation algorithm is used to ensure that the image maintains clarity during the scaling process. The processed image is saved in PNG format as a facial frame-by-frame image.

[0037] Preferably, the step of identifying facial skin features of facial frame-by-frame images in step S2 includes: Identify facial regions in face frame-by-frame images; Extract skin texture features of the facial area, identify the texture direction and texture density of the skin texture features, and generate skin texture data; Perform skin shape recognition on the facial area according to texture direction and texture density to generate skin shape data; Extract skin wrinkle features in the facial area, identify the length, depth and distribution direction of skin wrinkle features, and obtain skin wrinkle data; Extract skin sagging features in the facial area, identify the sagging amount of the skin sagging features, and obtain skin sagging data; Skin texture data, skin shape data, skin wrinkle data and skin sagging data are used to generate facial skin features.

[0038] In an embodiment of the present invention, face detection technology is used to process the collected facial images frame by frame to locate the facial area. Through OpenFace software or similar facial detection tools, the key points of the face in the image (such as eyes, nose, mouth, etc.) are identified, and the rectangular area containing the complete face is extracted; the extracted facial area is grayed to convert the color image into a gray image to highlight the texture information. The gray level co-occurrence matrix (GLCM) analysis method is used to calculate the texture direction and texture density. The texture direction is determined by analyzing the main direction of the gray level co-occurrence matrix, and the texture density is quantified by counting the texture repetition frequency; skin texture data is generated based on the texture direction and texture density. The texture direction is expressed in angles (such as 0°, 45°, 90°, etc.), and the texture density is expressed in pixel units for subsequent analysis; the skin shape features are identified by combining texture features and facial key point position information. By calculating the relative distance and angle between key points, the smoothness and symmetry of the facial contour are evaluated to generate skin shape data; an edge detection algorithm (such as the Canny algorithm) is used to identify the wrinkle edge and calculate the length, depth and distribution direction of the wrinkle. The length of wrinkles is obtained by measuring the length of the wrinkle edge, the depth is calculated by analyzing the grayscale change difference of the wrinkle area, and the distribution direction is determined by analyzing the direction of the wrinkles to obtain skin wrinkle data; the vertical displacement of facial key points (such as eyelids and corners of the mouth) is calculated to quantify the degree of skin relaxation. For example, the amount of eyelid droop or mouth corner droop is measured and expressed in pixels to obtain skin relaxation data; skin texture data, skin shape data, skin wrinkle data, and skin relaxation data are integrated to form facial skin features. After being standardized, these data are used as input features of the Parkinson's disease intelligent recognition model.

[0039] Preferably, the step S2 of performing texture wrinkle detection on facial skin features includes: Detect the texture lines of facial skin features and record the texture line spacing. When the texture line spacing is less than 1 mm, it is marked as wrinkle spacing data; Detect the texture depth of facial skin features. When the texture depth exceeds 0.5 mm, it is marked as wrinkle depth data. Detect the texture continuity of facial skin features, record the interruption position of texture continuity, and mark it as wrinkle interruption data when the interruption position exceeds twice; Detect the skin texture density of facial skin features and record the number of texture lines per unit area. When the texture density exceeds 10 per square centimeter, it is marked as wrinkle density data.

[0040] In an embodiment of the present invention, the facial image is grayed frame by frame, and the color image is converted into a gray image to highlight the texture information. The skin texture lines are identified using an edge detection algorithm (such as the Canny algorithm) and the spacing between the texture lines is calculated. When the spacing between the texture lines is less than 1 mm, the area is recorded as wrinkle spacing data. The texture spacing is detected by calculating the pixel distance between adjacent texture lines, and the pixel distance corresponding to 1 mm is obtained by conversion according to the image resolution; the gray image is subjected to local binary pattern (LBP) analysis to extract texture depth information. The texture depth is quantified by calculating the grayscale change gradient of the texture area. When the texture depth exceeds 0.5 mm, the area is marked as wrinkle depth data. The quantification of texture depth is based on the intensity of grayscale change, and the grayscale change threshold corresponding to 0.5 mm is determined by experimental calibration; the continuity of the texture lines is analyzed using a texture tracking algorithm, and the position of the texture interruption is recorded. When the interruption position exceeds twice, the area is marked as wrinkle interruption data. The detection of texture continuity is achieved by analyzing the directional consistency of the texture lines, and the identification of the interruption position is based on directional mutation or grayscale fault; the number of texture lines in a unit area is counted to calculate the texture density. When the texture density exceeds 10 per square centimeter, the area is marked as wrinkle density data. The number of texture lines per unit area is calculated by counting and converting the texture lines in a predefined skin area. The threshold of 10 per square centimeter is set based on the experience of the skin characteristics of Parkinson's patients.

[0041] Preferably, the step S2 of calculating the wrinkle layering of the skin texture wrinkle data includes: Quantifying the pixel grayscale values ​​of skin texture wrinkle data, dividing the pixel grayscale values ​​into depth levels, and obtaining texture wrinkle level data; Determine the wrinkle direction of the texture wrinkle grade data and measure the change angle of the wrinkle direction. When the change angle is less than 30 degrees, it is marked as a normal wrinkle area; when the change angle exceeds 30 degrees, it is marked as a complex wrinkle area; The number of wrinkle layers in normal wrinkle areas is identified and recorded as the number of normal wrinkle layers; the number of wrinkle layers in complex wrinkle areas is identified and recorded as the number of complex wrinkle layers; The stacking ratio of normal wrinkle layers and complex wrinkle layers is calculated to obtain the skin wrinkle stacking degree.

[0042] In an embodiment of the present invention, grayscale processing is performed on each frame of facial images to convert color images into grayscale images. Image processing software or tools are used to perform quantitative analysis on each pixel in the grayscale image. According to the range of pixel grayscale values, texture wrinkles are divided into different depth levels. For example, areas with grayscale values ​​between 0 and 50 are marked as shallow wrinkles, areas with grayscale values ​​between 51 and 150 are marked as medium wrinkles, and areas with grayscale values ​​between 151 and 255 are marked as deep wrinkles; image analysis tools such as OpenCV or MATLAB are used to calculate the main direction of the wrinkle area. The direction information of the wrinkles is quantified by calculating the gradient direction of the wrinkle lines or using Hough transform to detect the line direction. When the change angle of the wrinkle direction is less than 30 degrees, the area is marked as a normal wrinkle area; when the change angle exceeds 30 degrees, it is marked as a complex wrinkle area; for normal wrinkle areas and complex wrinkle areas, wrinkle layer identification is performed separately. An edge detection algorithm (such as the Canny algorithm) is used to extract wrinkle edges, and the number of wrinkle layers in each area is counted. For normal wrinkle areas, the number of wrinkle layers is recorded as the normal wrinkle layer number; for complex wrinkle areas, the number of wrinkle layers is recorded as the complex wrinkle layer number; the normal wrinkle layer number and the complex wrinkle layer number are compared and analyzed. The ratio of the complex wrinkle layer number to the normal wrinkle layer number is calculated to obtain the skin wrinkle layering degree. This ratio reflects the complexity of skin wrinkles and can be used for subsequent Parkinson's disease identification analysis.

[0043] Preferably, determining the skin texture smoothness based on the skin wrinkle stacking degree in step S2 includes: Mark the wrinkle contours of skin wrinkles and calculate the length of wrinkle contours; According to the length of wrinkle contour lines, the overlapping area of ​​skin wrinkles is segmented to obtain the wrinkle overlapping segmentation area; Identify the wrinkle layer ripple features of the wrinkle stack segmentation area, and determine the ripple bulge degree according to the wrinkle layer ripple features; The regional skin texture ridge mapping is performed based on the ripple ridge degree, and the skin texture smoothness is calculated using the regional skin texture ridge.

[0044] In an embodiment of the present invention, an image processing tool (such as OpenCV) is used to process the skin wrinkle stacking image. First, the contour of the wrinkle is extracted by the Canny edge detection algorithm. Then, the extracted contour is marked using the findContours function, and the length of each wrinkle contour is calculated using the arcLength function. The contour length is used as a quantitative indicator of the wrinkle contour length, and the contour length data of each wrinkle area is recorded; the wrinkle stacking area is segmented according to the wrinkle contour length. A length threshold is set. For example, when the wrinkle contour length exceeds a preset value (such as 10 pixels), the area is divided into an independent wrinkle stacking area. The wrinkle area is segmented by a semantic segmentation algorithm (such as U-Net or ENet), and the segmented area is marked as a wrinkle stacking segmentation area. The segmentation result is saved in the form of a binary image, in which the white area represents the wrinkle stacking area and the black area represents the non-wrinkle area; the segmented wrinkle stacking area is subjected to ripple feature analysis. The texture features of the wrinkle area are analyzed using local binary patterns (LBP) or gray-level co-occurrence matrix (GLCM). The degree of ripples in the wrinkle layer is identified by calculating the grayscale change gradient of the texture features. When the grayscale change gradient exceeds a certain threshold (such as the grayscale value change exceeds 50), the area is marked as a ripple ridge area. The degree of ripple ridge is quantified by the magnitude of the grayscale change gradient, and the ripple ridge degree data of each wrinkle stacking area is recorded; the skin texture ridge degree of the wrinkle stacking area is mapped according to the ripple ridge degree. The area with a higher degree of ripple ridge is marked as a high ridge area, and the area with a lower degree of ripple ridge is marked as a low ridge area. The smoothness of the skin texture is evaluated by calculating the area ratio of the high ridge area. When the area ratio of the high ridge area exceeds a certain threshold (such as 20%), the skin texture smoothness of the area is judged to be low.

[0045] Preferably, the step S3 of collecting speech data of Parkinson's disease patients and identifying prosodic feature data of the speech data includes: The microphone sampling rate was set to 44.1-54.1kHz, and the microphone was used to continuously collect speech data from Parkinson's patients; The speech data is segmented, with each segment being 5 seconds long and 2 seconds apart, to obtain speech segmentation data; Calculate the fundamental frequency jitter of the speech segment data, and count the average value, standard deviation and variation range of the fundamental frequency jitter to generate fundamental frequency jitter data; Calculating the intonation intensity of the speech segment data, and calculating the average value of the intonation intensity to generate intonation intensity data; Calculate the speech duration of the speech segment data, and count the variation range of the speech duration to generate speech duration data; The fundamental frequency jitter data, intonation intensity data and speech duration data are used as prosodic feature data.

[0046] In an embodiment of the present invention, the microphone sampling rate is set to 44.1-54.1kHz, and a high-quality microphone is used to continuously collect voice data of Parkinson's patients in a quiet environment. The collected voice data includes continuous vowels (such as / a / , / i / , / u / ), repeated syllables (such as / pa / , / ta / ) and situational dialogues to ensure the integrity and clarity of the voice signal; the collected voice data is segmented, and the length of each voice segment is set to 5 seconds, and the interval between segments is 2 seconds. Through voice processing software or tools, the continuous voice signal is divided into multiple 5-second voice segments to form voice segmented data; the fundamental frequency jitter (Jitter) of the voice segmented data is calculated. First, the fundamental frequency curve of the voice signal is extracted, and then the relative jitter value of the fundamental frequency curve is calculated. The specific operation includes: interpolating the fundamental frequency curve after smoothing, calculating the residual between the original fundamental frequency curve and the smoothed curve, and obtaining the fundamental frequency jitter value. The mean value, standard deviation and variation range of fundamental frequency jitter are counted to generate fundamental frequency jitter data; the intonation intensity analysis is performed on the speech segment data. The average intonation intensity of each speech segment is counted by calculating the short-time energy of the speech signal. The short-time energy calculation method is: calculate the square sum of the amplitudes of all sampling points in each frame of speech signal, and take the average value as the intonation intensity. Finally, the intonation intensity data is generated; the speech duration analysis is performed on the speech segment data. The variation range of the speech duration is counted by calculating the actual duration of each speech segment. The calculation formula of speech duration is: speech duration (seconds) = (speech data length / sampling rate). The duration data of each speech segment is recorded and its variation range is counted to generate speech duration data; the fundamental frequency jitter data, intonation intensity data and speech duration data are integrated into prosodic feature data. These data reflect the pitch and prosodic features of the speech signal.

[0047] Preferably, the step S3 of determining the facial expression loss feature of the facial texture index data according to the prosody feature data, and quantifying the facial stiffness degree data based on the facial expression loss feature comprises: Detecting facial movements of the patient in the pronunciation state according to the rhythmic feature data, and recording movement amplitude data of the patient's facial movements; The movement amplitude data is divided into amplitude levels to obtain the facial movement amplitude level; Perform facial expression mapping on the prosody feature data based on the level of facial movement amplitude to obtain facial expression features; Marking the texture change missing of the facial texture index data according to the facial expression characteristics to obtain the texture change missing data; and determining the facial detail missing data according to the texture change missing data; Recognize facial expression presentation features for facial detail missing data to generate facial expression presentation data; determine facial expression missing features based on the facial expression presentation data; Quantify the missing features of facial expressions to generate missing feature quantification data; The missing feature quantification data is mapped to the degree of facial stiffness to generate facial stiffness data.

[0048] In an embodiment of the present invention, a face detection technology (such as a face detection algorithm based on deep learning) is used to process the collected facial frame-by-frame images to locate facial key points (such as eyebrows, corners of the mouth, etc.). By analyzing the displacement changes of key points during the pronunciation process, facial movements in the pronunciation state are detected. The displacement amplitude of the key points is calculated and quantified into movement amplitude data in pixels; the facial movement amplitude is divided into different levels according to the size of the movement amplitude data. For example, the area with a movement amplitude less than 10 pixels is marked as a low amplitude level, 10-20 pixels is marked as a medium amplitude level, and greater than 20 pixels is marked as a high amplitude level; the facial movement amplitude level is associated with rhythmic feature data (such as fundamental frequency jitter, tone intensity). For example, when the fundamental frequency jitter is large and the facial movement amplitude is at a low level, it is mapped as a nervous or stiff expression feature; when the tone intensity is high and the facial movement amplitude is at a high level, it is mapped as an active expression feature; and the facial expression features are compared with facial texture index data (such as skin texture smoothness and wrinkle stacking). When the facial expression features show stiffness, mark the area with high wrinkle stacking or low skin texture smoothness in the facial texture index data as texture change missing data; analyze the texture change missing data to identify the area with missing facial details. For example, when the wrinkle stacking exceeds the preset threshold (such as more than 15 wrinkles per square centimeter) and the skin texture smoothness is lower than the threshold (such as smoothness less than 0.3), the area is determined to be facial detail missing data; use facial expression recognition technology (such as a facial expression recognition model based on a convolutional neural network) to analyze the facial detail missing data and identify the expression presentation features. For example, identify whether the patient has facial muscle stiffness or a single expression during the pronunciation process; analyze the expression missing features based on the facial expression presentation data. For example, when the facial expression presentation data shows that the patient has a single expression and lacks dynamic changes during the pronunciation process, it is determined to be a facial expression missing feature; quantitative analysis of the facial expression missing features is performed. For example, the area ratio and duration of the expression missing area are calculated to generate missing feature quantitative data. The quantitative data is expressed in percentage form, such as the facial expression missing area accounts for 30% of the total facial area; according to the quantitative data of missing features, the degree of facial stiffness is divided into mild, moderate and severe. For example, when the facial expression missing area accounts for less than 20%, it is marked as mild stiffness; 20%-40% is marked as moderate stiffness; more than 40% is marked as severe stiffness.

[0049] More importantly, the mapping of missing feature quantification data to facial stiffness includes: Detect missing features to quantify the duration of missing actions in the data; Detect missing motion amplitude values ​​of missing feature quantization data; Determine the frequency of missing action occurrence according to the missing action duration and the missing action amplitude value; The missing action duration, missing action amplitude value, and missing action occurrence frequency were marked as facial stiffness indicators; The facial stiffness indexes were weighted and calculated to obtain the degree of facial stiffness.

[0050] In an embodiment of the present invention, a time series analysis is performed on the missing feature quantification data to extract the duration of the missing action. By analyzing the timestamp of the facial expression data, the start and end time of each missing action is calculated to obtain the duration of the missing action. For example, if a missing action starts from the 5th second and lasts to the 10th second, the duration of the action is 5 seconds; the missing feature quantification data is subjected to amplitude analysis to extract the amplitude value of the missing action. By calculating the displacement change of facial key points (such as eyebrows, corners of the mouth, etc.) during the action, it is quantified as the amplitude value of the missing action. For example, if a key point moves 15 pixels from the initial position during the action, the amplitude value of the action is 15 pixels; the number of occurrences of the missing action in a unit time is counted to determine the frequency of occurrence of the missing action. For example, if a missing action occurs 6 times within a 30-second observation time, the frequency of occurrence of the action is 0.2 times per second. By comparing the duration and amplitude value of the missing action, its potential impact on Parkinson's disease is further evaluated; the above three parameters are marked as facial stiffness indicators for subsequent analysis. The duration of missing action reflects the continuity of action, the amplitude of missing action reflects the intensity of action, and the frequency of missing action reflects the frequency of action. These indicators together constitute the quantitative characteristics of facial stiffness; the facial stiffness indicators are weighted to obtain the final degree of facial stiffness. For example, the weight of missing action duration is set to 0.4, the weight of missing action amplitude is set to 0.3, and the weight of missing action frequency is set to 0.3.

[0051] As an example of the present invention, refer to Figure 3 As shown, in this example, step S4 includes: Step S41: performing wrinkle quantitative analysis on the facial texture index data to obtain Parkinson's disease wrinkle quantitative data; performing stiffness shape analysis on the facial stiffness degree data to obtain Parkinson's disease stiffness shape data; Step S42: performing Parkinson's disease state assessment according to the Parkinson's disease wrinkle quantification data and the Parkinson's disease stiffness shape data to obtain Parkinson's disease assessment data; Step S43: constructing a Parkinson's disease intelligent recognition model according to the Parkinson's disease assessment data to obtain an intelligent recognition pre-model; and training the intelligent recognition pre-model based on the Parkinson's disease assessment data to obtain an intelligent recognition training model; Step S44: performing model cross-validation evaluation on the intelligent recognition training model to obtain training model evaluation data; adjusting model parameters of the intelligent recognition training model using the training model evaluation data to obtain a Parkinson's disease intelligent recognition model.

[0052] In an embodiment of the present invention, image processing technology is used to perform wrinkle quantitative analysis on the collected facial texture index data. The depth and distribution density of wrinkles are quantified by calculating the grayscale change gradient and texture density of the wrinkle area. For example, the wrinkle depth can be measured by the intensity of grayscale change, and the texture density can be calculated by the number of wrinkle lines per unit area. Finally, Parkinson's wrinkle quantitative data is obtained; the facial stiffness data is subjected to shape analysis to identify the contour and shape characteristics of the facial stiffness area. The shape characteristics of facial stiffness are quantified by calculating the area, perimeter and shape factor (such as circularity or rectangularity) of the stiffness area. Finally, Parkinson's stiffness shape data is obtained; Parkinson's status is evaluated based on Parkinson's wrinkle quantitative data and Parkinson's stiffness shape data. The wrinkle quantitative data and stiffness shape data are comprehensively analyzed, and the patient's Parkinson's status is judged by setting a threshold (such as wrinkle depth exceeding 0.5 mm and stiffness area area exceeding 10%). Finally, Parkinson's evaluation data is obtained; based on the Parkinson's evaluation data, an intelligent recognition pre-model is constructed. Support vector machine (SVM) was selected as the classifier, and its kernel function was Gaussian radial basis kernel (RBF), the penalty parameter C was set to 1.0, and the kernel function parameter γ was set to 0.1. The evaluation data was divided into a training set and a test set, where the training set was used for model training and the test set was used for model verification; the intelligent recognition pre-model was trained using the training set. The model performance was optimized by adjusting the model parameters (such as the C value and γ value of SVM). Finally, the intelligent recognition training model was obtained; the intelligent recognition training model was cross-validated and evaluated. The 10-fold cross-validation method was used to calculate the accuracy, recall rate, and F1 score of the model. Finally, the training model evaluation data was obtained; according to the training model evaluation data, the parameters of the intelligent recognition training model were adjusted. For example, the C value of SVM was adjusted to 2.0 and the γ value was adjusted to 0.05 to improve the generalization ability and prediction accuracy of the model. Finally, the Parkinson's disease intelligent recognition model was obtained.

[0053] More importantly, step S42 includes the following steps: Step S421: Calculate the four parameters of the wrinkle area texture feature contrast, correlation, energy and homogeneity of the Parkinson's disease wrinkle quantification data, and calculate the average value and standard deviation of each parameter to obtain the wrinkle quantification feature vector; Step S422: Calculate the joint angle change data of the Parkinson's disease stiff shape data, and adjust the range of the joint angle change data to between 0 and 1, calculate the maximum angle change range, average angle change rate and standard deviation of the angle change of each joint within 10 seconds, and obtain the stiff shape feature vector; Step S423: setting the weight of the wrinkle quantization feature vector to 0.4 and the weight of the rigid shape feature vector to 0.6; and performing weighted summation on the wrinkle quantization feature vector and the rigid shape feature vector to obtain Parkinson's disease state score data; Step S424: Evaluate the Parkinson's disease status score data by score level to obtain Parkinson's disease evaluation data.

[0054] In an embodiment of the present invention, the texture features of Parkinson's disease wrinkle quantification data are calculated using the gray level co-occurrence matrix (GLCM) method. Specific parameters include contrast, correlation, energy and homogeneity. Contrast reflects the clarity of the texture, correlation measures the linear relationship between texture elements, energy represents the uniformity of the texture, and homogeneity reflects the consistency of the texture; the mean and standard deviation of each parameter are calculated. For example, the mean value of contrast is 0.45 and the standard deviation is 0.12; the mean value of correlation is 0.80 and the standard deviation is 0.05; the mean value of energy is 0.30 and the standard deviation is 0.08; the mean value of homogeneity is 0.75 and the standard deviation is 0.06. Finally, a wrinkle quantification feature vector is formed; motion capture technology or video analysis methods are used to record the angle changes of facial key points (such as eyebrows, corners of the mouth, etc.). The range of joint angle change data is adjusted to between 0 and 1 through normalization processing; the maximum angle change range, average angle change rate and standard deviation of angle change of each joint within 10 seconds are calculated. For example, the maximum angle change range is 0.8, the average angle change rate is 0.05, and the standard deviation of the angle change is 0.02. Finally, a rigid shape feature vector is formed; the weight of the wrinkle quantification feature vector is set to 0.4, and the weight of the rigid shape feature vector is set to 0.6; the wrinkle quantification feature vector and the rigid shape feature vector are weighted and summed to obtain the Parkinson's disease state score data. For example, the value of the wrinkle quantification feature vector is 0.5, and the value of the rigid shape feature vector is 0.7; the Parkinson's disease state score data is graded. Set a threshold interval, for example: a score less than 0.5 is mild, a score between 0.5 and 0.7 is moderate, and a score greater than 0.7 is severe; according to the score interval, the state score of 0.62 is evaluated as a moderate Parkinson's disease state, and finally the Parkinson's disease evaluation data is obtained.

[0055] The present invention also provides a system for constructing an intelligent recognition model for Parkinson's disease, which is used in the above-mentioned method for constructing an intelligent recognition model for Parkinson's disease. The system for constructing an intelligent recognition model for Parkinson's disease includes: The facial image video acquisition module is used to acquire facial image videos of Parkinson's disease patients; analyze the facial image videos frame by frame, and extract each frame of the facial image videos to generate facial frame-by-frame images; The facial texture feature recognition module is used to recognize facial skin features of facial frame-by-frame images; perform texture wrinkle detection on facial skin features to generate skin texture wrinkle data; perform wrinkle layering calculation on skin texture wrinkle data to obtain skin wrinkle layering; determine skin texture smoothness based on skin wrinkle layering; combine skin wrinkle layering and skin texture smoothness as texture indices to obtain facial texture index data; A facial stiffness quantification module is used to collect speech data of Parkinson's disease patients and identify rhythmic feature data of the speech data; determine facial expression loss features of facial texture index data according to the rhythmic feature data, and quantify facial stiffness data based on the facial expression loss features; The Parkinson's disease intelligent recognition model building module is used to evaluate the Parkinson's disease state based on facial texture index data and facial stiffness data to obtain Parkinson's disease evaluation data; and to build a Parkinson's disease intelligent recognition model based on the Parkinson's disease evaluation data.

[0056] The present invention collects facial image videos of Parkinson's patients and generates facial frame-by-frame images, providing high-quality original data for subsequent analysis, and ensuring the integrity and accuracy of facial feature extraction; identifies skin texture features of facial frame-by-frame images, generates skin texture fold data, wrinkle stacking and texture smoothness, and merges them into facial texture index data, to achieve quantitative analysis of facial skin features of Parkinson's patients; collects voice data and identifies rhythmic features, determines facial expression loss features based on facial texture index data, quantifies facial stiffness data, and achieves multimodal fusion analysis from voice and facial features; evaluates the state of Parkinson's disease based on facial texture index data and facial stiffness data, builds an intelligent recognition model, and achieves accurate state evaluation of Parkinson's disease.

[0057] Therefore, the embodiments should be regarded as illustrative and non-restrictive from all points, and the scope of the present invention is limited by the appended claims rather than the above description, and it is therefore intended that all changes falling within the meaning and range of equivalent elements of the application documents are included in the present invention.

[0058] The above description is only a specific embodiment of the present invention, so that those skilled in the art can understand or implement the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but should conform to the widest scope consistent with the principles and novel features invented herein.

Claims

1. A method for constructing an intelligent recognition model for Parkinson's disease, characterized in that: The following steps are involved: Step S1: collecting facial image videos of Parkinson's disease patients; Analyze the facial image video frame by frame, and extract each frame image in the facial image video to generate a facial frame-by-frame image; Step S2: identifying facial skin features of facial frame-by-frame images; performing texture wrinkle detection on facial skin features to generate skin texture wrinkle data; performing wrinkle layering calculation on the skin texture wrinkle data to obtain skin wrinkle layering; determining skin texture smoothness based on the skin wrinkle layering; merging skin wrinkle layering and skin texture smoothness as texture indices to obtain facial texture index data; Step S3: collecting speech data of Parkinson's disease patients and identifying rhythmic feature data of the speech data; determining facial expression loss features of facial texture index data according to the rhythmic feature data, and quantifying facial stiffness degree data based on the facial expression loss features; Step S4: Perform Parkinson's disease status assessment based on the facial texture index data and the facial stiffness degree data to obtain Parkinson's disease assessment data; and construct a Parkinson's disease intelligent recognition model based on the Parkinson's disease assessment data.

2. The method for constructing an intelligent recognition model for Parkinson's disease according to claim 1, characterized in that: Step S1 includes the following steps: Step S11: using a camera to collect data at a frame rate of 30 frames per second, the resolution of the camera is set to be no less than 1920×1080 pixels, the collection time is no less than 3 minutes each time, and the distance between the camera and the patient's face is maintained between 50-70 cm; Step S12: using multi-angle acquisition, including front, left and right sides, to acquire at least 1 minute of video from each angle to obtain a facial image video of the Parkinson's disease patient; Step S13: extracting each frame of the facial image video at a rate of 30 frames per second, and graying each frame to obtain grayscale image format data; Step S14: unify the image resolution of the grayscale image format data and save it as a frame-by-frame facial image.

3. The method for constructing an intelligent recognition model for Parkinson's disease according to claim 1, characterized in that: The step S2 of identifying facial skin features of facial frame-by-frame images includes: Identify facial regions in face frame-by-frame images; Extract skin texture features of the facial area, identify the texture direction and texture density of the skin texture features, and generate skin texture data; Perform skin shape recognition on the facial area according to texture direction and texture density to generate skin shape data; Extract skin wrinkle features in the facial area, identify the length, depth and distribution direction of skin wrinkle features, and obtain skin wrinkle data; Extract skin sagging features in the facial area, identify the sagging amount of the skin sagging features, and obtain skin sagging data; Skin texture data, skin shape data, skin wrinkle data and skin sagging data are used to generate facial skin features.

4. The method for constructing an intelligent recognition model for Parkinson's disease according to claim 1, characterized in that: The step S2 of detecting facial skin features by texture wrinkles includes: Detect the texture lines of facial skin features and record the texture line spacing. When the texture line spacing is less than 1 mm, it is marked as wrinkle spacing data; Detect the texture depth of facial skin features. When the texture depth exceeds 0.5 mm, it is marked as wrinkle depth data. Detect the texture continuity of facial skin features, record the interruption position of texture continuity, and mark it as wrinkle interruption data when the interruption position exceeds twice; Detect the skin texture density of facial skin features and record the number of texture lines per unit area. When the texture density exceeds 10 per square centimeter, it is marked as wrinkle density data.

5. The method for constructing an intelligent recognition model for Parkinson's disease according to claim 1, characterized in that: The step S2 of calculating the wrinkle layering degree of the skin texture wrinkle data includes: Quantifying the pixel grayscale values ​​of skin texture wrinkle data, dividing the pixel grayscale values ​​into depth levels, and obtaining texture wrinkle level data; Determine the wrinkle direction of the texture wrinkle grade data and measure the change angle of the wrinkle direction. When the change angle is less than 30 degrees, it is marked as a normal wrinkle area; when the change angle exceeds 30 degrees, it is marked as a complex wrinkle area; The number of wrinkle layers in normal wrinkle areas is identified and recorded as the number of normal wrinkle layers; the number of wrinkle layers in complex wrinkle areas is identified and recorded as the number of complex wrinkle layers; The stacking ratio of normal wrinkle layers and complex wrinkle layers is calculated to obtain the skin wrinkle stacking degree.

6. The method for constructing an intelligent recognition model for Parkinson's disease according to claim 1, characterized in that: Determining the skin texture smoothness based on the skin wrinkle stacking degree in step S2 includes: Mark the wrinkle contours of skin wrinkles and calculate the length of wrinkle contours; According to the length of wrinkle contour lines, the overlapping area of ​​skin wrinkles is segmented to obtain the wrinkle overlapping segmentation area; Identify the wrinkle layer ripple features of the wrinkle stack segmentation area, and determine the ripple bulge degree according to the wrinkle layer ripple features; The regional skin texture ridge mapping is performed based on the ripple ridge degree, and the skin texture smoothness is calculated using the regional skin texture ridge.

7. The method for constructing an intelligent recognition model for Parkinson's disease according to claim 1, characterized in that: The step S3 of collecting speech data of Parkinson's disease patients and identifying rhythmic feature data of the speech data includes: The microphone sampling rate was set to 44.1-54.1kHz, and the microphone was used to continuously collect speech data from Parkinson's patients; The speech data is segmented, with each segment being 5 seconds long and 2 seconds apart, to obtain speech segmentation data; Calculate the fundamental frequency jitter of the speech segment data, and count the average value, standard deviation and variation range of the fundamental frequency jitter to generate fundamental frequency jitter data; Calculating the intonation intensity of the speech segment data, and calculating the average value of the intonation intensity to generate intonation intensity data; Calculate the speech duration of the speech segment data, and count the variation range of the speech duration to generate speech duration data; The fundamental frequency jitter data, intonation intensity data and speech duration data are used as prosodic feature data.

8. The method for constructing an intelligent recognition model for Parkinson's disease according to claim 1, characterized in that: Determining the facial expression loss feature of the facial texture index data according to the prosody feature data in step S3, and quantifying the facial stiffness degree data based on the facial expression loss feature includes: Detecting facial movements of the patient in the pronunciation state according to the rhythmic feature data, and recording movement amplitude data of the patient's facial movements; The movement amplitude data is divided into amplitude levels to obtain the facial movement amplitude level; Perform facial expression mapping on the prosody feature data based on the level of facial movement amplitude to obtain facial expression features; Marking the texture change missing of the facial texture index data according to the facial expression characteristics to obtain the texture change missing data; and determining the facial detail missing data according to the texture change missing data; Recognize facial expression presentation features for facial detail missing data to generate facial expression presentation data; determine facial expression missing features based on the facial expression presentation data; Quantify the missing features of facial expressions to generate missing feature quantification data; The missing feature quantification data is mapped to the degree of facial stiffness to generate facial stiffness data.

9. The method for constructing an intelligent recognition model for Parkinson's disease according to claim 1, characterized in that: Step S4 includes the following steps: Step S41: performing wrinkle quantitative analysis on the facial texture index data to obtain Parkinson's disease wrinkle quantitative data; performing stiffness shape analysis on the facial stiffness degree data to obtain Parkinson's disease stiffness shape data; Step S42: performing Parkinson's disease state assessment according to the Parkinson's disease wrinkle quantification data and the Parkinson's disease stiffness shape data to obtain Parkinson's disease assessment data; Step S43: constructing a Parkinson's disease intelligent recognition model according to the Parkinson's disease assessment data to obtain an intelligent recognition pre-model; and training the intelligent recognition pre-model based on the Parkinson's disease assessment data to obtain an intelligent recognition training model; Step S44: performing model cross-validation evaluation on the intelligent recognition training model to obtain training model evaluation data; adjusting model parameters of the intelligent recognition training model using the training model evaluation data to obtain a Parkinson's disease intelligent recognition model.

10. A system for constructing an intelligent recognition model for Parkinson's disease, characterized in that: The method for constructing an intelligent recognition model for Parkinson's disease according to claim 1 is used to execute the construction system of the intelligent recognition model for Parkinson's disease, the construction system of the intelligent recognition model for Parkinson's disease comprises: The facial image video acquisition module is used to acquire facial image videos of Parkinson's disease patients; analyze the facial image videos frame by frame, and extract each frame of the facial image videos to generate facial frame-by-frame images; The facial texture feature recognition module is used to recognize facial skin features of facial frame-by-frame images; perform texture wrinkle detection on facial skin features to generate skin texture wrinkle data; perform wrinkle layering calculation on skin texture wrinkle data to obtain skin wrinkle layering; determine skin texture smoothness based on skin wrinkle layering; combine skin wrinkle layering and skin texture smoothness as texture indices to obtain facial texture index data; A facial stiffness quantification module is used to collect speech data of Parkinson's disease patients and identify rhythmic feature data of the speech data; determine facial expression loss features of facial texture index data according to the rhythmic feature data, and quantify facial stiffness data based on the facial expression loss features; The Parkinson's disease intelligent recognition model building module is used to evaluate the Parkinson's disease state based on facial texture index data and facial stiffness data to obtain Parkinson's disease evaluation data; and to build a Parkinson's disease intelligent recognition model based on the Parkinson's disease evaluation data.

Images