Machine learning-based sarcopenia auxiliary diagnosis system and electronic equipment
Through the multimodal feature fusion and decision-making level fusion methods, the problem of low accuracy of sarcopenia diagnosis is solved, and high-precision assisted sarcopenia diagnosis is achieved, which is suitable for rapid identification of places such as families, nursing homes and communities.
Patent Information
- Application Number
- CN202510596727.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-05-09
- Publication Date
- 2025-08-05
AI Technical Summary
In the prior art, the diagnostic accuracy of sarcopenia is low and trauma is high. The existing machine learning-based sarcopenia diagnostic system only analyzes and recognizes a single feature, resulting in poor diagnostic accuracy.
The multimodal feature fusion method is adopted to obtain voice information, walking video information and clinical biological information through the data acquisition module, and feature extraction and fusion are used for data processing module. The training module is used to build a classifier, and the decision-making level fusion is carried out in combination with exercise testing information to achieve accurate diagnosis of sarcopenia.
It improves the accuracy of sarcopenia diagnosis, and can quickly identify sarcopenia in places such as families, nursing homes and communities to prevent the occurrence of adverse outcomes.
Smart Images
Figure CN120432133A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of machine learning technology, and more specifically, to a sarcopenia diagnostic system and electronic device based on machine learning. Background Art
[0002] Sarcopenia is a common neuromuscular disease characterized by a gradual decrease in muscle mass and function, which seriously affects the quality of life of patients.
[0003] Currently, conventional sarcopenia diagnosis is primarily based on clinical symptoms, biochemical tests, and muscle tissue biopsies, which pose challenges such as low diagnostic accuracy and high invasiveness. Furthermore, with the advancement of machine learning, existing sarcopenia diagnostic systems based on machine learning only analyze and identify single features, resulting in poor diagnostic accuracy. Summary of the Invention
[0004] The purpose of this application is to provide a sarcopenia diagnosis system and electronic equipment based on machine learning to improve the accuracy of sarcopenia diagnosis.
[0005] In order to achieve the above objectives, the technical solutions adopted in this application are as follows:
[0006] In one aspect, the present application provides a sarcopenia diagnosis system based on machine learning, comprising:
[0007] a data acquisition module, configured to collect voice information, walking video information, and clinical biological information of multiple subjects; wherein the subjects include sarcopenia patients and healthy individuals, and the clinical biological information includes numerical clinical information and exercise test information;
[0008] a data processing module connected to the data acquisition module, configured to perform feature extraction on the voice information, walking video information, and numerical clinical information of each subject to obtain a voice feature vector, a video feature vector, and a clinical feature vector, and perform multimodal feature fusion on the voice feature vector, the video feature vector, and the clinical feature vector to obtain a first fused feature;
[0009] a training module connected to the data processing module, configured to construct a training set based on the plurality of first fusion features, train a classifier based on the training set, and use the trained classifier as a target classifier;
[0010] The data acquisition module is also used to collect voice information, walking video information and clinical biological information of the person to be diagnosed;
[0011] The data processing module is further configured to process the voice information, walking video information, and numerical clinical information of the person to be diagnosed to obtain a second fusion feature, and input the second fusion feature into the target classifier;
[0012] The diagnostic module is connected to the data acquisition module and the training module respectively, and is used to perform decision-level fusion based on the motion test information of the person to be diagnosed and the first classification result output by the target classifier according to the second fusion feature to obtain a second classification result, and determine the diagnostic result of the person to be diagnosed based on the second classification result.
[0013] Furthermore, in the process of extracting features from the speech information of each subject to obtain a speech feature vector, the data processing module is further configured to:
[0014] resampling the speech information of each subject to obtain target speech information;
[0015] The target speech information is subjected to feature extraction based on audio analysis technology and then subjected to standardization processing to obtain the speech feature vector.
[0016] Furthermore, the speech information includes a plurality of different types of vowel data; in the process of resampling the speech information of each subject to obtain the target speech information, the data processing module is further configured to:
[0017] trimming and removing the first and last segments of the voice information of each subject to obtain trimmed voice information;
[0018] The target voice information is obtained by calculating the average length of the vowel data of the same type in the plurality of cropped voice information, and resampling the vowel data of the same type in each of the cropped voice information to the same length based on the average length.
[0019] Furthermore, in the process of extracting features from the walking video information of each subject to obtain a video feature vector, the data processing module is further configured to:
[0020] A cycle is defined as the time from when the subject takes a step with the left foot to when the subject takes a step with the left foot next time. In each cycle, joint point information of the subject's arm swing and leg walking are extracted to obtain the video feature vector.
[0021] Furthermore, the video feature vector also includes the walking speed and acceleration of the subject during the entire walking video time period.
[0022] Furthermore, the numerical clinical information includes: upper arm circumference, grip strength, leg circumference, number of falls in the past year, eating assessment questionnaire score, and daily living ability score; in the process of extracting features from the numerical clinical information of each subject to obtain a clinical feature vector, the data processing module is further used to:
[0023] The upper arm circumference, grip strength, leg circumference, number of falls in the past year, eating assessment questionnaire score, and daily living ability score are standardized so that each feature is distributed between 0 and 1 to obtain the clinical feature vector.
[0024] Furthermore, the first fusion feature is a fusion feature vector or a fusion feature matrix; in the process of performing multimodal feature fusion on the speech feature vector, the video feature vector, and the clinical feature vector to obtain the first fusion feature, the data processing module is further used to:
[0025] Sequentially concatenating and fusing the speech feature vector, the video feature vector, and the clinical feature vector to obtain an initial feature vector, and performing dimensionality reduction on the initial feature vector using feature selection or principal component analysis to obtain the fused feature vector;
[0026] Alternatively, principal component analysis is used to reduce the dimension of the speech feature vector, the video feature vector and the clinical feature vector, and the dimensions of the reduced speech feature vector, the video feature vector and the clinical feature vector are ensured to be consistent, and then they are fused to obtain the fused feature matrix.
[0027] Furthermore, in the process of constructing a training set based on the plurality of first fusion features, training a classifier based on the training set, and using the trained classifier as a target classifier, the training module is further configured to:
[0028] If the first fused feature is the fused feature vector, selecting support vector machine or linear discriminant analysis as the initial classifier;
[0029] If the first fusion feature is the fusion feature matrix, selecting a convolutional neural network or a tensor neural network as an initial classifier;
[0030] The initial classifier is trained using a training set constructed using a plurality of the first fusion features until a target classifier is obtained.
[0031] Furthermore, the clinical biological information also includes basic information; in the process of performing multimodal feature fusion on the speech feature vector, the video feature vector, and the clinical feature vector to obtain a first fusion feature, the data processing module is further used to:
[0032] The speech feature vector, the video feature vector, and the clinical feature vector are weighted and transformed according to the basic information, and then multimodal feature fusion is performed to obtain the first fusion feature.
[0033] On the other hand, the present application also provides an electronic device comprising a processor and a memory, wherein the memory stores a computer program for implementing the sarcopenia auxiliary diagnosis system based on machine learning as described in any of the aforementioned embodiments, and the processor can execute the computer program.
[0034] Compared with the prior art, this application has the following beneficial effects:
[0035] The present application provides a sarcopenia auxiliary diagnosis system and electronic device based on machine learning. The system includes: a data acquisition module for collecting voice information, walking video information, and clinical biological information from multiple subjects; wherein the subjects include sarcopenia patients and healthy people, and the clinical biological information includes numerical clinical information and exercise test information. A data processing module connected to the data acquisition module is used to extract features from the voice information, walking video information, and numerical clinical information of each subject to obtain a voice feature vector, a video feature vector, and a clinical feature vector, and perform multimodal feature fusion on the voice feature vector, video feature vector, and clinical feature vector to obtain a first fused feature. A training module connected to the data processing module is used to construct a training set based on the multiple first fused features, train a classifier based on the training set, and use the trained classifier as a target classifier. The data acquisition module is also used to collect voice information, walking video information, and clinical biological information from the person to be diagnosed. The data processing module is also used to process the voice information, walking video information, and numerical clinical information of the person to be diagnosed to obtain a second fused feature, and input the second fused feature into the target classifier. The diagnostic module is connected to the data acquisition module and the training module respectively, and is used to perform decision-level fusion based on the motion test information of the person to be diagnosed and the first classification result output by the target classifier according to the second fusion feature to obtain a second classification result, and determine the diagnostic result of the person to be diagnosed based on the second classification result.
[0036] The machine learning-based sarcopenia auxiliary diagnosis system provided in this application performs multimodal feature fusion on the voice feature vector, video feature vector and clinical feature vector of each subject during the training process, and performs decision-level fusion on the first classification results and motion test information of the diagnosed person during the diagnosis process. Through two different fusion processes, a comprehensive analysis is performed, which greatly improves the sarcopenia diagnostic accuracy of the machine learning-based sarcopenia auxiliary diagnosis system, and can realize the rapid identification of sarcopenia in families, nursing homes, communities and other hospitals at all levels, and prevent the occurrence of adverse outcomes of sarcopenia. BRIEF DESCRIPTION OF THE DRAWINGS
[0037] In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all of the embodiments. The components of the embodiments of the present application generally described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the application for which protection is claimed, but merely represents selected embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.
[0038] Figure 1 A schematic diagram of the structure of a sarcopenia auxiliary diagnosis system based on machine learning provided in an embodiment of the present application;
[0039] Figure 2 A schematic diagram of the training process of a sarcopenia auxiliary diagnosis system based on machine learning provided in an embodiment of the present application;
[0040] Figure 3 A schematic diagram of the diagnostic process of a sarcopenia auxiliary diagnosis system based on machine learning provided in an embodiment of the present application;
[0041] Figure 4 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application.
[0042] Icon: 10-Sarcopenia auxiliary diagnosis system based on machine learning; 110-Data acquisition module; 120-Data processing module; 130-Training module; 140-Diagnosis module; 20-Electronic device; 210-Memory; 220-Processor. DETAILED DESCRIPTION
[0043] The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all of the embodiments. The components of the embodiments of the present application generally described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the application for protection, but merely represents the selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work fall within the scope of protection of the present application.
[0044] In the description of this application, it should be noted that relational terms such as first and second are used solely to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. The term "connected" should be understood broadly, for example, it can mean fixed connection, detachable connection, or integral connection; it can be directly connected or indirectly connected through an intermediate medium.
[0045] The following describes some embodiments of the present application in detail with reference to the accompanying drawings. The following embodiments and features thereof may be combined with each other unless there is any conflict.
[0046] Studies have found that changes in voice, gait, and other clinical information in the elderly are associated with sarcopenia. Conventional diagnosis of sarcopenia is mainly based on clinical symptoms, biochemical tests, and muscle tissue biopsies, which have problems such as low diagnostic accuracy and high trauma. In addition, with the development of machine learning, existing machine learning-based sarcopenia diagnosis systems only analyze and identify single features, resulting in poor diagnostic accuracy. Therefore, how to improve the accuracy of sarcopenia diagnosis is a technical problem that needs to be solved urgently by those skilled in the art.
[0047] To solve the above technical problems, please refer to Figure 1 The embodiment of the present application provides a sarcopenia auxiliary diagnosis system 10 based on machine learning, which includes: a data acquisition module 110, a data processing module 120, a training module 130, and a diagnosis module 140. The data processing module 120 is connected to the data acquisition module 110, and the data processing module 120 is also connected to the training module 130. The diagnosis module 140 is connected to the data acquisition module 110 and the training module 130 respectively.
[0048] It should be noted that the workflow of the sarcopenia auxiliary diagnosis system 10 based on machine learning provided in this application is divided into a training process and a diagnosis process. Figure 1 and Figure 2 , first, the training process of the sarcopenia auxiliary diagnosis system 10 based on machine learning is explained.
[0049] In the embodiment of the present application, the data collection module 110 collects voice information, walking video information, and clinical biological information of multiple subjects, including sarcopenia patients and healthy people.
[0050] The method for collecting speech information can be as follows: each subject is placed in a quiet room with consistent lighting and background conditions. A microphone array and a laryngeal vibration voice collector are used to collect specific speech data from each subject. The subject is required to pronounce the sound for as long as possible to collect sufficient data. Because vowels are relatively simple and stable speech units, they are easy to repeat and control under experimental conditions. Therefore, the subject can produce vowels such as "a," "o," and "u," ensuring that each subject's speech information contains data from a variety of different vowel types.
[0051] The method for collecting walking video information can be as follows: each subject walks back and forth in a corridor with consistent lighting and background environment (the distance can be set to 20 meters). A high-resolution video capture device is used to record each subject's walking video. The camera of the video capture device should be positioned approximately at the subject's waist, so that the subject's leg movements and arm swings can be fully seen in the video (and the subject should be fully visible in the video).
[0052] The clinical biological information collection method may be: collecting the clinical biological information of each subject through comprehensive evaluation, wherein the clinical biological information of the subject includes: numerical clinical information and exercise test information.
[0053] After the data collection is completed, the data collection module 110 sends the numerical clinical information in the voice information, walking video information, and clinical biological information corresponding to each subject to the data processing module 120 .
[0054] The data processing module 120 performs feature extraction on the speech information, walking video information and numerical clinical information of each subject to obtain a speech feature vector, a video feature vector and a clinical feature vector; and performs multimodal feature fusion on the speech feature vector, the video feature vector and the clinical feature vector to obtain a first fusion feature.
[0055] It can be seen that the data processing module 120 provided in the embodiment of the present application is divided into two processes during data processing: feature extraction and feature fusion.
[0056] For a better understanding, the following first explains the feature extraction process of voice information feature extraction, walking video information feature extraction, and numerical clinical information feature extraction.
[0057] As an optional implementation, in the process of extracting features from the speech information of each subject to obtain a speech feature vector, the data processing module 120 is further configured to:
[0058] The speech information of each subject is resampled to obtain the target speech information.
[0059] It should be noted that, because the subject's pronunciation time is required to be as long as possible during voice information collection, and different subjects have different pronunciation habits, the audio time lengths of the multiple voice information collected will be inconsistent. In addition, irrelevant noise will inevitably be collected at the beginning and end of each subject's voice information collection. Therefore, it is necessary to preprocess (i.e., resample) each subject's voice information and use the processed voice information as the target voice information.
[0060] Specifically, the speech information of each subject is resampled as follows:
[0061] First, the first and last segments of each subject's speech information are trimmed and removed to obtain trimmed speech information.
[0062] Next, the length average of the vowel data of the same type in the plurality of cropped voice information is calculated, and the vowel data of the same type in each cropped voice information is resampled to the same length based on the length average to obtain the target voice information.
[0063] Since the speech information of each subject includes a variety of different types of vowel data, the data of the same vowel type need to be resampled to the same length.
[0064] For example, assume that the audio time lengths of "a", "o", and "u" in the cropped voice information 1 corresponding to subject 1 are 4S, 6S, and 5S respectively, and the audio time lengths of "a", "o", and "u" in the cropped voice information 2 corresponding to subject 2 are 6S, 8S, and 7S respectively.
[0065] Then, the average length of the vowel data of type "a" = (4+6) / 2 = 5S. Based on 5S, the "a" vowel data in the cropped voice information 1 is upsampled, and the "a" vowel data in the cropped voice information 2 is downsampled to ensure that the length of the "a" vowel data in voice information 1 is the same as the length of the "a" vowel data in voice information 2.
[0066] Similarly, the mean length of vowel data of type "o" = (6 + 8) / 2 = 7 seconds, and the mean length of vowel data of type "u" = (5 + 7) / 2 = 6 seconds. Speech with an audio duration greater than the mean length is downsampled, while speech with an audio duration less than the mean length is upsampled to ensure that speech data of the same vowel type has consistent length.
[0067] After the resampling operation, the target speech information for each subject is obtained in WAV format. Audio analysis techniques are then used to extract multiple audio-related features from the target speech information. These features include frequency perturbations, amplitude perturbations, fundamental frequency, and period. These multiple audio features are then normalized (for example, using center normalization or feature dimensionality reduction) to produce a speech feature vector.
[0068] As another optional implementation, in the process of extracting features from the walking video information of each subject to obtain a video feature vector, the data processing module 120 is further configured to:
[0069] A cycle is defined as the time from when the subject takes a left step to when he / she takes the left foot next time. In each cycle, the joint information of the subject's arm swing and legs walking is extracted to obtain the video feature vector.
[0070] It is understandable that since each subject has a different stride length, the walking rhythm of each subject in the video is also different. Therefore, the embodiment of the application uses the gait information from the time the subject takes a left step to the next left step as one cycle for feature extraction.
[0071] Determine the joint information for arm swinging and leg walking during each cycle. Using a specific joint as the 3D coordinate center, determine the 3D xyz coordinates of the arm and leg joints and analyze the changes in the joint coordinates during a cycle.
[0072] Optionally, the video feature vector also includes the subject's walking speed and acceleration during the entire walking video period.
[0073] As another optional embodiment, the numerical clinical information in the clinical biological information includes: upper arm circumference, grip strength, leg circumference, number of falls in the past year, eating assessment questionnaire score, and daily living ability score. In the process of extracting features from the numerical clinical information of each subject to obtain a clinical feature vector, the data processing module 120 is further configured to:
[0074] Upper arm circumference, grip strength, leg circumference, number of falls in the past year, eating assessment questionnaire score, and daily living ability score were standardized so that each feature was distributed between 0 and 1 to obtain the clinical feature vector.
[0075] Based on the above process, the data processing module 120 performs feature extraction on each subject's voice information, walking video information, and numerical clinical information to obtain a voice feature vector, a video feature vector, and a clinical feature vector.
[0076] Next, the data processing module 120 also needs to perform feature fusion on the feature vectors of the three modalities and finally output a first fused feature. Specifically, the embodiment of the present application provides two feature fusion schemes. Different feature fusion schemes are used, and the types of first fused features obtained are also different, that is, the first fused feature is a fused feature vector or a fused feature matrix.
[0077] The first feature fusion scheme involves the data processing module 120 sequentially concatenating the speech feature vector, video feature vector, and clinical feature vector into a single, long vector to generate an initial feature vector. If the initial feature vector is too long, feature selection or principal component analysis can be used to reduce its dimensionality, selecting more effective features to improve recognition accuracy, thereby generating a fused feature vector.
[0078] The second feature fusion solution is: the data processing module 120 uses principal component analysis to reduce the dimension of the speech feature vector, video feature vector and clinical feature vector, ensures that the dimensions of the reduced speech feature vector, video feature vector and clinical feature vector are consistent, and then places them side by side to form a matrix, thereby obtaining a fused feature matrix.
[0079] Furthermore, each subject's clinical biometric information also includes basic information, which can be the subject's age, gender, and comorbidities. This basic information can be used as a weighted sum of the feature vectors of the three modalities mentioned above (i.e., voice feature vector, video feature vector, and clinical feature vector). For example, men and women differ in upper arm circumference, grip strength, and leg circumference, so different weightings are required for men and women during standardization.
[0080] That is, in the process of performing multimodal feature fusion on the speech feature vector, the video feature vector, and the clinical feature vector to obtain the first fusion feature, the data processing module 120 is further configured to:
[0081] According to basic information (ie, age, gender, and comorbidities), the speech feature vector, video feature vector, and clinical feature vector are weighted and transformed, and then multimodal feature fusion is performed to obtain the first fusion feature.
[0082] Based on the above design, the data processing module 120 processes the voice information, walking video information, and clinical biological information of multiple subjects, and outputs multiple first fusion features to the training module 130.
[0083] The training module 130 constructs a training set according to the plurality of first fusion features, trains a classifier according to the training set, and uses the trained classifier as a target classifier.
[0084] Specifically, in the process of constructing a training set according to the plurality of first fusion features, training a classifier according to the training set, and using the trained classifier as a target classifier, the training module 130 is further configured to:
[0085] If the first fused feature is a fused feature vector, support vector machine or linear discriminant analysis is selected as the initial classifier.
[0086] If the first fused feature is a fused feature matrix, a convolutional neural network or a tensor neural network is selected as the initial classifier.
[0087] The initial classifier is trained using a training set constructed using multiple first fusion features until a target classifier (ie, a trained classifier) is obtained.
[0088] Through the above-mentioned data acquisition module 110, data processing module 120 and training module 130, classifier training based on machine learning is implemented to obtain a target classifier, through which a more accurate sarcopenia diagnosis can be made for the person to be diagnosed.
[0089] The following combination Figure 1 and Figure 3 , the diagnostic process of the sarcopenia auxiliary diagnosis system 10 based on machine learning provided in an embodiment of the present application is introduced.
[0090] The data acquisition module 110 collects voice information, walking video information, and clinical biological information of the person to be diagnosed. Similarly, the clinical biological information of the person to be diagnosed also includes numerical clinical information and exercise test information. Among them, the exercise test information can be a relatively independent test such as a simple physical fitness test and a water drinking test.
[0091] Furthermore, the data acquisition module 110 sends the collected voice information, walking video information, and numerical clinical information of the person to be diagnosed to the data processing module 120 , and sends the motion test information of the person to be diagnosed to the diagnosis module 140 .
[0092] The data processing module 120 extracts and fuses the voice information, walking video information and numerical clinical information of the person to be diagnosed to obtain a second fusion feature, and inputs the second fusion feature into the target classifier.
[0093] The target classifier outputs a first classification result to the diagnosis module 140 according to the input second fusion feature.
[0094] The diagnosis module 140 performs decision-level fusion based on the received motion test information of the person to be diagnosed and the first classification result to obtain a second classification result, and determines the diagnosis result of the person to be diagnosed based on the second classification result.
[0095] It can be seen that the sarcopenia auxiliary diagnosis system 10 based on machine learning provided in the embodiment of the present application performs multimodal feature fusion on the voice feature vector, video feature vector and clinical feature vector of each subject during the training process, and performs decision-level fusion on the first classification results and motion test information of the diagnosed person during the diagnosis process. Through two different fusion processes and comprehensive analysis, the sarcopenia diagnosis accuracy of the sarcopenia auxiliary diagnosis system 10 based on machine learning is greatly improved, and it can realize the rapid identification of sarcopenia in families, nursing homes, communities and other hospitals at all levels, and prevent the occurrence of adverse outcomes of sarcopenia.
[0096] Further, see Figure 4 The present application also provides an electronic device 20, including a processor 220 and a memory 210. The processor 220 and the memory 210 are electrically connected to each other, directly or indirectly, to enable data transmission or interaction. For example, these components may be electrically connected to each other via one or more communication buses or signal lines.
[0097] Among them, the memory 210 stores a computer program for implementing the sarcopenia auxiliary diagnosis system 10 based on machine learning as described in any of the aforementioned embodiments, and the processor 220 can execute the computer program.
[0098] In addition, the memory 210 can be, but is not limited to, random access memory (RAM), read only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), etc.
[0099] It should be understood that Figure 4 The structure shown is only a schematic diagram of the structure of the electronic device 20. The electronic device 20 may also include Figure 4 More or fewer components than shown, or with Figure 4 Different configurations shown.
[0100] In summary, the embodiments of the present application provide a sarcopenia auxiliary diagnosis system and electronic device based on machine learning. The system includes: a data acquisition module for collecting voice information, walking video information, and clinical biological information of multiple subjects; wherein the subjects include sarcopenia patients and healthy people, and the clinical biological information includes numerical clinical information and exercise test information. A data processing module connected to the data acquisition module is used to extract features from the voice information, walking video information, and numerical clinical information of each subject to obtain a voice feature vector, a video feature vector, and a clinical feature vector, and perform multimodal feature fusion on the voice feature vector, video feature vector, and clinical feature vector to obtain a first fused feature. A training module connected to the data processing module is used to construct a training set based on the multiple first fused features, train a classifier based on the training set, and use the trained classifier as a target classifier. The data acquisition module is also used to collect voice information, walking video information, and clinical biological information of the person to be diagnosed. The data processing module is also used to process the voice information, walking video information, and numerical clinical information of the person to be diagnosed to obtain a second fused feature, and input the second fused feature into the target classifier. The diagnostic module is connected to the data acquisition module and the training module respectively, and is used to perform decision-level fusion based on the motion test information of the person to be diagnosed and the first classification result output by the target classifier according to the second fusion feature to obtain a second classification result, and determine the diagnostic result of the person to be diagnosed based on the second classification result.
[0101] The sarcopenia auxiliary diagnosis system based on machine learning provided in the embodiment of the present application performs multimodal feature fusion on the voice feature vector, video feature vector and clinical feature vector of each subject during the training process, and performs decision-level fusion on the first classification results and motion test information of the diagnosed person during the diagnosis process. Through two different fusion processes, a comprehensive analysis is performed, which greatly improves the sarcopenia diagnosis accuracy of the sarcopenia auxiliary diagnosis system 10 based on machine learning, and can realize the rapid identification of sarcopenia in families, nursing homes, communities and other hospitals at all levels, and prevent the occurrence of adverse outcomes of sarcopenia.
[0102] The above description is merely a preferred embodiment of the present application and is not intended to limit the present application. Various modifications and variations are possible for those skilled in the art. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present application shall be included within the scope of protection of the present application.
[0103] It will be apparent to those skilled in the art that the present application is not limited to the details of the exemplary embodiments described above and that the present application can be implemented in other specific forms without departing from the spirit or essential characteristics of the present application. Therefore, the embodiments should be considered in all respects as illustrative and non-restrictive, and the scope of the present application is defined by the appended claims, not the foregoing description, and all variations within the meaning and range of equivalents of the claims are intended to be included therein. Any reference sign in a claim should not be construed as limiting the claim to which it relates.
Claims
1. A sarcopenia auxiliary diagnosis system based on machine learning, characterized in that: include: a data acquisition module, configured to collect voice information, walking video information, and clinical biological information of multiple subjects; wherein the subjects include sarcopenia patients and healthy individuals, and the clinical biological information includes numerical clinical information and exercise test information; a data processing module connected to the data acquisition module, configured to perform feature extraction on the voice information, walking video information, and numerical clinical information of each subject to obtain a voice feature vector, a video feature vector, and a clinical feature vector, and perform multimodal feature fusion on the voice feature vector, the video feature vector, and the clinical feature vector to obtain a first fused feature; a training module connected to the data processing module, configured to construct a training set based on the plurality of first fusion features, train a classifier based on the training set, and use the trained classifier as a target classifier; The data acquisition module is also used to collect voice information, walking video information and clinical biological information of the person to be diagnosed; The data processing module is further configured to process the voice information, walking video information, and numerical clinical information of the person to be diagnosed to obtain a second fusion feature, and input the second fusion feature into the target classifier; The diagnostic module is connected to the data acquisition module and the training module respectively, and is used to perform decision-level fusion based on the motion test information of the person to be diagnosed and the first classification result output by the target classifier according to the second fusion feature to obtain a second classification result, and determine the diagnostic result of the person to be diagnosed based on the second classification result.
2. The sarcopenia auxiliary diagnosis system based on machine learning according to claim 1, characterized in that: In the process of extracting features from the speech information of each subject to obtain a speech feature vector, the data processing module is further configured to: resampling the speech information of each subject to obtain target speech information; The target speech information is subjected to feature extraction based on audio analysis technology and then subjected to standardization processing to obtain the speech feature vector.
3. The sarcopenia auxiliary diagnosis system based on machine learning according to claim 2, characterized in that: The speech information includes a plurality of different types of vowel data; in the process of resampling the speech information of each subject to obtain target speech information, the data processing module is further used to: trimming and removing the first and last segments of the voice information of each subject to obtain trimmed voice information; The target voice information is obtained by calculating the average length of the vowel data of the same type in the plurality of cropped voice information, and resampling the vowel data of the same type in each of the cropped voice information to the same length based on the average length.
4. The sarcopenia auxiliary diagnosis system based on machine learning according to claim 1, characterized in that: In the process of extracting features from the walking video information of each subject to obtain a video feature vector, the data processing module is further used to: A cycle is defined as the time from when the subject takes a step with the left foot to when the subject takes a step with the left foot next time. In each cycle, joint point information of the subject's arm swing and leg walking are extracted to obtain the video feature vector.
5. The sarcopenia auxiliary diagnosis system based on machine learning according to claim 4 is characterized in that: The video feature vector also includes the walking speed and acceleration of the subject during the entire walking video period.
6. The sarcopenia auxiliary diagnosis system based on machine learning according to claim 1, characterized in that: The numerical clinical information includes: upper arm circumference, grip strength, leg circumference, number of falls in the past year, eating assessment questionnaire score, and daily living ability score; in the process of extracting features from the numerical clinical information of each subject to obtain a clinical feature vector, the data processing module is further used to: The upper arm circumference, grip strength, leg circumference, number of falls in the past year, eating assessment questionnaire score, and daily living ability score are standardized so that each feature is distributed between 0 and 1 to obtain the clinical feature vector.
7. The sarcopenia auxiliary diagnosis system based on machine learning according to claim 1, characterized in that: The first fusion feature is a fusion feature vector or a fusion feature matrix; in the process of performing multimodal feature fusion on the speech feature vector, the video feature vector, and the clinical feature vector to obtain the first fusion feature, the data processing module is further used to: The speech feature vector, the video feature vector and the clinical feature vector are sequentially spliced and fused to obtain an initial feature vector, and the initial feature vector is subjected to dimensionality reduction by feature selection or principal component analysis to obtain the fused feature vector; or The speech feature vector, the video feature vector and the clinical feature vector are subjected to dimensionality reduction using principal component analysis, and the dimensions of the speech feature vector, the video feature vector and the clinical feature vector after dimensionality reduction are ensured to be consistent, and then fused to obtain the fused feature matrix.
8. The sarcopenia auxiliary diagnosis system based on machine learning according to claim 7, characterized in that: In the process of constructing a training set according to the plurality of first fusion features, training a classifier according to the training set, and using the trained classifier as a target classifier, the training module is further configured to: If the first fused feature is the fused feature vector, selecting support vector machine or linear discriminant analysis as the initial classifier; If the first fusion feature is the fusion feature matrix, selecting a convolutional neural network or a tensor neural network as an initial classifier; The initial classifier is trained using a training set constructed using a plurality of the first fusion features until a target classifier is obtained.
9. The sarcopenia auxiliary diagnosis system based on machine learning according to claim 1, characterized in that: The clinical biological information also includes basic information; in the process of performing multimodal feature fusion on the speech feature vector, the video feature vector, and the clinical feature vector to obtain a first fusion feature, the data processing module is further used to: The speech feature vector, the video feature vector, and the clinical feature vector are weighted and transformed according to the basic information, and then multimodal feature fusion is performed to obtain the first fusion feature.
10. An electronic device, characterized in that: It includes a processor and a memory, the memory stores a computer program for implementing the sarcopenia auxiliary diagnosis system based on machine learning as described in any one of claims 1 to 9, and the processor can execute the computer program.
Citation Information
Cited By
Multi-modal acquisition-based multi-center clinical data management system and method
CN120932921A