Intelligent vital sign detection and assisted decision system based on data fusion

Abstract

The application provides an intelligent sign detection and auxiliary decision system based on data fusion. The system is applied to the technical field of sign detection and auxiliary treatment, and comprises a collection and preprocessing module, a model construction and optimization module, a stimulation electrode adjustment module and a data visualization module.The collection and preprocessing module is used for collecting multi-modal human body sign data of a patient and performing preprocessing; the model construction and optimization module is used for constructing a model for training by combining a random forest and a ReliefF algorithm, extracting data features and layer strategies, and predicting a sign detection result; the stimulation electrode adjustment module is used for dynamically adjusting the intensity, frequency and position of a stimulation electrode according to the predicted sign detection result by using a feedback mechanism of a self-adaptive proportional-integral-derivative control algorithm; and the data visualization module is used for real-time presenting multi-modal human body sign data, a sign detection result and a real-time condition of electrode stimulation in a data visualization mode, to assist a doctor in operation and decision-making. The application improves the accuracy, practicality and scientificity of a medical health detection system.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of sign detection and auxiliary treatment, and particularly relates to an intelligent sign detection and auxiliary decision system based on data fusion. BACKGROUND

[0002] With the development of the medical and health field, sign detection and auxiliary decision systems play an increasingly important role in disease diagnosis, treatment optimization and health management. However, traditional sign detection methods have many limitations, such as low data processing efficiency, insufficient accuracy and difficulty in comprehensive analysis of multi-modal data. With the rapid development of wearable device technology, flexible sensors and stimulating electrodes can collect a large amount of medical data, providing rich information sources for sign detection and medical decision-making. However, how to efficiently process and analyze these data and extract valuable information to provide reliable basis for medical decision-making has become a problem to be solved.

[0003] Existing sign detection and auxiliary decision systems can only process single-modal data, and it is difficult to fully reflect the health status of patients. At the same time, due to the cross-domain difference and data scarcity of medical data, traditional machine learning algorithms perform well in disease classification and prediction, but their performance is often greatly reduced in the case of new class diseases or limited data. In addition, traditional electrode stimulation systems lack adaptive adjustment ability and cannot dynamically adjust according to physiological changes of patients, thereby affecting the treatment effect. SUMMARY

[0004] The present application provides an intelligent sign detection and auxiliary decision system based on data fusion to solve the technical problems that traditional methods cannot effectively process and analyze multi-modal data, have poor detection accuracy and precision when facing data scarcity or cross-domain difference, and traditional electrode stimulation systems cannot dynamically adjust according to physiological changes of patients and cannot meet the individual needs of different patients. The present application improves the accuracy and practicability of medical and health detection systems and enhances the precision of sign detection and the scientificity of medical decision-making by efficiently processing, feature layering, intelligent analysis and electrode adaptive adjustment of data collected by flexible pressure stimulation sensors.

[0005] To achieve the above-mentioned purpose, the following technical solution is implemented: an intelligent sign detection and auxiliary decision system based on data fusion, comprising:

[0006] An acquisition and preprocessing module is configured to acquire and preprocess multi-modal human sign data of a patient, wherein the multi-modal human sign data includes heart rate, blood pressure, skin resistance and electromyographic signals.

[0007] The model construction and optimization module is configured to construct a model trained by combining a random forest algorithm and a ReliefF algorithm and a layer strategy for data feature extraction, and integrate the model into a cross-domain low-sample self-centered identification task based on multi-modal input and unlabeled target data, so as to optimize a classification task of the target data, and obtain a sign detection result through dynamic mask generation and multiple inference integrated prediction.

[0008] The stimulation electrode adjustment module is configured to dynamically adjust intensity, frequency and position of the stimulation electrode according to the predicted sign detection result by using a feedback mechanism of an adaptive proportional-integral-derivative control algorithm.

[0009] The data visualization module is configured to present multi-modal human sign data, sign detection results and electrode stimulation real-time conditions in a data visualization manner in real time, to assist doctors in operation and decision-making.

[0010] Further, the model trained by combining the random forest algorithm and the ReliefF algorithm and the layer strategy for data feature extraction is integrated into the cross-domain low-sample self-centered identification task based on the multi-modal input and the unlabeled target data, so as to optimize the classification task of the target data, and the sign detection result is obtained through the dynamic mask generation and the multiple inference integrated prediction, which includes:

[0011] The cross-domain low-sample learning task is constructed, and a multi-modal human sign data set is divided into a labeled source data set D s and an unlabeled target data set D Tu The target data set D Tu is further divided into a support set S and a query set Q for model training and testing.

[0012] The data feature extraction and layering strategy combined with the random forest algorithm and the ReliefF algorithm is constructed to realize hierarchical screening of target data features.

[0013] The data feature extraction and layering strategy is applied to multiple data training stages, and domain adaptation and multi-modal distillation, spatiotemporal cube reconstruction and integrated mask inference are combined to optimize classification performance, and a sign detection result is predicted.

[0014] Further, the data feature extraction and layering strategy combined with the random forest algorithm and the ReliefF algorithm is constructed to realize hierarchical screening of target data features, which includes:

[0015] A plurality of subsets are randomly sampled from original sample data, each subset is used to train a decision tree, and finally a random forest is constructed.

[0016] After the random forest is constructed, the importance weight of the feature is calculated by using the ReliefF, hierarchical screening of the target data feature is realized, and the weight of the feature is obtained.

[0017] According to the feature weight ranking, the feature data extracted from the multi-modal human body sign data by the random forest and the ReliefF algorithm is divided into a high weight set, a medium weight set and a low weight set, and the features are uniformly sampled within each set.

[0018] Further, the data feature extraction is applied to a plurality of data training stages in combination with domain adaptation and multi-modal distillation, spatio-temporal cube reconstruction and integrated mask reasoning for optimizing classification performance, and a sign detection result is predicted, comprising:

[0019] A class discriminative feature composed of the sum of the RGB feature projected by the student encoder and the multi-modal feature projected by the teacher encoder is defined;

[0020] Self-supervised pre-training is performed by the mask autoencoder to optimize the reconstruction loss and the cross-entropy loss;

[0021] In the multi-modal distillation stage, random forest and ReliefF feature screening are introduced to enable the distillation process to adaptively select the most discriminative features, i.e., to extract class discriminative features from the preprocessed multi-modal data, optimize the decision boundary of disease classification, and thus realize efficient knowledge transfer and model optimization in the target domain, including: introducing a multi-modal distillation loss, and using the teacher encoder and the RGB student encoder to align the features;

[0022] The classifier is trained on the support set S, and an encoder-decoder architecture is introduced in the model to extract deep features of the data and optimize the classification loss through spatio-temporal cube reconstruction;

[0023] Dynamic mask generation and multiple reasoning integration are used to predict the query set Q and output the final prediction result, i.e., the sign detection result.

[0024] Further, the training of the classifier on the support set S and the introduction of the encoder-decoder architecture in the model to extract deep features of the data and optimize the classification loss through spatio-temporal cube reconstruction, comprising:

[0025] Based on the limited labeled samples in the support set S, an encoder-decoder architecture is introduced in the model;

[0026] Deep features of the data are extracted using the encoder-decoder architecture, and the classification loss is optimized through spatio-temporal cube reconstruction; wherein the specific method of the spatio-temporal cube reconstruction is:

[0027] Part of the input features are randomly occluded by the spatio-temporal mask to force the model to reason in the absence of information;

[0028] For the spatio-temporal cube data after masking, decoder reconstruction is adopted for data enhancement to restore the data blocked by the mask and generate new sample variants, so as to obtain the reconstructed data, that is, generate enhanced samples, which are used to expand the diversity of training data;

[0029] Features are further extracted from the reconstructed data by the encoder.

[0030] Further, the dynamic mask generation and multiple inference integration are used to predict the query set Q and output the final prediction result, that is, the vital sign detection result, which includes:

[0031] During inference, dynamic mask generation is adopted to generate multiple mask data according to the set mask ratio, and these mask data represent different blocking degrees of the query set samples;

[0032] Through multiple inferences, multiple prediction results corresponding to all mask variants are obtained;

[0033] The multiple prediction results corresponding to all mask variants are averaged to obtain the final prediction result.

[0034] Further, the feedback control formula of the adaptive proportional-integral-derivative control algorithm is:

[0035] ;

[0036] Wherein, is the electrode stimulation intensity at time ; is the error, that is, the difference between the current vital sign detection value and the target vital sign value, and ; is the predicted vital sign detection result, is the target vital sign value; , , : is the proportional-integral-derivative control parameter, which respectively represents the proportional, integral and derivative gain, which respectively adaptively adjusts the gain according to the change of the predicted vital sign detection result .

[0037] Further, the proportional, integral and derivative gains , , adaptively adjust the gain according to the change of the predicted vital sign detection result .

[0038] The control gain is adjusted by the confidence weighting of the vital sign data to adapt to the needs in different physiological states, and the gain adjustment formula is as follows:

[0039] ;

[0040] ;

[0041] ;

[0042] wherein, , , are updated proportional-integral-derivative control parameters, representing updated proportional, integral, derivative gains, respectively; , , are pre-updated proportional-integral-derivative control parameters, representing pre-updated proportional, integral, derivative gains, respectively; , and are the confidence scores calculated for proportional gain, integral gain and derivative gain, respectively; , and are the weighted sum of confidence scores calculated for proportional gain , integral gain and derivative gain , respectively; , and are the confidence scores of proportional gain , integral gain and derivative gain , calculated based on the detection result of the th vital sign.

[0043] Further, wherein the data visualization module is configured to present the multi-modal human body vital sign data, vital sign detection results and electrode stimulation real-time status in a data visualization manner in real time, to assist doctors in operation and decision-making, including:

[0044] The feature embedding space t-SNE method is used to show the distribution of multi-modal data, and the support vector machine kernel function mapping is used to display the classification decision boundary, and the visualization result is dynamically updated to assist medical decision-making.

[0045] Further, wherein the data visualization process includes:

[0046] The feature embedding space t-SNE method is used to project the high-dimensional features of the class-discriminative features extracted from the pre-processed multi-modal data to two-dimensional space through t-SNE, to show the distribution relationship of different modal data in the embedding space, and to help doctors understand the model decision basis;

[0047] ​The feature space of the class discriminative feature F(x) is mapped by the kernel function of the support vector machine, and the decision boundary of disease classification can be directly displayed, which assists doctors in evaluating the classification logic of the model, wherein the kernel function mapping of the support vector machine is used for nonlinear decision boundary display of high-dimensional data, the decision boundary is projected onto a two-dimensional plane, and different colors are used to identify different categories of physical data, which assists doctors in quickly identifying high-risk cases.

[0048] When the patient's physical data changes, the model immediately updates the visualization result, realizes dynamic real-time updating of the visualization result, and provides the most timely clinical decision support.

[0049] Compared with the prior art, the present application has the following beneficial effects:

[0050] (1) The present application integrates data from different sensors (such as heart rate, blood pressure, skin resistance, electromyographic signals, etc.) through multi-modal data fusion technology, so that the system can more comprehensively reflect the health status of the patient. Using an improved cross-domain low-sample learning method, the system can still accurately classify new category diseases even if there are cross-domain differences or limited data in medical data, thereby improving the accuracy of physical detection. Combined with random forest and ReliefF algorithm for feature extraction and layering, redundant features are effectively removed, improving the effectiveness and computational efficiency of the features.

[0051] (2) The present application generates dynamic masks and integrates multiple inferences to improve the robustness and accuracy of the model while ensuring computational efficiency, reducing the bias and instability that may be caused by single inference. Combined with domain adaptation and multi-modal distillation strategy, the system can achieve efficient knowledge transfer and model optimization in the target domain, improving the adaptability of the model in new environments.

[0052] (3) According to the predicted physical detection results, the present application uses a self-adaptive proportional-integral-derivative control algorithm to dynamically adjust the intensity, frequency and position of the stimulating electrode, ensuring the accuracy and safety of the stimulation. By adjusting the control gain with the confidence weighting of the physical data, the system can achieve personalized electrode stimulation adjustment according to the needs of different physiological states, thereby improving the treatment effect.

[0053] (4) The present application establishes an individualized physiological model, combines real-time physical data and historical data, and realizes personalized adjustment of stimulating electrode parameters to meet the individual needs of different patients. An intuitive data visualization interface is provided to enable doctors to better understand the health status and treatment effect of patients, thereby developing more personalized treatment plans.

[0054] It is to be understood that the description in the summary is not intended to identify key or essential features of embodiments of the application, nor is it intended to limit the scope of the application. Other features, aspects, and advantages of the application will become apparent from the following description, which is given for the purpose of illustration and is not intended to limit the scope of what the inventors regard as their application. BRIEF DESCRIPTION OF DRAWINGS

[0055] The above and other features, aspects, and advantages of embodiments of the present application will become more apparent from the following description when taken in conjunction with the accompanying drawings. The drawings are intended to be not limiting and are provided for purposes of illustration only. In the drawings:

[0056] Figure 1 is a system flow diagram of an intelligent vital sign detection and auxiliary decision system based on data fusion according to an embodiment of the application;

[0057] Figure 2 is a module schematic diagram of an intelligent vital sign detection and auxiliary decision system based on data fusion according to an embodiment of the application. DETAILED DESCRIPTION

[0058] In order to make the objects, technical solutions and advantages of embodiments of the present application clearer, the following will clearly and completely describe the technical solutions in the embodiments of the present application with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are some but not all of the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art without creative work fall within the scope of protection of the present application.

[0059] In addition, the term "and / or" in this paper is only to describe the association relationship of the associated objects, which means that there can be three relationships, for example, A and / or B, which can represent the three cases of A alone, A and B together, and B alone. In addition, the character " / " in this paper generally represents an "or" relationship between the front and rear associated objects.

[0060] The present application provides an intelligent vital sign detection and auxiliary decision system based on data fusion, and the core technology includes the following aspects:

[0061] Through multi-modal data fusion, improved cross-domain low sample learning is adopted, the model is trained by using labeled source data and unlabeled target data, multi-weight features are divided by combining random forest and ReliefF algorithm, and integrated mask reasoning is performed by combining self-defined domain adaptation, multi-modal distillation strategy, and spatiotemporal cube reconstruction, so that in the case of cross-domain difference or limited data of medical data, new class diseases can still be accurately classified;

[0062] According to the sign detection result, an improved feedback control mechanism method is used to adaptively adjust the electrode stimulation intensity, the position, intensity and frequency of the electrode are dynamically adjusted through self-defined gain adjustment, and it is ensured that the electrode stimulation scheme can be optimally adjusted according to the changes of the sign data and the individualized physiological model;

[0063] Combined with various data visualization technologies, the complexity of high-dimensional data is converted into intuitive graphical dynamic display, and when the patient sign data changes, the model can immediately update the visualization result to assist the doctor in operation and decision-making.

[0064] Figure 1 A flow chart of an intelligent sign detection and auxiliary decision-making system based on data fusion is shown; Figure 2 A module schematic diagram of an intelligent sign detection and auxiliary decision-making system based on data fusion is shown. As Figure 1 and Figure 2 As shown in the figure, an intelligent sign detection and auxiliary decision-making system 100 based on data fusion includes:

[0065] The acquisition and preprocessing module 101 is used for acquiring and preprocessing the multi-modal human sign data of the patient, and the multi-modal human sign data includes heart rate, blood pressure, skin resistance, electromyographic signal and other physiological signals, etc., thereby forming a set of rich multi-modal data stream, which not only covers the traditional vital sign monitoring parameters, but also introduces more detailed stimulation electrode feedback signals.

[0066] In order to efficiently process these data from different devices and different modalities, the present application supports large-scale data integration and management through a data warehouse. First, through data extraction, the system can acquire sign data from multiple data sources (including flexible sensors and stimulation electrodes) in real time or periodically. After conversion processing, the raw data are standardized, denoised based on an adaptive filter, missing value filled, etc., to ensure the consistency and integrity of the data.

[0067] The model construction and optimization module 102 is used for model training combined with data feature extraction and layer strategy of random forest and ReliefF algorithm, and is integrated into a cross-domain low-sample self-centered recognition task based on multi-modal input and unlabeled target data, so as to optimize the classification task of the target data, and obtain the sign detection result through dynamic mask generation and multiple inference integrated prediction;

[0068] The stimulation electrode adjustment module 103 is used for dynamically adjusting the intensity, frequency and position of the stimulation electrode according to the predicted sign detection result by using a feedback mechanism of an adaptive proportional-integral-derivative control algorithm;

[0069] The data visualization module 104 is used for real-time presentation of the multi-modal human body sign data, the sign detection result and the electrode stimulation real-time condition in a data visualization manner, to assist doctors in operation and decision-making.

[0070] Further, the data feature extraction and layer strategy combined with the random forest and the ReliefF algorithm is used for model training, and is integrated into the cross-domain low sample self-centered identification task based on the multi-modal input and the unlabeled target data, to optimize the classification task of the target data, and the sign detection result is obtained through dynamic mask generation and multiple inference integrated prediction, including:

[0071] (1) A cross-domain low sample learning task is constructed, and a multi-modal human body sign data set is divided into a labeled source data set Ds and an unlabeled target data set D Tu The cross-domain low sample learning task is used for training by using the labeled source data set Ds and the unlabeled target data D Tu , so as to classify new classes of the target data set D , and the data is divided according to the mode, and the classes of the source data set and the target data set do not overlap; the target data set D Tu is further divided into a support set S (containing classes, and data in each class) and a query set Q (the same as the support set class), which are used for model training and testing.

[0072] On this basis, the data feature extraction and layer strategy described below is used to optimize the model training and improve the classification accuracy.

[0073] (2) A data feature extraction and layer strategy combined with the random forest and the ReliefF algorithm is constructed, to realize hierarchical screening of the target data features.

[0074] In order to enhance the modal alignment and feature adaptation ability of the source data and the target data, the feature layer strategy is constructed based on the random forest and the ReliefF algorithm.

[0075] In the feature learning stage of the source data set D and the unlabeled target data set D , the Bootstrap self-sampling method is introduced, a plurality of subsets are randomly sampled from the original sample data, each subset is used for training a decision tree, and finally a random forest is constructed. After the random forest is constructed, the importance weight of the feature is calculated by using the ReliefF, so as to realize hierarchical screening of the target data features:

[0076] ;

[0077] Wherein, : the weight of the feature ;

[0078] : number of samples in query set Q;

[0079] : feature difference with samples of the same category;

[0080] : feature difference with samples of different categories;

[0081] : number of nearest neighbor samples.

[0082] According to feature weight Sorting, the feature data extracted from the multi-modal human body sign data by the random forest and the ReliefF algorithm is divided into: a high-weight set (ranking top 30%), a medium-weight set (30%-70%), and a low-weight set (ranking last 30%).

[0083] Uniformly sampling features within each set avoids excessive influence of low-weight features on the model and improves the discrimination ability of the target data set.

[0084] (3) The data feature extraction and hierarchical strategy is applied to multiple data training stages, combined with domain adaptation and multi-modal distillation, spatiotemporal cube reconstruction and integrated mask reasoning to optimize classification performance, and a sign detection result is predicted.

[0085] In the data training process, the data feature extraction and hierarchical strategy described above is applied to multiple data training stages to optimize classification performance.

[0086] Specifically, the data feature extraction and hierarchical strategy is applied to multiple data training stages, combined with domain adaptation and multi-modal distillation, spatiotemporal cube reconstruction and integrated mask reasoning to optimize classification performance, and a sign detection result is predicted, including:

[0087] (3.1) defining a category discriminative feature composed of the sum of the RGB feature of the student encoder and the multi-modal feature projected by the teacher encoder, to improve the cross-domain adaptation ability;

[0088] Cross-domain low-sample learning is adopted, which aims to use source or labeled data and target domain unlabeled data to classify target domain data . Define the category discriminative feature :

[0089] ;

[0090] wherein, is the feature extracted by the RGB student encoder for the input data x, and the RGB student encoder is a model trained for disease classification; is the feature extracted by the teacher encoder for the m-th modality data x, and the teacher encoder is a pre-trained model for extracting effective feature representation from multi-modal data; is the feature projection layer for projecting the features from different modalities into the same feature space for subsequent fusion and classification tasks; m is the modality index for traversing all modalities to fuse the features from different modalities; represents the fused class discriminative feature, which is obtained by fusing the feature extracted by the student encoder and the multi-modal feature extracted by the teacher encoder, and this fused feature is used for subsequent classification tasks.

[0091] (3.2) Self-supervised pre-training by mask autoencoder, optimizing reconstruction loss (including source data reconstruction loss and target data reconstruction loss) and cross-entropy loss;

[0092] In the self-supervised pre-training process based on the mask autoencoder, the model is jointly trained by the source data set and the target data set to optimize the objective function:

[0093] ;

[0094] wherein, is the source data reconstruction loss; is the target data reconstruction loss; is the cross-entropy loss on the source data; : trade-off hyperparameter.

[0095] wherein:

[0096] ;

[0097] ;

[0098] ;

[0099] wherein, is the original input data; is the reconstructed data; and are the sample sizes of the source data set and the target data set , respectively; For the source dataset the true label of the nth sample (usually a one-hot encoding or a numerical representation of the class label) in the source dataset For the source dataset the predicted probability (or predicted output) of the nth sample in the source dataset, usually a probability distribution representing the probability of the sample belonging to each class. In a classification task, if the model is a softmax regression or a similar probability model, is the value of the element in the probability vector output by the model corresponding to the true class label (i.e., the probability that the model predicts the sample belongs to the true class).

[0100] (3.3) In the multi-modal distillation stage, random forest and ReliefF feature selection are introduced to enable the distillation process to adaptively select the most discriminative features, i.e., to extract class discriminative features from the pre-processed multi-modal data, optimize the decision boundary of disease classification, and thus realize efficient knowledge transfer and model optimization in the target domain, including: introducing a multi-modal distillation loss, using the teacher encoder and the RGB student encoder for feature alignment;

[0101] In the multi-modal distillation stage, the present application introduces random forest and ReliefF feature selection to enable the distillation process to adaptively select the most discriminative features, thereby realizing efficient knowledge transfer and model optimization in the target domain, while ensuring the generalization ability of the model, to improve the discriminative ability of the target domain RGB student encoder, and using L2 norm to optimize the distillation loss:

[0102] ;

[0103] wherein, is the feature extracted by the RGB student encoder; is the real modality feature extracted by the teacher encoder; is the multi-modal distillation loss.

[0104] (3.4) Train the classifier on the support set S, and introduce an encoder-decoder architecture in the model to extract deep features of the data and optimize the classification loss through spatiotemporal cube reconstruction, including:

[0105] Based on the limited labeled samples in the support set S, an encoder-decoder architecture is introduced in the model, which not only extracts deep features of the data, but also processes the sample scarcity problem caused by label scarcity through spatiotemporal cube reconstruction.

[0106] wherein, the specific way of spatiotemporal cube reconstruction is:

[0107] During the training process, part of the input features is randomly masked by a spatio-temporal mask, forcing the model to reason in the absence of information. The randomness of this mask not only simulates the common data missing problem in practical applications, but also prompts the model to extract as much implicit information as possible from the data with limited information.

[0108] For the masked spatio-temporal cube data, decoder reconstruction is used for data enhancement. In this process, the decoder not only restores the data masked by the mask, but also generates new sample variants, and then obtains the reconstructed data, i.e. generates enhanced samples, which are used to expand the diversity of training data.

[0109] The reconstructed data is further extracted by the encoder, and the original sample is used to participate in the training of the classifier, ensuring that the model can learn more effective features with limited samples.

[0110] Specifically, the mask reconstruction loss is optimized by the following formula:

[0111] ;

[0112] Wherein, is the newly trained classifier; is the encoded feature of the support set; is the cross-entropy loss.

[0113] (3.5) Dynamic mask generation and multiple reasoning integration are used to predict the query set Q and output the final prediction result, i.e. the physical detection result.

[0114] To reduce the calculation cost, the present application balances the consumption of computing resources and the requirement of prediction accuracy by dynamic mask generation and multiple reasoning integration. It not only reduces the redundant information of input data through the mask mechanism, but also further enhances the robustness and accuracy of the model by integrating the prediction results of multiple mask variants.

[0115] Specifically, during reasoning, the present application uses dynamic mask generation, and according to the set mask proportion Multiple mask data are generated, which represent different degrees of masking of the query set samples. Specifically, the mask proportion determines which features are randomly masked and which features are retained when inputting data each time. In this way, the present application can simulate the incomplete or partially missing data that may be encountered in the real world while ensuring calculation efficiency. Through multiple reasoning, multiple prediction results corresponding to all mask variants are obtained, and the deviation and instability that may be caused by single reasoning are reduced by integrating multiple prediction results.

[0116] The samples of the query set Q are generated with a mask proportion ​ The final prediction result is obtained by averaging the output results of all mask variants:

[0117] ;

[0118] wherein, is the final prediction result or the final prediction probability; is the input data (i.e., the samples in the query set Q) processed by different masks, in the inference process, by dynamic mask generation technology, different masks are applied to the original input data to generate multiple mask variants ; is the feature extracted by the RGB student encoder for the jth mask variant ; is the number of mask variants for integrated inference, i.e., the number of dynamically generated mask data; is a newly trained classifier based on the encoder-decoder architecture, which is used to classify the input data.

[0119] The final prediction result is obtained by integrating the prediction results of multiple mask variants , which is used to represent the final classification result of the model for the samples in the query set Q, or as the basis for subsequent decision-making. The above formula can improve the robustness and accuracy of the model by generating different mask data multiple times, performing multiple inferences, and then integrating the prediction results of multiple mask variants.

[0120] According to the predicted sign detection result, the stimulation electrode adjustment module 103 adopts a feedback mechanism of an adaptive proportional-integral-derivative control algorithm to dynamically adjust the intensity, frequency and position of the stimulation electrode, specifically including:

[0121] In the present application, in order to realize personalized stimulation electrode adjustment, a dynamic feedback control mechanism is proposed. Based on real-time sign detection results such as muscle response, neural activity or heart rate fluctuation, the mechanism automatically adjusts the intensity of the stimulation electrode to ensure the accuracy and safety of the stimulation and maximize the treatment effect.

[0122] Specifically, the aforementioned predicted sign detection result reflects the individual's health status or physiological signals, including but not limited to muscle electrical activity, heart rate variability or other neurophysiological signals. Through the analysis of these sign data, the physiological state of the individual can be monitored in real time, and accurate feedback can be provided for the adjustment of the stimulation electrode.

[0123] To achieve this goal, the present invention introduces a feedback mechanism based on adaptive proportional-integral-derivative control algorithm, which not only can dynamically adjust the intensity, frequency and position of the stimulating electrode according to the results of physical detection, but also has good response speed and flexibility. Unlike traditional proportional-integral-derivative control, the control mechanism of the present invention considers the nonlinear characteristics and time-varying nature of individual physiological changes, making the electrode adjustment more intelligent and personalized. The feedback control formula is as follows:

[0124] ;

[0125] where, is the electrode stimulation intensity at time ; is the error, i.e. the difference between the current physical detection value and the target physical value, and ; is the predicted physical detection result (such as heart rate, blood pressure, muscle electrical signal, neural activity, etc.), is the target physical value or expected physical value; , , : are proportional-integral-derivative control parameters, representing proportional, integral, and derivative gains respectively, which are respectively adjusted according to the adaptive gain adjustment of the predicted physical detection result .

[0126] Through real-time error feedback, the intensity of electrode stimulation can be adjusted according to the real-time changes of individual physiological signals. For example, when the heart rate fluctuates greatly, the control system will automatically reduce the stimulation intensity to avoid excessive stimulation to the body; when the muscle reaction is too tired or unstable, the system will automatically adjust the frequency and intensity to avoid side effects caused by excessive stimulation.

[0127] To improve the sensitivity of adjustment, , , can be adjusted according to the adaptive gain adjustment of (combined with dynamic weight), by weighting the confidence of physical data, adjusting the control gain to adapt to the needs in different physiological states. For example, when the fluctuation of physical signal is large or there is uncertainty, the system will automatically reduce the gain value, thereby reducing the risk of excessive adjustment; while the signal is stable, the gain will automatically increase to ensure more accurate stimulation adjustment. The gain adjustment formula is as follows:

[0128] ;

[0129] ;

[0130] ;

[0131] wherein, , , are updated proportional-integral-derivative control parameters, representing updated proportional, integral, and derivative gains, respectively; , , are pre-updated proportional-integral-derivative control parameters, representing pre-updated proportional, integral, and derivative gains, respectively; , and are the confidence scores calculated for the proportional gain, integral gain, and derivative gain, respectively; , and are the confidence scores calculated for the proportional gain , integral gain , and derivative gain , respectively; , and are the confidence scores calculated for the proportional gain , integral gain , and derivative gain , respectively, based on the th vital sign detection result .

[0132] wherein, the optimization of electrode parameters, i.e., the dynamic adjustment of the intensity, frequency, and position of the stimulating electrode, is as follows:

[0133] By establishing an individualized physiological model, the system can simulate the response of different physiological signals to electrode stimulation and then make predictive adjustments. This physiological model not only can be trained according to the individual's historical data, but also can be optimized in combination with real-time vital sign signals, thereby providing a more personalized and precise electrode adjustment scheme. By combining real-time vital sign data and a physiological model, the multiple parameters of the electrode are adjusted through gradient optimization to ensure that the electrode stimulation always maintains the best effect throughout the treatment process:

[0134] ;

[0135] , : represent the pre-updated and updated electrode parameters (position, intensity, frequency), respectively; : learning rate; : electrode parameter optimization loss function, related to vital sign detection result ;

[0136] ;

[0137] , : respectively represent the prediction result of the individual sign detection and the expected sign value.

[0138] For the data visualization module 104, the multi-modal human body sign data, sign detection results and electrode stimulation real-time situation are presented in a data visualization manner in real time to assist doctors in operation and decision-making, including:

[0139] The multi-modal data distribution is displayed by using the feature embedding space t-SNE method, and the classification decision boundary is displayed by using the support vector machine kernel function mapping, and the visualization result is dynamically updated to assist medical decision-making.

[0140] t-SNE (t-Distributed Stochastic Neighbor Embedding) is a nonlinear dimensionality reduction technique, which is particularly suitable for the visualization of high-dimensional data. It can project high-dimensional data into low-dimensional space (usually two-dimensional or three-dimensional), while preserving the local and global structure between data points.

[0141] The class discriminative feature F(x) is a feature representation obtained by cross-domain low-sample learning from source data and target data, which contains key information for classification. The F(x) of the application is a high-dimensional feature vector, which contains features from different modalities (such as heart rate, blood pressure, skin resistance, electromyographic signals, etc.). The feature obtained after projecting F(x) to two-dimensional space by t-SNE method is called embedding feature, and this projection process preserves the key structural information in F(x), so that doctors can intuitively observe the distribution relationship of different modal data on the two-dimensional plane.

[0142] Support vector machine (SVM) is a binary classification model, which finds a hyperplane to separate data points of different classes and maximizes the distance (i.e. margin) of the hyperplane to the nearest data point. For non-linearly separable data, SVM can map data to high-dimensional space by introducing kernel functions, so as to find a linearly separable hyperplane. In the application, the kernel function (such as radial basis function RBF) of support vector machine is used to map the class discriminative feature F(x) to high-dimensional space. This mapping process preserves the nonlinear structure information in F(x), so that SVM can learn more complex decision boundaries.

[0143] Further, the data visualization process of the data visualization module 104 includes:

[0144] The high-dimensional feature of the class discriminative feature extracted from the preprocessed multi-modal data is projected to two-dimensional space by t-SNE to show the distribution relationship of different modal data in the embedding space, which helps doctors understand the model decision basis;​

[0145] Optionally, the specific process of t-SNE projection includes: (1) data preparation: extract the class discriminative features F(x) from the classification model as the input data of t-SNE. (2) Parameter setting: set the parameters of t-SNE, such as perplexity, number of iterations, learning rate, etc. Perplexity is a key parameter that controls the balance between preserving local structure and global structure for t-SNE. (3) Run t-SNE: input F(x) into the t-SNE algorithm to get the two-dimensional embedding features. (4) Visualization display: plot the two-dimensional embedding features on a two-dimensional plane, and different colors or shapes are used to represent different categories of data points. Doctors can understand the relationship between different modal data and the model's ability to distinguish different categories by observing the distribution of these data points.

[0146] The feature space of the class discriminative features F(x) is mapped by the kernel function of the support vector machine, and the decision boundary of the disease classification can be displayed intuitively to assist doctors in evaluating the classification logic of the model. The kernel function mapping of the support vector machine is used for nonlinear decision boundary display of high-dimensional data, the decision boundary is projected onto a two-dimensional plane, and different colors are used to identify different categories of sign data to assist doctors in quickly identifying high-risk cases.

[0147] Optionally, the visualization process of the classification boundary is as follows: (1) Data preparation: similar to t-SNE, extract the class discriminative features F(x) from the classification model. (2) Train SVM: use F(x) as input data to train an SVM model. In the training process, select appropriate kernel function and parameters (such as penalty parameter C, kernel function parameter γ, etc.). (3) Decision boundary projection: project the decision boundary of SVM onto a two-dimensional plane. This is usually achieved by selecting two feature dimensions (such as the two dimensions after t-SNE projection) and calculating the decision boundary in these dimensions. (4) Visualization display: plot the data points of different categories and the decision boundary on a two-dimensional plane, and use different colors to identify the data points of different categories. Doctors can evaluate the classification logic and performance of the model by observing the position and shape of the decision boundary.

[0148] When the patient's sign data changes, the model immediately updates the visualization results, achieving dynamic real-time updating of the visualization results and providing the most timely clinical decision support. Specifically, input the new F(x) into the t-SNE and SVM models to get new two-dimensional embedding features and decision boundaries. Then, update the visualization interface to display the latest data distribution and decision boundary. Doctors can adjust and optimize the treatment plan according to the updated visualization results. At the same time, the system can also further train and optimize the model according to the doctor's feedback.

[0149] According to the above-mentioned embodiments of the present application, through the multi-modal data fusion and cross-domain low sample learning technology, comprehensive analysis and accurate classification of multi-modal data are realized; through dynamic mask generation and multiple inference integration, the robustness and accuracy of the system are improved; the adaptive proportional-integral-derivative control algorithm is adopted to realize adaptive adjustment of the electrode stimulation intensity, which can meet the individualized treatment needs of patients; through the combination of various data visualization technologies, intuitive and dynamic data visualization support is provided.

[0150] It should be understood that the various forms of flow shown above can be reordered, added to, or deleted from without departing from the scope of the present application. For example, the steps recited in the present application can be performed in parallel, in series, or in a different order, as long as the desired results of the technical solutions disclosed in the present application can be achieved, and the present application is not limited herein.

[0151] The above detailed description does not constitute a limitation on the scope of protection of the present application. Those skilled in the art should understand that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modifications, equivalent replacements and improvements made within the spirit and principles of the present application shall be included in the scope of protection of the present application.

Claims

1. A data fusion-based intelligent vital sign detection and decision support system, characterized in that, include: The acquisition and preprocessing module is used to acquire and preprocess the patient's multimodal human vital signs data, which includes heart rate, blood pressure, skin resistance and electromyography signals. The model building and optimization module is used to construct a model for training by combining data feature extraction and hierarchical strategies with random forest and ReliefF algorithms. This model is then integrated into a cross-domain low-sample egocentricity recognition task based on multimodal input and unlabeled target data to optimize the target data classification task. Furthermore, it obtains vital sign detection results through dynamic mask generation and multi-inference ensemble prediction. This includes: constructing a cross-domain low-sample learning task, dividing the multimodal human vital sign dataset into labeled source datasets D. s and unlabeled target dataset D Tu Target dataset D Tu The model is further divided into a support set S and a query set Q for model training and testing. A data feature extraction and hierarchical strategy combining random forest and ReliefF algorithms is constructed to achieve hierarchical screening of target data features. The data feature extraction and hierarchical strategy is applied to multiple data training stages, and the classification performance is optimized by combining domain adaptation and multimodal distillation, spatiotemporal cube reconstruction and ensemble mask inference to predict the vital sign detection results. The strategy for data feature extraction and hierarchical analysis, which combines random forest and ReliefF algorithms, is designed to achieve hierarchical filtering of target data features. This includes: randomly sampling multiple subsets from the original sample data, with each subset used to train a decision tree, ultimately constructing a random forest; after constructing the random forest, using ReliefF to calculate the importance weights of the features, thus achieving hierarchical filtering of target data features and obtaining the feature weights; and based on the feature weights, dividing the feature data extracted from multimodal human physical characteristic data using random forest and ReliefF algorithms into high-weight, medium-weight, and low-weight sets, and uniformly sampling features within each set. The data feature extraction and hierarchical strategy is applied to multiple data training stages. Classification performance is optimized by combining domain adaptation, multimodal distillation, spatiotemporal cube reconstruction, and ensemble mask inference to predict vital sign detection results. This includes: defining a class-discriminative feature composed of the sum of the projected RGB features of the student encoder and the multimodal features of the teacher encoder; performing self-supervised pre-training using a masked autoencoder to optimize reconstruction loss and cross-entropy loss; in the multimodal distillation stage, random forest and ReliefF feature selection are introduced to adaptively select the most discriminative features, i.e., extracting class-discriminative features from the preprocessed multimodal data, optimizing the decision boundary for disease classification, thereby achieving efficient knowledge transfer and model optimization in the target domain. This includes: introducing multimodal distillation loss and using the teacher encoder and RGB student encoder for feature alignment; training a classifier on the support set S, introducing an encoder-decoder architecture in the model to extract deep-level features of the data, and optimizing the classification loss through spatiotemporal cube reconstruction; using dynamic mask generation and multiple inference ensembles to predict the query set Q, outputting the final prediction result, i.e., the vital sign detection result. The stimulation electrode adjustment module is used to dynamically adjust the intensity, frequency, and position of the stimulation electrode based on the predicted vital sign detection results and using a feedback mechanism of an adaptive proportional-integral-derivative control algorithm. The data visualization module is used to present multimodal human vital signs data, vital sign detection results, and real-time status of electrode stimulation in a data visualization manner to assist doctors in operation and decision-making.

2. The intelligent vital sign detection and auxiliary decision-making system based on data fusion according to claim 1, characterized in that, in, The process of training a classifier on the support set S, introducing an encoder-decoder architecture in the model to extract deep features from the data, and optimizing the classification loss through spatiotemporal cube reconstruction includes: Based on support set Given a limited number of labeled samples, an encoder-decoder architecture is introduced into the model; The encoder-decoder architecture is used to extract deep features from the data, and spatiotemporal cube reconstruction is performed to optimize the classification loss; wherein, the specific method of spatiotemporal cube reconstruction is as follows: By randomly occluding some input features using a spatiotemporal mask, the model is forced to make inferences with missing information. For the masked spatiotemporal cube data, a decoder reconstruction is used to augment the data in order to recover the data that was masked and generate new sample variants, resulting in reconstructed data, i.e., augmented samples, which are used to expand the diversity of training data. The reconstructed data is further processed by an encoder to extract features.

3. The intelligent vital sign detection and auxiliary decision-making system based on data fusion according to claim 1 or 2, characterized in that, in, The method employs dynamic mask generation and multiple inference ensembles to predict the query set Q, outputting the final prediction result, i.e., the vital sign detection result, including: During inference, dynamic mask generation is used to generate multiple mask data according to the set mask ratio. These mask data represent different degrees of occlusion of the query set samples. Through multiple inferences, multiple prediction results corresponding to all mask variants are obtained; The final prediction result is obtained by averaging the multiple prediction results corresponding to all mask variants.

4. The intelligent vital sign detection and auxiliary decision-making system based on data fusion according to claim 3, characterized in that, in, The feedback control formula of the adaptive proportional-integral-derivative control algorithm is as follows: ； in, For a moment Electrode stimulation intensity at the location; The error is the difference between the current vital sign value and the target vital sign value. ; For the predicted vital signs test results, The target vital sign value; : These are the proportional-integral-derivative control parameters, representing the proportional, integral, and derivative gains, respectively, which are based on the predicted vital sign detection results. Adaptive gain adjustment is performed based on the changes.

5. The intelligent vital sign detection and auxiliary decision-making system based on data fusion according to claim 4, characterized in that, in, Proportional, integral, and differential gain Based on the predicted vital signs test results The adaptive gain adjustment process for the changes is as follows: The control gain is adjusted by weighting the confidence level of vital signs data to adapt to the needs of different physiological states. The gain adjustment formula is as follows: ； ； ； in, The updated proportional-integral-derivative control parameters represent the updated proportional, integral, and derivative gains, respectively. The parameters are the proportional-integral-derivative control parameters before the update, representing the proportional, integral, and derivative gains before the update, respectively. and These are the confidence levels calculated for proportional gain, integral gain, and differential gain, respectively. , and These are for proportional gain Integral gain and differential gain Calculate the weighted sum of confidence levels; , and Based on the first Individual trait test results Calculated proportional gain Integral gain and differential gain The confidence level.

6. The intelligent vital sign detection and auxiliary decision-making system based on data fusion according to claim 1, characterized in that, in, The data visualization module is used to present multimodal human vital sign data, vital sign detection results, and real-time electrode stimulation status in a data visualization manner in real time, assisting doctors in operation and decision-making, including: The t-SNE method in the feature embedding space is used to display the distribution of multimodal data, and the classification decision boundary is displayed by the kernel function mapping of support vector machine. The visualization results are dynamically updated to assist medical decision-making.

7. The intelligent vital sign detection and auxiliary decision-making system based on data fusion according to claim 6, characterized in that, in, The data visualization process includes: The t-SNE method is used to project high-dimensional features of class discriminative features extracted from preprocessed multimodal data into a two-dimensional space through t-SNE. This is used to show the distribution relationship of different modal data in the embedding space and help doctors understand the basis of model decision-making. After the feature space of the class discriminative feature F(x) is mapped by the kernel function of the support vector machine, the decision boundary of disease classification can be displayed intuitively, which helps doctors evaluate the classification logic of the model. The kernel function mapping of the support vector machine is used to display the nonlinear decision boundary of high-dimensional data, projecting the decision boundary onto a two-dimensional plane and using different colors to mark the vital signs data of different categories, which helps doctors quickly identify high-risk cases. When a patient's vital signs change, the model immediately updates the visualization results, enabling dynamic and real-time updates and providing the most timely clinical decision support.

Images