Multi-parameter fusion intelligent internal medicine physical sign monitoring system and method

Abstract

The invention provides a multi-parameter fusion intelligent internal medicine physical sign monitoring system and method, and relates to the technical field of medical detection, and the method comprises the steps: carrying out the synchronous collection and time-space fusion of multi-source physical sign data, and obtaining a fusion data set; extracting physical sign waveform features from the fusion data set, and performing modal decomposition on the physical sign waveform features to obtain sub-segment modal signals; carrying out fusion modeling through the thoracic displacement track of the monitored object and all the sub-segment modal signals to obtain sign reconstruction parameters, and carrying out echo suppression on the sign reconstruction parameters to obtain a sign joint monitoring field; determining a physical sign matching feature vector according to the physical sign joint monitoring field and the real-time physical sign phase, and performing physical sign mode recognition on the physical sign matching feature vector through a classification model to obtain a trigger factor when the physical sign is abnormal; according to the method and the device, deep fusion and abnormal intelligent identification of multi-source sign data can be realized, so that the clinical adaptability of internal medicine sign monitoring is improved.

Description

Technical Field

[0001] The present application relates to the field of medical detection technology, and more specifically, to an intelligent medical vital sign monitoring system and method with multi-parameter fusion. Background Art

[0002] With the continuous development of smart medical and remote monitoring technologies, non-contact monitoring technology of physiological signs has been widely used in scenarios such as clinical auxiliary diagnosis, chronic disease management, and home health care. Especially in the field of internal medicine, vital sign parameters such as heart rate, respiratory rate, and heart rate variability (HRV) have important clinical value in judging the patient's circulation and respiratory status and identifying abnormal patterns (such as tidal respiration and intermittent breathing).

[0003] Traditional vital sign monitoring methods rely primarily on contact sensors, such as electrocardiogram (ECG) electrodes and chest-strap breathing belts. These methods not only compromise patient comfort but also lack adaptability in specific scenarios such as emergency care, infectious disease monitoring, or sleep monitoring. The construction of intelligent, multi-parameter fusion internal medicine vital sign monitoring methods by integrating non-contact sensors such as radar, physiological images, and surface vibration has become a research hotspot. However, while existing multimodal fusion vital sign monitoring methods have achieved non-contact acquisition and collaborative analysis of multi-channel signals to a certain extent, they still lack a unified spatiotemporal reference between multimodal data, remaining at the feature superposition level and lacking a modeling mechanism based on physical consistency. Sign pattern recognition often uses static thresholds, resulting in an imperfect abnormal response mechanism. Therefore, how to achieve deep fusion of multi-source vital sign data and intelligent anomaly identification to improve the clinical adaptability of internal medicine vital sign monitoring is a challenge facing the industry. Summary of the Invention

[0004] The present application provides an intelligent internal medicine vital sign monitoring system and method with multi-parameter fusion, which can realize the deep fusion of multi-source vital sign data and intelligent identification of abnormalities, so as to improve the clinical adaptability of internal medicine vital sign monitoring.

[0005] In a first aspect, the present application provides a multi-parameter fusion intelligent internal medicine vital sign monitoring system and method, the monitoring method comprising the following steps: Synchronously collect and spatially fuse the multi-source vital sign data of the monitored object to obtain a fused data set; Extracting vital sign waveform features from the fused data set, and then performing modal decomposition on the vital sign waveform features to obtain sub-segment modal signals of different vital signs; Collecting the chest displacement trajectory of the monitored object, performing fusion modeling on the chest displacement trajectory and all sub-segment modal signals to obtain vital sign reconstruction parameters, performing echo suppression on the vital sign reconstruction parameters, and thereby obtaining a vital sign joint monitoring field; Determining a physical sign matching feature vector based on the physical sign combined monitoring field and the real-time physical sign phase of the monitored object, and then performing physical sign pattern recognition on the physical sign matching feature vector using a preset classification model to obtain a trigger factor when the physical sign is abnormal; An abnormal response is performed based on the triggering factor to obtain abnormal physical sign results of the monitored object.

[0006] In this embodiment, the multi-source vital sign data of the monitored object are synchronously collected and spatiotemporally fused to obtain a fused data set, specifically including: Use multiple intelligent terminals to synchronously collect multi-source vital sign data of the monitored object to obtain a multi-source vital sign data set; The multi-source vital sign datasets are temporally and spatially fused using a multimodal fusion algorithm to obtain a fused dataset.

[0007] In this embodiment, extracting the vital sign waveform features from the fused data set specifically includes: performing bandpass filtering and envelope modulation operations on the radar signals in the fused data set to obtain a vital sign waveform sequence; The signal-to-noise ratio enhancement and normalization processing are performed on the vital sign waveform sequence to obtain vital sign waveform features.

[0008] In this embodiment, modal decomposition is performed on the waveform characteristics of the vital signs to obtain sub-segment modal signals of different vital signs, specifically including: Extracting signal components of different frequency bands from the vital sign waveform features based on a modal decomposition algorithm; The signal components of each frequency band are subjected to frequency band screening to obtain sub-segment modal signals of different signs.

[0009] In this embodiment, the chest displacement trajectory of the monitored object is collected by a visual sensor, and the visual sensor is used to track the surface motion characteristics of the chest area of ​​the monitored object.

[0010] In this embodiment, the chest displacement trajectory and all sub-segment modal signals are fused and modeled to obtain the physical sign reconstruction parameters, which specifically include: Performing dynamic time-frequency analysis on the thoracic displacement trajectory to obtain frequency characteristics; Performing cross-correlation analysis on the frequency characteristics and the modal signals of each sub-segment to obtain correlation coefficients of the modal signals of each sub-segment; Construct a joint regression model based on all correlation coefficients; The change in the chest displacement trajectory is used as the independent variable of the joint regression model, and then the physical sign reconstruction parameters are determined through the joint regression model.

[0011] In this embodiment, performing echo suppression on the vital sign reconstruction parameters to obtain the vital sign joint monitoring field specifically includes: performing interference identification on the waveform components in the vital sign reconstruction parameters to obtain an echo interference section; performing suppressive filtering on the echo interference section based on an energy threshold to obtain an echo suppression signal; performing spatial domain fusion on the modal components of the echo suppression signal and the chest displacement trajectory according to the vital sign reconstruction parameters to obtain vital sign joint monitoring parameters; A vital sign joint monitoring field is generated by using the spatial coupling function and the vital sign joint monitoring parameters.

[0012] In this embodiment, determining the physical sign matching feature vector according to the physical sign combined monitoring field and the real-time physical sign phase of the monitored object specifically includes: In the joint vital sign monitoring field, spatial amplitude distribution and modal response characteristics are extracted based on the real-time vital sign phase of the monitored object; determining a phase matching sampling point set based on the spatial amplitude distribution and the modal response characteristics; Perform multimodal feature fusion on the phase matching sampling point set to obtain a local fusion feature set; The local fusion feature sets are spliced ​​and normalized to obtain the physical sign matching feature vector.

[0013] In this embodiment, the preset classification model refers to a support vector machine classification model pre-trained based on a multi-parameter vital sign sample data set.

[0014] In a second aspect, the present application provides a multi-parameter fusion intelligent internal medicine vital sign monitoring system for performing a multi-parameter fusion intelligent internal medicine vital sign monitoring method, the monitoring system comprising: The acquisition and fusion module is used to synchronously acquire and spatially fuse the multi-source vital sign data of the monitored object to obtain a fused data set; a feature decomposition module, configured to extract vital sign waveform features from the fused data set, and then perform modal decomposition on the vital sign waveform features to obtain sub-segment modal signals of different vital signs; A monitoring fusion module is used to collect the chest displacement trajectory of the monitored object, perform fusion modeling on the chest displacement trajectory and all sub-segment modal signals to obtain vital sign reconstruction parameters, perform echo suppression on the vital sign reconstruction parameters, and thus obtain a vital sign joint monitoring field; An abnormality trigger module is used to determine a physical sign matching feature vector based on the physical sign combined monitoring field and the real-time physical sign phase of the monitored object, and then perform physical sign pattern recognition on the physical sign matching feature vector through a preset classification model to obtain a trigger factor when the physical sign is abnormal; The abnormal response module is used to perform an abnormal response based on the trigger factor to obtain abnormal physical sign results of the monitored object.

[0015] The technical solutions provided by the embodiments disclosed in this application have the following beneficial effects: A fused data set is obtained by synchronously collecting and spatiotemporally fusing multi-source vital sign data of the monitored object; vital sign waveform features are extracted from the fused data set, and then modal decomposition is performed on the vital sign waveform features to obtain sub-segment modal signals of different vital signs; the chest displacement trajectory of the monitored object is collected, and a fusion model is performed by fusing the chest displacement trajectory and all sub-segment modal signals to obtain vital sign reconstruction parameters, and echo suppression is performed on the vital sign reconstruction parameters to obtain a vital sign joint monitoring field; a vital sign matching feature vector is determined based on the vital sign joint monitoring field and the real-time vital sign phase of the monitored object, and then a vital sign pattern recognition is performed on the vital sign matching feature vector through a preset classification model to obtain a trigger factor when the vital sign is abnormal; an abnormal response is performed based on the trigger factor to obtain an abnormal vital sign result of the monitored object.

[0016] It can be seen that in this application, deep fusion of multi-source vital sign data and intelligent recognition of abnormalities can be achieved. First, the synchronous collection and fusion processing of multi-source vital sign data avoids the problems of inconsistency and time dislocation of multi-modal data in traditional vital sign collection systems, and improves the integrity and timeliness of vital sign data; by constructing a fusion data set, a high-quality data foundation is laid for subsequent unified modeling and feature extraction, and by extracting waveform features and modal decomposition of fused vital sign data, the coupling interference between different vital sign sources can be effectively removed, and the sub-segment signal representing a single physiological indicator can be extracted, thereby improving the specificity of subsequent modeling and the accuracy of recognition; secondly, the chest displacement trajectory is introduced as a dynamic reference dimension, and fused with the sub-modal vital sign signal for modeling, which is beneficial In order to realize the dynamic reconstruction and interference suppression of individualized vital signs, the echo suppression mechanism can reduce the interference of abnormal signals such as motion artifacts, and construct a joint vital sign monitoring field with a higher signal-to-noise ratio; then, the phase perception mechanism is used to realize the time-series tracking of vital sign states, extract the vital sign matching feature vectors, and use the preset intelligent classification model to perform vital sign pattern recognition, so as to realize the rapid detection of potential abnormal signals and the discrimination of trigger factors, dynamically perceive the vital sign fluctuation trend, and improve the sensitivity of early detection of abnormal events; finally, the abnormal response mechanism is executed through the trigger factor, so as to realize the clinical interpretability of the monitoring results. By mapping the vital sign recognition results into abnormal vital sign outputs, it is beneficial to improve the early warning ability of the monitoring system in sudden situations and complex symptoms.

[0017] In summary, the technical solution adopted in this application can realize the deep fusion of multi-source vital sign data and intelligent identification of abnormalities, so as to improve the clinical adaptability of internal medicine vital sign monitoring. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following is a brief introduction to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only the embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0019] Figure 1 This is a flow chart of a multi-parameter fusion intelligent medical vital sign monitoring method provided by the present application; Figure 2 is an exemplary flow chart for determining vital sign waveform characteristics provided by the present application; Figure 3 is an exemplary flow chart for determining physical sign reconstruction parameters provided by the present application; Figure 4 It is a module structure diagram of the monitoring system provided by this application. DETAILED DESCRIPTION

[0020] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0021] The embodiment of the present application provides an intelligent internal medicine vital sign monitoring system and method with multi-parameter fusion. The core of the system and method is to obtain a fused data set by synchronously collecting and spatiotemporally fusing multi-source vital sign data of the monitored object; extracting vital sign waveform features from the fused data set, and then performing modal decomposition on the vital sign waveform features to obtain sub-segment modal signals of different vital signs; collecting the chest displacement trajectory of the monitored object, and fusing the chest displacement trajectory and all sub-segment modal signals to obtain vital sign reconstruction parameters, and performing echo suppression on the vital sign reconstruction parameters to obtain a vital sign joint monitoring field; determining a vital sign matching feature vector based on the vital sign joint monitoring field and the real-time vital sign phase of the monitored object, and then performing sign pattern recognition on the vital sign matching feature vector through a preset classification model to obtain a trigger factor when the vital sign is abnormal; performing an abnormal response based on the trigger factor to obtain an abnormal vital sign result of the monitored object.

[0022] Example 1: In order to better understand the above technical solution, the above technical solution will be described in detail below with reference to the accompanying drawings and specific implementation methods. Figure 1 As shown in FIG, this figure is an exemplary flow chart of an intelligent internal medicine vital sign monitoring method with multi-parameter fusion according to this embodiment of the present application, and the monitoring method includes the following steps: In step S1, multi-source vital sign data of the monitored object are synchronously collected and spatiotemporally fused to obtain a fused data set.

[0023] In this embodiment, the following methods are used to synchronously collect and spatiotemporally fuse the multi-source vital sign data of the monitored object to obtain a fused data set, namely: Use multiple intelligent terminals to synchronously collect multi-source vital sign data of the monitored object to obtain a multi-source vital sign data set; The multi-source vital sign datasets are temporally and spatially fused using a multimodal fusion algorithm to obtain a fused dataset.

[0024] In specific implementation, the multi-source vital sign data of the monitored object include: radar signals, visual image signals and surface microseismic signals, among which: radar signals can be collected by radar sensors, and the sampling frequency can be set between 20-50Hz, which is used to capture tiny displacement changes in the chest area of ​​the monitored object, and the radar signal includes the waveform characteristics of breathing and heartbeat; visual image signals can be collected by structured light depth cameras, and the sampling frequency can be set between 25-30fps, which can collect the surface motion trajectory of the chest area of ​​the monitored object; surface microseismic signals can be collected by contact with piezoelectric films, and the sampling frequency can be between 100-200Hz. The surface microseismic signals are used to reflect events such as body movement, coughing, snoring, etc. of the monitored object. Vibration information, the radar signals, visual image signals and surface microseismic signals collected by the above-mentioned radar sensor, structured light depth camera and piezoelectric film are synchronized with a unified timestamp through the NTP protocol, and then the set of radar signals, visual image signals and surface microseismic signals with unified timestamps is used as a multi-source vital sign data set; then, each modal signal is preprocessed by standardization, detrending, de-drifting and signal alignment through existing preprocessing technology, and the preprocessed multi-source vital sign data set is fused using a multimodal spatiotemporal fusion algorithm. Preferably, the multimodal spatiotemporal fusion algorithm can select the multi-channel convolution-attention fusion model in the existing technology to extract the time dependency and spatial synergy of various types of data.

[0025] It should be noted that the multi-source vital sign data in this application refers to physiological information channels from different sources, including non-contact radar fluctuation data, visual image motion data and vibration sensor output data. The collection of multi-source vital sign data can construct a comprehensive description of the vital sign status in multiple signal dimensions; the fused data set is the data basis for performing key steps such as waveform feature extraction, modal decomposition, and joint modeling; in addition, in actual deployment, the sensor layout should ensure that the radar, vision, and vibration equipment collection areas are highly overlapping, and clock synchronization testing should be performed during system initialization to avoid fusion deviation caused by time misalignment of multi-source data.

[0026] In step S2, vital sign waveform features are extracted from the fused data set, and then modal decomposition is performed on the vital sign waveform features to obtain sub-segment modal signals of different vital signs.

[0027] Preferably, in this embodiment, reference Figure 2 As shown in FIG. 1 , this figure is an exemplary flow chart for determining vital sign waveform features according to the present application. In this embodiment, extracting vital sign waveform features from the fused data set can be implemented by the following steps: In step S21, bandpass filtering and envelope modulation operations are performed on the radar signals in the fused data set to obtain a vital sign waveform sequence; In step S22, the signal-to-noise ratio enhancement and normalization processing are performed on the vital sign waveform sequence to obtain vital sign waveform features.

[0028] In the specific implementation, first, the radar signal in the fusion data set includes the waveform characteristics of breathing and heartbeat, and the radar signal is filtered by the bandpass filtering algorithm to obtain the amplitude change curve of the radar signal. Among them, the filtering processing frequency can be set between 0.1-0.4Hz, which is recommended to retain the waveform characteristic signals of breathing and heartbeat. Then, the envelope line in the amplitude change curve is filtered by the sliding window extreme value connection method to obtain the vital sign waveform sequence; then, the vital sign waveform sequence is denoised by wavelet, and the denoised vital sign waveform sequence is normalized by the standard deviation normalization method, and the processed vital sign waveform sequence is used as the vital sign waveform feature.

[0029] It should be noted that the vital sign waveform sequence refers to a periodic waveform curve representing the vital signs of the monitored object obtained after filtering and envelope extraction of the radar signal. The periodic waveform curve represents the time domain structure of the micro-motion of the chest surface. In addition, in this embodiment, low-frequency drift and high-frequency noise can be filtered out through bandpass filtering, and the complex oscillation waveform can be simplified into a waveform contour line that is easy to model through envelope adjustment operation. Clutter, environmental vibration and instrument noise can be removed through signal-to-noise ratio enhancement, and normalization processing can effectively solve the problems of excessive signal strength difference and inconsistent sample amplitude distribution.

[0030] In this embodiment, modal decomposition is performed on the waveform characteristics of the vital signs to obtain sub-segment modal signals of different vital signs in the following manner, namely: Extracting signal components of different frequency bands from the vital sign waveform features based on a modal decomposition algorithm; The signal components of each frequency band are subjected to frequency band screening to obtain sub-segment modal signals of different signs.

[0031] In the specific implementation, first, the variational modal decomposition algorithm is used to extract multiple modal components from the vital sign waveform characteristics to obtain signal components in different frequency bands, where the modal components represent the characteristic changes of the vital sign signal in different frequency bands. The above process can be completed using the vmdpy library in the open source Python tool library; then, the main frequency value of the signal component of each frequency band can be calculated by fast Fourier transform, and the frequency band can be filtered by the preset frequency band range to obtain sub-segment modal signals of different vital signs, which can effectively eliminate the interference of non-target factors such as body movement and vibration. Among them, the preset frequency band range can refer to the following range, for example: the screening condition for the respiratory modal component is: the main frequency value is between 0.1-0.5Hz, and the screening condition for the heartbeat modal component is: the main frequency value is between 0.8-2.0Hz. Other modal components can be set according to clinical experience and are not limited here.

[0032] It should be noted that the sub-segment modal signal in the present application is an independent waveform segment related to a single vital sign variable (such as breathing or heartbeat). The sub-segment modal signal can be jointly analyzed with the chest trajectory in the subsequent fusion modeling stage to provide a data basis for personalized dynamic modeling; the modal decomposition algorithm is a mathematical method that decomposes a complex nonlinear signal into several modal components with physical significance. In this embodiment, it can be used to separate different frequency components such as breathing, heartbeat, and interference in the vital sign waveform characteristics to facilitate subsequent identification; in addition, the variational modal decomposition algorithm used in this application has a stronger frequency separation capability.

[0033] In step S3, the chest displacement trajectory of the monitored object is collected, and a fusion model is performed by the chest displacement trajectory and all sub-segment modal signals to obtain a vital sign reconstruction parameter, and echo suppression is performed on the vital sign reconstruction parameter to obtain a vital sign joint monitoring field.

[0034] In specific implementation, the chest displacement trajectory of the monitored object is collected through a visual sensor, and the visual sensor is used to track the surface motion characteristics of the chest area of ​​the monitored object. First, the visual image signal of the monitored object is obtained, and then the surface motion of the visual image signal is estimated based on the optical flow method, that is: the optical flow field of the chest area is calculated by the pixel changes between consecutive frames in the visual image signal, and then the overall motion vector is extracted to obtain the chest displacement trajectory. It should be noted that the chest displacement trajectory refers to the continuous motion curve of the chest surface of the monitored object on the time axis, which can represent a two-dimensional sequence of motion vectors of key points of the chest changing with time. The surface motion characteristics include the breathing amplitude, frequency, rhythm and body movement characteristics of the monitored object; in addition, in actual implementation, the background segmentation algorithm can be used to limit the region of interest (i.e., ROI limitation) of the visual image signal to ensure stable tracking effect, which is conducive to improving the accuracy of chest displacement extraction.

[0035] Preferably, in this embodiment, reference Figure 3 As shown in FIG. 1 , this figure is an exemplary flow chart for determining vital sign reconstruction parameters according to the present application. In this embodiment, the vital sign reconstruction parameters are obtained by fusion modeling the chest displacement trajectory and all sub-segment modal signals, which can be specifically implemented by the following steps: In step S31, dynamic time-frequency analysis is performed on the thoracic displacement trajectory to obtain frequency characteristics; In step S32, cross-correlation analysis is performed on the frequency characteristics and the modal signals of each sub-segment to obtain the correlation coefficient of the modal signals of each sub-segment; In step S33, a joint regression model is constructed based on all correlation coefficients; In step S34, the change in the chest displacement trajectory is used as an independent variable of the joint regression model, and then the physical sign reconstruction parameters are determined through the joint regression model.

[0036] In the specific implementation, first, the chest displacement trajectory is subjected to continuous wavelet transform, and the chest displacement trajectory is converted into a time-frequency two-dimensional matrix. The main frequency components and amplitude information are extracted from the time-frequency two-dimensional matrix through the principal component analysis algorithm, and then the vector composed of the main frequency components and amplitude information is used as the frequency feature, that is: frequency feature = (main frequency component, amplitude information), wherein the main frequency components include the main frequency of breathing and the motion interference frequency, which can be extracted by the principal component analysis algorithm; secondly, for each sub-segment modal signal, the mean of the sub-segment modal signal is extracted, and then the mean and the frequency feature are calculated by the Pearson correlation coefficient calculation formula to obtain the correlation coefficient of the modal signal of this segment. The above method can be used to obtain the modal signal of each sub-segment. The correlation coefficient can be used to quantify the influence of the sub-segment modal signal on the change of chest displacement; then, a regression model is initialized, and the ratio of the correlation coefficients of all sub-segment modal signals is used as the feature weight factor to perform weighted linear regression processing on the regression model, and then the regression model after weighted linear regression processing is used as a joint regression model; finally, the change in the chest displacement trajectory is used as the independent variable of the joint regression model. The joint regression model maps the chest displacement trajectory to the response intensity of each sign modality to obtain the sign reconstruction parameters for characterizing physiological signals such as breathing and heartbeat, among which the change in the chest displacement trajectory can be obtained by calculating the Euclidean distance of the chest key points between consecutive frames, and the change reflects the movement amplitude of the chest area per unit time.

[0037] It should be noted that the vital sign reconstruction parameters are the intensity data of the vital sign modal signal during the vital sign monitoring period, which can be used to judge the depth of breathing, the strength of the heartbeat, etc., and are the core data for constructing a high signal-to-noise ratio monitoring field; in this embodiment, the time domain signal is converted into a joint representation of time and frequency through time-frequency transformation, which can be used to extract the frequency change characteristics of the signal in different time periods, and can analyze the periodic respiratory rhythm, sudden movements, etc. in the chest movement; the joint regression model is a method of comprehensively modeling the contributions of multiple sub-modal signals, which can be used to establish a mathematical mapping relationship between chest movement and multi-frequency physiological signals.

[0038] In this embodiment, echo suppression is performed on the vital sign reconstruction parameters to obtain the vital sign joint monitoring field in the following manner, namely: performing interference identification on the waveform components in the vital sign reconstruction parameters to obtain an echo interference section; performing suppressive filtering on the echo interference section based on an energy threshold to obtain an echo suppression signal; performing spatial domain fusion on the modal components of the echo suppression signal and the chest displacement trajectory according to the vital sign reconstruction parameters to obtain vital sign joint monitoring parameters; A vital sign joint monitoring field is generated by using the spatial coupling function and the vital sign joint monitoring parameters.

[0039] In the specific implementation, first, the instantaneous energy of each signal segment in the vital sign reconstruction parameters is calculated with a 2s time window through the moving window energy analysis method. When the instantaneous energy is greater than 1.5 times the average energy, the echo segment corresponding to the instantaneous energy is used as the echo interference segment; secondly, the echo interference segment is denoised by the wavelet shrinkage method, and the smoothed signal obtained after processing is used as the echo suppression signal; then, the window sliding algorithm can be used to align the modal components of the echo suppression signal and the chest displacement trajectory to ensure that the two types of signals have a corresponding relationship under the same time reference, and then the modal components of the aligned echo suppression signal and the chest displacement trajectory are fused by the dot product attention method, that is, : Use the attention adjustment mechanism to dynamically adjust the degree of influence of each modal input on the fusion result according to the reconstruction parameters, and then determine the weight value according to the influence degree, and then use the weight value to weightedly output the modal components of the echo suppression signal and the chest displacement trajectory to obtain the vital sign joint monitoring parameters; finally, obtain the two-dimensional coordinates of the monitoring area, and use the chest center of the monitored object as the coordinate center, and use the spatial coupling function to interpolate and fuse the vital sign joint monitoring parameters in each spatial area to generate a spatial distribution map, and then use the spatial distribution map as the vital sign joint monitoring field, wherein the spatial coupling function can use the bidirectional Gaussian kernel function in the existing technology to identify the coordinate space corresponding to the vital sign joint monitoring parameters.

[0040] It should be noted that the joint monitoring of vital signs in this application refers to a multi-parameter dynamic distribution map constructed in the dimensions of time, space and modal fusion, which can be used to dynamically monitor the changing trends of vital sign signals in different time and space regions; in addition, echo interference in this embodiment refers to abnormal fluctuations caused by non-physiological factors such as body movement, environmental vibration or electromagnetic interference in non-contact monitoring; echo suppression signal refers to the effective signal part retained after the echo interference is processed by targeted filtering or denoising algorithm; spatial domain fusion refers to mapping time-synchronized multi-source signals into a unified spatial domain representation, combining chest deformation and multi-modal vital sign signals to construct a parameter set with spatial consistency and dynamic change characteristics; vital sign joint monitoring parameters are signal parameters obtained by fusion of the signal after echo suppression and the spatial coupling model.

[0041] In step S4, the sign matching feature vector is determined according to the physical sign combined monitoring field and the real-time physical sign phase of the monitored object, and then the physical sign matching feature vector is subjected to physical sign pattern recognition through a preset classification model to obtain the trigger factor when the physical sign is abnormal.

[0042] In this embodiment, the physical sign matching feature vector may be determined based on the physical sign combined monitoring field and the real-time physical sign phase of the monitored object in the following manner, namely: In the joint vital sign monitoring field, spatial amplitude distribution and modal response characteristics are extracted based on the real-time vital sign phase of the monitored object; determining a phase matching sampling point set based on the spatial amplitude distribution and the modal response characteristics; Perform multimodal feature fusion on the phase matching sampling point set to obtain a local fusion feature set; The local fusion feature sets are spliced ​​and normalized to obtain the physical sign matching feature vector.

[0043] In a specific implementation, in the joint vital sign monitoring field, first, based on the time axis in the joint vital sign monitoring field, the vital sign phase information of the monitored object at the current moment is obtained in real time. The phase information can be obtained by performing Hilbert transform calculation on the heartbeat or respiratory signal. Under the current phase information, the signal values ​​of the vital signs in different monitoring areas (such as the left, right, upper and lower parts of the chest) are used as spatial amplitude distributions, and the response changes of each modal signal at the corresponding spatial point are used as modal response features; secondly, the overlapping points of the spatial amplitude distribution and the modal response features in different monitoring areas are used as phase matching sampling points, and then all phase matching sampling points are obtained, and the set composed of all phase matching sampling points is used as phase matching. Sampling point set; then, a one-dimensional convolutional neural network is used to compress and extract the spatial amplitude distribution and modal response features of each point, and the compressed and extracted spatial amplitude distribution and modal response features are fused through the attention mechanism to obtain local fusion features, and then the set of all local fusion features is used as the local fusion feature set; finally, the local fusion features in the local fusion feature set are numbered, and the numbered local fusion features are vector spliced, that is: (local fusion feature 1, local fusion feature 2, local fusion feature 3, local fusion feature 4), and the spliced ​​vector is normalized by standard deviation, so that the standardized vector is used as the sign matching feature vector.

[0044] It should be noted that the vital sign matching feature vector in this application refers to a multidimensional vector formed by fusing the spatial amplitude distribution information extracted from the vital sign joint monitoring field with the modal response characteristics at a specific time phase point, which is used to comprehensively express the feature combination of the current vital sign state in the spatial, temporal and modal dimensions; the phase information in this embodiment represents the phase position of the current vital sign (such as breathing, heartbeat) in its periodic waveform, which is used to synchronously match the spatial features in the joint monitoring field; in addition, in this embodiment, the phase matching sampling point set can capture the current vital sign state and reflect the trend of the vital sign, thereby improving the model's ability to recognize non-periodic abnormalities.

[0045] In this embodiment, the physical sign pattern recognition is performed on the physical sign matching feature vector using a preset classification model to obtain the trigger factor when the physical sign is abnormal. Specifically, the following method can be used, namely: Inputting the physical sign matching feature vector into a preset classification model to perform pattern recognition reasoning to obtain a classification label and classification confidence; A trigger factor when a physical sign is abnormal is determined according to the classification label and the classification confidence.

[0046] It should be noted that, in the present application, the preset classification model refers to a support vector machine classification model pre-trained based on a multi-parameter vital sign sample data set, wherein the multi-parameter vital sign sample data set is collected from a large number of monitored subjects under different vital sign states (such as normal breathing, shallow breathing, apnea, arrhythmia, etc.), including a set of labeled vital sign matching feature vector samples, wherein each sample vector contains multiple feature dimensions, corresponding to fused vital sign features at different spatial positions, modal signal responses and phase points, and is associated with a corresponding vital sign state label. The present application adopts a pre-trained support vector machine classification model, which can match and classify the vital signs of the monitored subjects, make decision scores and determine abnormalities, and can effectively improve the recognition accuracy of vital sign patterns, wherein the pre-training is conventional machine learning training.

[0047] In specific implementation, first, the vital sign matching feature vector is input into a preset classification model to perform pattern recognition reasoning. During the reasoning process, the classification model will calculate the vital sign matching feature vector to obtain a classification label to represent the vital sign status category corresponding to the current feature, wherein the classification labels include: normal, abnormal-apnea, abnormal-tachypnea and abnormal-arrhythmia. In the classification model, the classification label decision function score is used as the classification confidence; then, the trigger factor in this application refers to the trigger signal generated when the vital sign status at the current moment is determined to be an abnormal state after pattern recognition is performed on the preset classification model, which is used to start the subsequent abnormal response mechanism. The trigger factors when the vital signs are abnormal include: abnormal type, abnormal confidence, trigger time and belonging modality (such as breathing or heartbeat). In the classification model, the classification label and the classification confidence perform abnormal judgment to obtain the trigger factor when the vital signs are abnormal.

[0048] It should be noted that the trigger factors in this application can identify whether the current physical signs are abnormal, including information such as abnormal type, confidence, and abnormal time, which is convenient for clinical tracing; the introduction of classification confidence makes the system more flexible in abnormality detection, which is conducive to the realization of risk grading and response strategy differentiation; in addition, in this embodiment, through the discrimination ability of the support vector machine, the physical sign status in the high-dimensional fusion feature space can be accurately divided.

[0049] In step S5, an abnormal response is performed based on the triggering factor to obtain abnormal physical sign results of the monitored object.

[0050] In this embodiment, the abnormal response based on the triggering factor to obtain the abnormal physical sign results of the monitored object can be specifically carried out in the following manner, namely: Determining the abnormal level of physical signs and the response mode according to the triggering factors through a preset clinical event response rule library; The abnormal physical sign level, the response pattern and the electronic medical record data of the detected object are correlated and analyzed to generate abnormal physical sign results of the monitored object.

[0051] In specific implementation, the clinical event response rule base can be constructed according to medical institution standards or industry guidelines, support customized updates, and ensure flexible response capabilities to different types of abnormal physical signs; first, the response rules matching the trigger factors are retrieved in the clinical event response rule base, and the abnormality level (such as low risk, medium risk, high risk) and response mode (such as enhanced monitoring, alarm, remote notification) are obtained through the response rules; then, the identified abnormal physical sign level and the existing electronic medical record information of the monitored object are fused, where the data fused information includes: medical history, age, BMI, basal heart rate, respiratory rate and physical sign trends, etc., and the electronic medical record information after data fusion is used as the abnormal physical sign result of the monitored object.

[0052] It should be noted that the abnormal vital signs results in this application refer to the structured diagnostic output generated by the system based on clinical rules, patient background and real-time data after the trigger factor is determined to be an abnormal state, which is used to guide medical staff in subsequent intervention or record clinical events; in this embodiment, by introducing electronic medical record data association analysis, the system's ability to understand the background of abnormal events can be improved, avoiding misjudgment of abnormal vital signs in high-risk chronic disease patients, and enhancing the clinical interpretability of abnormal results; in addition, by connecting the abnormal level with the alarm mechanism or telemedicine interface, a rapid response to critical conditions can be achieved.

[0053] In summary, the technical solution adopted in this application can realize the deep fusion of multi-source vital sign data and intelligent identification of abnormalities, so as to improve the clinical adaptability of internal medicine vital sign monitoring.

[0054] Example 2: This application provides a multi-parameter fusion intelligent internal medicine vital signs monitoring system, referring to Figure 4 As shown, this figure is a module structure diagram of the monitoring system shown in this embodiment of the present application, and the monitoring system includes: The acquisition and fusion module 100 is used to synchronously acquire and spatially fuse the multi-source vital sign data of the monitored object to obtain a fused data set; A feature decomposition module 200 is used to extract vital sign waveform features from the fused data set, and then perform modal decomposition on the vital sign waveform features to obtain sub-segment modal signals of different vital signs; The monitoring fusion module 300 is used to collect the chest displacement trajectory of the monitored object, perform fusion modeling on the chest displacement trajectory and all sub-segment modal signals to obtain vital sign reconstruction parameters, perform echo suppression on the vital sign reconstruction parameters, and thus obtain a vital sign joint monitoring field; The abnormal trigger module 400 is used to determine a physical sign matching feature vector based on the physical sign combined monitoring field and the real-time physical sign phase of the monitored object, and then perform physical sign pattern recognition on the physical sign matching feature vector using a preset classification model to obtain a trigger factor when the physical sign is abnormal; The abnormal response module 500 is used to perform an abnormal response based on the triggering factor to obtain abnormal physical sign results of the monitored object.

[0055] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems) and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0056] Those skilled in the art will appreciate that all or part of the steps in the various methods of the above embodiments can be completed by instructing related hardware through a program. The program can be stored in a computer-readable storage medium, including a read-only memory (ROM), a random access memory (RAM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), a one-time programmable read-only memory (OTPROM), an electronically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM), or other optical disc storage, magnetic disk storage, or magnetic tape storage, or any other computer-readable medium capable of carrying or storing data.

[0057] It should also be noted that the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, commodity, or apparatus that includes a series of elements includes not only those elements but also other elements not explicitly listed, or includes elements inherent to such process, method, commodity, or apparatus. In the absence of further limitations, an element defined by the phrase "comprises a..." does not exclude the presence of other identical elements in the process, method, commodity, or apparatus that includes the element.

Claims

1. A multi-parameter fusion intelligent internal medicine vital sign monitoring method, characterized in that: The monitoring method comprises the following steps: Synchronously collect and spatially fuse the multi-source vital sign data of the monitored object to obtain a fused data set; Extracting vital sign waveform features from the fused data set, and then performing modal decomposition on the vital sign waveform features to obtain sub-segment modal signals of different vital signs; Collecting the chest displacement trajectory of the monitored object, performing fusion modeling on the chest displacement trajectory and all sub-segment modal signals to obtain vital sign reconstruction parameters, performing echo suppression on the vital sign reconstruction parameters, and thereby obtaining a vital sign joint monitoring field; Determining a physical sign matching feature vector based on the physical sign combined monitoring field and the real-time physical sign phase of the monitored object, and then performing physical sign pattern recognition on the physical sign matching feature vector using a preset classification model to obtain a trigger factor when the physical sign is abnormal; An abnormal response is performed based on the triggering factor to obtain abnormal physical sign results of the monitored object.

2. The multi-parameter fusion intelligent medical vital sign monitoring method according to claim 1, characterized in that: The multi-source vital sign data of the monitored object are synchronously collected and spatially fused to obtain a fused data set including: Use multiple intelligent terminals to synchronously collect multi-source vital sign data of the monitored object to obtain a multi-source vital sign data set; The multi-source vital sign datasets are temporally and spatially fused using a multimodal fusion algorithm to obtain a fused dataset.

3. The multi-parameter fusion intelligent internal medicine vital sign monitoring method according to claim 1, characterized in that: Extracting the vital sign waveform features from the fused data set specifically includes: performing bandpass filtering and envelope modulation operations on the radar signals in the fused data set to obtain a vital sign waveform sequence; The signal-to-noise ratio enhancement and normalization processing are performed on the vital sign waveform sequence to obtain vital sign waveform features.

4. The multi-parameter fusion intelligent internal medicine vital sign monitoring method according to claim 1, characterized in that: Performing modal decomposition on the waveform characteristics of the physical signs to obtain sub-segment modal signals of different physical signs specifically includes: Extracting signal components of different frequency bands from the vital sign waveform features based on a modal decomposition algorithm; The signal components of each frequency band are subjected to frequency band screening to obtain sub-segment modal signals of different signs.

5. The multi-parameter fusion intelligent internal medicine vital sign monitoring method according to claim 1, characterized in that: The chest displacement trajectory of the monitored object is collected by a visual sensor, and the visual sensor is used to track the surface motion characteristics of the chest area of ​​the monitored object.

6. The multi-parameter fusion intelligent internal medicine vital sign monitoring method according to claim 1, characterized in that: By fusion modeling the chest displacement trajectory and all sub-segment modal signals, the physical sign reconstruction parameters obtained specifically include: Performing dynamic time-frequency analysis on the thoracic displacement trajectory to obtain frequency characteristics; Performing cross-correlation analysis on the frequency characteristics and the modal signals of each sub-segment to obtain correlation coefficients of the modal signals of each sub-segment; Construct a joint regression model based on all correlation coefficients; The change in the chest displacement trajectory is used as the independent variable of the joint regression model, and then the physical sign reconstruction parameters are determined through the joint regression model.

7. The multi-parameter fusion intelligent internal medicine vital sign monitoring method according to claim 1, characterized in that: Performing echo suppression on the vital sign reconstruction parameters to obtain a vital sign joint monitoring field specifically includes: performing interference identification on the waveform components in the vital sign reconstruction parameters to obtain an echo interference section; performing suppressive filtering on the echo interference section based on an energy threshold to obtain an echo suppression signal; performing spatial domain fusion on the modal components of the echo suppression signal and the chest displacement trajectory according to the vital sign reconstruction parameters to obtain vital sign joint monitoring parameters; A vital sign joint monitoring field is generated by using the spatial coupling function and the vital sign joint monitoring parameters.

8. The multi-parameter fusion intelligent medical vital sign monitoring method according to claim 1, characterized in that: Determining the physical sign matching feature vector according to the physical sign combined monitoring field and the real-time physical sign phase of the monitored object specifically includes: In the joint vital sign monitoring field, spatial amplitude distribution and modal response characteristics are extracted based on the real-time vital sign phase of the monitored object; determining a phase matching sampling point set based on the spatial amplitude distribution and the modal response characteristics; Perform multimodal feature fusion on the phase matching sampling point set to obtain a local fusion feature set; The local fusion feature sets are spliced ​​and normalized to obtain the physical sign matching feature vector.

9. The multi-parameter fusion intelligent medical vital sign monitoring method according to claim 1, characterized in that: The preset classification model refers to a support vector machine classification model pre-trained based on a multi-parameter vital sign sample data set.

10. A multi-parameter fusion intelligent internal medicine vital sign monitoring system, used to execute the multi-parameter fusion intelligent internal medicine vital sign monitoring method according to any one of claims 1 to 9, characterized in that: The monitoring system comprises: The acquisition and fusion module is used to synchronously acquire and spatially fuse the multi-source vital sign data of the monitored object to obtain a fused data set; a feature decomposition module, configured to extract vital sign waveform features from the fused data set, and then perform modal decomposition on the vital sign waveform features to obtain sub-segment modal signals of different vital signs; A monitoring fusion module is used to collect the chest displacement trajectory of the monitored object, perform fusion modeling on the chest displacement trajectory and all sub-segment modal signals to obtain vital sign reconstruction parameters, perform echo suppression on the vital sign reconstruction parameters, and thus obtain a vital sign joint monitoring field; An abnormality trigger module is used to determine a physical sign matching feature vector based on the physical sign combined monitoring field and the real-time physical sign phase of the monitored object, and then perform physical sign pattern recognition on the physical sign matching feature vector through a preset classification model to obtain a trigger factor when the physical sign is abnormal; The abnormal response module is used to perform an abnormal response based on the trigger factor to obtain abnormal physical sign results of the monitored object.

Images