A multi-parameter fusion intelligent internal medicine sign monitoring system and method

Abstract

The application provides a multi-parameter fusion intelligent internal medicine sign monitoring system and method, relates to the technical field of medical detection, obtains a fusion data set by synchronously collecting and spatio-temporal fusion of multi-source sign data; extracts sign waveform features from the fusion data set, decomposes the sign waveform features in modes to obtain sub-section modal signals; then fuses and models the thoracic displacement trajectory of a monitoring object and all the sub-section modal signals to obtain sign reconstruction parameters, performs echo suppression on the sign reconstruction parameters to obtain a sign joint monitoring field; determines sign matching feature vectors according to the sign joint monitoring field and real-time sign phases, performs sign mode recognition on the sign matching feature vectors through a classification model to obtain trigger factors in the case of sign abnormalities; performs abnormal response based on the trigger factors to obtain abnormal sign results, and the application can realize deep fusion and abnormal intelligent recognition of multi-source sign data to improve the clinical adaptability of internal medicine sign monitoring.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of medical detection, more specifically, the present application relates to a multi-parameter fusion intelligent internal medicine sign monitoring system and method. BACKGROUND

[0002] With the continuous development of intelligent medical treatment and remote monitoring technology, non-contact monitoring technology of physiological signs has been widely used in clinical auxiliary diagnosis, chronic disease management and home health care scenes, especially in the field of internal medicine, heart rate, respiratory rate, heart rate variability (HRV) and other vital sign parameters have important clinical value for judging the circulation and respiratory state of patients and identifying abnormal patterns (such as tidal breathing, intermittent breathing).

[0003] Traditional sign monitoring methods mainly rely on contact sensors such as electrocardiogram electrodes, chest strap respiratory belts, etc., which not only affect patient comfort, but also have insufficient adaptability in specific scenarios such as emergency, infectious disease monitoring or sleep monitoring. Therefore, it has become a research hotspot to build a multi-parameter fusion intelligent internal medicine sign monitoring method by fusing radar, physiological images and surface vibration non-contact sensors; however, although the existing multi-modal fusion sign monitoring realizes non-contact acquisition and multi-channel signal collaborative analysis to a certain extent, it still lacks a unified time and space reference between multi-modal data, stays at the feature superposition level, and lacks a modeling mechanism based on physical consistency, and the sign pattern recognition mainly uses static threshold, resulting in imperfect abnormal response mechanism. Therefore, how to realize deep fusion and intelligent identification of multi-source sign data to improve the clinical adaptability of internal medicine sign monitoring is a difficult problem faced by the industry. SUMMARY

[0004] The present application provides a multi-parameter fusion intelligent internal medicine sign monitoring system and method, which can realize deep fusion and intelligent identification of multi-source sign data to improve the clinical adaptability of internal medicine sign monitoring.

[0005] In a first aspect, the present application provides a multi-parameter fusion intelligent internal medicine sign monitoring system and method, the monitoring method comprising the following steps:

[0006] Synchronously collecting and spatio-temporally fusing multi-source sign data of a monitoring object to obtain a fusion data set;

[0007] Extracting sign waveform features from the fusion data set, and then modally decomposing the sign waveform features to obtain sub-section modal signals of different signs;

[0008] Collecting a thoracic displacement trajectory of the monitoring object, fusing and modeling the thoracic displacement trajectory and all sub-section modal signals to obtain sign reconstruction parameters, and performing echo suppression on the sign reconstruction parameters to obtain a sign joint monitoring field;

[0009] Determine a sign matching feature vector according to the sign joint monitoring field and the real-time sign phase of the monitoring object, and further perform sign pattern recognition on the sign matching feature vector through a preset classification model to obtain a trigger factor in a sign abnormality;

[0010] Perform an abnormal response based on the trigger factor to obtain an abnormal sign result of the monitoring object.

[0011] In the embodiment, multi-source sign data of the monitoring object are synchronously collected and spatio-temporal fused to obtain a fused data set, which specifically includes:

[0012] The multi-source sign data of the monitoring object are synchronously collected by a multi-intelligent terminal to obtain a multi-source sign data set;

[0013] The multi-source sign data set is spatio-temporal fused by a multi-modal fusion algorithm to obtain a fused data set.

[0014] In the embodiment, sign waveform features are extracted from the fused data set, which specifically includes:

[0015] Band-pass filtering and envelope adjustment operations are performed on the radar signal in the fused data set to obtain a sign waveform sequence;

[0016] The sign waveform sequence is subjected to signal-to-noise ratio enhancement and normalization processing to obtain sign waveform features.

[0017] In the embodiment, the sign waveform features are subjected to modal decomposition to obtain sub-section modal signals of different signs, which specifically includes:

[0018] Signal components of different frequency bands are extracted from the sign waveform features based on a modal decomposition algorithm;

[0019] The signal components of each frequency band are subjected to frequency band screening to obtain sub-section modal signals of different signs.

[0020] In the embodiment, a thoracic displacement trajectory of the monitoring object is collected by a visual sensor, which is used to track surface motion features of the chest region of the monitoring object.

[0021] In the embodiment, a sign reconstruction parameter is obtained by fusion modeling of the thoracic displacement trajectory and all sub-section modal signals, which specifically includes:

[0022] The thoracic displacement trajectory is subjected to dynamic time-frequency analysis to obtain a frequency feature;

[0023] The frequency feature and each sub-section modal signal are subjected to cross-correlation analysis to obtain a correlation coefficient of each sub-section modal signal;

[0024] Construct a joint regression model based on all the correlation coefficients;

[0025] Take the change amount of the thoracic displacement trajectory as an independent variable of the joint regression model, and then determine the sign reestablishment parameter through the joint regression model.

[0026] In this embodiment, the sign reestablishment parameter is subjected to echo suppression, and then a sign joint monitoring field is obtained, which specifically includes:

[0027] The wave forming part in the sign reestablishment parameter is subjected to interference identification, and an echo interference section is obtained.

[0028] Based on an energy threshold, the echo interference section is subjected to suppressive filtering processing, and an echo suppression signal is obtained.

[0029] According to the sign reestablishment parameter, the echo suppression signal and the modal component of the thoracic displacement trajectory are subjected to spatial domain fusion, and a sign joint monitoring parameter is obtained.

[0030] The sign joint monitoring field is generated through a spatial coupling function and the sign joint monitoring parameter.

[0031] In this embodiment, according to the sign joint monitoring field and the real-time sign phase of the monitoring object, a sign matching feature vector is determined, which specifically includes:

[0032] In the sign joint monitoring field, based on the real-time sign phase of the monitoring object, a spatial amplitude distribution and a modal response feature are extracted.

[0033] The phase matching sampling point set is determined through the spatial amplitude distribution and the modal response feature.

[0034] The multi-modal feature fusion is performed on the phase matching sampling point set, and a local fusion feature set is obtained.

[0035] The local fusion feature set is spliced and normalized to obtain the sign matching feature vector.

[0036] In this embodiment, the preset classification model refers to a support vector machine classification model pre-trained based on a multi-parameter sign sample data set.

[0037] In a second aspect, the present application provides a multi-parameter fusion intelligent internal medicine sign monitoring system for executing a multi-parameter fusion intelligent internal medicine sign monitoring method, and the monitoring system includes:

[0038] The acquisition fusion module is configured to synchronously acquire and spatiotemporally fuse the multi-source sign data of the monitoring object to obtain a fusion data set.

[0039] The feature decomposition module is configured to extract a sign waveform feature from the fusion data set, and then perform modal decomposition on the sign waveform feature to obtain a sub-section modal signal of different signs;

[0040] The monitoring fusion module is configured to collect a thoracic displacement trajectory of the monitoring object, perform fusion modeling on the thoracic displacement trajectory and all sub-section modal signals, obtain a sign reconstruction parameter, perform echo suppression on the sign reconstruction parameter, and then obtain a sign joint monitoring field;

[0041] The abnormal trigger module is configured to determine a sign matching feature vector according to the sign joint monitoring field and a real-time sign phase of the monitoring object, perform sign pattern recognition on the sign matching feature vector through a preset classification model, and obtain a trigger factor in a sign abnormal state.

[0042] The abnormal response module is configured to perform abnormal response based on the trigger factor, and obtain an abnormal sign result of the monitoring object.

[0043] The technical scheme provided by the embodiments disclosed in the application has the following beneficial effects:

[0044] The multi-source sign data of the monitoring object is synchronously collected and spatio-temporally fused to obtain a fusion data set. The sign waveform feature is extracted from the fusion data set, and then the sign waveform feature is subjected to modal decomposition to obtain a sub-section modal signal of different signs. The thoracic displacement trajectory of the monitoring object is collected, and fusion modeling is performed on the thoracic displacement trajectory and all sub-section modal signals to obtain a sign reconstruction parameter. The sign reconstruction parameter is subjected to echo suppression, and then a sign joint monitoring field is obtained. The sign matching feature vector is determined according to the sign joint monitoring field and a real-time sign phase of the monitoring object. The sign pattern recognition is performed on the sign matching feature vector through a preset classification model, and the trigger factor in the sign abnormal state is obtained. The abnormal response is performed based on the trigger factor, and the abnormal sign result of the monitoring object is obtained.

[0045] It can be seen that in the present application, deep fusion and abnormal intelligent recognition of multi-source sign data can be realized. First, the synchronous acquisition and fusion processing of multi-source sign data avoids the problem of inconsistent multi-modal data and time dislocation in the traditional sign acquisition system, improves the integrity and timeliness of the sign data; by constructing a fusion data set, a high-quality data foundation is laid for subsequent unified modeling and feature extraction, and through waveform feature extraction and modal decomposition of the fusion sign data, the coupling interference between different sign sources can be effectively stripped, the sub-section signal representing a single physiological indicator is extracted, and the specificity of subsequent modeling and the accuracy of recognition are improved; secondly, the thoracic displacement trajectory is introduced as a dynamic reference dimension, which is fused with the sub-modal sign signal for modeling, which is beneficial to realize individualized sign dynamic reconstruction and interference suppression, and through the echo suppression mechanism, the abnormal signal interference such as motion artifact can be reduced, and a sign joint monitoring field with higher signal-to-noise ratio is constructed; then, the phase perception mechanism is used to realize the time sequence tracking of the sign state, and the sign matching feature vector is extracted, and the preset intelligent classification model is used for sign pattern recognition, so that the rapid detection and trigger factor discrimination of potential abnormal signals can be realized, the sign fluctuation trend can be dynamically perceived, and the sensitivity of early detection of abnormal events can be improved; finally, the abnormal response mechanism is executed through the trigger factor, so that the clinical interpretability of the monitoring result is realized, and the sign recognition result is mapped as an abnormal sign output, which is beneficial to improve the early warning ability of the monitoring system under sudden conditions and complex symptoms.

[0046] In summary, the technical scheme adopted by the present application can realize deep fusion and abnormal intelligent recognition of multi-source sign data, so as to improve the clinical adaptability of internal medicine sign monitoring. BRIEF DESCRIPTION OF DRAWINGS

[0047] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments or prior art description will be briefly introduced. Obviously, the drawings in the following description are only the embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0048] Figure 1 is a flowchart of a multi-parameter fusion intelligent internal medicine sign monitoring method provided by the present application;

[0049] Figure 2 is an exemplary flowchart for determining sign waveform features according to the present application;

[0050] Figure 3 is an exemplary flowchart for determining sign reconstruction parameters according to the present application;

[0051] Figure 4 is a module structure diagram of a monitoring system according to the present application. DETAILED DESCRIPTION

[0052] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art without creative effort belong to the scope of protection of the present application.

[0053] The embodiments of the present application provide a multi-parameter fusion intelligent internal medicine sign monitoring system and method. The core is to obtain a fusion data set by synchronously collecting and spatio-temporal fusion of multi-source sign data of a monitoring object; extract sign waveform features from the fusion data set, and then perform modal decomposition on the sign waveform features to obtain sub-section modal signals of different signs; collect a thoracic displacement trajectory of the monitoring object, perform fusion modeling on the thoracic displacement trajectory and all sub-section modal signals to obtain sign reconstruction parameters, perform echo suppression on the sign reconstruction parameters, and then obtain a sign joint monitoring field; determine a sign matching feature vector according to the sign joint monitoring field and a real-time sign phase of the monitoring object, and then perform sign pattern recognition on the sign matching feature vector through a preset classification model to obtain a trigger factor in a sign abnormality; perform an abnormal response based on the trigger factor to obtain an abnormal sign result of the monitoring object.

[0054] Embodiment one, in order to better understand the above technical solutions, the above technical solutions will be described in detail below with reference to the drawings and specific embodiments of the specification. Referring to Figure 1 The figure is an exemplary flowchart of a multi-parameter fusion intelligent internal medicine sign monitoring method according to the embodiments of the present application. The monitoring method includes the following steps:

[0055] In step S1, multi-source sign data of a monitoring object is synchronously collected and spatio-temporal fused to obtain a fusion data set.

[0056] In the embodiment, the multi-source sign data of the monitoring object is synchronously collected and spatio-temporal fused to obtain a fusion data set. The following methods can be used, namely:

[0057] The multi-source sign data of the monitoring object is synchronously collected by a multi-intelligent terminal to obtain a multi-source sign data set.

[0058] The multi-source sign data set is spatio-temporally fused by a multi-modal fusion algorithm to obtain a fusion data set.

[0059] In a specific implementation, the multi-source vital sign data of the monitoring object includes radar signals, visual image signals and surface microseismic signals, wherein the radar signals can be collected by a radar sensor, the sampling frequency can be set between 20-50 Hz, and the radar signals include waveform features of breathing and heartbeat; the visual image signals can be collected by a structured light depth camera, the sampling frequency can be set between 25-30 fps, and the structured light depth camera can collect surface motion trajectories of the monitoring object; the surface microseismic signals can be collected by a piezoelectric film, the sampling frequency can be between 100-200 Hz, and the surface microseismic signals are used to reflect vibration information of events such as body movement, coughing and snoring of the monitoring object; the radar signals, the visual image signals and the surface microseismic signals collected by the radar sensor, the structured light depth camera and the piezoelectric film are synchronized by an NTP protocol to obtain a timestamp, and then the radar signals, the visual image signals and the surface microseismic signals with the synchronized timestamp are combined to form a multi-source vital sign data set; then, existing preprocessing techniques are used to standardize, detrend, remove drift and align the signals, and a multi-modal spatio-temporal fusion algorithm is used to fuse the preprocessed multi-source vital sign data set, and preferably, the multi-modal spatio-temporal fusion algorithm can use a multi-channel convolution-attention fusion model in the prior art to extract the time dependence and spatial correlation of various data.

[0060] It should be noted that the multi-source vital sign data in the present application refers to different physiological information channels, including non-contact radar wave data, visual image motion data and vibration sensor output data, and the multi-source vital sign data can be collected in multiple signal dimensions to comprehensively describe the vital sign state; the fused data set is the data basis for performing key steps such as waveform feature extraction, modal decomposition and joint modeling; in addition, in actual deployment, the sensor arrangement should ensure that the radar, visual and vibration devices have a highly overlapping collection area, and clock synchronization testing should be performed during system initialization to avoid fusion deviation caused by time misplacement of multi-source data.

[0061] In step S2, vital sign waveform features are extracted from the fused data set, and then modal decomposition is performed on the vital sign waveform features to obtain sub-modal signals of different vital signs.

[0062] Preferably, in the present embodiment, reference Figure 2 As shown in the figure, the figure is an exemplary flowchart for determining vital sign waveform features according to the present application, and the extraction of vital sign waveform features from the fused data set in the present embodiment can be implemented by the following steps:

[0063] In step S21, band-pass filtering and envelope adjustment operations are performed on the radar signals in the fused data set to obtain vital sign waveform sequences;

[0064] In step S22, the sequence of the physical waveform is subjected to signal-to-noise ratio enhancement and normalization processing to obtain a physical waveform feature.

[0065] In a specific implementation, first, the radar signal in the fusion dataset includes waveform features of respiration and heartbeat, the radar signal is subjected to filter processing by a band-pass filter algorithm to obtain an amplitude variation curve of the radar signal, wherein the filter processing frequency can be set to be in a filter range recommendation of 0.1-0.4 Hz, so as to facilitate reservation of the waveform feature signals of respiration and heartbeat, and then the envelope line in the amplitude variation curve is filtered by a sliding window extreme value connection method to obtain a sequence of physical waveforms; then, the sequence of physical waveforms is subjected to wavelet denoising, and the sequence of physical waveforms after denoising is subjected to normalization processing by a standard deviation normalization method, and then the sequence of physical waveforms after processing is taken as a physical waveform feature.

[0066] It should be noted that the sequence of physical waveforms refers to a periodic waveform curve representing a sign of a monitoring object after filter processing and envelope extraction of the radar signal, and the periodic waveform curve represents a time domain structure of chest surface micro-motion; in addition, in this embodiment, the band-pass filter can filter low-frequency drift and high-frequency noise, the envelope extraction operation can simplify a complex oscillation waveform into a waveform contour line which is easy to model, the signal-to-noise ratio enhancement can remove clutter, environmental vibration and instrument noise, and the normalization processing can effectively solve the problems of too large signal intensity difference and inconsistent sample amplitude distribution.

[0067] In this embodiment, the physical waveform feature is subjected to modal decomposition to obtain a sub-section modal signal of different signs, which can be implemented in the following manner, that is:

[0068] Extracting a signal component of different frequency bands from the physical waveform feature based on a modal decomposition algorithm;

[0069] Performing frequency band screening on the signal components of different frequency bands to obtain a sub-section modal signal of different signs.

[0070] In a specific implementation, first, a plurality of modal components are extracted from the characteristic of the physical waveform using a variational modal decomposition algorithm to obtain signal components in different frequency bands, wherein the modal component represents the characteristic change of the physical signal in different frequency bands, and the above process can be completed using the vmdpy library in the open source Python tool library; then, the main frequency value of the signal component in each frequency band can be calculated by fast Fourier transform, and the frequency band is filtered according to the preset frequency band range to obtain the sub-band modal signal of different signs, which can effectively exclude the interference of non-target factors such as body movement and vibration, wherein the preset frequency band range can refer to the following range, for example: the filtering condition of the respiratory modal component is that the main frequency value is between 0.1-0.5Hz, the filtering condition of the heartbeat modal component is that the main frequency value is between 0.8-2.0Hz, and other modal components can be set according to clinical experience, which is not limited here.

[0071] It should be noted that the sub-band modal signal in the present application is an independent waveform segment related to a single sign variable (such as respiration or heartbeat), and the sub-band modal signal can be jointly analyzed with the thoracic trajectory in the subsequent fusion modeling stage to provide a data basis for personalized dynamic modeling; the modal decomposition algorithm is a mathematical method for decomposing complex nonlinear signals into a plurality of modal components with physical significance, which can be used in the present embodiment to separate different frequency components such as respiration, heartbeat and interference in the characteristic of the physical waveform, facilitating subsequent identification; in addition, the variational modal decomposition algorithm used in the present application has stronger frequency separation capability.

[0072] In step S3, the thoracic displacement trajectory of the monitoring object is collected, and fusion modeling is performed on the thoracic displacement trajectory and all sub-band modal signals to obtain a sign reconstruction parameter, and echo suppression is performed on the sign reconstruction parameter to obtain a sign joint monitoring field.

[0073] In a specific implementation, the thoracic displacement trajectory of the monitoring object is collected by a visual sensor, and the visual sensor is used to track the surface motion characteristics of the chest region of the monitoring object. First, the visual image signal of the monitoring object is acquired, and then the surface motion of the visual image signal is estimated based on the optical flow method, that is, the optical flow field of the chest region is calculated by the pixel change between the continuous frames in the visual image signal, and then the overall motion vector is extracted and the thoracic displacement trajectory is obtained. It should be noted that the thoracic displacement trajectory refers to the continuous motion curve of the chest surface of the monitoring object on the time axis, which can represent the two-dimensional sequence of the motion vector of the chest key point changing with time, and the surface motion characteristics include the breathing amplitude, frequency, rhythm and body movement characteristics of the monitoring object. In addition, in actual implementation, a background segmentation algorithm can be used to limit the region of interest (ROI) of the visual image signal to ensure stable tracking effect and improve the accuracy of chest displacement extraction.

[0074] Preferably, in the present embodiment, the thoracic displacement trajectory is obtained by tracking the surface motion characteristics of the chest region of the monitoring object using a visual sensor, and the surface motion characteristics include the breathing amplitude, frequency, rhythm and body movement characteristics of the monitoring object.Figure 3 As shown in the figure, the figure is an example flow chart for determining a sign reconstruction parameter provided in the present application, and in the embodiment, a sign reconstruction parameter is obtained by fusing modeling of the thoracic displacement trajectory and all sub-section modal signals, and can be realized by the following steps:

[0075] In step S31, dynamic time-frequency analysis is performed on the thoracic displacement trajectory to obtain a frequency feature;

[0076] In step S32, cross-correlation analysis is performed on the frequency feature and each sub-section modal signal to obtain a correlation coefficient of each sub-section modal signal;

[0077] In step S33, a joint regression model is constructed based on all correlation coefficients;

[0078] In step S34, the change amount of the thoracic displacement trajectory is taken as an independent variable of the joint regression model, and then the sign reconstruction parameter is determined by the joint regression model.

[0079] In a specific implementation, first, continuous wavelet transform is performed on the thoracic displacement trajectory, the thoracic displacement trajectory is converted into a time-frequency two-dimensional matrix, main frequency components and amplitude information are extracted from the time-frequency two-dimensional matrix by a principal component analysis algorithm, and then a vector composed of the main frequency components and the amplitude information is taken as a frequency feature, i.e., frequency feature=(main frequency component, amplitude information), wherein the main frequency components include a respiratory main frequency and a motion interference frequency, and both can be extracted by the principal component analysis algorithm; second, for each sub-section modal signal, a mean value of the sub-section modal signal is extracted, and then the mean value and the frequency feature are used to calculate a correlation coefficient of the sub-section modal signal by a Pearson correlation coefficient calculation formula, and by the above manner, the correlation coefficient of each sub-section modal signal can be obtained, which can be used to quantify the influence degree of the sub-section modal signal on the thoracic displacement change; third, a regression model is initialized, and the proportion of the correlation coefficients of all sub-section modal signals is taken as a feature weight factor to perform weighted linear regression processing on the regression model, and then the regression model after the weighted linear regression processing is taken as a joint regression model; and finally, the change amount of the thoracic displacement trajectory is taken as an independent variable of the joint regression model, the joint regression model maps the thoracic displacement trajectory to a response strength of each sign modality to obtain a sign reconstruction parameter for representing physiological signals such as respiration and heartbeat, wherein the change amount of the thoracic displacement trajectory can be obtained by calculating the Euclidean distance between key points of the thoracic displacement trajectory between consecutive frames, and the change amount reflects the motion amplitude of the thoracic region per unit time.

[0080] It should be noted that the sign reconstruction parameter is the intensity data of the sign modal signal in the sign monitoring period, which can be used to judge the depth of breathing, the strength of heartbeat and the like, and is the core data for constructing a high signal-to-noise ratio monitoring field; in the embodiment, the time-frequency transformation is used to convert the time-domain signal into a joint representation of time and frequency, which can be used to extract the frequency variation characteristics of the signal in different time periods, and can analyze the periodic respiratory rhythm and sudden action existing in the chest movement; the joint regression model is a modeling method that combines the contributions of multiple sub-modal signals, which can be used to establish a mathematical mapping relationship between the chest movement and the multi-frequency physiological signal.

[0081] In the embodiment, the sign reconstruction parameter is echo suppressed, and the sign joint monitoring field is obtained in the following manner, that is:

[0082] The wave component in the sign reconstruction parameter is identified for interference to obtain an echo interference section;

[0083] The echo interference section is subjected to suppressive filtering processing based on an energy threshold to obtain an echo suppression signal;

[0084] The echo suppression signal and the modal component of the chest displacement trajectory are fused in a spatial domain according to the sign reconstruction parameter to obtain a sign joint monitoring parameter;

[0085] The sign joint monitoring field is generated by a spatial coupling function and the sign joint monitoring parameter.

[0086] In a specific implementation, first, the instantaneous energy of each signal segment in the sign parameter is calculated by the moving window energy analysis method with a time window of 2s, and when the instantaneous energy is greater than 1.5 times the average energy, the echo segment corresponding to the instantaneous energy is taken as an echo interference segment; second, the echo interference segment is denoised by the wavelet shrinkage method, and then the smoothed signal obtained after processing is taken as an echo suppression signal; then, the window sliding algorithm can be used to align the echo suppression signal and the modal component of the thoracic displacement trajectory to ensure that the two types of signals have a corresponding relationship under the same time reference, and then the dot product attention method is used to fuse the aligned echo suppression signal and the modal component of the thoracic displacement trajectory, that is, the influence degree of each modal input on the fusion result is dynamically adjusted according to the reconstruction parameter using the attention adjustment mechanism, and then the weight value is determined according to the influence degree, and then the echo suppression signal and the modal component of the thoracic displacement trajectory are weighted and output through the weight value to obtain a sign joint monitoring parameter; finally, the two-dimensional coordinates of the monitoring area are obtained, the center of the chest of the monitoring object is taken as the coordinate center, and the sign joint monitoring parameter in each spatial region is interpolated and fused using a spatial coupling function to generate a spatial distribution map, and then the spatial distribution map is taken as a sign joint monitoring field, wherein the spatial coupling function can use a bidirectional Gaussian kernel function in the prior art, and the coordinate space corresponding to the sign joint monitoring parameter can be identified.

[0087] It should be noted that the sign joint monitoring in the present application refers to a multi-parameter dynamic distribution map constructed in the time, space and modal fusion dimensions, which can be used for dynamically monitoring the change trend of the sign signal in different time and space regions; in addition, the echo interference in the present embodiment refers to abnormal fluctuations caused by non-physiological factors such as body movement, environmental vibration or electromagnetic interference in non-contact monitoring; the echo suppression signal refers to the effective signal part retained after processing the echo interference by a targeted filtering or denoising algorithm; the spatial domain fusion refers to mapping time-synchronized multi-source signals into a unified spatial domain representation, combining thoracic deformation and multi-modal sign signals to construct a parameter set with spatial consistency and dynamic change characteristics; the sign joint monitoring parameter is a signal parameter obtained by fusing the echo-suppressed signal and the spatial coupling model.

[0088] In step S4, a sign matching feature vector is determined according to the sign joint monitoring field and the real-time sign phase of the monitoring object, and then a sign mode recognition is performed on the sign matching feature vector through a pre-set classification model to obtain a trigger factor when the sign is abnormal.

[0089] In the present embodiment, the sign matching feature vector can be determined according to the sign joint monitoring field and the real-time sign phase of the monitoring object in the following manner, that is:

[0090] In the sign joint monitoring field, the spatial amplitude distribution and the modal response feature are extracted based on the real-time sign phase of the monitoring object;

[0091] The phase matching sampling point set is determined through the spatial amplitude distribution and the modal response feature;

[0092] The multi-modal feature fusion is performed on the phase matching sampling point set to obtain a local fusion feature set;

[0093] The local fusion feature set is spliced and normalized to obtain a sign matching feature vector.

[0094] In the specific implementation, in the sign joint monitoring field, first, based on the time axis in the sign joint monitoring field, the sign phase information of the monitoring object at the current time is obtained in real time. The phase information can be calculated by performing Hilbert transform on the heartbeat or breathing signal. Under the current phase information, the signal values of the signs of different monitoring regions (such as the left, right, upper, and lower parts of the chest) are taken as the spatial amplitude distribution, and the response changes of the modal signals on the corresponding spatial points are taken as the modal response feature. Second, the coincident points of the spatial amplitude distribution and the modal response feature in different monitoring regions are taken as the phase matching sampling points, and then all the phase matching sampling points are obtained, and the set composed of all the phase matching sampling points is taken as the phase matching sampling point set. Then, the spatial amplitude distribution and the modal response feature of each point are compressed and extracted by using a one-dimensional convolutional neural network, and the spatial amplitude distribution and the modal response feature obtained by compression and extraction are fused through an attention mechanism, so that the local fusion feature can be obtained, and the set composed of all the local fusion features is taken as the local fusion feature set. Finally, the local fusion features in the local fusion feature set are numbered, and the numbered local fusion features are spliced into a vector, that is, (local fusion feature 1, local fusion feature 2, local fusion feature 3, local fusion feature 4), and the spliced vector is standardized by standard deviation, so that the standardized vector is taken as the sign matching feature vector.

[0095] It should be noted that the sign matching feature vector in the present application refers to a multi-dimensional vector formed by fusing the spatial amplitude distribution information and the modal response feature extracted in the sign joint monitoring field at a specific time phase point, which is used to comprehensively express the feature combination of the current sign state in the spatial, time and modal dimensions. The phase information in the present embodiment represents the phase position of the current sign (such as breathing and heartbeat) in its periodic waveform, which is used to synchronize and match the spatial features in the joint monitoring field. In addition, the phase matching sampling point set in the present embodiment can capture the current sign state and reflect the trend of the sign, thereby improving the recognition ability of the model for non-periodic abnormalities.

[0096] In the embodiment, the trigger factor when the sign is abnormal is obtained by performing sign pattern recognition on the sign matching feature vector through a preset classification model. Specifically, the following method can be used, that is:

[0097] The sign matching feature vector is input into a preset classification model to perform pattern recognition inference, and a classification label and a classification confidence are obtained.

[0098] The trigger factor when the sign is abnormal is determined according to the classification label and the classification confidence.

[0099] It should be noted that in the present application, the preset classification model refers to a support vector machine classification model pre-trained based on a multi-parameter sign sample data set, wherein the multi-parameter sign sample data set is obtained by collecting a large number of monitoring objects under different sign states (such as normal breathing, shallow breathing, apnea, arrhythmia, etc.), and includes a labeled sign matching feature vector sample set, wherein each sample vector includes multiple feature dimensions corresponding to fusion sign features under different spatial positions, modal signal responses and phase points, and is associated with a corresponding sign state label. The pre-trained support vector machine classification model can match, classify, score and determine the sign of the monitoring object, and can effectively improve the recognition accuracy of the sign pattern, wherein the pre-training is a conventional machine learning training.

[0100] In specific implementation, first, the sign matching feature vector is input into a preset classification model to perform pattern recognition inference. In the inference process, the classification model calculates the sign matching feature vector to obtain a classification label, which is used to represent the sign state category corresponding to the current feature, wherein the classification label includes normal, abnormal-apnea, abnormal-tachypnea and abnormal-arrhythmia. In the classification model, the classification label decision function score is used as the classification confidence. Then, the trigger factor in the present application refers to a trigger signal generated when the sign state at the current time is determined to be abnormal by performing pattern recognition on the sign state through a preset classification model, which is used to start the subsequent abnormal response mechanism. The trigger factor when the sign is abnormal includes abnormal type, abnormal confidence, trigger time and belonging modality (such as breathing or heartbeat). In the classification model, the classification label and the classification confidence perform abnormal determination to obtain the trigger factor when the sign is abnormal.

[0101] It should be noted that the trigger factor in the present application can identify whether the current sign is abnormal, and includes abnormal type, confidence, abnormal time and other information, which is convenient for clinical tracing. The introduction of the classification confidence makes the system more flexible in abnormal detection, which is beneficial to realize risk grading and response strategy differentiation. In addition, in the embodiment, the discrimination ability of the support vector machine can accurately divide the sign state in the high-dimensional fusion feature space.

[0102] In step S5, an abnormal response is performed based on the trigger factor, and an abnormal sign result of the monitoring object is obtained.

[0103] In this embodiment, the abnormal response based on the trigger factor to obtain the abnormal sign result of the monitoring object can be implemented in the following manner, that is:

[0104] The sign abnormality level and the response mode are determined by the preset clinical event response rule library according to the trigger factor;

[0105] The sign abnormality level, the response mode, and the electronic medical record data of the monitoring object are associated and analyzed to generate the abnormal sign result of the monitoring object.

[0106] In specific implementation, the clinical event response rule library can be constructed according to medical institution standards or industry guidelines, supports custom updates, and can ensure flexible response capability for different types of sign abnormalities; first, the response rule matched with the trigger factor is searched in the clinical event response rule library, and the abnormality level (such as low risk, medium risk, and high risk) and the response mode (such as monitoring enhancement, alarm, and remote notification) are obtained through the response rule; then, the identified sign abnormality level and the existing electronic medical record information of the monitoring object are data fused, wherein the data fused information includes medical history, age, BMI, basic heart rate, respiratory rate, and sign trend, and the data fused electronic medical record information is taken as the abnormal sign result of the monitoring object.

[0107] It should be noted that the abnormal sign result in the present application refers to the structured diagnostic output generated by the system based on clinical rules, patient background, and real-time data after the trigger factor is determined to be an abnormal state, which is used to guide subsequent intervention or record clinical events of medical staff; in this embodiment, the understanding ability of the system to the abnormal event background can be improved by introducing the electronic medical record data associated analysis, the misjudgment of the sign abnormality of the high-risk chronic disease patient can be avoided, and the clinical explainability of the abnormal result can be enhanced; in addition, by connecting the abnormal level with the alarm mechanism or the remote medical interface, the rapid response to the critical state can be realized.

[0108] In summary, the technical solution adopted in the present application can realize deep fusion of multi-source sign data and intelligent identification of abnormalities, so as to improve the clinical adaptability of internal medicine sign monitoring.

[0109] Embodiment two, the present application provides a kind of intelligent internal medicine sign monitoring system of multi-parameter fusion, reference Figure 4 As shown in the figure, it is the module structure diagram of the monitoring system shown in the embodiment of the present application, and the monitoring system comprises:

[0110] The acquisition fusion module 100 is used for synchronously acquiring and spatio-temporally fusing multi-source sign data of a monitoring object to obtain a fusion data set;

[0111] The feature decomposition module 200 is used for extracting sign waveform features from the fusion data set, and further decomposing the sign waveform features to obtain sub-segment modal signals of different signs;

[0112] The monitoring fusion module 300 is used for acquiring a thoracic displacement trajectory of the monitoring object, fusing modeling through the thoracic displacement trajectory and all the sub-segment modal signals to obtain sign reconstruction parameters, performing echo suppression on the sign reconstruction parameters, and further obtaining a sign joint monitoring field;

[0113] The abnormal trigger module 400 is used for determining a sign matching feature vector according to the sign joint monitoring field and a real-time sign phase of the monitoring object, further performing sign pattern recognition on the sign matching feature vector through a preset classification model to obtain a trigger factor in a sign abnormal state;

[0114] The abnormal response module 500 is used for performing abnormal response based on the trigger factor to obtain an abnormal sign result of the monitoring object.

[0115] The present application is described with reference to flowcharts and / or block diagrams of the method, equipment (system) and computer program product according to the embodiments of the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and the combination of the flows and / or blocks in the flowcharts and / or block diagrams can be implemented by computer program instructions. These computer program instructions can be provided to a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing devices to produce a machine, so that the instructions executed by the computer or other programmable data processing devices produce a device for implementing the functions specified in the flowcharts and / or block diagrams. Figure 1 The device for implementing the functions specified in one flow or multiple flows and / or blocks Figure 1 The device for implementing the functions specified in one flow or multiple flows and / or blocks

[0116] Those skilled in the art can understand that all or part of the steps of various methods in the above embodiments can be completed by instructing the relevant hardware by means of a program, and the program can be stored in a computer readable storage medium, including Read-Only Memory (ROM), Random Access Memory (RAM), Programmable Read-Only Memory (PROM), Erasable Programmable Read-Only Memory (EPROM), One-time Programmable Read-Only Memory (OTPROM), Electrically-Erasable Programmable Read-Only Memory (EEPROM), Compact Disc Read-Only Memory (CD-ROM) or other optical disk storage, magnetic disk storage, magnetic tape storage, or any other medium that can be used to carry or store data in a computer readable manner.

[0117] It should also be noted that the terms "comprising", "containing", or any other variant thereof are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements, but can also include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the statement "comprising a" does not exclude the existence of additional identical elements in the process, method, article, or apparatus that includes the element.

Claims

1. A multi-parameter fusion intelligent internal medicine vital sign monitoring method, characterized in that, The monitoring method includes the following steps: Simultaneous collection and spatiotemporal fusion of multi-source vital sign data of the monitored objects yield a fused dataset. The vital sign waveform features are extracted from the fused dataset, and then modal decomposition is performed on the vital sign waveform features to obtain sub-segment modal signals of different vital signs. The chest displacement trajectory of the monitored object is collected, and the chest displacement trajectory and all sub-segment modal signals are fused and modeled to obtain vital sign reconstruction parameters. Echo suppression is applied to the vital sign reconstruction parameters to obtain a joint vital sign monitoring field. Based on the real-time vital sign phase of the joint vital sign monitoring field and the monitored object, a vital sign matching feature vector is determined. Then, a pre-set classification model is used to perform vital sign pattern recognition on the vital sign matching feature vector to obtain the triggering factor when vital signs are abnormal. Based on the triggering factor, an abnormal response is performed to obtain the abnormal physical signs of the monitored object; Specifically, performing modal decomposition on the waveform features of the vital signs to obtain sub-segment modal signals of different vital signs includes: Based on the mode decomposition algorithm, signal components of different frequency bands are extracted from the waveform features of the vital signs; By performing frequency band filtering on the signal components of each frequency band, sub-segment modal signals with different characteristics are obtained; Specifically, the vital sign reconstruction parameters obtained by fusing the thoracic displacement trajectory and all sub-segment modal signals include: Dynamic time-frequency analysis was performed on the thoracic displacement trajectory to obtain frequency characteristics; Cross-correlation analysis is performed on the frequency characteristics and the modal signals of each sub-segment to obtain the correlation coefficient of the modal signals of each sub-segment; Construct a joint regression model based on all correlation coefficients; The change in the thoracic displacement trajectory is used as the independent variable of the joint regression model, and then the vital sign reconstruction parameters are determined through the joint regression model. Specifically, the echo suppression of the reconstructed vital signs parameters to obtain the joint monitoring field of vital signs includes: Interference identification is performed on the waveform components in the reconstructed vital signs parameters to obtain the echo interference segment; Based on the energy threshold, suppressive filtering is performed on the echo interference section to obtain the echo suppression signal; Based on the reconstructed vital signs parameters, the modal components of the echo suppression signal and the thoracic displacement trajectory are spatially fused to obtain joint monitoring parameters of vital signs. A joint monitoring field for vital signs is generated by using a spatial coupling function and the joint monitoring parameters for vital signs. Specifically, determining the vital sign matching feature vector based on the real-time vital sign phase of the joint vital sign monitoring field and the monitored object includes: In the joint monitoring field of vital signs, spatial amplitude distribution and modal response characteristics are extracted based on the real-time vital sign phases of the monitored objects; The phase-matched sampling point set is determined by the spatial amplitude distribution and the modal response characteristics; Multimodal feature fusion is performed on the phase-matched sampling point set to obtain a local fused feature set; The local fusion feature sets are concatenated and normalized to obtain the vital sign matching feature vector.

2. The intelligent internal medicine vital sign monitoring method with multi-parameter fusion as described in claim 1, characterized in that, The multi-source vital signs data of the monitored objects are collected synchronously and fused in time and space to obtain the fused dataset, which specifically includes: Multi-source vital signs data of the monitored objects are collected synchronously using multiple intelligent terminals to obtain a multi-source vital signs dataset; The multi-source vital signs dataset is spatiotemporally fused using a multimodal fusion algorithm to obtain a fused dataset.

3. The intelligent internal medicine vital sign monitoring method with multi-parameter fusion as described in claim 1, characterized in that, Extracting vital sign waveform features from the fused dataset specifically includes: Bandpass filtering and envelope modulation are performed on the radar signals in the fused dataset to obtain a sequence of vital sign waveforms. The signal-to-noise ratio of the waveform sequence is enhanced and normalized to obtain the waveform features of the vital signs.

4. The intelligent internal medicine vital sign monitoring method with multi-parameter fusion as described in claim 1, characterized in that, The chest displacement trajectory of the monitored object is acquired by a visual sensor, which is used to track the surface motion characteristics of the chest area of ​​the monitored object.

5. The intelligent internal medicine vital sign monitoring method with multi-parameter fusion as described in claim 1, characterized in that, The preset classification model refers to a support vector machine classification model pre-trained based on a multi-parameter physical characteristic sample dataset.

6. A multi-parameter fusion intelligent internal medicine vital signs monitoring system, used to execute the multi-parameter fusion intelligent internal medicine vital signs monitoring method as described in any one of claims 1 to 5, characterized in that, The monitoring system includes: The data acquisition and fusion module is used to simultaneously acquire and spatiotemporally fuse multi-source vital sign data of the monitored objects to obtain a fused dataset. The feature decomposition module is used to extract vital sign waveform features from the fused dataset, and then perform modal decomposition on the vital sign waveform features to obtain sub-segment modal signals of different vital signs. The monitoring fusion module is used to collect the thoracic displacement trajectory of the monitored object, fuse the thoracic displacement trajectory and all sub-segment modal signals to form a model, obtain vital sign reconstruction parameters, suppress echoes on the vital sign reconstruction parameters, and then obtain a joint vital sign monitoring field. The abnormality triggering module is used to determine the vital sign matching feature vector based on the real-time vital sign phase of the joint vital sign monitoring field and the monitored object, and then perform vital sign pattern recognition on the vital sign matching feature vector through a preset classification model to obtain the triggering factor when vital signs are abnormal. An anomaly response module is used to respond to anomalies based on the triggering factor and obtain the results of abnormal physical signs of the monitored object.

Images