LSTM neural network-based multi-signature fusion lung respiration monitoring system and method

Abstract

This invention discloses a multi-sign fusion lung respiratory monitoring system and method based on LSTM neural network. The invention achieves dynamic monitoring of multiple sign parameters through the collection of user physiological parameters and lung breath sound signals. Simultaneously, based on the collected multimodal sign data and combined with standard physiological indicator data, multimodal feature data is generated. Then, an LSTM model is used to generate the user's lung respiratory health probability. This provides an effective data processing method for monitoring multiple sign parameters. Finally, the invention utilizes the user's historical probability threshold data to determine the user's probability threshold within the current monitoring period. Based on this, different probability thresholds can be generated for different users, thereby achieving adaptive changes in alarm thresholds based on the patient. Finally, the probability thresholds and lung respiratory health probability are sent to a medical terminal, providing accurate auxiliary information for medical personnel.

Description

Technical Field

[0001] This invention belongs to the field of medical and health monitoring technology based on artificial intelligence, and specifically relates to a lung respiratory monitoring system and method based on LSTM neural network for multi-sign fusion. Background Technology

[0002] Currently, traditional methods of assessing sputum suctioning largely rely on physicians' subjective judgment based on lung auscultation or patients' complaints of "excessive sputum and difficulty breathing." For patients with aphasia, in a vegetative state, or those who have undergone tracheotomy, due to their limited communication skills, they cannot proactively request suctioning. Therefore, assessment requires physician and nurse rounds or the use of pulse oximetry equipment. This often leads to delayed suctioning, worsening of lung infections, and in severe cases, sputum plugging, resulting in patient death. The primary reason for this is the lack of dynamic monitoring and effective management methods for the coordinated changes in multiple vital signs and parameters in clinical practice, failing to provide healthcare professionals with adequate support. Accurate auxiliary evidence is needed; currently, clinical assessment of sputum suctioning in aphasic and dementia patients generally relies solely on finger-clip pulse oximeters, which provide limited and outdated data. Furthermore, the fixed alarm threshold for pulmonary suctioning fails to consider individual patient differences, resulting in all patients using the same threshold, thus limiting its applicability and further reducing the accuracy of the auxiliary evidence provided to medical staff. Therefore, given these shortcomings, providing a pulmonary respiratory monitoring system based on LSTM neural networks that enables dynamic monitoring and effective processing of multiple vital signs, with alarm thresholds adaptively changing according to the patient, has become an urgent problem to be solved. Summary of the Invention

[0003] The purpose of this invention is to provide a lung respiratory monitoring system and method based on LSTM neural network fusion, in order to solve the problems of single and lagging monitoring data, lack of dynamic monitoring and effective processing of the coordinated changes of multiple vital signs parameters, and the inability to provide accurate auxiliary information for medical staff due to fixed alarm thresholds in the existing technology.

[0004] To achieve the above objectives, the present invention adopts the following technical solution:

[0005] Firstly, a lung respiratory monitoring system based on LSTM neural network fusion of multiple signs is provided, including:

[0006] A physiological parameter monitoring device and a respiratory signal acquisition device are provided, wherein the physiological parameter monitoring device is used to collect the physiological parameters of a target user during the current monitoring period, and the respiratory signal acquisition device is used to collect the lung breath sound signal of the target user during the current monitoring period. The physiological parameter monitoring device and the respiratory signal acquisition device are communicatively connected to a cloud server for transmitting the physiological parameters and the lung breath sound signal to the cloud server.

[0007] A cloud server is used to acquire standard physiological indicator data and generate multimodal feature data based on the standard physiological indicator data, the physiological parameters, and the lung breath sound signals.

[0008] A cloud server is used to input the multimodal feature data into the trained LSTM model to obtain the probability of the target user's lung respiratory health in the current period.

[0009] The cloud server is also used to generate a probability threshold for the current monitoring period based on the historical probability threshold data of the target user within a preset historical time period, and to send the lung respiratory health probability and the probability threshold to the medical terminal.

[0010] Based on the above disclosure, the respiratory monitoring system provided by this invention is equipped with a physiological parameter monitoring device and a respiratory signal acquisition device. The physiological parameter monitoring device collects the physiological parameters of the target user during the current monitoring period, while the respiratory signal acquisition device collects the lung breath sound signals during the current monitoring period. Thus, compared to traditional technologies that only collect blood oxygen saturation, this invention achieves dynamic monitoring of multiple vital signs of the user. Simultaneously, after the aforementioned data is collected, it is transmitted to a cloud server for processing. The specific process is as follows: first, standard physiological indicator data is obtained; then, based on the standard physiological indicator data, as well as the aforementioned collected physiological parameters and lung breath sound signals, a respiratory monitoring system is generated. Multimodal feature data is generated and then input into a trained LSTM model to obtain the probability of lung respiratory health of the target user within the current monitoring period. Thus, by generating multimodal feature data and using an LSTM model for data processing, an effective data processing method is provided for monitoring multiple vital signs. Next, this invention utilizes historical probability threshold data of the target user within a preset historical time period to generate the probability threshold for the current monitoring period, thereby generating individual probability thresholds for different users. Finally, the probability threshold and lung respiratory health probability for the current monitoring period are sent to the medical terminal, providing accurate auxiliary information for medical personnel.

[0011] Through the above design, this invention achieves dynamic monitoring of multiple vital signs by collecting users' physiological parameters and lung breath sound signals. Simultaneously, based on the aforementioned collected multimodal vital sign data and combined with standard physiological indicator data, multimodal feature data is generated. Then, an LSTM model is used to generate the user's lung respiratory health probability. Thus, by generating multimodal feature data and using an LSTM model for data processing, an effective data processing method is provided for monitoring multiple vital signs. Finally, this invention utilizes the user's historical probability threshold data to determine the user's probability threshold within the current monitoring period. Based on this, individual probability thresholds can be generated for different users, thereby achieving adaptive changes in alarm thresholds based on the patient. Finally, sending the probability thresholds and lung respiratory health probability to the medical terminal provides accurate auxiliary information for medical personnel.

[0012] In one possible design, a cloud server is used to align the physiological parameters and the lung breath sound signals with standard physiological index data to obtain standard values ​​corresponding to the physiological parameters and the lung breath sound signals, respectively.

[0013] A cloud server is used to embed the standard values ​​corresponding to the physiological parameters and the standard values ​​corresponding to the lung breath sound signals into an embedding vector.

[0014] A cloud server is used to normalize the physiological parameters and the lung breath sound signals to obtain a normalized physiological parameter vector and a normalized lung breath sound vector.

[0015] The cloud server is also used to concatenate the embedded vector, the normalized physiological parameter vector, and the normalized lung breath sound vector to obtain the multimodal feature data.

[0016] In one possible design, the LSTM model is trained by taking the multimodal feature data of several sample users as input and the lung respiratory health probability value of each sample user as output, and the trained LSTM model is obtained.

[0017] The loss function of the trained LSTM model is:

[0018]

[0019] In equation (1), γ represents the loss function, p i,c Let represent the probability that the i-th sample user in a set of sample users output by the LSTM model belongs to the c-th class, where C represents the total number of classes. ,c P represents the true label of the i-th sample user belonging to the c-th class. tP represents the maximum probability value among all categories for the i-th sample user at time t. t-1 Let λ represent the maximum probability value among all categories for the i-th sample user at time t-1, T represent the collection duration of the multimodal feature data for the i-th sample user, N represent the total number of sample users, and λ represent the smoothing coefficient.

[0020] In one possible design, a cloud server is used to obtain the probability of a target user's lung respiratory health over a preset period of time prior to the current moment, and to determine the false alarm rate and the false negative rate based on the probability of the target user's lung respiratory health.

[0021] The cloud server is used to calculate the updated parameters based on the false positive rate and the false negative rate.

[0022] A cloud server is used to construct a probability update objective function based on the historical probability threshold data;

[0023] The cloud server is also used to determine the probability threshold within the current monitoring period using the update parameters and the probability update objective function.

[0024] In one possible design, a cloud server is used to calculate a weighted sum of the false positive rate and the false negative rate to obtain the updated parameters.

[0025] In one possible design, the probability update objective function is:

[0026] u(x)=μ(x)+κ·σ(x) (2)

[0027] In equation (2), u(x) represents the probability update objective function, μ(x) represents the probability threshold prediction mean, σ(x) represents the probability threshold prediction standard deviation, where μ(x) and σ(x) are obtained from historical probability threshold data, and κ represents the exploration coefficient.

[0028] In one possible design, the historical probability threshold data includes historical probability thresholds corresponding to each historical monitoring period;

[0029] The cloud server is used to calculate the gradient of the probability update objective function.

[0030] The cloud server is used to obtain the learning rate and filter out the historical probability threshold corresponding to the most recent historical monitoring period from the historical probability thresholds corresponding to each historical monitoring period, so as to use it as the target probability threshold.

[0031] The cloud server is also used to calculate the probability threshold within the current monitoring period based on the learning rate, the gradient of the probability-updated objective function, the update parameters, and the target probability threshold.

[0032] In one possible design, the cloud server is used to calculate the probability threshold within the current monitoring period according to the following formula (3);

[0033]

[0034] In equation (3), τnew represents the probability threshold within the current monitoring period, τ old This represents the target probability threshold. The gradient of the probability update objective function is represented by η, where η represents the learning rate and Objective represents the update parameters.

[0035] In one possible design, the physiological parameters of the target user during the current monitoring period include: the target user's heart rate data, blood oxygen data, respiratory rate data, and body temperature data during the current monitoring period.

[0036] Secondly, a method for lung respiratory monitoring based on LSTM neural network fusion of multiple vital signs is provided, executed on a cloud server within the lung respiratory monitoring system based on LSTM neural network fusion of multiple vital signs fusion, which is designed according to the first aspect or any possible design of the first aspect, and the method includes:

[0037] Acquire the physiological parameters of the target user during the current monitoring period uploaded by the physiological parameter monitoring device, and the lung breath sound signals of the target user during the current monitoring period uploaded by the respiratory signal acquisition device;

[0038] Obtain standard data of physiological indicators, and generate multimodal feature data based on the standard data of physiological indicators, the physiological parameters and the lung breath sound signals;

[0039] The multimodal feature data is input into the trained LSTM model to obtain the probability of the target user's lung respiratory health in the current period;

[0040] Based on the historical probability threshold data of the target user within a preset historical time period, a probability threshold for the current monitoring period is generated, and the lung respiratory health probability and the probability threshold are sent to the medical terminal.

[0041] Thirdly, an electronic device is provided, comprising a memory, a processor, and a transceiver connected in sequence and communication, wherein the memory is used to store a computer program, the transceiver is used to send and receive messages, and the processor is used to read the computer program and execute the lung respiratory monitoring method based on LSTM neural network fusion as described in the second aspect.

[0042] Fourthly, a storage medium is provided, on which instructions are stored, which, when executed on a computer, perform the lung respiratory monitoring method based on LSTM neural network fusion as described in the second aspect.

[0043] Fifthly, a computer program product containing instructions is provided, which, when executed on a computer, causes the computer to perform the lung respiratory monitoring method based on LSTM neural network fusion as described in the second aspect.

[0044] Beneficial effects:

[0045] (1) This invention achieves dynamic monitoring of multiple vital signs by collecting users' physiological parameters and lung breath sound signals. Simultaneously, based on the collected multimodal vital sign data and combined with standard physiological index data, multimodal feature data is generated. Then, an LSTM model is used to generate the user's lung respiratory health probability. Thus, by generating multimodal feature data and using an LSTM model for data processing, an effective data processing method is provided for the monitored multimodal vital signs. Finally, this invention uses the user's historical probability threshold data to determine the user's probability threshold in the current monitoring period. Based on this, different probability thresholds can be generated for different users, thereby achieving adaptive changes based on the patient's alarm threshold. Finally, the probability threshold and lung respiratory health probability are sent to the medical terminal, which can provide accurate auxiliary information for medical staff. Attached Figure Description

[0046] Figure 1 A schematic diagram of the architecture of a lung respiratory monitoring system based on LSTM neural network fusion provided in an embodiment of the present invention;

[0047] Figure 2 This is a network structure diagram of the LSTM model provided in an embodiment of the present invention;

[0048] Figure 3 This is a flowchart illustrating the steps of a lung respiratory monitoring method based on LSTM neural network fusion, as provided in an embodiment of the present invention. Detailed Implementation

[0049] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the present invention will be briefly introduced below in conjunction with the accompanying drawings and descriptions of the embodiments or the prior art. Obviously, the following description of the structure of the accompanying drawings is only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. It should be noted that the description of these embodiments is for the purpose of helping to understand the present invention, but does not constitute a limitation of the present invention.

[0050] It should be understood that although the terms first, second, etc., may be used herein to describe various units, these units should not be limited by these terms. These terms are only used to distinguish one unit from another. For example, a first unit may be referred to as a second unit, and similarly, a second unit may be referred to as a first unit, without departing from the scope of the exemplary embodiments of the invention.

[0051] It should be understood that the term "and / or" that may appear in this document is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can mean: A exists alone, B exists alone, and A and B exist simultaneously. The term " / and" that may appear in this document describes another relationship between related objects, indicating that two relationships can exist. For example, A / and B can mean: A exists alone, and A and B exist alone. In addition, the character " / " that may appear in this document generally indicates that the related objects before and after it are in an "or" relationship.

[0052] Example:

[0053] See Figure 1 As shown, the LSTM neural network-based multi-sign fusion lung respiratory monitoring system provided in this embodiment may include, but is not limited to, a physiological parameter monitoring device, a respiratory signal acquisition device, a cloud server, and a medical terminal. The physiological parameter monitoring device and the respiratory signal acquisition device serve as front-end devices for the collaborative acquisition of user multimodal vital sign data. The cloud server performs data processing, generating the user's lung respiratory health probability based on the data collected by the front-end devices, and using the user's historical probability threshold data to generate a probability threshold for each user. Finally, the aforementioned processing results are sent to the medical terminal for visualization, thereby providing accurate auxiliary information for medical personnel.

[0054] Specifically, the working process of each of the aforementioned devices is as follows:

[0055] The physiological parameter monitoring device is used to collect the physiological parameters of the target user during the current monitoring period, and the respiratory signal acquisition device is used to collect the lung breath sound signals of the target user during the current monitoring period. For example, the physiological parameters of the target user during the current monitoring period may include, but are not limited to, heart rate data, blood oxygen data, respiratory rate data, and body temperature data. For example, the aforementioned monitoring period may be set to, but is not limited to, 60 seconds. Of course, it can be specifically set according to actual use and is not limited to the aforementioned examples.

[0056] Meanwhile, the aforementioned physiological parameter monitoring device can, but is not limited to, use a low-power medical-grade smartwatch (which integrates PPG, ECG, temperature sensors, etc.) with a sampling frequency of 100Hz; while the respiratory signal acquisition device can, but is not limited to, use a respiratory sound sensor, which is attached to the lung auscultation area of ​​the target user according to the body surface markings during acquisition; thus, by using the aforementioned two devices, the collaborative acquisition of multimodal vital sign data of the target user can be achieved.

[0057] In this embodiment, the physiological parameter monitoring device and the respiratory signal acquisition device are also communicatively connected to a cloud server to realize data transmission, that is, to transmit the physiological parameters and the lung breath sound signals to the cloud server so that the cloud server can perform data processing based on the received data; Optionally, for example, the aforementioned physiological parameter monitoring device and respiratory signal acquisition device can communicate with the cloud server via WIFI, 4G and / or 5G network.

[0058] Thus, after receiving the multimodal vital sign data of the target user, the cloud server can generate the target user's lung respiratory health probability based on this data. The specific working process is as follows:

[0059] The cloud server is first used to acquire standard physiological indicator data, and based on the standard physiological indicator data, the physiological parameters and the lung breath sound signal, multimodal feature data is generated; then, the multimodal feature data is input into the trained LSTM model to obtain the probability of the target user's lung respiratory health in the current period.

[0060] In specific implementation, the aforementioned physiological indicator standard data refers to the preset clinical scores (such as CPIS and ADL scores). These clinical scores correspond to standard values ​​for various physiological parameters and lung breath sound signals at different ranges, such as standard values ​​corresponding to the normal physiological indicator range (i.e., standard values ​​corresponding to the signal intensity of various physiological parameters and the aforementioned lung breath sound signals within the normal range), and standard values ​​corresponding to the abnormal physiological indicator range when suctioning is required (i.e., standard values ​​corresponding to the signal intensity of various physiological parameters and the aforementioned lung breath sound signals within the suctioning range). Thus, this embodiment requires data alignment, normalization, and other preprocessing procedures before generating multimodal feature data.

[0061] Specifically, the generation process of multimodal feature data is as follows: A cloud server first aligns the physiological parameters and the lung breath sound signal with standard physiological index data to obtain standard values ​​for the physiological parameters and the lung breath sound signal, respectively. As previously explained, the standard physiological index data package contains standard values ​​for each physiological parameter and the lung breath sound signal within different ranges. Therefore, during use, based on the received values ​​of each physiological parameter and the signal intensity of the lung breath sound signal (which can be the average intensity of the entire signal), the standard physiological index data is aligned with the standard data. The corresponding standard values ​​can be obtained by matching the data. After data alignment is completed, the cloud server embeds the standard values ​​corresponding to the physiological parameters and the standard values ​​corresponding to the lung breath sound signals to obtain an embedding vector. Then, the physiological parameters and the lung breath sound signals are normalized to obtain a normalized physiological parameter vector and a normalized lung breath sound vector. Finally, the cloud server concatenates the embedding vector, the normalized physiological parameter vector, and the normalized lung breath sound vector to obtain the multimodal feature data.

[0062] In this embodiment, the clinical score, namely the standard values ​​of the aforementioned physiological parameters and lung breath sound signals (discrete classification data), is transformed into a continuous vector through an embedding layer. This vector is then concatenated with the normalized physiological parameters (numerical time-series data) to form multimodal input features.

[0063] Meanwhile, taking lung breath sound signals as an example, the aforementioned normalization process is illustrated as follows: First, the mean and standard deviation of the lung breath sound signal intensity are calculated; then, for any signal point in the lung breath sound signal, the mean signal intensity is subtracted from the signal intensity of that signal point, and then the difference is divided by the standard deviation to obtain the normalized signal point; of course, the normalization process for each physiological parameter in the current monitoring period is the same, and will not be elaborated here.

[0064] In this way, integrating physiological parameters (numerical time-series data) with clinical scores (discrete data) through feature embedding can improve the model's ability to represent complex rehabilitation states.

[0065] After integrating the multimodal vital signs data, the probability of lung respiratory health can be calculated by inputting the multimodal features into the trained LSTM model to obtain the probability of lung respiratory health of the target user in the current period.

[0066] In practical applications, this embodiment uses the multimodal feature data of several sample users as input and the lung respiratory health probability value of each sample user as output to train the LSTM model and obtain the trained LSTM model. The process of generating the multimodal feature data is the same as the process of generating the multimodal feature data mentioned above, and will not be repeated here.

[0067] In this embodiment, the model structure of the LSTM model can be found in [reference needed]. Figure 2 As shown, it includes an input layer, an LSTM layer, and a fully connected layer. Physiological parameters and clinical scale data (i.e., the aforementioned standard physiological indicator data) are used as inputs. After time-series data preprocessing, namely the aforementioned data alignment, normalization, embedding vectorization, and vector concatenation, the data is input into the aforementioned input layer. Then, it is transmitted to the LSTM layer for processing. Finally, the corresponding probability of lung respiratory health is output through the fully connected layer.

[0068] Furthermore, for example, the LSTM layer uses two stacked layers, each with 128 neurons, and the activation function is tanh; at the same time, the fully connected layer outputs three probabilities of lung respiratory health, each probability corresponding to a category, and this embodiment takes the highest probability as the lung respiratory health probability of the target user.

[0069] Furthermore, the loss function of the publicly trained LSTM model is shown in Equation (1) below.

[0070]

[0071] In equation (1), γ represents the loss function, p i,c Let y represent the probability that the i-th sample user in a set of sample users output by the LSTM model belongs to the c-th class, where C represents the total number of classes. i,c P represents the true label of the i-th sample user belonging to the c-th class. t P represents the maximum probability value among all categories for the i-th sample user at time t. t-1Let λ represent the maximum probability value among all categories for the i-th sample user at time t-1, T represent the collection duration of the multimodal feature data for the i-th sample user, N represent the total number of sample users, and λ represent the smoothing coefficient.

[0072] In practical applications, as explained above, physiological parameters and lung sound signals are collected according to a monitoring cycle. Therefore, when the LSTM model processes the data, it divides the entire input data (i.e., multimodal feature data) into multiple data segments. Each data segment corresponds to a monitoring time window within the monitoring cycle (the length of which is set according to actual use). Thus, the LSTM model processes the data in chronological order. That is, for each monitoring time window (in this embodiment, time is the unit), it will obtain three categories of lung respiratory health probability values. Then, the highest probability value is taken as the probability value of the sample user in the current monitoring time window (i.e., time t).

[0073] Thus, as can be seen from the aforementioned formula (1), this embodiment uses cross-entropy loss + temporal consistency constraint term to construct the loss function of the LSTM model; based on this, the LSTM model is trained based on the aforementioned loss function to obtain the trained LSTM model; finally, the multimodal feature data of the target user is input into the trained LSTM model to obtain the probability of the target user's lung respiratory health in the current monitoring period.

[0074] After obtaining the probability of lung respiratory health, this embodiment can also generate a probability threshold for each individual, thereby avoiding the drawback of low applicability of traditional technologies that use a fixed threshold for all users; the process of generating the probability threshold is as follows:

[0075] The cloud server is also used to generate the probability threshold for the current monitoring period based on the historical probability threshold data of the target user within a preset historical time period. In specific applications, the historical probability threshold data includes the historical probability thresholds corresponding to each historical monitoring period. The preset historical time period can be set to one day, such as the historical probability threshold data for each historical monitoring period within the past day before the current monitoring period. Thus, due to individual differences among different users, their historical probability thresholds within each historical monitoring period are different. Therefore, deriving the probability threshold for the current monitoring period based on the historical probability threshold is equivalent to considering the individual differences among users to generate a probability threshold for each user.

[0076] Specifically, this embodiment uses the Bayesian optimization principle to update the probability threshold. The calculation process for the probability threshold within the current monitoring period is as follows:

[0077] The cloud server first obtains the target user's lung respiratory health probability within a preset time period prior to the current moment, and determines the false alarm rate and false negative rate based on the lung respiratory health probability. In this embodiment, the current moment is the start time of the current monitoring period, and its preset time period can be set to 24 hours. At the same time, within the preset time period prior to the current moment, medical staff can derive the true lung respiratory health category based on the monitored physiological data and lung breath sound signals. Therefore, by combining the true data and the predicted health probability (as explained above, one probability corresponds to one category, and the highest probability is taken as the final model output, thus obtaining the lung respiratory health probability is equivalent to obtaining the respiratory health category), the false alarm rate and false negative rate are obtained.

[0078] After obtaining the aforementioned false positive rate and false negative rate, the cloud server can use them to calculate the update parameters. For example, but not limited to, the weighted sum of the false positive rate and false negative rate can be calculated to obtain the update parameters. That is, the weight of the false positive rate is set to α, and the weight of the false negative rate is 1-α. Then, the two are multiplied by various weights and the sum is obtained to obtain the update parameters.

[0079] After obtaining the update parameters, the objective function for probability updates can be constructed so that the probability threshold can be updated based on the constructed objective function, that is, the probability threshold for the current monitoring period can be obtained. Specifically, the cloud server is used to construct the probability update objective function based on the historical probability threshold data; then, the probability threshold within the current monitoring period is determined using the update parameters and the probability update objective function.

[0080] Optionally, the aforementioned probability update objective function can be, but is not limited to, the one shown in Equation (2) below.

[0081] u(x)=μ(x)+κ·σ(x) (2)

[0082] In equation (2), u(x) represents the probability update objective function, μ(x) represents the probability threshold prediction mean, σ(x) represents the probability threshold prediction standard deviation, and κ represents the exploration coefficient. In this embodiment, μ(x) and σ(x) are obtained based on 24-hour historical probability threshold data. Specifically, the aforementioned equation (2) is essentially a function with the probability threshold x as the variable. The calculation process of the prediction mean and standard deviation is as follows:

[0083] (1) Obtain n historical observation data points, denoted as:

[0084] In the formula, x i Let y represent the i-th historical probability threshold. iIt is based on the loss within the monitoring period corresponding to the i-th historical probability threshold, which is the updated parameter within the monitoring period corresponding to the i-th historical probability threshold, i.e., the weighted sum of the false alarm rate and the false negative rate within the monitoring period corresponding to the i-th historical probability threshold.

[0085] (2) Stack all inputs and outputs into vectors and matrices respectively:

[0086] X = [x1, x2, ..., x n ] T Y = [y1, y2, ..., y n ] T

[0087] (3) Choose a kernel function to define the covariance between two points;

[0088] Let be the signal variance, and let l be the length scale, controlling the smoothness of the function's change. In the formula, x and x′ are elements in X, i.e., distinct from x, to calculate the covariance between two points.

[0089] (4) The prediction function for the mean and standard deviation is as follows;

[0090]

[0091] in:

[0092] It is the inverse matrix of the sum of the covariance matrix and the noise variance matrix of the historical data points; where the noise variance matrix is ​​the interference term, which can be taken as an empirical value or removed.

[0093] k(x) is an nx1 covariance vector.

[0094] k(x)=[k(x,x1),k(x,x2),...,k(x,x n )] T

[0095] K is an n x n kernel matrix (covariance matrix).

[0096]

[0097] y is a vector of historical observations, which is the vector formed by the losses within the monitoring period corresponding to each historical probability threshold.

[0098] Therefore, after constructing the probability update objective function, the probability threshold within the current monitoring period can be determined by combining the aforementioned update parameters; the detailed calculation process is disclosed below:

[0099] In this embodiment, the cloud server is first used to calculate the gradient of the probability update objective function; then, it is used to obtain the learning rate, and from the historical probability thresholds corresponding to each historical monitoring period, it selects the historical probability threshold corresponding to the most recent historical monitoring period as the target probability threshold; finally, it can be used to calculate the probability threshold within the current monitoring period based on the learning rate, the gradient of the probability update objective function, the update parameters, and the target probability threshold.

[0100] Optionally, the derivative of the probability update objective function with respect to x can be calculated to obtain the aforementioned gradient. After calculating the gradient of the probability update objective function, the historical probability threshold in the previous monitoring period can be combined to update the threshold, that is, to calculate the probability threshold in the current monitoring period. For example, a cloud server can be used, but is not limited to, to calculate the probability threshold in the current monitoring period according to the following formula (3).

[0101]

[0102] In equation (3), τ new τ represents the probability threshold within the current monitoring period. old This represents the target probability threshold. The gradient of the probability update objective function is represented by η, where η represents the learning rate and Objective represents the update parameters.

[0103] Therefore, by using the aforementioned method, the probability threshold for each user in the current monitoring period can be calculated, and the probability threshold for each user is dynamically adjusted; thus, the applicability of the method can be improved.

[0104] Thus, after obtaining the probability threshold of the target user within the current monitoring period, the probability threshold and the aforementioned probability of lung respiratory health can be sent to the medical terminal to provide accurate auxiliary information for medical staff.

[0105] Furthermore, the aforementioned system can also issue early warnings based on probability thresholds, the process of which is as follows:

[0106] Based on the range of the lung respiratory health probability, different prompt messages are generated. Specifically, when the lung respiratory health probability is less than 0.3, the lung respiratory health probability and corresponding multimodal vital signs data of the target user are automatically recorded, i.e., recorded to the electronic medical record. When the lung respiratory health probability is greater than or equal to 0.3 and less than 0.7, a follow-up examination prompt message is sent to the medical terminal. When the lung respiratory health probability is greater than or equal to 0.7, i.e., higher than the probability threshold, an audible and visual alarm is triggered, and an emergency plan is pushed to the medical terminal.

[0107] In addition, the entire system can also interface with the hospital's HIS system to update user status in real time.

[0108] Therefore, through the detailed description of the lung respiratory monitoring system based on LSTM neural network fusion, the present invention has the following beneficial effects:

[0109] (1) It realizes the dynamic monitoring of multiple vital signs parameters in coordination; at the same time, by generating multimodal feature data and using the LSTM model for data processing, it provides an effective data processing method for monitoring multiple vital signs parameters.

[0110] (2) Multimodal data fusion: Physiological parameters (numerical time series data) and clinical scores (discrete data) are integrated through feature embedding to improve the model's ability to represent complex rehabilitation states.

[0111] (3) Personalized dynamic threshold: Based on the health status probability distribution output by LSTM, combined with Bayesian optimization algorithm, the warning threshold is dynamically adjusted to realize the dynamic adjustment of the warning threshold based on the individual, thereby improving the applicability.

[0112] (4) End-to-end closed-loop system: integrates wearable devices, cloud servers and medical terminals to achieve full-process automation of "monitoring-analysis-early warning-rescue".

[0113] In one possible design, see Figure 3 As shown, the second aspect of this embodiment provides a lung respiratory monitoring method based on the fusion of multiple signs using an LSTM neural network. The method is executed on a cloud server in the lung respiratory monitoring system based on the fusion of multiple signs using an LSTM neural network described in the first aspect of the embodiment, and the operation of the method may be, but is not limited to, the steps S1 to S4 below.

[0114] S1. Acquire the physiological parameters of the target user during the current monitoring period uploaded by the physiological parameter monitoring device, and the lung breath sound signals of the target user during the current monitoring period uploaded by the respiratory signal acquisition device.

[0115] S2. Obtain standard physiological indicator data, and generate multimodal feature data based on the standard physiological indicator data, the physiological parameters, and the lung breath sound signals.

[0116] S3. Input the multimodal feature data into the trained LSTM model to obtain the probability of the target user's lung respiratory health in the current period.

[0117] S4. Based on the historical probability threshold data of the target user within a preset historical time period, generate the probability threshold for the current monitoring period, and send the lung respiratory health probability and the probability threshold to the medical terminal.

[0118] The working process, working details and technical effects of this embodiment can be found in the first aspect of the embodiment, and will not be repeated here.

[0119] The third aspect of this embodiment provides an electronic device as an example, including: a memory, a processor, and a transceiver connected in sequence, wherein the memory is used to store a computer program, the transceiver is used to send and receive messages, and the processor is used to read the computer program and execute the lung respiratory monitoring method based on LSTM neural network fusion as described in the second aspect of the embodiment.

[0120] For specific examples, the memory may include, but is not limited to, random access memory (RAM), read-only memory (ROM), flash memory, first-in-first-out (FIFO) memory, and / or first-in-last-out (FILO) memory, etc.; specifically, the processor may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor may be implemented using at least one hardware form of DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), PLA (Programmable Logic Array). The processor may also include a main processor and a coprocessor. The main processor, also known as the CPU (Central Processing Unit), is used to process data in the wake-up state; the coprocessor is a low-power processor used to process data in the standby state.

[0121] In some embodiments, the processor may integrate a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the screen. For example, the processor may not be limited to microprocessors of the STM32F105 series, reduced instruction set computer (RISC) microprocessors, x86 architecture processors, or processors with integrated neural network processing units (NPUs). The transceiver may be, but is not limited to, a Wi-Fi transceiver, a Bluetooth transceiver, a General Packet Radio Service (GPRS) transceiver, a ZigBee (a low-power LAN protocol based on the IEEE 802.15.4 standard) transceiver, a 3G transceiver, a 4G transceiver, and / or a 5G transceiver. Furthermore, the device may also include, but is not limited to, a power module, a display screen, and other necessary components.

[0122] The working process, working details and technical effects of the electronic device provided in this embodiment can be found in the first aspect of the embodiment, and will not be repeated here.

[0123] The fourth aspect of this embodiment provides a storage medium for storing instructions containing the lung respiratory monitoring method based on LSTM neural network fusion as described in the second aspect of the embodiment. That is, the storage medium stores instructions that, when executed on a computer, perform the lung respiratory monitoring method based on LSTM neural network fusion as described in the second aspect of the embodiment.

[0124] The storage medium refers to a carrier for storing data, which may include, but is not limited to, floppy disks, optical disks, hard disks, flash memory, USB flash drives, and / or memory sticks. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices.

[0125] The working process, working details and technical effects of the storage medium provided in this embodiment can be found in the first aspect of the embodiment, and will not be repeated here.

[0126] The fifth aspect of this embodiment provides a computer program product containing instructions that, when executed on a computer, cause the computer to perform the lung respiratory monitoring method based on LSTM neural network fusion as described in the second aspect of this embodiment, wherein the computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device.

[0127] Finally, it should be noted that the above description is merely a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A lung respiratory monitoring system based on LSTM neural network fusion of multiple signs, characterized in that, include: A physiological parameter monitoring device and a respiratory signal acquisition device are provided. The physiological parameter monitoring device is used to collect physiological parameters of a target user during the current monitoring period, and the respiratory signal acquisition device is used to collect lung breath sound signals of the target user during the current monitoring period. The physiological parameter monitoring device and the respiratory signal acquisition device are communicatively connected to a cloud server for transmitting the physiological parameters and the lung breath sound signals to the cloud server. The physiological parameters of the target user during the current monitoring period include: heart rate data, blood oxygen data, respiratory rate data, and body temperature data of the target user during the current monitoring period. A cloud server is used to acquire standard physiological indicator data and generate multimodal feature data based on the standard physiological indicator data, the physiological parameters, and the lung breath sound signals. The cloud server is used to align the physiological parameters and the lung breath sound signals with the standard data of physiological indicators to obtain the standard values ​​corresponding to the physiological parameters and the lung breath sound signals, respectively. The standard data of physiological indicators refers to a preset clinical score, which corresponds to the standard values ​​of each physiological parameter and the lung breath sound signal at different ranges. The standard data of physiological indicators is matched with the received values ​​of each physiological parameter and the signal intensity of the lung breath sound signal to obtain the standard values ​​corresponding to the physiological parameters and the lung breath sound signal. A cloud server is used to embed the standard values ​​corresponding to the physiological parameters and the standard values ​​corresponding to the lung breath sound signals into an embedding vector. A cloud server is used to normalize the physiological parameters and the lung breath sound signals to obtain a normalized physiological parameter vector and a normalized lung breath sound vector. A cloud server is used to concatenate the embedded vector, the normalized physiological parameter vector, and the normalized lung breath sound vector to obtain the multimodal feature data. A cloud server is used to input the multimodal feature data into the trained LSTM model to obtain the probability of the target user's lung respiratory health in the current period. The cloud server is also used to generate a probability threshold for the current monitoring period based on the historical probability threshold data of the target user within a preset historical time period, and to send the lung respiratory health probability and the probability threshold to the medical terminal.

2. The lung respiratory monitoring system based on LSTM neural network fusion according to claim 1, characterized in that, The LSTM model is trained by taking the multimodal feature data of several sample users as input and the lung respiratory health probability value of each sample user as output, and the trained LSTM model is obtained. The loss function of the trained LSTM model is: （1） In equation (1), Denotes the loss function, This represents the probability that the i-th sample user in a set of sample users output by the LSTM model belongs to the c-th class. This indicates the total number of categories. This represents the true label of the i-th sample user belonging to the c-th class. Let represent the maximum probability value among all categories for the i-th sample user at time t. Let represent the maximum probability value among all categories for the i-th sample user at time t-1. This represents the collection time of the multimodal feature data corresponding to the i-th sample user. This represents the total number of sample users, and This represents the smoothing coefficient.

3. The lung respiratory monitoring system based on LSTM neural network fusion according to claim 1, characterized in that, A cloud server is used to obtain the probability of a target user's lung respiratory health over a preset period of time prior to the current moment, and to determine the false alarm rate and false negative rate based on the probability of the target user's lung respiratory health. The cloud server is used to calculate the updated parameters based on the false positive rate and the false negative rate. A cloud server is used to construct a probability update objective function based on the historical probability threshold data; The cloud server is also used to determine the probability threshold within the current monitoring period using the update parameters and the probability update objective function.

4. The lung respiratory monitoring system based on LSTM neural network fusion according to claim 3, characterized in that, A cloud server is used to calculate the weighted sum of the false positive rate and the false negative rate to obtain the updated parameters.

5. A lung respiratory monitoring system based on LSTM neural network fusion according to claim 3, characterized in that, The probability update objective function is: （2） In equation (2), This represents the probability update objective function. This represents the probability threshold prediction mean. This represents the standard deviation of the probability threshold prediction, where... and It was obtained based on historical probability threshold data, and This represents the exploration coefficient.

6. A lung respiratory monitoring system based on LSTM neural network fusion according to claim 3, characterized in that, The historical probability threshold data includes the historical probability thresholds corresponding to each historical monitoring period. The cloud server is used to calculate the gradient of the probability update objective function. The cloud server is used to obtain the learning rate and filter out the historical probability threshold corresponding to the most recent historical monitoring period from the historical probability thresholds corresponding to each historical monitoring period, so as to use it as the target probability threshold. The cloud server is also used to calculate the probability threshold within the current monitoring period based on the learning rate, the gradient of the probability-updated objective function, the update parameters, and the target probability threshold.

7. A lung respiratory monitoring system based on LSTM neural network fusion according to claim 6, characterized in that, The cloud server is used to calculate the probability threshold within the current monitoring period according to the following formula (3); （3） In equation (3), This represents the probability threshold within the current monitoring period. This represents the target probability threshold. This represents the gradient of the probability update objective function. Indicates the learning rate. This refers to the updated parameters.

8. A method for monitoring lung respiration based on LSTM neural network fusion of multiple signs, characterized in that, The method is executed on a cloud server in the LSTM neural network-based multi-sign fusion lung respiratory monitoring system according to any one of claims 1 to 7, wherein the method includes: Acquire the physiological parameters of the target user uploaded by the physiological parameter monitoring device during the current monitoring period, and the lung breath sound signals of the target user uploaded by the respiratory signal acquisition device during the current monitoring period; Obtain standard data of physiological indicators, and generate multimodal feature data based on the standard data of physiological indicators, the physiological parameters and the lung breath sound signals; The multimodal feature data is input into the trained LSTM model to obtain the probability of the target user's lung respiratory health in the current period; Based on the historical probability threshold data of the target user within a preset historical time period, a probability threshold for the current monitoring period is generated, and the lung respiratory health probability and the probability threshold are sent to the medical terminal.

Images