Vital sign monitoring method and device and electronic equipment
By extracting the time and frequency domain characteristics of fiber sensor signals and combining with machine learning models, the problem of low accuracy of fiber sensor monitoring results is solved, and more accurate and reliable vital sign monitoring is achieved.
Patent Information
- Application Number
- CN202411491873.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-24
- Publication Date
- 2024-11-22
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
When monitoring vital signs based on fiber optic sensors, the signal is processed only through filtering, resulting in low accuracy of monitoring results, and failure to fully consider the signal characteristics of the time and frequency domains, affecting the accuracy of the monitoring results.
By obtaining the vital sign signals collected by optical fiber sensors, extracting the time domain signal and frequency domain signal characteristics, combining the vital sign monitoring model obtained by machine learning, the time domain and frequency domain signals are processed by convolutional neural network to improve the monitoring results accuracy.
It significantly improves the accuracy and robustness of vital sign monitoring results, and can accurately monitor heart rate and breathing rate under complex environments and individual differences, enhancing the adaptability and robustness of the system.
Smart Images

Figure CN118986307A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of vital sign monitoring, and in particular to a vital sign monitoring method, device and electronic equipment. Background Art
[0002] Fiber optic sensors have the advantages of high sensitivity, anti-electromagnetic interference, small size, safety, and versatility. They are widely used in industry, medical care, communications and other fields. Vital sign monitoring technology based on fiber optic sensors is a relatively new type of non-skin contact vital sign monitoring technology. Its small size and easy placement also make it suitable for most people's usage scenarios and actual needs. However, in related technologies, when monitoring vital signs based on fiber optic sensors, vital sign signals are only processed by filtering, and the factors considered are not comprehensive, resulting in low accuracy of the life-enhancing monitoring results.
[0003] To address the above-mentioned problems, no effective solution has been proposed yet. Summary of the invention
[0004] The embodiments of the present invention provide a vital sign monitoring method, device and electronic device to at least solve the technical problem of low accuracy of monitoring results existing in the vital sign monitoring method in the related art.
[0005] According to one aspect of an embodiment of the present invention, a vital sign monitoring method is provided, including: acquiring a vital sign signal of a target object collected based on an optical fiber sensor; extracting time domain signal features in the vital sign signal to obtain a target time domain signal; extracting frequency domain signal features in the vital sign signal to obtain a target frequency domain signal; and obtaining a vital sign monitoring result of the target object using a vital sign monitoring model based on the target time domain signal and the target frequency domain signal, wherein the vital sign monitoring model is obtained through machine learning based on time domain signals, frequency domain signals and vital sign monitoring results respectively corresponding to multiple vital sign signals.
[0006] According to another aspect of an embodiment of the present invention, a vital sign monitoring device is also provided, including: a signal acquisition module, used to acquire a vital sign signal of a target object collected based on an optical fiber sensor; a first signal extraction module, used to extract time domain signal features in the vital sign signal to obtain a target time domain signal; a second signal extraction module, used to extract frequency domain signal features in the vital sign signal to obtain a target frequency domain signal; a result acquisition module, used to obtain a vital sign monitoring result of the target object based on the target time domain signal and the target frequency domain signal using a vital sign monitoring model, wherein the vital sign monitoring model is obtained through machine learning based on the time domain signals, frequency domain signals and vital sign monitoring results respectively corresponding to multiple vital sign signals.
[0007] According to another aspect of an embodiment of the present invention, a non-volatile storage medium is provided, wherein the non-volatile storage medium stores a plurality of instructions, wherein the instructions are suitable for being loaded by a processor and executing any one of the vital sign monitoring methods.
[0008] According to another aspect of an embodiment of the present invention, there is also provided an electronic device, comprising one or more processors and a memory, wherein the memory is used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement any one of the vital signs monitoring methods.
[0009] In an embodiment of the present invention, a vital sign signal of a target object collected based on an optical fiber sensor is acquired; time domain signal features in the vital sign signal are extracted to obtain a target time domain signal; frequency domain signal features in the vital sign signal are extracted to obtain a target frequency domain signal; based on the target time domain signal and the target frequency domain signal, a vital sign monitoring model is adopted to obtain a vital sign monitoring result of the target object, wherein the vital sign monitoring model is based on time domain signals, frequency domain signals and vital sign monitoring results respectively corresponding to multiple vital sign signals, and is obtained through machine learning, thereby achieving the purpose of using the time domain signal and frequency domain signal extracted from the vital sign signal as the input of the model and obtaining the vital sign monitoring result by way of model prediction, thereby achieving the technical effect of improving the accuracy of the vital sign monitoring result, and further solving the technical problem of low accuracy of monitoring results existing in the vital sign monitoring method in the related art. BRIEF DESCRIPTION OF THE DRAWINGS
[0010] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:
[0011] Figure 1 is a flow chart of a vital sign monitoring method according to an embodiment of the present invention;
[0012] Figure 2 is a flow chart of an optional vital sign monitoring method according to an embodiment of the present invention;
[0013] Figure 3 is a schematic diagram of an optional vital sign signal waveform according to an embodiment of the present invention;
[0014] Figure 4 is a schematic diagram of an optional time domain signal waveform based on breathing according to an embodiment of the present invention;
[0015] Figure 5 is a schematic diagram of an optional time domain signal waveform based on heart rate according to an embodiment of the present invention;
[0016] Figure 6 is a schematic diagram of an optional FFT sequence according to an embodiment of the present invention;
[0017] Figure 7 is a schematic diagram of a vital sign monitoring device according to an embodiment of the present invention. DETAILED DESCRIPTION
[0018] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.
[0019] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.
[0020] First, to facilitate understanding of the embodiments of the present invention, some terms or nouns involved in the present invention are explained below:
[0021] The convolution layer slides the convolution kernel over the signal to identify and extract local features in the time domain signal. These features can be specific waveforms, frequency patterns, or intensity changes of the signal, especially features related to heart rate. The convolution operation can capture the periodicity and temporal dependence of the signal.
[0022] The batch normalization layer (Batch Normalization, BN) is used to normalize the features of the convolutional layer output to make the distribution of features more stable, reduce the problem of internal covariate shift, speed up network learning, help the network converge better during training, and improve the stability of the model.
[0023] The pooling layer reduces the dimension of the feature map by performing local operations on the feature map, such as maximum pooling or average pooling, thereby reducing the amount of computation and memory requirements while retaining the most important feature information. The pooling operation can further enhance the model's anti-interference ability, allowing it to maintain good performance on input signals of different scales.
[0024] Fast Fourier Transform (FFT) is a fast algorithm for calculating discrete Fourier transform. It can convert signals from time domain to frequency domain and is used to analyze the spectral characteristics of signals. The FFT algorithm makes frequency domain analysis of large-scale data more efficient and faster by reducing the amount of calculation and improving efficiency.
[0025] Vital sign monitoring is the operation of monitoring human physiological characteristics, such as heart rate, respiratory rate, body temperature and blood pressure. At present, vital sign monitoring technology is mainly divided into skin contact and non-skin contact vital sign sensing technology. Compared with contact technology, non-contact vital sign monitoring technology is not as accurate and has weaker anti-interference ability than contact technology. Skin contact instruments generally need to be worn or worn on the user and may cause user discomfort in the case of long-term monitoring (such as sleep monitoring of polygraphs). In addition, some technologies require the use of specific occasions and equipment and the operation of professionals to complete the monitoring. Compared with contact technology, non-skin contact vital sign sensing technology does not require direct contact with the body, does not require the restriction of professionals or places, increases practicality, reduces user discomfort, improves comfort, and provides a more convenient and feasible monitoring method, which is suitable for the needs of ordinary users and the mass market.
[0026] Fiber optic sensors have the advantages of high sensitivity, anti-electromagnetic interference, small size, safety, and versatility. They are widely used in industry, medical care, communications and other fields. Vital sign monitoring technology based on fiber optic sensors is a relatively new type of non-skin contact vital sign monitoring technology. Its small size and easy placement also make it suitable for most people's usage scenarios and actual needs. However, in related technologies, when monitoring vital signs based on fiber optic sensors, vital sign signals are only processed by filtering, and the factors considered are not comprehensive, resulting in low accuracy of the life-enhancing monitoring results.
[0027] A method for analyzing and processing vital sign signals is proposed in the related art. The method uses a filter to denoise the collected original signal to obtain a time domain signal to calculate the first vital sign parameter, performs time-frequency transformation on the time domain signal of a preset time length to obtain a frequency domain signal to calculate the second vital sign parameter, and combines the first vital sign parameter with the second vital sign parameter according to the quality of the time domain signal and the frequency domain signal to obtain the final vital sign parameter. Although the scheme also combines the time domain and frequency domain signal processing methods, in the time domain signal processing, in order to obtain the ballistocardiography (BCG) heart rate time domain signal, the original signal is only filtered and denoised by a filter in the physiological frequency bandwidth range required for heart rate calculation; and the denoised original signal is scaled to obtain the BCG time domain signal; when the actual user is in a non-stationary state or the user's heart rate is different from that of ordinary people or in a real collection situation with a thick mattress and environmental noise, it is difficult to obtain an excellent heart rate time domain waveform signal through this simple method, and it is also difficult to perform the subsequent time domain related calculation operation of the scheme in the related art, and the final heart rate accuracy will also be reduced.
[0028] A method for processing vital signs signals is also proposed in the related art. After collecting the vital signs signals of the tested person, the method first performs Infinite Impulse Response (IIR) low-pass filtering to improve the signal-to-noise ratio of the vital signs signals; then the interference signal is processed by IIR low-pass filtering, baseline removal and Fourier transformation in sequence to obtain the maximum value of the spectrum in the interference signal, and a notch filter is selected according to the maximum value of the spectrum to process the vital signs signal, thereby eliminating the influence of interference on the vital signs signal. Then, the distinguishability of the vital signs signal in the time domain is improved by adaptive enhancement processing of the autocorrelation of the processed respiratory signal and the adaptive spectral line enhancement algorithm. Although this scheme also combines the two signal processing methods of time domain and frequency domain, its main design purpose is to determine the life status of trapped people in the wild. The main technical purpose is also to determine the maximum possible frequency of the environmental signal in the wild and then perform notch processing, and then extract the respiratory characteristics of the human body after strengthening the periodic signal to determine the life status of the person.
[0029] From the above, it can be seen that in the related art, when vital signs monitoring is performed based on optical fiber sensors, the vital signs signals are only processed by filtering, and the factors considered are not comprehensive, resulting in low accuracy of the life-enhancing monitoring results.
[0030] In response to the above problems, an embodiment of the present invention provides a method embodiment of vital signs monitoring. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.
[0031] Figure 1 is a flow chart of a method for monitoring vital signs according to an embodiment of the present invention. Figure 1 As shown, the method comprises the following steps:
[0032] Step S102, acquiring a vital sign signal of the target object collected by the optical fiber sensor;
[0033] Optionally, by using an optical fiber sensor instead of a traditional skin contact sensor to collect the target object's vital sign signal, direct contact with the user's body is reduced, and the comfort of the monitoring process is improved, which is particularly suitable for bedridden scenarios or other situations where contact monitoring is inconvenient. The vital sign signal can be a fragment of the original optical power signal collected by the optical fiber sensor.
[0034] Step S104, extracting the time domain signal features in the vital sign signal to obtain the target time domain signal;
[0035] Optionally, the time domain signal feature may be a time domain signal feature based on breathing, or may be a time domain signal feature based on heart rate.
[0036] In an optional embodiment, when the time domain signal feature is a time domain signal feature based on breathing, the time domain signal feature in the vital sign signal is extracted to obtain a target time domain signal, including: performing low-pass filtering on the vital sign signal to obtain a target time domain signal.
[0037] It is understandable that the respiratory frequency is usually low, generally between 0.1 Hz and 1 Hz. Low-pass filtering can effectively remove noise signals above the respiratory frequency range, such as heart rate, muscle activity, environmental vibration, etc., so that the extracted signal is more focused on the respiratory characteristics, improving the purity and pertinence of the signal. Through low-pass filtering, high-frequency noise can be effectively reduced or eliminated, and the signal-to-noise ratio of the respiratory signal can be improved. In practical applications, especially in non-ideal environments, such as bedridden scenes, various interference sources (such as the elasticity of the mattress, environmental vibration, etc.) may cause signal quality to deteriorate. Low-pass filtering ensures the clarity of the respiratory signal by filtering out these high-frequency interference signals, providing a more reliable basis for subsequent analysis and calculation.
[0038] Optionally, the vital sign signal can be subjected to low-pass filtering below 0.8 Hz, but is not limited to filtering out high-frequency components above 0.8 Hz, retaining low-frequency components, and obtaining a time domain signal of the characteristic components of the respiratory waveform as the target time domain signal. This can remove high-frequency noise, make the signal smoother, and highlight low-frequency features.
[0039] In an optional embodiment, when the time domain signal feature is a time domain signal feature based on heart rate, the time domain signal feature in the vital sign signal is extracted to obtain a target time domain signal, including: performing bandpass filtering on the vital sign signal to obtain a target time domain signal.
[0040] Optionally, the vital sign signal may be bandpass filtered using, but not limited to, a bandpass Butterworth filter. The vital sign signal may be bandpass filtered at 5Hz-9Hz to obtain a time domain signal containing characteristic components of the heart rate waveform for determining the target frequency domain signal. By performing a 5Hz-9Hz bandpass Butterworth filter on the vital sign signal, only the frequency components between 5Hz and 9Hz can be retained, and signals of other frequencies can be filtered out. This can highlight the frequency characteristics within the range of 5Hz to 9Hz, filter out interference signals of other frequencies, and make the signal clearer and more accurate.
[0041] It should be noted that the bandpass Butterworth filter can selectively retain the frequency components between 5Hz and 9Hz from the original signal, which is the typical frequency range of the heart rate signal. In the above way, low-frequency interference (such as breathing frequency) below 5Hz and high-frequency noise above 9Hz can be effectively removed, thereby purifying the signal and making it more suitable for subsequent processing. The design characteristics of the Butterworth filter ensure that the signal distortion within the passband is small, which helps to maintain the original waveform characteristics of the heart rate signal and ensure that the extracted heart rate signal is both accurate and has good waveform quality. Through the bandpass Butterworth filtering process, the accuracy of calculating the heart rate from the fiber optic sensor data can be significantly improved. The filtered signal is clearer, and the characteristic peaks and troughs of the heart rate are more obvious, which provides higher quality input for subsequent feature extraction and neural network training, thereby improving the accuracy of model prediction. In addition, because the Butterworth filter can resist the influence of environmental noise and other interference signals to a certain extent, it can extract effective heart rate signals even in the case of poor signal quality or complex environment. This enables the entire vital signs monitoring system to maintain high performance and stability in different scenarios, enhancing the robustness of the system.
[0042] Step S106, extracting frequency domain signal features from the vital sign signal to obtain a target frequency domain signal;
[0043] Optionally, the frequency domain signal feature may be a frequency domain signal feature based on breathing, or may be a frequency domain signal feature based on heart rate.
[0044] In an optional embodiment, frequency domain signal features in the vital sign signal are extracted to obtain a target frequency domain signal, including: adding a Hamming window to the vital sign signal to obtain an added result; and performing fast Fourier transform processing on the added result to obtain a target frequency domain signal.
[0045] Optionally, adding a Hamming window to the vital sign signal before performing a Fast Fourier Transform (FFT) can reduce spectral leakage caused by signal truncation. Spectral leakage refers to the fact that truncation of the signal in the time domain introduces additional frequency components, causing the original single-frequency signal to appear as multiple frequencies in the frequency domain, thereby reducing the frequency resolution and signal clarity. The Hamming window can smooth the edges of the signal, reduce this leakage, and make the spectrum more accurate. The use of the Hamming window helps to improve the frequency resolution of the FFT result, making it possible to more accurately identify specific frequency components in the vital sign signal, which is crucial for accurately extracting information such as heart rate and respiratory rate.
[0046] By using FFT to convert the vital signs signal in the time domain into a frequency domain signal, the frequency components and energy distribution of the signal in the frequency domain become more obvious, which is convenient for subsequent feature analysis and signal recognition. Through FFT transformation, the respiratory frequency and heart rate frequency in the signal can be identified, which can be achieved by observing the frequency points with the most concentrated energy in the spectrum. For vital signs monitoring, FFT results can help quickly locate and identify key frequency information corresponding to heart rate and respiratory rate.
[0047] In an optional embodiment, the added result is processed by fast Fourier transform to obtain a target frequency domain signal, including: performing fast Fourier transform processing on the added result to obtain a first frequency domain signal; when the frequency domain signal characteristic is a frequency domain signal characteristic based on breathing, a signal in the first frequency domain signal that is within a preset first frequency range is used as the target frequency domain signal; or when the frequency domain signal characteristic is a frequency domain signal characteristic based on heart rate, a signal in the first frequency domain signal that is within a preset second frequency range is used as the target frequency domain signal, wherein the lower limit value of the preset second frequency range is greater than or equal to the upper limit value of the preset first frequency range.
[0048] Optionally, the preset first frequency range may be below 0.5 Hz, and the preset second frequency range may be above 0.5 Hz. It should be noted that the frequency of breathing is usually in a lower frequency band (approximately 0.12 Hz to 0.33 Hz), which means that the energy of the respiratory signal in the frequency domain is mainly concentrated on these frequencies. By screening the signal in the preset first frequency range in the first frequency domain signal, the respiratory signal can be more accurately identified and extracted, thereby improving the accuracy of the respiratory frequency calculation. This screening can filter out the interference of other frequency components, making the respiratory signal more prominent and clear. Compared with the respiratory signal, the frequency of the heart rate signal is usually in a higher frequency band (approximately 0.8 Hz to 3 Hz). This means that when extracting the heart rate signal, the frequency domain signal features that need to be paid attention to are in different frequency ranges. By setting a preset second frequency range and extracting the signal from it, the heart rate signal can be effectively separated, the interference of other frequency components (such as the respiratory signal) can be reduced, and the accuracy of the heart rate calculation can be improved. The lower limit value of the preset second frequency range is greater than or equal to the upper limit value of the preset first frequency range, ensuring that the respiratory and heart rate signals do not overlap in the frequency domain, thereby enhancing the distinguishability and recognition capabilities of each signal. In the above manner, the features of the respiratory and heart rate signals can be effectively separated and extracted from complex vital sign signals, and then more accurate vital sign parameter calculations can be performed through subsequent signal processing steps (such as convolutional neural network analysis), thereby improving the robustness and accuracy of monitoring.
[0049] Optionally, an FFT operation is performed on the vital sign signal after adding a Hamming window, and only the part before 3 Hz is considered according to the normal range of human respiration and heart rate, to obtain an FFT sequence D (two-dimensional including frequency sequence and energy sequence) as the first frequency domain signal. If the frequency domain signal feature is a frequency domain signal feature based on respiration, that is, the respiration frequency domain signal is calculated, then the sequence D takes the part of the preset first frequency range (such as below 0.5 Hz) as the target frequency domain signal; if the frequency domain signal feature is a frequency domain signal feature based on heart rate, that is, the heart rate frequency domain signal is calculated, then the sequence D only takes the part of the preset second frequency range (such as above 0.5 Hz).
[0050] Step S108, based on the target time domain signal and the target frequency domain signal, a vital sign monitoring model is used to obtain the vital sign monitoring results of the target object, wherein the vital sign monitoring model is obtained through machine learning based on the time domain signals, frequency domain signals and vital sign monitoring results corresponding to multiple vital sign signals.
[0051] Optionally, the vital signs monitoring model not only considers time domain signals, but also combines frequency domain signals, which enables the model to capture vital signs information from different angles, enhancing the robustness of the model, and accurately monitoring vital signs even in complex environments or individual differences among users, to obtain more accurate and reliable vital signs monitoring results. By combining time domain and frequency domain features, and using intelligent processing of neural networks, vital signs information such as heart rate and / or respiratory rate can be more accurately extracted from the signals of optical fiber sensors as vital signs monitoring results, and the accuracy of the monitoring results is significantly improved compared to the method of using only a single domain signal in related technologies.
[0052] Optionally, it can be based on but not limited to the time domain signals, frequency domain signals and vital sign monitoring results corresponding to multiple vital sign signals, and use the vital sign monitoring results as labels. By accumulating a large amount of real labeled data and training, the vital sign monitoring model can adapt to the vital sign signals of different users and have good adaptability and generalization ability. Even when the user is in a non-stationary state or the signal is affected by environmental noise, the heart rate and respiratory rate can still be accurately calculated.
[0053] In an optional embodiment, when the vital sign monitoring model includes a first convolutional network, a second convolutional network and a fully connected network, the vital sign monitoring model is used based on the target time domain signal and the target frequency domain signal to obtain the vital sign monitoring results of the target object, including: using the first convolutional network to extract features of the target time domain signal to obtain a first feature tensor; using the second convolutional network to extract features of the target frequency domain signal to obtain a second feature tensor; splicing the first feature tensor and the second feature tensor to obtain a spliced feature tensor; based on the spliced feature tensor, using a fully connected network to obtain the vital sign monitoring results of the target object.
[0054] Optionally, the first convolutional network is a one-dimensional convolutional network; and the second convolutional network is a two-dimensional convolutional network.
[0055] Optionally, the first convolutional network (one-dimensional convolution) focuses on capturing local features in the time domain signal, such as pulse shape, frequency, and intensity in the heart rate signal. Through the convolution operation, the network can identify key features related to vital signs in the time domain signal and convert them into high-dimensional feature representations (first feature tensor). This improves the efficiency and accuracy of feature extraction, helps the model better understand the vital signs information in the time domain signal, and thus improves monitoring accuracy. The second convolutional network (two-dimensional convolution) processes the frequency domain signal, which contains the frequency component information of the signal, which is particularly important for identifying frequency-based signs such as respiratory rate and heart rate. Through two-dimensional convolution, the model can capture the characteristics of different frequency components and how these components are distributed on the spectrum.
[0056] By fusing the feature tensors extracted in the time domain and frequency domain, we can combine the advantages of the two signal processing methods to form a more comprehensive description of vital signs. The spliced feature tensor (i.e., spliced feature tensor) contains complementary information of time domain and frequency domain features. Such feature representation is conducive to the model making more accurate decisions. Through this vital sign monitoring model, we can not only identify the shape and changes of the waveform, but also understand the performance of these changes in frequency, thereby improving the accuracy and comprehensiveness of vital sign monitoring results.
[0057] The fully connected network is used to convert the concatenated feature tensors into specific vital signs monitoring results, such as heart rate values or respiratory rate, etc. Through dense neuron connections, the network can learn the complex relationship between features and achieve mapping from features to results. The use of the fully connected network enables the vital signs monitoring model to learn the comprehensive judgment logic of vital signs from the concatenated high-dimensional feature tensors, so that it can accurately calculate vital signs parameters such as heart rate and respiratory rate, improving the intelligence and accuracy of the overall vital signs monitoring process.
[0058] In an optional embodiment, when the first convolutional network includes a first convolutional layer, a first batch of normalization layers, and a first pooling layer, the first convolutional network is used to perform feature extraction on the target time domain signal to obtain a first feature tensor, including: using the first convolutional layer to perform feature extraction on the target time domain signal to obtain a first feature extraction result; using the first batch of normalization layers to perform standardization processing on the first feature extraction result to obtain a first normalization result; using the first pooling layer to perform dimensionality reduction processing on the first normalization result to obtain a first feature tensor; when the second convolutional network includes a second convolutional layer, a second batch of normalization layers, and a second pooling layer, the second convolutional network is used to perform feature extraction on the target frequency domain signal to obtain a second feature tensor, including: using the second convolutional layer to perform feature extraction on the target frequency domain signal to obtain a second feature extraction result; using the second batch of normalization layers to perform standardization processing on the second feature extraction result to obtain a second normalization result; using the second pooling layer to perform dimensionality reduction processing on the second normalization result to obtain a second feature tensor.
[0059] Optionally, by setting the first convolutional network to include the first convolutional layer, the first batch of normalization layers, and the first pooling layer, and the second convolutional network to include the second convolutional layer, the second batch of normalization layers, and the second pooling layer, the first convolutional layer is used to extract features of the target time domain signal, and the periodicity and time-dependent features of the target time domain signal are captured to obtain the first feature extraction result; the first batch of normalization layers are further used to normalize the first feature extraction result, so that the distribution of features is more stable, the internal covariate offset problem is reduced, the network learning speed is accelerated, and the network is better converged during the training process, and the stability of the model is improved; the first pooling layer is further used to perform dimensionality reduction processing on the first normalization result obtained by the normalization processing, thereby reducing the amount of calculation and memory requirements, while retaining the most important feature information, and obtaining the first feature tensor. Based on the second convolutional layer, the second batch of normalization layers, and the second pooling layer included in the second convolutional network, the target frequency domain signal is subjected to the same operation as the target time domain feature to obtain the second feature tensor, and the time domain and frequency domain signals are processed by the first convolutional network and the second convolutional network respectively, so that the features of the signal can be captured from both the time and frequency perspectives. These features are subsequently integrated together through a splicing operation to form a more comprehensive input, allowing the model to use information from both the time domain and the frequency domain when making predictions, thereby improving the accuracy and robustness of the heart rate and respiratory rate calculations.
[0060] Through the above steps S102 to S108, the time domain signal and frequency domain signal extracted from the vital sign signal can be used as the input of the model together, and the vital sign monitoring result can be obtained by model prediction, thereby achieving the technical effect of improving the accuracy of the vital sign monitoring result, and further solving the technical problem of low accuracy of monitoring results existing in the vital sign monitoring method in the related technology.
[0061] Based on the above embodiments and optional embodiments, the present invention proposes an optional implementation mode: Figure 2 is a flow chart of an optional vital sign monitoring method according to an embodiment of the present invention. Figure 2 As shown, the method includes:
[0062] Step S1: The original optical power signal segment with a duration of t seconds and a sampling rate of fs collected by the optical fiber sensor is recorded as a vital sign signal A, such as Figure 3 As shown, Figure 3 : is an optional waveform diagram of a vital sign signal according to an embodiment of the present invention. The vital sign signal A is subjected to low-pass filtering below 0.8 Hz and band-pass Butterworth filtering of 5 Hz-9 Hz, respectively, to obtain a time domain signal B containing a characteristic component of a respiratory waveform, and a time domain signal C containing a characteristic component of a heart rate waveform, wherein both the time domain signal B and the time domain signal C are one-dimensional time domain waveform sequences, Figure 4is a schematic diagram of an optional time domain signal waveform based on breathing according to an embodiment of the present invention, Figure 5 The figure is a schematic diagram of an optional heart rate-based time domain signal waveform according to an embodiment of the present invention.
[0063] Step S2: After adding a Hamming window to the vital sign signal A, perform FFT operation, and only consider the part before 3Hz according to the normal range of human breathing and heart rate, and obtain the FFT sequence D (two-dimensional including frequency sequence and energy sequence, that is, two-dimensional frequency domain sequence). Figure 6 It is a schematic diagram of an optional FFT sequence according to an embodiment of the present invention.
[0064] Step S3: The following calculation logic is exactly the same for heart rate and respiration. In the following, heart rate calculation is used as an example. If heart rate is calculated, sequence D only takes the part above 0.5 Hz. If respiration is calculated, sequence D takes the part below 0.5 Hz.
[0065] Step S4: Perform continuous one-dimensional convolution operations on the time domain signal C to extract features. The one-dimensional convolution operation includes three parts: convolution layer, BN layer and pooling layer. After continuous convolution, a high-dimensional feature tensor output E is obtained.
[0066] Step S5: Perform continuous two-dimensional convolution operations on the frequency domain signal D to extract features. The two-dimensional convolution operation includes three parts: convolution layer, BN layer and pooling layer. After continuous convolution, a high-dimensional feature tensor output F is obtained.
[0067] Step S6: concatenate the high-dimensional features E and F respectively formed after convolution of the frequency domain signal D and the time domain signal C to form a new feature tensor G. Perform a fully connected layer operation on the feature tensor G to obtain the final output predict as the vital sign monitoring result.
[0068] Step S7: Build the Torch Pytorch model for the above steps S1 to S6, and train it with the accumulated data set. The PyTorch model is a neural network model built using the PyTorch deep learning framework. PyTorch is an open source deep learning framework that provides flexible tensor calculation and automatic differentiation functions, allowing users to easily build and train neural network models.
[0069] In this embodiment, by using optical fiber sensors instead of traditional skin contact sensors to collect vital sign signals, direct contact with the user's body is reduced, the comfort of the monitoring process is improved, and it is particularly suitable for bedridden scenes or other situations where contact monitoring is inconvenient. Extracting time domain signal and frequency domain signal features, this step includes filtering (low pass and band pass) and FFT transformation, the purpose of which is to remove noise, highlight the waveform characteristics of breathing and heart rate, and ensure the signal quality of subsequent processing, thereby improving the accuracy and reliability of the monitoring results. The vital sign monitoring model not only considers time domain signals, but also combines frequency domain signals, which enables the model to capture vital sign information from different angles and enhances the robustness of the model, that is, it can still accurately monitor vital signs in the face of complex environments or individual differences among users. The vital sign monitoring model obtained by machine learning, especially the neural network based on one-dimensional and two-dimensional convolution, can automatically learn the patterns and features in the signal without manually setting complex thresholds or parameters, reducing the difficulty of technical implementation and improving the intelligence level and accuracy of monitoring. This embodiment accumulates a large amount of real label data and conducts training, so that the vital sign monitoring model can adapt to the vital sign signals of different users, has good adaptability and generalization ability, and can still accurately calculate the heart rate and respiratory rate even when the user is in a non-stationary state or the signal is affected by environmental noise. By combining time domain and frequency domain features, and using intelligent processing of neural networks, this embodiment can more accurately extract heart rate and respiratory rate from the signal of the optical fiber sensor, and the accuracy of its monitoring results is significantly improved compared with the traditional method that only uses a single domain signal. The use of convolutional neural networks simplifies the feature extraction and calculation process, can quickly process a large amount of data, improves the efficiency of vital sign monitoring, and is suitable for real-time monitoring and complex data processing scenarios. In summary, this embodiment aims to provide an efficient, accurate and comfortable vital sign monitoring method through a non-contact optical fiber sensor, combined with time domain and frequency domain signal processing, and an intelligent model based on machine learning, especially in monitoring heart rate and respiratory rate. There are significant improvements and advantages.
[0070] In this embodiment, a vital sign monitoring device is also provided, which is used to implement the above-mentioned embodiments and preferred implementation modes, and the descriptions that have been made will not be repeated. As used below, the terms "module" and "device" can be a combination of software and / or hardware that implements a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.
[0071] According to an embodiment of the present invention, there is also provided an embodiment of a device for implementing the above-mentioned vital sign monitoring method. Figure 7 is a schematic diagram of the structure of a vital sign monitoring device according to an embodiment of the present invention. Figure 7As shown, the above-mentioned vital sign monitoring device includes: a signal acquisition module 200, a first signal extraction module 202, a second signal extraction module 204, and a result acquisition module 206, wherein:
[0072] The signal acquisition module 200 is used to acquire the vital sign signal of the target object collected by the optical fiber sensor;
[0073] A first signal extraction module 202, connected to the signal acquisition module 200, is used to extract time domain signal features from the vital sign signal to obtain a target time domain signal;
[0074] The second signal extraction module 204 is connected to the first signal extraction module 202 and is used to extract frequency domain signal features in the vital sign signal to obtain a target frequency domain signal;
[0075] The result acquisition module 206 is connected to the second signal extraction module 204, and is used to obtain the vital sign monitoring results of the target object based on the target time domain signal and the target frequency domain signal by using a vital sign monitoring model, wherein the vital sign monitoring model is based on the time domain signals, frequency domain signals and vital sign monitoring results corresponding to multiple vital sign signals, and is obtained through machine learning.
[0076] In the embodiment of the present invention, a signal acquisition module 200 is provided to acquire a vital sign signal of a target object collected based on an optical fiber sensor; a first signal extraction module 202 is connected to the signal acquisition module 200 to extract time domain signal features in the vital sign signal to obtain a target time domain signal; a second signal extraction module 204 is connected to the first signal extraction module 202 to extract frequency domain signal features in the vital sign signal to obtain a target frequency domain signal; a result acquisition module 206 is connected to the second signal extraction module 204 to obtain a vital sign monitoring result of the target object by adopting a vital sign monitoring model based on a target time domain signal and a target frequency domain signal, wherein the vital sign monitoring model is obtained by machine learning based on time domain signals, frequency domain signals and vital sign monitoring results corresponding to a plurality of vital sign signals, respectively, so as to achieve the purpose of using the time domain signal and frequency domain signal extracted from the vital sign signal as inputs of the model and obtaining the vital sign monitoring result by means of model prediction, thereby achieving the technical effect of improving the accuracy of the vital sign monitoring result, and further solving the technical problem of low accuracy of the monitoring result existing in the vital sign monitoring method in the related art.
[0077] It should be noted that the above modules can be implemented by software or hardware. For example, for the latter, it can be implemented in the following ways: the above modules can be located in the same processor; or the above modules can be located in different processors in any combination.
[0078] It should be noted that the signal acquisition module 200, the first signal extraction module 202, the second signal extraction module 204, and the result acquisition module 206 correspond to steps S102 to S108 in the embodiment, and the examples and application scenarios implemented by the modules and the corresponding steps are the same, but are not limited to the contents disclosed in the above embodiments. It should be noted that the modules as part of the device can be run in a computer terminal.
[0079] It should be noted that the optional or preferred implementation of this embodiment can refer to the relevant description in the embodiment, which will not be repeated here.
[0080] The above-mentioned vital signs monitoring device may also include a processor and a memory. The above-mentioned signal acquisition module 200, the first signal extraction module 202, the second signal extraction module 204, the result acquisition module 206, etc. are all stored in the memory as program modules, and the processor executes the above-mentioned program modules stored in the memory to realize the corresponding functions.
[0081] The processor includes a kernel, which retrieves the corresponding program module from the memory. The kernel may be one or more. The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. The memory includes at least one memory chip.
[0082] According to an embodiment of the present application, an embodiment of a non-volatile storage medium is also provided. Optionally, in this embodiment, the non-volatile storage medium includes a stored program, wherein when the program is executed, the device where the non-volatile storage medium is located is controlled to execute any of the above-mentioned vital sign monitoring methods.
[0083] Optionally, in this embodiment, the non-volatile storage medium may be located in any computer terminal in a computer terminal group in a computer network, or in any mobile terminal in a mobile terminal group, and the non-volatile storage medium includes a stored program.
[0084] Optionally, when the program is running, the device where the non-volatile storage medium is located is controlled to perform the following functions: obtain the vital sign signal of the target object collected based on the optical fiber sensor; extract the time domain signal characteristics in the vital sign signal to obtain the target time domain signal; extract the frequency domain signal characteristics in the vital sign signal to obtain the target frequency domain signal; based on the target time domain signal and the target frequency domain signal, use a vital sign monitoring model to obtain the vital sign monitoring result of the target object, wherein the vital sign monitoring model is based on the time domain signals, frequency domain signals and vital sign monitoring results corresponding to multiple vital sign signals, and is obtained through machine learning.
[0085] According to an embodiment of the present application, an embodiment of a processor is also provided. Optionally, in this embodiment, the processor is used to run a program, wherein the program executes any one of the above-mentioned vital sign monitoring methods when it is run.
[0086] According to an embodiment of the present application, an embodiment of a computer program product is also provided, which, when executed on a data processing device, is suitable for executing a program that initializes any one of the steps of the above-mentioned vital sign monitoring method.
[0087] Optionally, the above-mentioned computer program product, when executed on a data processing device, is suitable for executing a program that initializes the following method steps: obtaining a vital sign signal of a target object collected based on an optical fiber sensor; extracting time domain signal features in the vital sign signal to obtain a target time domain signal; extracting frequency domain signal features in the vital sign signal to obtain a target frequency domain signal; based on the target time domain signal and the target frequency domain signal, using a vital sign monitoring model to obtain a vital sign monitoring result of the target object, wherein the vital sign monitoring model is obtained through machine learning based on the time domain signals, frequency domain signals and vital sign monitoring results corresponding to multiple vital sign signals.
[0088] An embodiment of the present invention provides an electronic device, which includes a processor, a memory, and a program stored in the memory and executable on the processor. When the processor executes the program, the following steps are implemented: acquiring a vital sign signal of a target object collected based on an optical fiber sensor; extracting time domain signal features in the vital sign signal to obtain a target time domain signal; extracting frequency domain signal features in the vital sign signal to obtain a target frequency domain signal; and based on the target time domain signal and the target frequency domain signal, using a vital sign monitoring model to obtain a vital sign monitoring result of the target object, wherein the vital sign monitoring model is obtained through machine learning based on the time domain signals, frequency domain signals, and vital sign monitoring results corresponding to multiple vital sign signals.
[0089] The above sequence of the embodiments of the present invention is for description only and does not represent the superiority or inferiority of the embodiments.
[0090] In the above embodiments of the present invention, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.
[0091] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the above modules can be a logical function division. There may be other division methods in actual implementation, such as multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, modules or indirect coupling or communication connection of modules, which can be electrical or other forms.
[0092] The modules described above as separate components may or may not be physically separated, and the components shown as modules may or may not be physical modules, that is, they may be located in one place or distributed on multiple modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the present embodiment.
[0093] In addition, each functional module in each embodiment of the present invention may be integrated into one processing module, or each module may exist physically separately, or two or more modules may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of software functional modules.
[0094] If the above-mentioned integrated module is implemented in the form of a software function module and sold or used as an independent product, it can be stored in a computer-readable non-volatile storage medium. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art or the whole or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a non-volatile storage medium, including several instructions to enable a computer device (which can be a personal computer, server or network device, etc.) to perform all or part of the steps of the various embodiments of the present invention. The aforementioned non-volatile storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, disk or optical disk and other media that can store program codes.
[0095] The above are only preferred embodiments of the present invention. It should be pointed out that, for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.
Claims
1. A method for monitoring vital signs, characterized in that: include: Acquire vital sign signals of a target object collected by an optical fiber sensor; Extracting time domain signal features from the vital sign signal to obtain a target time domain signal; Extracting frequency domain signal features from the vital sign signal to obtain a target frequency domain signal; Based on the target time domain signal and the target frequency domain signal, a vital sign monitoring model is used to obtain the vital sign monitoring results of the target object, wherein the vital sign monitoring model is obtained through machine learning based on the time domain signals, frequency domain signals and vital sign monitoring results respectively corresponding to multiple vital sign signals.
2. The method according to claim 1, characterized in that In the case where the time domain signal feature is a time domain signal feature based on breathing, extracting the time domain signal feature in the vital sign signal to obtain a target time domain signal includes: The vital sign signal is subjected to low-pass filtering to obtain the target time domain signal.
3. The method according to claim 1, characterized in that In the case where the time domain signal feature is a time domain signal feature based on heart rate, extracting the time domain signal feature in the vital sign signal to obtain a target time domain signal includes: The vital sign signal is subjected to bandpass filtering to obtain the target time domain signal.
4. The method according to claim 1, characterized in that: Extracting frequency domain signal features from the vital sign signal to obtain a target frequency domain signal includes: Adding a Hamming window to the vital sign signal to obtain an added result; The adding result is subjected to fast Fourier transform processing to obtain the target frequency domain signal.
5. The method according to claim 4, characterized in that Performing fast Fourier transform processing on the added result to obtain the target frequency domain signal, including: Performing fast Fourier transform processing on the adding result to obtain a first frequency domain signal; In a case where the frequency domain signal feature is a frequency domain signal feature based on breathing, a signal in the first frequency domain signal that is within a preset first frequency range is used as the target frequency domain signal; or In the case where the frequency domain signal feature is a frequency domain signal feature based on heart rate, the signal in the first frequency domain signal within a preset second frequency range is used as the target frequency domain signal, wherein the lower limit value of the preset second frequency range is greater than or equal to the upper limit value of the preset first frequency range.
6. The method according to any one of claims 1 to 5, characterized in that In the case where the vital sign monitoring model includes a first convolutional network, a second convolutional network, and a fully connected network, based on the target time domain signal and the target frequency domain signal, the vital sign monitoring model is used to obtain a vital sign monitoring result of the target object, including: Using the first convolutional network to extract features from the target time domain signal to obtain a first feature tensor; Using the second convolutional network to extract features of the target frequency domain signal to obtain a second feature tensor; Performing concatenation processing on the first feature tensor and the second feature tensor to obtain a concatenated feature tensor; Based on the concatenated feature tensor, the fully connected network is used to obtain the vital sign monitoring results of the target object.
7. The method according to claim 6, characterized in that The first convolutional network is a one-dimensional convolutional network; the second convolutional network is a two-dimensional convolutional network.
8. The method according to claim 6, characterized in that In the case where the first convolutional network includes a first convolutional layer, a first batch of normalization layers, and a first pooling layer, the using the first convolutional network to perform feature extraction on the target time domain signal to obtain a first feature tensor includes: using the first convolutional layer to perform feature extraction on the target time domain signal to obtain a first feature extraction result; using the first batch of normalization layers to perform normalization processing on the first feature extraction result to obtain a first normalization result; using the first pooling layer to perform dimensionality reduction processing on the first normalization result to obtain the first feature tensor; In the case that the second convolutional network includes a second convolutional layer, a second batch normalization layer, and a second pooling layer, the using the second convolutional network to perform feature extraction on the target frequency domain signal to obtain a second feature tensor includes: using the second convolutional layer to perform feature extraction on the target frequency domain signal to obtain a second feature extraction result; using the second batch normalization layer to perform standardization processing on the second feature extraction result to obtain a second standardized result; using the second pooling layer to perform dimensionality reduction processing on the second standardized result to obtain the second feature tensor.
9. A vital signs monitoring device, characterized in that: include: A signal acquisition module, used to acquire the vital sign signal of the target object based on the optical fiber sensor; A first signal extraction module, used to extract the time domain signal features in the vital sign signal to obtain a target time domain signal; A second signal extraction module is used to extract frequency domain signal features in the vital sign signal to obtain a target frequency domain signal; The result acquisition module is used to obtain the vital sign monitoring results of the target object based on the target time domain signal and the target frequency domain signal using a vital sign monitoring model, wherein the vital sign monitoring model is obtained through machine learning based on the time domain signals, frequency domain signals and vital sign monitoring results corresponding to multiple vital sign signals.
10. An electronic device, characterized in that: It includes one or more processors and a memory, wherein the memory is used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the vital sign monitoring method described in any one of claims 1 to 8.
Citation Information
Patent Citations
Method and device of signal analyzing and processing of vital signs and vital signs monitoring equipment
CN108056769A
Physical sign signal analysis method and device and storage medium
CN114587301A
