Non-contact user state monitoring system and method

Abstract

The invention discloses a non-contact user state monitoring system and method. The system comprises a body temperature monitoring module, a heart rate monitoring module, a video acquisition module, a control module, a signal processing module, a storage module, a communication module, a display module, a power supply module, an alarm module and a mobile terminal. The signal processing module is used for processing the body temperature data information, the heart rate data information and the video data information to obtain a user state monitoring result, and the user state monitoring result is sent to the control module and the communication module. The user state monitoring result is sent to the mobile terminal through the communication module; the user state monitoring result is sent to the display module and the alarm module through the control module; using a display module to display the user state monitoring result; and the alarm module is used for carrying out alarm processing on the user state monitoring result. According to the method, the privacy of the user can be protected, the risk of cross infection is avoided, the operation of a monitoring instrument is reduced, and the safety is high.

Description

Technical Field

[0001] The present invention relates to the technical field of user status monitoring, and in particular to a non-contact user status monitoring system and method. Background Art

[0002] Currently, most of the commonly used user status monitoring devices are contact-based, and the measuring equipment will come into direct contact with human body parts, mainly relying on manual monitoring. However, this monitoring method has problems such as heavy nursing burden, low safety, and low management efficiency. Non-contact millimeter-wave radar and time-frequency monitoring data have obvious advantages. They can protect the privacy of users and do not contact users. They greatly facilitate people who need long-term continuous status monitoring, avoid the risk of cross-infection, and reduce the operation of monitoring instruments. They are particularly suitable for status monitoring of the elderly, children, mentally ill patients, and patients with large-area burns. Therefore, it is of great significance to study non-contact user status monitoring methods. Summary of the invention

[0003] The technical problem to be solved by the present invention is to provide a non-contact user status monitoring system and method, which utilizes non-contact body temperature monitoring, heart rate monitoring and video acquisition methods to obtain the user's body temperature data, heart rate data, blood pressure data and motion recognition results, and processes the body temperature data, heart rate data, blood pressure data and motion recognition results to obtain the user status monitoring results.

[0004] In order to solve the above technical problems, the first aspect of the embodiment of the present invention discloses a non-contact user status monitoring system, the system includes a body temperature monitoring module, a heart rate monitoring module, a video acquisition module, a control module, a signal processing module, a storage module, a communication module, a display module, a power module, an alarm module and a mobile terminal;

[0005] The control module is data-connected with the body temperature monitoring module, the heart rate monitoring module, the video acquisition module, the signal processing module, the storage module, the display module, the power module, and the alarm module for system control;

[0006] The body temperature monitoring module is data-connected to the control module and is used to collect the user's body temperature data information;

[0007] The heart rate monitoring module is data-connected to the control module and is used to collect the user's heart rate data information;

[0008] The video acquisition module is data-connected with the control module and is used to acquire video data information of the user;

[0009] The signal processing module is data-connected with the control module and the communication module, and is used to process the body temperature data information, the heart rate data information and the video data information to obtain a user status monitoring result, and send the user status monitoring result to the control module and the communication module;

[0010] The communication module is data-connected with the signal processing module and the mobile terminal, and is used to send the user status monitoring result to the mobile terminal;

[0011] The display module is data-connected to the control module and is used to display the user status monitoring result;

[0012] The power supply module is data-connected to the control module and is used to supply power to the user status monitoring system;

[0013] The alarm module is data-connected to the control module and is used to generate an alarm based on the user status monitoring result;

[0014] The storage module is data-connected to the control module and is used to store data;

[0015] The mobile terminal is data-connected to the communication module, and is used to receive and process the user status monitoring result;

[0016] The steps of the non-contact user status monitoring method applied to the non-contact user status monitoring system include:

[0017] S1, using the body temperature monitoring module to collect the user's body temperature data information, and then caching the body temperature data information and sending it to the signal processing module;

[0018] S2, using the heart rate monitoring module to collect the user's heart rate data information, and buffering the heart rate data information and sending it to the signal processing module;

[0019] S3, using the video acquisition module to collect the user's video data information, and then caching the video data information and sending it to the signal processing module;

[0020] S4, using a signal processing module to process the body temperature data information, the heart rate data information and the video data information to obtain a user state monitoring result, and send the user state monitoring result to a control module and a communication module; the user state monitoring result includes a physiological parameter recognition result and a motion recognition result, including:

[0021] S41, processing the body temperature data information, the heart rate data information and the video data information to obtain a physiological parameter recognition result, including:

[0022] S411, extracting features from the body temperature data information to obtain body temperature feature parameter information;

[0023] S412, processing the heart rate data information to obtain heart rate characteristic parameter information;

[0024] S413, processing the video data information to obtain blood pressure characteristic parameter information;

[0025] S414, performing feature fusion on the body temperature feature parameter information, the heart rate feature parameter information, and the blood pressure feature parameter information to obtain fused feature parameter information;

[0026] S415, using the fused characteristic parameter information, training a preset physiological parameter recognition model to obtain an optimized physiological parameter recognition model;

[0027] S416, using the optimized physiological parameter recognition model, processing the fused feature parameter information to be processed to obtain a physiological parameter recognition result;

[0028] S42, processing the video data information to obtain an action recognition result

[0029] S5, using the communication module to send the user status monitoring result to the mobile terminal;

[0030] S6, using the control module to send the user status monitoring result to the display module and the alarm module;

[0031] S7, using the display module to display the user status monitoring result;

[0032] S8, utilizing the alarm module to perform alarm processing on the user status monitoring result.

[0033] As an optional implementation, in an embodiment of the present invention, the body temperature monitoring module includes an infrared temperature sensor, which detects the user's body temperature through infrared energy radiated by the user;

[0034] The heart rate monitoring module includes a radar sensor, which measures the user's heart rate by receiving the user's reflected electromagnetic waves.

[0035] As an optional implementation manner, in an embodiment of the present invention, the feature extraction of the body temperature data information to obtain body temperature feature parameter information includes:

[0036] S4111, performing short-time Fourier transform on the body temperature data information to obtain frequency domain feature information;

[0037] S4112, using a preset denoising network model, processing the frequency domain feature information to obtain body temperature feature parameter information.

[0038] As an optional implementation manner, in an embodiment of the present invention, the processing of the heart rate data information to obtain the heart rate characteristic parameter information includes:

[0039] S4121, processing the heart rate data information to obtain first characteristic parameter information, second characteristic parameter information and third characteristic parameter information;

[0040] The calculation formula of the first characteristic parameter information is:

[0041] S1=20log 10 |X(k)| 2

[0042] in, S1 is the first characteristic parameter information, x(n) is the heart rate data information, N is the length of x(n), x(n) = [x1, x2, ..., x N-1 ], x i , i=1, 2, ..., N-1 is the i-th sample point in x(n);

[0043] The calculation formula of the second characteristic parameter information is:

[0044]

[0045] In the formula, j represents the imaginary unit, φ(x k ) represents x k The instantaneous phase, S k is the kth value in the second characteristic parameter information S2, x k , k = 1, 2, ..., N-1 is the kth sample point in x(n);

[0046] The calculation method of the third characteristic parameter information is:

[0047]

[0048] Where S3 is the third characteristic parameter information;

[0049] S4122: Integrate the first characteristic parameter information, the second characteristic parameter information and the third characteristic parameter information to obtain heart rate characteristic parameter information.

[0050] As an optional implementation manner, in an embodiment of the present invention, the first characteristic parameter information, the second characteristic parameter information and the third characteristic parameter information are integrated to obtain the heart rate characteristic parameter information, including:

[0051] Using a feature integration model, integrating the first feature parameter information, the second feature parameter information and the third feature parameter information to obtain heart rate feature parameter information;

[0052] The feature integration model expression is:

[0053] t=WQ+b

[0054] Among them, t is the characteristic parameter information of heart rate, W is the weight information, b is the bias information, W and b are set by experiments, Q is the input, Represents a tensor product operation.

[0055] As an optional implementation, in an embodiment of the present invention, the processing of the video data information to obtain blood pressure characteristic parameter information includes:

[0056] S4131, preprocessing the video data information to obtain preprocessed video data information;

[0057] S4132, extracting features from the pre-processed video data information to obtain video feature parameter information;

[0058] S4133, using the video feature parameter information, training a preset blood pressure measurement model to obtain an optimized blood pressure measurement model;

[0059] S4134: Using the optimized blood pressure measurement model, the video feature parameter information to be processed is processed to obtain blood pressure feature parameter information.

[0060] As an optional implementation manner, in an embodiment of the present invention, the processing of the video data information to obtain the action recognition result includes:

[0061] S421, processing the video data information to obtain video frame image information;

[0062] S422, performing human body detection on the video frame image information to obtain human body image information;

[0063] S423, extracting joint points from the human body image information to obtain joint data information;

[0064] S424, stacking the joint data information along the time dimension to obtain joint point sequence data information;

[0065] S425, using the joint point sequence data information, training a preset action recognition model to obtain an optimized action recognition model;

[0066] S426, using the optimized action recognition model, processing the joint point sequence data information to be processed to obtain an action recognition result.

[0067] Compared with the prior art, the embodiments of the present invention have the following beneficial effects:

[0068] The present invention proposes a non-contact user status monitoring system and solution, provides non-contact millimeter wave radar, temperature, and video monitoring data, monitors the user's status, obtains physiological parameter recognition results and action recognition results, and uses an alarm system to make comprehensive decisions and alarms. The method of the present invention can protect the privacy of users, does not contact users, greatly facilitates people who need to perform long-term continuous status monitoring, avoids the risk of cross infection, reduces the operation of monitoring instruments, has high safety, and improves management efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0069] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0070] Figure 1 It is a structural schematic diagram of a non-contact user status monitoring system disclosed in an embodiment of the present invention;

[0071] Figure 2 It is a flow chart of a non-contact user status monitoring method disclosed in an embodiment of the present invention. DETAILED DESCRIPTION

[0072] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0073] The terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, device, product or equipment that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units that are not listed, or may optionally include other steps or units that are inherent to these processes, methods, products or equipment.

[0074] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present invention. The appearance of the phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0075] The present invention discloses a non-contact user status monitoring system and method, the system includes a body temperature monitoring module, a heart rate monitoring module, a video acquisition module, a control module, a signal processing module, a storage module, a communication module, a display module, a power module, an alarm module and a mobile terminal; the body temperature data information, the heart rate data information and the video data information are processed by the signal processing module to obtain the user status monitoring result, and the user status monitoring result is sent to the control module and the communication module. The communication module is used to send the user status monitoring result to the mobile terminal; the control module is used to send the user status monitoring result to the display module and the alarm module; the display module is used to display the user status monitoring result; and the alarm module is used to perform alarm processing on the user status monitoring result. The method of the present invention can protect the privacy of the user, avoid the risk of cross infection, reduce the operation of the monitoring instrument, and has high safety. The following are detailed descriptions.

[0076] Embodiment 1

[0077] See also Figure 1 , Figure 1 Schematic diagram of a non-contact user status monitoring system disclosed in an embodiment of the present invention. Figure 1 The non-contact user status monitoring system described is applied to the field of user status monitoring technology, and the embodiments of the present invention do not limit this. Figure 1 As shown, the non-contact user status monitoring system includes a body temperature monitoring module, a heart rate monitoring module, a video acquisition module, a control module, a signal processing module, a storage module, a communication module, a display module, a power module, an alarm module and a mobile terminal;

[0078] The control module is data-connected with the body temperature monitoring module, the heart rate monitoring module, the video acquisition module, the signal processing module, the storage module, the display module, the power module, and the alarm module for system control;

[0079] The body temperature monitoring module is data-connected to the control module and is used to collect the user's body temperature data information;

[0080] The heart rate monitoring module is data-connected to the control module and is used to collect the user's heart rate data information;

[0081] The video acquisition module is data-connected with the control module and is used to acquire video data information of the user;

[0082] The signal processing module is data-connected with the control module and the communication module, and is used to process the body temperature data information, the heart rate data information and the video data information to obtain a user status monitoring result, and send the user status monitoring result to the control module and the communication module;

[0083] The communication module is data-connected with the signal processing module and the mobile terminal, and is used to send the user status monitoring result to the mobile terminal;

[0084] The display module is data-connected to the control module and is used to display the user status monitoring result;

[0085] The power supply module is data-connected to the control module and is used to supply power to the user status monitoring system;

[0086] The alarm module is data-connected to the control module and is used to generate an alarm based on the user status monitoring result;

[0087] The storage module is data-connected to the control module and is used to store data;

[0088] The mobile terminal is data-connected to the communication module for receiving and processing the user status monitoring result.

[0089] Optionally, the body temperature monitoring module includes a LU90614 infrared temperature sensor, which detects the user's body temperature through the infrared energy radiated by the user, and has the characteristics of fast response speed, non-contact, high accuracy and wide application field.

[0090] The heart rate monitoring module includes an R60ABH1 radar sensor, which measures the user's heart rate by receiving the user's reflected electromagnetic waves.

[0091] Specifically, the radar transmits electromagnetic waves in the 60GHz frequency band. The part of the human body being measured will emit electromagnetic waves. The millimeter-wave radar module receives the electromagnetic wave signals reflected by the human body, demodulates the electromagnetic wave signals, and then processes the signals through signal amplification, filtering, AD conversion, etc. The MCU of the millimeter-wave radar receives the demodulated electromagnetic wave signals and calculates the demodulated electromagnetic wave signals to obtain the user's heart rate.

[0092] In this embodiment, the storage module is a DDR3 storage module.

[0093] In this embodiment, the video acquisition module is a camera placed indoors, and the angle needs to be adjusted when in use so that it faces the monitored user.

[0094] The control module is an FPGA development board, using ALINX's AX7035;

[0095] Optionally, the signal processing module can be completed by a host computer or by a dedicated DSP signal processing board;

[0096] The communication module is completed by a wireless communication module or other communication modules, which is not limited in this embodiment;

[0097] The display module and the power module are common technologies in this field.

[0098] It can be seen that the present invention proposes a non-contact user status monitoring system and solution, provides non-contact millimeter wave radar, temperature, and video monitoring data, monitors the user's status, obtains physiological parameter recognition results and action recognition results, and uses the alarm system to make comprehensive decisions and alarms. The method of the present invention can protect the privacy of users, does not contact users, greatly facilitates people who need to perform long-term continuous status monitoring, avoids the risk of cross infection, reduces the operation of monitoring instruments, has high safety, and improves management efficiency.

[0099] Embodiment 2

[0100] See also Figure 2 , Figure 2 : is a flow chart of a non-contact user status monitoring method disclosed in an embodiment of the present invention. Figure 2 The non-contact user status monitoring method described is applied to the field of user status monitoring technology, and the embodiments of the present invention do not limit this. Figure 2 As shown, the non-contact user status monitoring method includes:

[0101] S1, using the body temperature monitoring module to collect the user's body temperature data information, and then caching the body temperature data information and sending it to the signal processing module;

[0102] S2, using the heart rate monitoring module to collect the user's heart rate data information, and buffering the heart rate data information and sending it to the signal processing module;

[0103] S3, using the video acquisition module to collect the user's video data information, and then caching the video data information and sending it to the signal processing module;

[0104] S4, using a signal processing module to process the body temperature data information, the heart rate data information and the video data information to obtain a user state monitoring result, and sending the user state monitoring result to a control module and a communication module; the user state monitoring result includes a physiological parameter recognition result and a motion recognition result;

[0105] S5, using the communication module to send the user status monitoring result to the mobile terminal;

[0106] The guardian may not be with the monitored user sometimes, and in this case the guardian can receive the user status monitoring result on the mobile terminal.

[0107] S6, using the control module to send the user status monitoring result to the display module and the alarm module;

[0108] S7, using the display module to display the user status monitoring result;

[0109] The display module displays the physiological parameter recognition results and the action recognition results for people to view and perform necessary processing.

[0110] S8, utilizing the alarm module to perform alarm processing on the user status monitoring result.

[0111] The alarm module receives the physiological parameter recognition results and the action recognition results, and uses the trained neural network model based on Bi-GRU and dual attention mechanism to process the physiological parameter recognition results and the action recognition results respectively to obtain the physiological parameter recognition confusion matrix and the action recognition confusion matrix. The physiological parameter recognition confusion matrix and the action recognition confusion matrix are processed to obtain a comprehensive evaluation result. The comprehensive evaluation result includes four levels: normal, relatively dangerous, dangerous and critical, and different indicator lights and alarm sounds are used for prompts.

[0112] Optionally, the signal processing module is used to process the body temperature data information, the heart rate data information and the video data information to obtain a user status monitoring result, including:

[0113] S41, processing the body temperature data information, the heart rate data information and the video data information to obtain a physiological parameter recognition result;

[0114] S42, processing the video data information to obtain an action recognition result.

[0115] Optionally, the processing of the body temperature data information, the heart rate data information and the video data information to obtain a physiological parameter recognition result includes:

[0116] S411, extracting features from the body temperature data information to obtain body temperature feature parameter information;

[0117] S412, processing the heart rate data information to obtain heart rate characteristic parameter information;

[0118] S413, processing the video data information to obtain blood pressure characteristic parameter information;

[0119] S414, performing feature fusion on the body temperature feature parameter information, the heart rate feature parameter information, and the blood pressure feature parameter information to obtain fused feature parameter information;

[0120] Optional, fusion method:

[0121] Get the cluster feature information X and cluster feature information Y to be fused; X is an n′×m′ dimensional matrix, Y is a p×m′ dimensional matrix, m′ represents the number of samples, n′ and p represent the dimensions of the two features, project the two matrices to 1 dimension for linear representation, corresponding to the projection vectors a1 and b1 respectively, then the feature matrix after projection becomes:

[0122]

[0123] The purpose is to maximize the correlation coefficient between X′ and Y′, so as to obtain the projection vectors a1 and b1 when the correlation coefficient is maximum, that is:

[0124]

[0125] Before projection, the data is standardized. The purpose of standardization is to make the mean of the data 0 and the variance 1. We can get:

[0126] cov(X′, Y′)=cov(a1 T X,b1 T Y)=E( <a1 T X,b1 T Y>)=E((a1 T X)(b1 T Y) T )=a1 T E(XY T )b1

[0127]

[0128] D(X)=cov(X,X)=E(X T X)

[0129] D(Y)=cov(Y,Y)=E(Y T Y)

[0130] cov(X,Y)=E(XY T ), cov(Y,X)=E(YX T )

[0131] S XX =cov(X, X), then the solution objective is transformed into:

[0132]

[0133] Step 1: Calculate the variance S of X and Y XX and S YY , the covariance S of XY and YX XY =S YX T ;

[0134] Step 2: Calculate the matrix

[0135] Step 3: Solve the singular values ​​of M′ to obtain the largest singular value and its preceding and following singular vectors u, v;

[0136] Step 4: The projection vectors a1 and b1 of X and Y are:

[0137]

[0138] Step 5: The fusion feature vector Z of X and Y is

[0139] According to the above method, the body temperature characteristic parameter information and the heart rate characteristic parameter information are fused to obtain the first fused information; the first fused information and the blood pressure characteristic parameter information are feature fused to obtain the fused characteristic parameter information.

[0140] S415, using the fused characteristic parameter information, training a preset physiological parameter recognition model to obtain an optimized physiological parameter recognition model;

[0141] The preset physiological parameter recognition model is a convolutional neural network model based on global weighting, which is a global weighted structure;

[0142] The global weighted structure includes adding a global weighted module to the convolutional layer of the convolutional neural network model;

[0143] The global weighting module eliminates noise of the convolutional neural network model using an adaptive threshold;

[0144] The convolution kernel of the global weighted convolutional neural network model performs convolution operation by means of feature superposition;

[0145] The loss function L of the global weighted convolutional neural network model is:

[0146]

[0147] Among them, the λ parameter is used to control the degree of dispersion between classes, x i is the predicted value of the current neuron output, c yi is the true sample label value, is the weight of the neuron, b j is the bias of the neuron, n is the number of neurons, and m is the number of features of the input sample.

[0148] S416, using the optimized physiological parameter recognition model, the fused characteristic parameter information to be processed is processed to obtain a physiological parameter recognition result. The physiological parameter recognition result is divided into four levels: excellent, good, medium, and poor.

[0149] Optionally, the extracting features of the body temperature data information to obtain body temperature feature parameter information includes:

[0150] S4111, performing short-time Fourier transform on the body temperature data information to obtain frequency domain feature information;

[0151] S4112, using a preset denoising network model, processing the frequency domain feature information to obtain body temperature feature parameter information.

[0152] Denoising network model: Use STFT (Short Time Fourier Transform) to convert the signal to be processed into the time-frequency domain, using a window length of 256 samples, an overlap rate of 75%, and a Hamming window. By discarding frequency samples corresponding to negative frequencies, the length of the spectrum vector can be reduced to 129. Define a fully convolutional network consisting of 16 convolutional layers. The size of the convolution kernel of the first layer is 9×8, with 18 convolution kernels. The second to 13th convolutional layers are groups of 3 layers repeated 4 times, with convolution kernel widths of 5, 9, and 9, respectively, and the number of convolution kernels is 64. The convolution kernel width of the last convolution layer is 129 and there is only 1 convolution kernel. In this network, convolution is performed only in the frequency dimension, and for all layers except the first layer, the convolution kernel width along the time dimension is set to 1. The convolution layer is followed by a batch normalization layer and a ReLU activation function layer, and finally output through a regression layer.

[0153] A convolutional neural network is established to complete the mapping F between the STFT spectrum X(m, k) of the signal to be processed and the STFT spectrum S(m, k) of the clean signal to be processed, so as to achieve the purpose of denoising. The input prediction variable signal and the network target signal are the amplitude spectrum of the received noisy signal to be processed and the clean signal to be processed respectively. The goal of the network is to learn the mapping F so that the network outputs the amplitude spectrum of the noisy signal to be processed. The regression network uses the prediction variable input to minimize the mean square error between its output and the input target. Therefore, the loss function of the denoising network model can be expressed as

[0154]

[0155] Where B represents the batch size, M and K represent the total number of STFT time frames and frequency bins, respectively.

[0156] Optionally, the processing of the heart rate data information to obtain heart rate characteristic parameter information includes:

[0157] S4121, processing the heart rate data information to obtain first characteristic parameter information, second characteristic parameter information and third characteristic parameter information;

[0158] The calculation formula of the first characteristic parameter information is:

[0159] S1=20log 10 |X(k)| 2

[0160] in, S1 is the first characteristic parameter information, x(n) is the heart rate data information, N is the length of x(n), x(n) = [x1, x2, ..., x N-1 ], x i ,i=1,2,…,N-1 is the i-th sample point in x(n);

[0161] The calculation formula of the second characteristic parameter information is:

[0162]

[0163] In the formula, j represents the imaginary unit, φ(x k ) represents x k The instantaneous phase, S k is the kth feature parameter in the second feature parameter information S2, x k , k=1,2,…,N-1 is the kth sample point in x(n);

[0164] The calculation method of the third characteristic parameter information is:

[0165]

[0166] Where S3 is the third characteristic parameter information;

[0167] S4122: Integrate the first characteristic parameter information, the second characteristic parameter information and the third characteristic parameter information to obtain heart rate characteristic parameter information.

[0168] Optionally, the integrating the first characteristic parameter information, the second characteristic parameter information and the third characteristic parameter information to obtain the heart rate characteristic parameter information includes:

[0169] Using a feature integration model, integrating the first feature parameter information, the second feature parameter information and the third feature parameter information to obtain heart rate feature parameter information;

[0170] The feature integration model expression is:

[0171] t=WQ+b

[0172] Among them, t is the characteristic parameter information of heart rate, W is the weight information, b is the bias information, W and b are set by experiments, Q is the input, Represents a tensor product operation.

[0173] Optionally, the processing of the video data information to obtain blood pressure characteristic parameter information includes:

[0174] S4131, preprocessing the video data information to obtain preprocessed video data information;

[0175] The method includes extracting an imaging photoelectric mapping (IPPG) signal from the video data information, performing sliding average processing on the IPPG signal, and obtaining a smooth IPPG signal;

[0176] removing outliers from the smoothed IPPG signal to obtain a first smoothed IPPG signal;

[0177] Processing the first smoothed IPPG signal to remove the signal segment with abnormal heart rate, thereby obtaining a second smoothed IPPG signal;

[0178] Processing the second smoothed IPPG signal to remove signal segments with serious discontinuity, thereby obtaining a third smoothed IPPG signal;

[0179] Performing autocorrelation calculation on the third smoothed IPPG signal, removing high-variation signal segments according to the autocorrelation value, and obtaining preprocessed video data information;

[0180] S4132, extracting features from the pre-processed video data information to obtain video feature parameter information;

[0181] S41321, using a fuzzy parameter calculation model, processing the pre-processed video data information to obtain fuzzy parameter information;

[0182] The fuzzy parameter calculation model expression is:

[0183]

[0184]

[0185] Among them, x(t) is the pre-processed video data information, A x (θ, τ) is the fuzzy parameter information, t is the time variable, τ is the displacement variable, θ is the frequency variable corresponding to τ, and * indicates taking the conjugate.

[0186] S41322, performing Hadamard product calculation on the fuzzy parameter information and a preset kernel function to obtain an optimized kernel function;

[0187] The optimization kernel function expression is:

[0188] k(θ, τ) = A x (θ,τ)g(θ,τ)

[0189] Among them, k(θ, τ) is the optimization kernel function, g(θ, τ) is the preset kernel function, a=0.001,β=50;

[0190] S41323, processing the pre-processed video data information and the optimized kernel function to obtain video feature parameter information;

[0191] The video feature parameter information calculation formula is:

[0192]

[0193] Among them, C x is the video feature parameter information, the integration range is -∞~∞, u, t is the time variable, τ is the time shift variable, Ω is the frequency variable corresponding to u, and θ is the frequency variable corresponding to τ.

[0194] S4133, using the video feature parameter information, training a preset blood pressure measurement model to obtain an optimized blood pressure measurement model;

[0195] The preset blood pressure measurement model is the CTPN model;

[0196] The CTPN model includes a VGG16 convolutional model, a bidirectional LSTM model, and a fully connected layer;

[0197] Inputting the video feature parameter information into the VGG16 convolution model; the VGG16 network includes 13 convolutional layers, 5 maximum pooling layers and 3 fully connected layers;

[0198] In the convolution layer, a two-dimensional convolution kernel w∈R is used. 3×3 Extract the video feature parameter information to obtain the feature matrix C n ;

[0199]

[0200] Among them, n represents the number of convolution operations, m represents the number of convolution kernels, and p i represents the i-th feature matrix obtained, f represents the nonlinear activation function, · represents the corresponding operation of the shared weight of the convolution kernel and the feature matrix, w represents the weight of the convolution kernel, b represents the bias value, R3×3 represents a 3×3 real matrix;

[0201] The maximum pooling method is used in the pooling layer, and the feature extraction formula is expressed as follows:

[0202] p u =Max 2×2 [C n ]

[0203] Among them, u represents the number of pooling times, Max 2×2 represents the maximum pooling operation method of a 2×2 matrix, p u is the extracted feature; after multiple convolution and pooling operations, the Reshape processed data stream is input into the bidirectional LSTM model to obtain a feature vector with time series attributes, which is expressed as follows:

[0204] o t =g(V st +V′ sT′ )

[0205]

[0206] Among them, s t represents the output of the forward sequence at time t, s′ t Represents the output of the reverse sequence at time t, U Xt Represents the initial input of the forward sequence, U′ Xt Represents the initial input of the reverse sequence, Represents the input at the previous moment in the forward sequence. The input of the next moment in reverse time sequence, o t Represents the output at time t. After feature extraction by the time series model, the spatial + sequence feature vector is input into the RPN network and passes through two feature extraction networks at the same time. One of them obtains positive feedback and negative feedback classification through the softmax classification feature vector set, and the other is used to calculate the boundary value regression offset for the feature vector set to obtain accurate measurement results.

[0207] S4134: Using the optimized blood pressure measurement model, the video feature parameter information to be processed is processed to obtain blood pressure feature parameter information.

[0208] Optionally, the processing the video data information to obtain the action recognition result includes:

[0209] S421, processing the video data information to obtain video frame image information;

[0210] S422, performing human body detection on the video frame image information to obtain human body image information;

[0211] S423, extracting joint points from the human body image information to obtain joint data information;

[0212] S424, stacking the joint data information along the time dimension to obtain joint point sequence data information;

[0213] S425, using the joint point sequence data information, training a preset action recognition model to obtain an optimized action recognition model;

[0214] The action recognition model is a multi-scale temporal convolution model, which consists of two parts: multi-scale feature extraction and multi-scale feature fusion. The feature extraction part consists of a multi-scale pyramid convolution module, which extracts different temporal scale features on multiple branches to obtain a multi-scale feature map on the channel. The multi-scale feature fusion module consists of three steps: first, the SEWeight module is used to extract the multi-scale feature weight matrix on different channels; then, the multi-scale feature weight matrix is ​​normalized using Softmax to ensure that the importance of each feature channel is reasonably considered and a more effective feature fusion result is produced; finally, the multi-scale feature map on different channels is element-by-element multiplied with the channel feature weight matrix, so that the multi-scale features with higher weights are more prominent and the multi-scale features with lower weights are weakened.

[0215] S426, using the optimized action recognition model, processing the joint point sequence data information to be processed to obtain action recognition results. The action recognition results include 16 action categories such as standing, walking, sitting, and falling.

[0216] It can be seen that the present invention proposes a non-contact user status monitoring system and solution, provides non-contact millimeter wave radar, temperature, and video monitoring data, monitors the user's status, obtains physiological parameter recognition results and action recognition results, and uses the alarm system to make comprehensive decisions and alarms. The method of the present invention can protect the privacy of users, does not contact users, greatly facilitates people who need to perform long-term continuous status monitoring, avoids the risk of cross infection, reduces the operation of monitoring instruments, has high safety, and improves management efficiency.

[0217] The device embodiments described above are only illustrative, wherein the modules described as separate components may or may not be physically separated, and the components displayed as modules may or may not be physical modules, i.e., they may be located in one place, or they may be distributed on multiple network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Those of ordinary skill in the art may understand and implement it without creative work.

[0218] Through the specific description of the above embodiments, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on such an understanding, the above technical solution can be essentially or partly contributed to the prior art in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, and the storage medium includes a read-only memory (ROM), a random access memory (RAM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), a one-time programmable read-only memory (OTPROM), an electronically erasable rewritable read-only memory (EEPROM), a compact disc (CD-ROM) or other optical disc storage, magnetic disk storage, magnetic tape storage, or any other computer-readable medium that can be used to carry or store data.

[0219] Finally, it should be noted that the non-contact user status monitoring system and method disclosed in the embodiments of the present invention disclose only the preferred embodiments of the present invention, which are only used to illustrate the technical solution of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments can still be modified, or some of the technical features therein can be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A non-contact user status monitoring system, characterized in that: The system includes a body temperature monitoring module, a heart rate monitoring module, a video acquisition module, a control module, a signal processing module, a storage module, a communication module, a display module, a power module, an alarm module and a mobile terminal; The control module is data-connected with the body temperature monitoring module, the heart rate monitoring module, the video acquisition module, the signal processing module, the storage module, the display module, the power module, and the alarm module for system control; The body temperature monitoring module is data-connected to the control module and is used to collect the user's body temperature data information; The heart rate monitoring module is data-connected to the control module and is used to collect the user's heart rate data information; The video acquisition module is data-connected with the control module and is used to acquire video data information of the user; The signal processing module is data-connected with the control module and the communication module, and is used to process the body temperature data information, the heart rate data information and the video data information to obtain a user status monitoring result, and send the user status monitoring result to the control module and the communication module; The communication module is data-connected with the signal processing module and the mobile terminal, and is used to send the user status monitoring result to the mobile terminal; The display module is data-connected to the control module and is used to display the user status monitoring result; The power supply module is data-connected to the control module and is used to supply power to the user status monitoring system; The alarm module is data-connected to the control module and is used to generate an alarm based on the user status monitoring result; The storage module is data-connected to the control module and is used to store data; The mobile terminal is data-connected to the communication module, and is used to receive and process the user status monitoring result; The steps of the non-contact user status monitoring method applied to the non-contact user status monitoring system include: S1, using the body temperature monitoring module to collect the user's body temperature data information, and then caching the body temperature data information and sending it to the signal processing module; S2, using the heart rate monitoring module to collect the user's heart rate data information, and buffering the heart rate data information and sending it to the signal processing module; S3, using the video acquisition module to collect the user's video data information, and then caching the video data information and sending it to the signal processing module; S4, using a signal processing module to process the body temperature data information, the heart rate data information and the video data information to obtain a user state monitoring result, and send the user state monitoring result to a control module and a communication module; the user state monitoring result includes a physiological parameter recognition result and a motion recognition result, including: S41, processing the body temperature data information, the heart rate data information and the video data information to obtain a physiological parameter recognition result, including: S411, extracting features from the body temperature data information to obtain body temperature feature parameter information; S412, processing the heart rate data information to obtain heart rate characteristic parameter information; S413, processing the video data information to obtain blood pressure characteristic parameter information; S414, performing feature fusion on the body temperature feature parameter information, the heart rate feature parameter information, and the blood pressure feature parameter information to obtain fused feature parameter information; S415, using the fused characteristic parameter information, training a preset physiological parameter recognition model to obtain an optimized physiological parameter recognition model; S416, using the optimized physiological parameter recognition model, processing the fused feature parameter information to be processed to obtain a physiological parameter recognition result; S42, processing the video data information to obtain an action recognition result S5, using the communication module to send the user status monitoring result to the mobile terminal; S6, using the control module to send the user status monitoring result to the display module and the alarm module; S7, using the display module to display the user status monitoring result; S8, utilizing the alarm module to perform alarm processing on the user status monitoring result.

2. The non-contact user status monitoring system according to claim 1, characterized in that: The body temperature monitoring module includes an infrared temperature sensor, which detects the user's body temperature through the infrared energy radiated by the user; The heart rate monitoring module includes a radar sensor, which measures the user's heart rate by receiving the user's reflected electromagnetic waves.

3. The non-contact user status monitoring system according to claim 1, characterized in that: The extracting features of the body temperature data information to obtain body temperature feature parameter information includes: S4111, performing short-time Fourier transform on the body temperature data information to obtain frequency domain feature information; S4112, using a preset denoising network model, processing the frequency domain feature information to obtain body temperature feature parameter information.

4. The non-contact user status monitoring system according to claim 1, characterized in that: The processing of the heart rate data information to obtain the heart rate characteristic parameter information includes: S4121, processing the heart rate data information to obtain first characteristic parameter information, second characteristic parameter information and third characteristic parameter information; The calculation formula of the first characteristic parameter information is: S1=20log 10 |X(k) 2 in, S1 is the first characteristic parameter information, x(n) is the heart rate data information, N is the length of x(n), x(n) = [x1, x2, ..., x N-1 ], x i ,i=1,2,…,N-1 is the i-th sample point in x(n); The calculation formula of the second characteristic parameter information is: In the formula, j represents the imaginary unit, φ(x k ) represents x k The instantaneous phase, S k is the kth value in the second characteristic parameter information S2, x k , k=1,2,…,N-1 is the kth sample point in x(n); The calculation method of the third characteristic parameter information is: Wherein S3 is the third characteristic parameter information; S4122: Integrate the first characteristic parameter information, the second characteristic parameter information and the third characteristic parameter information to obtain heart rate characteristic parameter information.

5. The non-contact user status monitoring system according to claim 4, characterized in that: The step of integrating the first characteristic parameter information, the second characteristic parameter information and the third characteristic parameter information to obtain the heart rate characteristic parameter information includes: Using a feature integration model, integrating the first feature parameter information, the second feature parameter information and the third feature parameter information to obtain heart rate feature parameter information; The feature integration model expression is: t=WQ+b Among them, t is the characteristic parameter information of heart rate, W is the weight information, b is the bias information, W and b are set by experiments, Q is the input, Represents a tensor product operation.

6. The non-contact user status monitoring system according to claim 1, characterized in that: The processing of the video data information to obtain blood pressure characteristic parameter information includes: S4131, preprocessing the video data information to obtain preprocessed video data information; S4132, extracting features from the pre-processed video data information to obtain video feature parameter information; S4133, using the video feature parameter information, training a preset blood pressure measurement model to obtain an optimized blood pressure measurement model; S4134: Using the optimized blood pressure measurement model, the video feature parameter information to be processed is processed to obtain blood pressure feature parameter information.

7. The non-contact user status monitoring system according to claim 1, characterized in that: The processing of the video data information to obtain the action recognition result includes: S421, processing the video data information to obtain video frame image information; S422, performing human body detection on the video frame image information to obtain human body image information; S423, extracting joint points from the human body image information to obtain joint data information; S424, stacking the joint data information along the time dimension to obtain joint point sequence data information; S425, using the joint point sequence data information, training a preset action recognition model to obtain an optimized action recognition model; S426, using the optimized action recognition model, processing the joint point sequence data information to be processed to obtain an action recognition result.

Images