A monitoring method, device, storage medium and computer device
By using non-contact detection equipment and time-delay neural networks to process monitoring images and physiological feature sequences, the problems of high difficulty and low accuracy in monitoring by contact-based wearable devices have been solved, enabling efficient and accurate monitoring and timely alarm for vulnerable groups.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- DONGGUAN ZKTECO ELECTRONICS TECH
- Filing Date
- 2022-05-24
- Publication Date
- 2026-06-02
AI Technical Summary
In existing technologies, contact-based wearable devices for detecting various motor physiological signals of vulnerable groups face challenges in monitoring and have low accuracy.
Non-contact detection equipment is used to collect monitoring images and human physiological characteristic sequences. These are then processed using a time-delay neural network to predict the physical condition of the monitored person and to issue an alarm when a fall is detected.
Wearable devices that do not require contact can accurately monitor the physical condition of vulnerable groups, reducing the difficulty of monitoring, improving detection accuracy, and providing timely alarms to ensure personal safety.
Smart Images
Figure CN114947800B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of mobile monitoring technology, and in particular to a monitoring method, device, storage medium and computer equipment. Background Technology
[0002] Because vulnerable groups such as the elderly, infants, and people with disabilities often have mobility issues and may not be able to seek help in a timely manner when they encounter problems, relevant guardians will take certain guardianship measures to supervise the elderly, infants, or people with disabilities.
[0003] Currently, when conducting various motor physiological signal detections on vulnerable groups, contact wearable devices are mostly used. These devices need to come into contact with the skin. If skin contact is obstructed, or if a contact wearable device is not being carried, it is impossible to conduct various motor physiological signal detections on vulnerable groups in a timely manner, thus increasing the difficulty of monitoring. Furthermore, contact wearable devices can only provide single motor physiological signal detection, and the detection accuracy is not high. Summary of the Invention
[0004] The purpose of this application is to at least address one of the aforementioned technical deficiencies, particularly the technical shortcomings of existing technologies that use contact-based wearable devices to detect various motor physiological signals of vulnerable groups, which presents significant challenges in monitoring and results in low detection accuracy.
[0005] This application provides a monitoring method, the method comprising:
[0006] Acquire monitoring images and human physiological characteristic sequences of monitored persons during their activities in the monitored area, collected by non-contact detection equipment;
[0007] The human physiological feature sequence is input into a pre-configured time-delay neural network to obtain the physical state of the monitored person output by the time-delay neural network; wherein, the time-delay neural network is trained using the training feature sequence as training samples and the actual physical state of the monitored person as sample labels;
[0008] If the physical condition includes a fall, the fall condition and the corresponding time period of the monitoring footage will be sent to the monitoring personnel to issue an alarm.
[0009] Optionally, the human physiological characteristic sequence includes a sequence of attribute characteristics in multiple dimensions related to the physical state of the monitored person;
[0010] The step of inputting the human physiological feature sequence into a pre-configured time-delay neural network to obtain the physical state of the monitored person output by the time-delay neural network includes:
[0011] The attribute feature sequence of each dimension related to the physical state of the monitored person is input into a pre-configured time-delay neural network, so that the time-delay neural network predicts the classification result of the attribute feature sequence of each dimension, and comprehensively judges and outputs the physical state of the monitored person based on the classification results of the attribute feature sequences of each dimension.
[0012] Optionally, the process of determining the attribute feature sequence includes:
[0013] The human physiological feature sequence is decomposed to obtain a multi-dimensional attribute feature sequence;
[0014] The attribute feature sequences of each dimension are filtered to obtain filtered attribute feature sequences, and the filtered attribute feature sequences are used as attribute feature sequences of multiple dimensions related to the physical state of the monitored person.
[0015] Optionally, the decomposition of the human physiological feature sequence to obtain a multi-dimensional attribute feature sequence includes:
[0016] The human physiological feature sequence is subjected to various forms of Fourier transform to obtain the Fourier transform results under each form;
[0017] The Fourier transform results of each form are used as the attribute feature sequence of each dimension to obtain the attribute feature sequence of multiple dimensions.
[0018] Optionally, the step of filtering the attribute feature sequences of each dimension to obtain the filtered attribute feature sequences includes:
[0019] Constant false alarm rate detection is performed on the attribute feature sequences of each dimension to determine whether the attribute feature sequence of each dimension is an interference signal.
[0020] The attribute feature sequences that are interference signals in each dimension are filtered to obtain the filtered attribute feature sequences.
[0021] Optionally, the training process of the time-delay neural network includes:
[0022] Acquire training feature sequences of monitored persons during their activities in the monitored area, collected by non-contact detection equipment, as well as the actual physical state of the monitored persons;
[0023] The training feature sequence is input into an initial time-delay neural network to obtain the predicted physical state of the monitored person as output by the initial time-delay neural network;
[0024] The network parameters of the initial time-delay neural network are updated with the training objective of making the predicted physical state as close as possible to the actual physical state.
[0025] When the initial time-delay neural network meets the preset training conditions, the network parameters are stopped being updated, and the final time-delay neural network is obtained.
[0026] Optionally, the acquisition of monitoring footage and human physiological characteristic sequences of the monitored person during their activities in the monitored area, collected by the non-contact detection device, includes:
[0027] The system acquires monitoring images of monitored individuals moving within a monitored area, captured by a visible light image sensor, and sequences of human physiological characteristics, captured by a millimeter-wave radar sensor.
[0028] This application also provides a monitoring device, including:
[0029] The data acquisition module is used to acquire monitoring images and human physiological characteristic sequences of the monitored person during their activities in the monitored area, collected by non-contact detection equipment.
[0030] The state prediction module is used to input the human physiological feature sequence into a pre-configured time-delay neural network to obtain the physical state of the monitored person output by the time-delay neural network; wherein, the time-delay neural network is trained using the training feature sequence as training samples and the actual physical state of the monitored person as sample labels;
[0031] The monitoring module is used to send the fall status and the monitoring footage of the corresponding time period to the monitoring personnel to provide an alarm if the physical status includes a fall status.
[0032] This application also provides a storage medium storing computer-readable instructions that, when executed by one or more processors, cause the one or more processors to perform the steps of the monitoring method as described in any of the above embodiments.
[0033] This application also provides a computer device, including: one or more processors, and memory;
[0034] The memory stores computer-readable instructions, which, when executed by the one or more processors, perform the steps of the monitoring method as described in any of the above embodiments.
[0035] As can be seen from the above technical solutions, the embodiments of this application have the following advantages:
[0036] This application provides a monitoring method, device, storage medium, and computer equipment. When monitoring a person under supervision, it can acquire monitoring images and human physiological characteristic sequences of the person under supervision during their activities in the monitored area, collected by a non-contact detection device. This eliminates the need for the person under supervision to wear any contact-based wearable devices, thus avoiding issues such as obstructed skin contact or the absence of such devices, effectively reducing the difficulty of monitoring. Furthermore, this application can input the human physiological characteristic sequences into a pre-configured time-delay neural network. Since time-delay neural networks have a certain memory function, they are more suitable for processing sequence information. After processing the sequence information through the time-delay neural network, the accuracy of obtaining the person under supervision's physical status is high. Therefore, when this application obtains the person under supervision's physical status output by the time-delay neural network and determines that the physical status includes a fall, it can directly send the fall status and the corresponding monitoring footage to the monitor for an alarm notification. In this way, the monitor can quickly and accurately understand the person under supervision from multiple perspectives and take timely measures based on the alarm notification, thereby effectively protecting the personal safety of vulnerable groups while reducing the difficulty of monitoring. Attached Figure Description
[0037] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0038] Figure 1 A schematic flowchart illustrating a monitoring method provided in an embodiment of this application;
[0039] Figure 2 This is a schematic diagram of the detection range of the non-contact testing equipment of this application;
[0040] Figure 3 This is a schematic diagram of the structure of a monitoring device provided in an embodiment of this application;
[0041] Figure 4 This is a schematic diagram of the internal structure of a computer device provided in an embodiment of this application. Detailed Implementation
[0042] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.
[0043] Currently, when conducting various motor physiological signal detections on vulnerable groups, contact wearable devices are mostly used. These devices need to come into contact with the skin. If skin contact is obstructed, or if a contact wearable device is not being carried, it is impossible to conduct various motor physiological signal detections on vulnerable groups in a timely manner, thus increasing the difficulty of monitoring. Furthermore, contact wearable devices can only provide single motor physiological signal detection, and the detection accuracy is not high.
[0044] Based on this, this application proposes the following technical solution, as detailed below:
[0045] In one embodiment, such as Figure 1 As shown, Figure 1 This is a flowchart illustrating a monitoring method provided in an embodiment of this application; this application provides a monitoring method, which may include:
[0046] S110: Acquire monitoring images and human physiological characteristic sequences of monitored persons during their activities in the monitored area, collected by non-contact detection equipment.
[0047] In this step, when monitoring the person under supervision, with the consent of the person under supervision, non-contact detection equipment can be installed in advance in the key activity areas of the person under supervision or in areas where accidents may occur. The non-contact detection equipment can be used to collect monitoring images and human physiological characteristic sequences of the person under supervision when they are active in the monitored area.
[0048] For example, when the person under guardianship is an elderly person living alone, non-contact detection devices can be installed in the elderly person's bedroom, living room and other areas. This will not affect the elderly person's daily life, and the guardian, such as children, can check the elderly person's home situation and physical condition at any time so as to exercise guardianship responsibilities when necessary.
[0049] It is understood that the non-contact detection device here can be a centimeter-wave radar sensor, a millimeter-wave radar sensor, a visible light image sensor, or a combination of a millimeter-wave radar sensor and a visible light image sensor, etc., without any restrictions.
[0050] The human physiological characteristic sequence here refers to the radar echo signal reflected from the human body surface of the monitored person by non-contact detection equipment, such as centimeter-wave radar sensors or millimeter-wave radar sensors, which emits linear frequency modulated continuous wave signals. This radar echo signal contains the human physiological characteristics of the monitored person, such as breathing and heart rate.
[0051] In a specific implementation, such as Figure 2 As shown, Figure 2 This is a schematic diagram of the detection range of the non-contact testing equipment of this application; Figure 2 In this case, non-contact detection equipment (devices) can be installed on a wall 2000mm above the ground. The detection range is a sector area with a radius of 3000mm and an angle of 120 degrees. A single device can meet the detection needs of a 3*3 square meter area. In addition, if a larger monitoring range is required, multiple detection devices can be set up without any restrictions.
[0052] S120: Input the human physiological feature sequence into a pre-configured time-delay neural network to obtain the physical status of the monitored person output by the time-delay neural network.
[0053] In this step, after acquiring the monitoring footage of the monitored person moving in the monitoring area and the human physiological feature sequence collected by the non-contact detection device through S110, the human physiological feature sequence can be input into the pre-configured time-delay neural network to output the physical status of the monitored person through the time-delay neural network.
[0054] It can be understood that the Temporal Delay Neural Network (TDNN) in this application can essentially be viewed as a one-dimensional convolutional neural network. This convolutional neural network can adapt to dynamic changes in temporal features and has fewer parameters, thereby reducing training time. Furthermore, the hidden layer features of the TDNN are not only related to the input at the current time step, but also to the inputs at past and future time steps. The input of each layer of the TDNN is obtained through the context window of the lower layer, thus describing the temporal relationship between nodes in the upper and lower layers. This allows the TDNN to possess both temporal information and maintain non-fully connected nature, thereby reducing network complexity while improving the accuracy of processing sequence information.
[0055] Specifically, the time-delay neural network in this application is trained using training feature sequences as training samples and the actual physical state of the monitored person as sample labels, thereby enabling the trained time-delay neural network to predict the physical state of the monitored person. When the obtained human physiological feature sequences are input into the pre-configured time-delay neural network, the physical state of the monitored person output by the time-delay neural network can be obtained, and the physical state can be used to determine whether the monitored person has fallen.
[0056] Furthermore, in this application, the physical state of the person under guardianship refers to the body posture and vital signs information under that posture, such as whether the person under guardianship is in a fall and the acceleration during the fall.
[0057] S130: If the physical condition includes a fall, the fall condition and the corresponding time period of the monitoring footage will be sent to the monitoring personnel for alarm notification.
[0058] In this step, the human physiological feature sequence is input into the pre-configured time-delay neural network via S120. After obtaining the physical state of the monitored person output by the time-delay neural network, it can be determined whether the physical state includes a fall. If a fall is included, the fall and the corresponding time period of the monitoring screen can be sent to the monitor for alarm prompts. This allows the monitor to take relevant measures quickly based on the monitoring screen and the fall state of the monitored person, thereby effectively ensuring the personal safety of the monitored person.
[0059] It is understandable that when the time-delay neural network in this application determines that the monitored person is in a fall state at a certain period of time in the monitoring screen, since the fall state has a certain duration, when the monitored person's fall state is sent to the monitor via the network, the monitoring screen corresponding to the fall state can also be sent to the monitor, so that the monitor can more intuitively understand the monitored person's condition and judge whether relevant measures need to be taken, or the timing of taking relevant measures, based on the content displayed in the monitoring screen.
[0060] In addition, this application can save the collected monitoring images and human physiological characteristic sequences during routine monitoring so that monitoring personnel can access and view them at any time.
[0061] In the above embodiments, when monitoring a person under supervision, monitoring images and human physiological characteristic sequences of the person under supervision collected by non-contact detection devices can be obtained. This eliminates the need for the person under supervision to wear any contact-based wearable devices, thus avoiding issues such as obstructed skin contact or the absence of such devices and effectively reducing the difficulty of monitoring. Next, the application can input the human physiological characteristic sequence into a pre-configured time-delay neural network. Since the time-delay neural network has a certain memory function, it is more suitable for processing sequence information. After processing the sequence information through the time-delay neural network, the accuracy of obtaining the person under supervision's physical state is high. Therefore, when the application obtains the physical state of the person under supervision output by the time-delay neural network and determines that the physical state is a fall, it can directly send the fall state and the corresponding monitoring images of the time period to the monitor for an alarm notification. In this way, the monitor can quickly and accurately understand the physical state of the person under supervision from multiple perspectives and take relevant measures in a timely manner through alarm notifications, thereby effectively protecting the personal safety of vulnerable groups while reducing the difficulty of monitoring.
[0062] In one embodiment, the human physiological characteristic sequence may include a sequence of attribute characteristics in multiple dimensions related to the physical state of the monitored person.
[0063] In S120, the human physiological feature sequence is input into a pre-configured time-delay neural network to obtain the physical state of the monitored person output by the time-delay neural network, which may include:
[0064] S121: Input the attribute feature sequence of each dimension related to the physical state of the monitored person into a pre-configured time-delay neural network, so that the time-delay neural network predicts the classification result of the attribute feature sequence of each dimension, and comprehensively judges and outputs the physical state of the monitored person based on the classification results of the attribute feature sequences of each dimension.
[0065] In this embodiment, a time-delay neural network can be used to predict the physical condition of the monitored person. This application can input the attribute feature sequence of each dimension related to the physical condition of the monitored person into a pre-configured time-delay neural network, so that the time-delay neural network can calculate the attribute feature sequence of each dimension and obtain a classification result. Then, the time-delay neural network can comprehensively judge and output the physical condition of the monitored person based on the classification results of the attribute feature sequences of each dimension.
[0066] Specifically, since the attribute feature sequence is continuously variable at each time step, it must be calculated and an ideal signal output is generated through a time-delay neural network. The time-delay neural network uses delay unit structures between neurons in layer i and neuron node j in layer i+1, and so on. Each subsequent node is a weighted sum of the outputs of the previous layer's neuron node i at times t, t-1, ..., t-T1. The number of delay units is determined by the time-varying characteristics of the free mode. Finally, based on the precise ideal value of the algorithm's output function, the instantaneous error is also output. Then, using an error cancellation function, the output vector is calculated. This output vector is the classification result in this application. Next, the classification results of the attribute feature sequences in each dimension are comprehensively judged to output the final physical condition of the monitored person.
[0067] It is understandable that the output of the time-delay neural network is the digital signal corresponding to the physical state of the monitored person. Because the time-delay neural network adds memory function to the network, it is more suitable for processing sequence information and plays an important role in improving the accuracy of judgment, and has great value in the whole process.
[0068] In one embodiment, the process of determining the attribute feature sequence may include:
[0069] S111: Decompose the human physiological feature sequence to obtain a multi-dimensional attribute feature sequence.
[0070] S112: Filter the attribute feature sequences of each dimension to obtain the filtered attribute feature sequences, and use the filtered attribute feature sequences as the attribute feature sequences of multiple dimensions related to the physical state of the monitored person.
[0071] In this embodiment, the human physiological feature sequence refers to the radar echo signal reflected from the surface of the monitored person's body by non-contact detection devices, such as centimeter-wave radar sensors or millimeter-wave radar sensors, which emit linear frequency modulated continuous wave signals. This radar echo signal contains the monitored person's physiological characteristics such as respiration and heart rate. Therefore, before inputting the human physiological feature sequence into the time-delay neural network, this application can segment and model the acquired human physiological feature sequence to distinguish between attribute feature sequences of different dimensions within the human body feature sequence.
[0072] Specifically, this application can decompose the human physiological characteristic sequence to obtain a multi-dimensional attribute characteristic sequence related to the physical state of the monitored person, such as the radial distance, radial velocity, radial angle, and RCS (radar cross section) of the monitored person.
[0073] In addition, the echoes received by non-contact detection equipment are sometimes interference and sometimes targets. For example, if it is necessary to detect moving vehicles, ground clutter, noise and human interference are interference items. If an area of the ground is imaged, then ground clutter can be regarded as a target. Therefore, the echoes need to be filtered to obtain the final target.
[0074] In one embodiment, decomposing the human physiological feature sequence in S111 to obtain a multi-dimensional attribute feature sequence may include:
[0075] S1111: Perform various forms of Fourier transform on the human physiological feature sequence to obtain the Fourier transform results under each form.
[0076] S1112: Use the Fourier transform results of each form as the attribute feature sequence of each dimension to obtain the attribute feature sequence of multiple dimensions.
[0077] In this embodiment, when decomposing the human physiological feature sequence, the frequency of the monitored person's action signal can be calculated using continuous wavelet transform and peak detection method. This technique refers to performing wavelet transform on a time-varying signal using Fourier function. In this way, the wavelet transform will generate a maximum value at the inherent frequency of the original signal. The frequency corresponding to the generated extreme value is the frequency contained in this interval of the original signal. The wavelet is similar to a bandpass filter, which only allows signals with frequencies close to the wavelet center frequency (after scaling) to pass through.
[0078] In one specific implementation, assume that each chirp signal (non-stationary signal) acquires M sample points, N chirs are transmitted per frame, and there are P acquisition channels. Range FFT is used to obtain the distance information of the monitored person. The AD sampling data on each chirp is subjected to a Range FFT operation and stored as row vectors of a matrix. If there are N chirs, an N×M matrix is obtained, with each column called a Range Bin. If there are P receiving channels, a P×N×M cube is obtained. Doppler FFT (two-dimensional FFT) is used to obtain the velocity information of the monitored person. An FFT transformation is performed on each distance dimension, resulting in a matrix with row vectors representing distance and column vectors representing velocity. If there are P receiving channels, a P×N×M cube is obtained. Furthermore, since there are multiple receiving channels, uncorrelated accumulation is required. Therefore, the modulus of each element of the P complex matrices is first calculated to obtain an amplitude matrix, which is then accumulated and averaged.
[0079] Once this application obtains Fourier transform results in multiple forms, these results can be used as a sequence of attribute features related to the physical state of the monitored person in multiple dimensions, and the physical state of the monitored person can be predicted using this sequence of attribute features.
[0080] In one embodiment, filtering the attribute feature sequences of each dimension in S112 to obtain the filtered attribute feature sequences may include:
[0081] S1121: Perform constant false alarm rate detection on the attribute feature sequences of each dimension to determine whether the attribute feature sequence of each dimension is an interference signal.
[0082] S1122: Filter the attribute feature sequences that are interference signals in the attribute feature sequences of each dimension to obtain the filtered attribute feature sequences.
[0083] In this embodiment, since the received radar echo signal is sometimes an interference signal and sometimes a signal reflected by the target, before predicting the attribute feature sequences of each dimension, the attribute feature sequences of each dimension can be filtered by constant false alarm rate detection, and it can be determined whether the attribute feature sequence of each dimension is an interference signal. If it is an interference signal, the attribute feature sequence of that dimension is filtered, and the attribute feature sequences of non-interference signals are retained as the input of the time delay neural network.
[0084] The constant false alarm rate (CFAR) detection method works as follows: The CFAR detector first processes the input noise and determines a threshold. This threshold is compared with the input signal. If the input signal exceeds the threshold, a target is detected; otherwise, no target is detected. Generally, the signal is emitted by the signal source, encounters various interferences during propagation, and after reaching the receiver, it is processed and output to the detector. The detector then makes a decision on the input signal based on appropriate criteria.
[0085] Assuming the signal at the receiver output is represented by x(t), there are two cases:
[0086] (1) Noise and signal coexist: x(t) = s(t) + n(t)
[0087] (2) Only noise exists: x(t) = n(t)x(t) = n(t)x(t) = n(t);
[0088] Then, H0H0 and H1H1 can be used to represent the assumptions of no signal input and signal input of the receiver, respectively; D0D0 and D1D1 can be used to represent the decision results of the detector when there is no signal and when there is a signal, respectively.
[0089] Therefore, there are four possible scenarios for the receiver input and the detector's decision:
[0090] (1) If H 0H_{0}H0 is true, it is judged as D 0D_{0}D0, that is, the receiver has no signal input and the detector judges that there is no signal, which is called correct non-detection;
[0091] (2) H 0H_{0}H0 is true, and it is judged as D1D_{1}D1, that is, the receiver has no signal input, but the detector judges that there is a signal, which is called a false alarm;
[0092] (3) If H 1H_{1}H1 is true, it is judged as D 0D_{0}D0, that is, the receiver has a signal input, but the detector judges that there is no signal, which is called a missed alarm.
[0093] (4) If H 1H_{1}H1 is true, it is judged as D1D_{1}D1, that is, the receiver has a signal input and the detector judges that there is a signal, which is called a correct detection;
[0094] The first and fourth cases are correct judgments, while the other two are incorrect judgments.
[0095] In addition, this application can also use p(z|H0)p(z|{undefined{H}_{0}})p(z|H0) and p(z|H1)p(z|{undefined{H}_{1}})p(z|H1) to represent the probability density functions of the signal level at the receiver output when there is no signal input and when there is a signal input, respectively. Z0Z_{0}Z0 and Z1Z_{1}Z1 are used to represent the decision regions where the detector makes a decision of no signal and a decision of a signal, respectively. When the input level is in the Z0Z_{0}Z0 region, it is judged as no signal, and when it is in the Z1Z_{1}Z1 region, it is judged as a signal.
[0096] In one embodiment, the training process of the time-delay neural network may include:
[0097] S141: Obtain the training feature sequence of the monitored person during their activities in the monitored area, collected by the non-contact detection device, as well as the actual physical state of the monitored person.
[0098] S142: Input the training feature sequence into the initial time delay neural network to obtain the predicted physical state of the monitored person output by the initial time delay neural network.
[0099] S143: Update the network parameters of the initial time delay neural network with the training objective of the predicted body state approaching the actual body state.
[0100] S144: When the initial time-delay neural network meets the preset training conditions, the network parameters are stopped from being updated, and the final time-delay neural network is obtained.
[0101] In this embodiment, when training the time-delay neural network, a large number of training feature sequences of monitored persons during their activities in the monitored area can be acquired by non-contact detection devices, as well as the actual physical state of the monitored persons. Then, each acquired training feature sequence is input as a training sample into the initial time-delay neural network to obtain the predicted physical state of the monitored persons output by the initial time-delay neural network. Then, the actual physical state corresponding to the predicted physical state that is close to the input training feature sequence can be used as the training target to update the network parameters of the initial time-delay neural network. When the initial time-delay neural network meets the preset training conditions, the network parameters can be stopped, and the final time-delay neural network is obtained.
[0102] Furthermore, in the process of using the actual body state corresponding to the training feature sequence that predicts the body state as close as possible to the input as the training target, the mean squared error loss function can be used to calculate the loss value between the predicted body state and the actual body state, and the network parameters of the initial time delay neural network can be updated according to the loss value.
[0103] In one embodiment, acquiring the monitoring footage and human physiological characteristic sequence of the monitored person during their activities in the monitored area, collected by the non-contact detection device, in step S110 may include:
[0104] The system acquires monitoring images of monitored individuals moving within a monitored area, captured by a visible light image sensor, and sequences of human physiological characteristics, captured by a millimeter-wave radar sensor.
[0105] In this embodiment, when collecting monitoring images and real-time human physiological signals of the monitored person during their activities in the monitored area through non-contact detection equipment, a visible light image sensor can be used to collect monitoring images of the monitored person during their activities in the monitored area, and a millimeter-wave radar sensor can be used to collect the human physiological characteristic sequence of the monitored person.
[0106] It is understandable that the millimeter waves used in millimeter-wave radar sensors typically refer to those in the 30–300 GHz frequency range (wavelength 1–10 mm). Millimeter waves have wavelengths between centimeter waves and light waves, thus combining the advantages of microwave guidance and photoelectric guidance. Compared to centimeter-wave seekers, millimeter-wave seekers are smaller, lighter, and have higher spatial resolution. Compared to infrared, laser, and television optical seekers, millimeter-wave seekers have stronger penetration capabilities through fog, smoke, and dust, and are suitable for all weather conditions (except heavy rain). Furthermore, millimeter-wave seekers have superior anti-jamming and anti-stealth capabilities compared to other microwave seekers.
[0107] This application employs millimeter-wave linked video surveillance technology, providing 4D stereoscopic data such as depth and motion information for monitoring images from the same perspective. This solves the problem of weak information detection capability of video in low light and thunderstorm weather. The multi-sensor dual-detection algorithm improves the accuracy of AI intelligent recognition and reduces false alarms.
[0108] The monitoring device provided in the embodiments of this application is described below. The monitoring device described below can be referred to in correspondence with the monitoring method described above.
[0109] In one embodiment, such as Figure 3 As shown, Figure 3 This is a schematic diagram of a monitoring device provided in an embodiment of this application; this application also provides a monitoring device, which may include a data acquisition module 210, a status prediction module 220, and a monitoring alarm module 230, specifically including the following:
[0110] The data acquisition module 210 is used to acquire monitoring images and human physiological characteristic sequences of the monitored person when they are active in the monitored area, collected by the non-contact detection equipment.
[0111] The state prediction module 220 is used to input the human physiological feature sequence into a pre-configured time-delay neural network to obtain the physical state of the monitored person output by the time-delay neural network; wherein, the time-delay neural network is trained with training feature sequences as training samples and the actual physical state of the monitored person as sample labels.
[0112] The monitoring module 230 is used to send the fall status and the monitoring screen of the corresponding time period to the monitoring personnel to provide an alarm prompt if the physical status includes a fall status.
[0113] In the above embodiments, when monitoring a person under supervision, monitoring images and human physiological characteristic sequences of the person under supervision collected by non-contact detection devices can be obtained. This eliminates the need for the person under supervision to wear any contact-based wearable devices, thus avoiding issues such as obstructed skin contact or the absence of such devices and effectively reducing the difficulty of monitoring. Next, the application can input the human physiological characteristic sequence into a pre-configured time-delay neural network. Since the time-delay neural network has a certain memory function, it is more suitable for processing sequence information. After processing the sequence information through the time-delay neural network, the accuracy of obtaining the person under supervision's physical state is high. Therefore, when the application obtains the physical state of the person under supervision output by the time-delay neural network and determines that the physical state includes a fall, it can directly send the fall state and the monitoring image of the corresponding time period to the monitor for an alarm notification. In this way, the monitor can quickly and accurately understand the physical state of the person under supervision from multiple perspectives and take relevant measures in a timely manner through alarm notifications, thereby effectively protecting the personal safety of vulnerable groups while reducing the difficulty of monitoring.
[0114] In one embodiment, the human physiological characteristic sequence includes a multi-dimensional sequence of attribute characteristics related to the physical state of the monitored person.
[0115] The state prediction module 220 may include:
[0116] The state prediction submodule is used to input the attribute feature sequence of each dimension related to the physical state of the monitored person into a pre-configured time-delay neural network, so that the time-delay neural network predicts the classification result of the attribute feature sequence of each dimension, and comprehensively judges and outputs the physical state of the monitored person based on the classification results of the attribute feature sequences of each dimension.
[0117] In one embodiment, the monitoring device may further include:
[0118] The signal decomposition module is used to decompose the human physiological feature sequence to obtain a multi-dimensional attribute feature sequence.
[0119] The signal filtering module is used to filter the attribute feature sequences of each dimension to obtain the filtered attribute feature sequences, and to use the filtered attribute feature sequences as the attribute feature sequences of multiple dimensions related to the physical state of the monitored person.
[0120] In one embodiment, the signal decomposition module may include:
[0121] The wavelet transform module is used to perform various forms of Fourier transform on the human physiological feature sequence to obtain the Fourier transform results under each form.
[0122] The attribute determination module is used to take the Fourier transform results of each form as the attribute feature sequence of each dimension, so as to obtain the attribute feature sequence of multiple dimensions.
[0123] In one embodiment, the signal filtering module may include:
[0124] The signal detection module is used to perform constant false alarm rate detection on the attribute feature sequences of each dimension to determine whether the attribute feature sequence of each dimension is an interference signal.
[0125] The signal removal module is used to filter out the attribute feature sequences that are interference signals in the attribute feature sequences of each dimension, so as to obtain the filtered attribute feature sequences.
[0126] In one embodiment, the monitoring device may further include:
[0127] The sample acquisition module is used to acquire the training physical feature sequence of the monitored person when he / she is active in the monitored area, collected by the non-contact detection device, as well as the actual physical state of the monitored person.
[0128] The training module is used to input the training feature sequence into the initial time-delay neural network to obtain the predicted physical state of the monitored person output by the initial time-delay neural network.
[0129] The parameter update module is used to update the network parameters of the initial delay neural network with the training objective of the predicted body state approaching the actual body state.
[0130] The stop update module is used to stop updating the network parameters when the initial time-delay neural network meets the preset training conditions, thereby obtaining the final time-delay neural network.
[0131] In one embodiment, the data acquisition module 210 may include:
[0132] The data acquisition submodule is used to acquire monitoring images of the monitored person moving in the monitored area, collected by the visible light image sensor, and human physiological characteristic sequences collected by the millimeter-wave radar sensor.
[0133] In one embodiment, this application also provides a storage medium storing computer-readable instructions that, when executed by one or more processors, cause the one or more processors to perform the steps of the monitoring method as described in any of the above embodiments.
[0134] In one embodiment, this application also provides a computer device, including: one or more processors, and memory.
[0135] The memory stores computer-readable instructions, which, when executed by the one or more processors, perform the steps of the monitoring method as described in any of the above embodiments.
[0136] Indicatively, such as Figure 4 As shown, Figure 4 This is a schematic diagram of the internal structure of a computer device 300 provided in an embodiment of this application. The computer device 300 can be provided as a server. (Refer to...) Figure 4 The computer device 300 includes a processing component 302, which further includes one or more processors, and memory resources represented by memory 301 for storing instructions, such as application programs, that can be executed by the processing component 302. The application programs stored in memory 301 may include one or more modules, each corresponding to a set of instructions. Furthermore, the processing component 302 is configured to execute instructions to perform the monitoring methods of any of the above embodiments.
[0137] The computer device 300 may also include a power supply component 303 configured to perform power management of the computer device 300, a wired or wireless network interface 304 configured to connect the computer device 300 to a network, and an input / output (I / O) interface 305. The computer device 300 may operate on an operating system stored in memory 301, such as Windows Server™, Mac OS X™, Unix™, Linux™, Free BSD™, or similar.
[0138] Those skilled in the art will understand that Figure 4 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.
[0139] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.
[0140] The various embodiments in this specification are described in a progressive manner. Each embodiment focuses on the differences from other embodiments. The various embodiments can be combined as needed, and the same or similar parts can be referred to each other.
[0141] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims
1. A monitoring method, characterized in that, The method includes: The system acquires monitoring images of the monitored person moving within the monitored area, captured by a visible light image sensor, and human physiological feature sequences, captured by a millimeter-wave radar sensor. The human physiological feature sequences include attribute feature sequences of multiple dimensions related to the physical state of the monitored person. The process of determining the attribute feature sequence includes: The human physiological feature sequence is subjected to various forms of Fourier transform to obtain the Fourier transform results under each form; The Fourier transform results under each form are used as the attribute feature sequence for each dimension to obtain the attribute feature sequence for multiple dimensions. Constant false alarm rate detection is performed on the attribute feature sequences of each dimension to determine whether the attribute feature sequence of each dimension is an interference signal; The attribute feature sequences that are interference signals in the attribute feature sequences of each dimension are filtered to obtain the filtered attribute feature sequences, and the filtered attribute feature sequences are used as the attribute feature sequences of multiple dimensions related to the physical state of the monitored person. The human physiological feature sequence is input into a pre-configured time-delay neural network to obtain the physical state of the monitored person output by the time-delay neural network; wherein, the training process of the time-delay neural network includes: Acquire training feature sequences of monitored persons during their activities in the monitored area, collected by non-contact detection equipment, as well as the actual physical state of the monitored persons; The training feature sequence is input into an initial time-delay neural network to obtain the predicted physical state of the monitored person as output by the initial time-delay neural network; The network parameters of the initial time-delay neural network are updated with the training objective of making the predicted physical state approximate the actual physical state. When the initial time-delay neural network meets the preset training conditions, the network parameters are stopped from being updated, and the final time-delay neural network is obtained. If the monitored person's physical condition includes a fall, the fall and the corresponding time period of the monitoring footage will be sent to the monitor as an alarm notification.
2. The monitoring method according to claim 1, characterized in that, The step of inputting the human physiological feature sequence into a pre-configured time-delay neural network to obtain the physical state of the monitored person output by the time-delay neural network includes: The attribute feature sequence of each dimension related to the physical state of the monitored person is input into a pre-configured time-delay neural network, so that the time-delay neural network predicts the classification result of the attribute feature sequence of each dimension, and comprehensively judges and outputs the physical state of the monitored person based on the classification results of the attribute feature sequences of each dimension.
3. A monitoring device, characterized in that, include: The data acquisition module is used to acquire monitoring images of the monitored person moving in the monitored area collected by the visible light image sensor, as well as human physiological characteristic sequences collected by the millimeter-wave radar sensor. The human physiological characteristic sequence includes a multi-dimensional attribute characteristic sequence related to the physical state of the monitored person; The process of determining the attribute feature sequence includes: The human physiological feature sequence is subjected to various forms of Fourier transform to obtain the Fourier transform results under each form; The Fourier transform results under each form are used as the attribute feature sequence for each dimension to obtain the attribute feature sequence for multiple dimensions. Constant false alarm rate detection is performed on the attribute feature sequences of each dimension to determine whether the attribute feature sequence of each dimension is an interference signal; The attribute feature sequences that are interference signals in the attribute feature sequences of each dimension are filtered to obtain the filtered attribute feature sequences, and the filtered attribute feature sequences are used as the attribute feature sequences of multiple dimensions related to the physical state of the monitored person. A state prediction module is used to input the human physiological feature sequence into a pre-configured time-delay neural network to obtain the physical state of the monitored person output by the time-delay neural network; wherein, the training process of the time-delay neural network includes: Acquire training feature sequences of monitored persons during their activities in the monitored area, collected by non-contact detection equipment, as well as the actual physical state of the monitored persons; The training feature sequence is input into an initial time-delay neural network to obtain the predicted physical state of the monitored person as output by the initial time-delay neural network; The network parameters of the initial time-delay neural network are updated with the training objective of making the predicted physical state approximate the actual physical state. When the initial time-delay neural network meets the preset training conditions, the network parameters are stopped from being updated, and the final time-delay neural network is obtained. The monitoring module is used to send the fall status and the monitoring footage of the corresponding time period to the monitoring personnel as an alarm prompt if the physical state of the monitored person includes a fall status.
4. A storage medium, characterized in that, The storage medium stores computer-readable instructions that, when executed by one or more processors, cause the one or more processors to perform the steps of the monitoring method as described in any one of claims 1 to 2.
5. A computer device, characterized in that, include: One or more processors, and memory; The memory stores computer-readable instructions that, when executed by the one or more processors, perform the steps of the monitoring method as described in any one of claims 1 to 2.