Health monitoring method based on millimeter wave radar and infrared imaging, computer and storage medium

Abstract

The invention relates to the technical field of health monitoring, in particular to a health monitoring method based on millimeter wave radar and infrared imaging, a computer and a storage medium, and the method comprises the steps: building a tracking algorithm when a plurality of targets coincide, and enabling each target to obtain a plurality of associated confidence values, each target in the first set selects a target whose association confidence value is most close to 1 as a tracking target; according to the method, the moving targets in the time period before coincidence and the time period after coincidence are selected for tracking, so that statistical errors caused by target confusion due to wrong patient identification by millimeter-wave radar and infrared imaging after movement coincidence of patients are avoided; the tracking algorithm is established, and the Euclidean norm is utilized, so that the vector when the target moves can be included, and the accuracy of target tracking is further improved; through the adjustable variance parameter, as the speed change probability is relatively large, the respiration change probability is lower or is described to have hysteresis, and the tracking accuracy can be improved.

Description

Technical Field

[0001] The present invention relates to the technical field of health monitoring, and in particular to a health monitoring method, a computer, and a storage medium based on millimeter wave radar and infrared imaging. Background Art

[0002] With the rapid development of information technology, especially from the Internet to mobile Internet and then to the Internet of Things, an interconnected lifestyle has been created, which also provides a technical foundation for patient health monitoring.

[0003] At present, in the field of health detection, it is mainly carried out face-to-face in hospitals and medical staff. After the medical staff actually measures the relevant data of the tester through the physical sign information collector, they fill in the form and submit it to the doctor for processing. This process not only increases the inconvenience of the tester, but also increases the burden on the medical staff. In the prior art, there are technical solutions for monitoring patients based on the combination of millimeter-wave radar and infrared sensors. Infrared sensors are used to make up for the problem that millimeter-wave radar cannot sense body temperature, but this creates new problems. There is not only one patient in a ward. Most wards have multiple patients in one ward. This leads to the overlapping of heat sources of infrared sensors in multi-target scenarios, making it difficult to distinguish individuals. The confusion of targets can easily cause statistical errors, which in turn causes unnecessary waste of medical resources or misdiagnosis by doctors. Therefore, a health monitoring method, computer, and storage medium based on millimeter-wave radar and infrared imaging that can achieve separate target tracking in multi-patient scenarios are needed. Summary of the invention

[0004] The technical problem to be solved by the present invention is to provide a health monitoring method, a computer and a storage medium based on millimeter wave radar and infrared imaging, which can realize target tracking in a multi-patient scenario.

[0005] In order to solve the above technical problems, the first technical solution adopted by the present invention is:

[0006] A health monitoring method based on millimeter wave radar and infrared imaging, comprising:

[0007] S1. Determine multiple targets in the video and their respective locations and breathing frequencies through the video of the infrared imaging device, and obtain the movement speed of each target through the millimeter wave radar;

[0008] S2. Determine whether there are multiple targets overlapping in the infrared imaging video. If not, do not take any action. If yes, establish a tracking algorithm. The time period before multiple targets move to overlap in the infrared imaging video is time period i, and the time period after overlap and then separation is time period j. The tracking algorithm includes:

[0009]

[0010] Among them, C i，j represents the association confidence of the target in the i-th period and the j-th period, and Δv i is the average motion speed of target i, and Δv j is the average motion speed of target j; Δf resp,i is the average breathing frequency of the target in the i-th period, and Δf resp,j is the average breathing frequency of the target in the j-th period; σ v is the adjustable variance parameter of the motion speed v, and σ f is the adjustable variance parameter of the breathing frequency f; ∥Δv i -Δv j ∥ represents the Euclidean norm of the vector, that is, the similarity of the target's motion state in the i-th period and the j-th period;

[0011] S3. Obtain several targets with moving actions in the i-th period as the first set, and obtain several targets with moving actions in the j-th period as the second set. Each target in the first set is calculated one by one with each target in the second set using a tracking algorithm. Each target in the first set will obtain an association confidence value with the same number as the number of targets in the second set. Each target in the first set selects the target with the association confidence value closest to 1 as the tracking target;

[0012] S4. If multiple targets in the first set select the target with the association confidence value closest to 1 as the tracking target, determine whether there is a repetition when different targets in the first set select the tracking target. If not, end the process. If so, lower σ f and re-execute S2 - S4.

[0013] Preferably, establish the motion and breathing curve of each target, with the horizontal axis being the motion speed and the vertical axis being the breathing frequency;

[0014] If σ is lowered f and there is still a repetition when different targets in the first set select the tracking target after re-executing S2 - S4, directly calculate the Δv j and Δf resp,j of the target in the second set, and then track after finding the motion and breathing curve with the highest coincidence degree.

[0015] Preferably, if the Δv i and Δf resp,i of the target in the i-th period or the Δv j and Δf resp,j in the j-th period deviate from the motion and breathing curve by more than a preset value, an alarm is issued.

[0016] Preferably, the lengths of the i-th period and the j-th period do not exceed 10 seconds.

[0017] Preferably, the millimeter-wave radar also acquires the breathing frequency of each target.

[0018] Preferably, the average value of the breathing frequency is calculated as the average of the breathing frequency obtained by the infrared imaging device and the breathing frequency obtained by the millimeter-wave radar.

[0019] Preferably, if the coincidence time exceeds a preset value, an alarm is directly issued.

[0020] Preferably, the infrared imaging device and the millimeter-wave radar are arranged on the ceiling.

[0021] To solve the above technical problems, the second technical solution adopted by the present invention is:

[0022] A computer, which includes a memory and a processor. A computer program is stored in the memory, and the processor executes the computer program to implement the health monitoring method based on millimeter-wave radar and infrared imaging as described above.

[0023] To solve the above technical problems, the third technical solution adopted by the present invention is:

[0024] A computer-readable storage medium, on which a computer program is stored, and the processor executes the computer program to implement the health monitoring method based on millimeter-wave radar and infrared imaging as described above.

[0025] The beneficial effects of the present invention are as follows: By selecting moving targets in the i-th period and the j-th period for tracking, there is no need to track non-moving targets, avoiding misidentification of patients by the millimeter-wave radar and infrared imaging after the patient moves and coincides. The confusion of targets is likely to cause statistical errors, resulting in unnecessary waste of medical resources or misdiagnosis by doctors, and reducing errors during identity switching; By establishing a tracking algorithm and using the Euclidean norm, the vector during the movement of the target can be included, thereby improving the accuracy of target tracking. Combined with the application scenario of the present application, that is, in a hospital or other medical institutions, the number of people in each ward is small, the space is small, and the speed will not be fast. Tracking is achieved through changes in speed and breathing frequency, without introducing too many various situations, resulting in a complex algorithm and improving the calculation speed; And by adjusting the variance parameter σ f, since the probability of speed change is relatively high and the probability of respiratory change is a little lower or described as having hysteresis, when the tracked targets overlap, adjusting the variance and recalculating can improve the tracking accuracy, thereby avoiding registration errors in various information collected among patients; and by using the Gaussian radial basis function for transformation to obtain two parameters, speed and respiratory rate, for comparison, multi-feature fusion is achieved to ensure flexibility; by means of the average value, the interference of various unexpected situations is eliminated to ensure accuracy. Detailed implementation manner

[0026] To describe in detail the technical content, achieved purpose and effects of the present invention, the following is described in conjunction with the implementation manners.

[0027] A health monitoring method based on millimeter-wave radar and infrared imaging, comprising:

[0028] S1. Judging multiple targets in the video and the respective positions and respiratory rates of the multiple targets through the video of the infrared imaging device, and obtaining the movement speed of each target through the millimeter-wave radar;

[0029] S2. Judging whether there is a situation where multiple targets overlap in the video of the infrared imaging. If not, no action is taken. If so, a tracking algorithm is established. The period before the multiple targets move to overlap in the video of the infrared imaging is the i period, and the period after overlapping and then separating is the j period. The tracking algorithm includes:

[0030]

[0031] Among them, C i，j represents the association confidence of the target in the i period and the j period, Δv i is the average movement speed of target i, Δv j is the average movement speed of target j; Δf resp,i is the average respiratory rate of the target in the i period, Δf resp,j is the average respiratory rate of the target in the j period; σ v is the adjustable variance parameter of the movement speed v, σ f is the adjustable variance parameter of the respiratory rate f; ∥Δv i -Δv j ∥ represents the Euclidean norm of the vector, that is, the similarity of the movement states of the target in the i period and the j period;

[0032] S3. Obtain a number of targets with movement actions during the i period as the first set, and obtain a number of targets with movement actions during the j period as the second set. Each target in the first set is calculated one by one with each target in the second set using a tracking algorithm. Each target in the first set will obtain an associated confidence value with the same number as the number of targets in the second set. Each target in the first set selects the target with the associated confidence value closest to 1 as the tracking target;

[0033] Similarity of motion characteristics: The smaller the difference in target speed, the higher the associated confidence. Similarity of physiological characteristics: The smaller the difference in breathing frequency, the higher the associated confidence. Comprehensive association: Only when both are similar, the target is considered to be the same entity. By multiplying the similarity of motion characteristics and the similarity of physiological characteristics, it is equivalent to a kind of amplification operation, avoiding misjudgment due to too large a single similarity.

[0034] S4. If multiple targets in the first set select the target with the associated confidence value closest to 1 as the tracking target, determine whether there is repetition when different targets in the first set select the tracking target. If not, end the process; if so, lower σ f And re - execute S2 - S4.

[0035] As can be seen from the above description, by selecting moving targets during the i period and the j period for tracking, there is no need to track non - moving targets, avoiding misidentifying patients by millimeter - wave radar and infrared imaging after the patients' movements overlap. The confusion of targets is likely to cause statistical errors, resulting in unnecessary waste of medical resources or misdiagnosis by physicians, and reducing errors during identity switching; by establishing a tracking algorithm and using the Euclidean norm, it can include the vectors during target movement, thereby improving the accuracy of target tracking. Combined with the application scenario of this application, that is, in hospitals or other medical institutions, the number of people in each ward is small, the space is small, and the speed will not be fast. Tracking is achieved through changes in speed and breathing frequency, without introducing too many various situations that lead to complex algorithms and improving the calculation speed; and through the adjustable variance parameter σ f , since the probability of speed change is relatively large and the probability of breathing change is a little lower or described as having hysteresis, therefore, when there is an overlap of the tracked targets, adjusting the variance and recalculating can improve the accuracy of tracking, thereby avoiding registration errors in the various information collected between patients; and by using the Gaussian radial basis function to transform and obtain two parameters of speed and breathing frequency for comparison, multi - feature fusion is realized to ensure flexibility; by means of the average value, the interference of various unexpected situations is eliminated to ensure accuracy.

[0036] Further, establish the motion - breathing curve of each target, with the horizontal axis being the motion speed and the vertical axis being the breathing frequency;

[0037] If σ is loweredf If there is still a repetition when selecting and tracking different targets within the first set after re - executing S2 - S4, directly calculate the Δv of the targets in the second set during the j - period j and Δf resp,j Then track by finding the motion - respiration curve with the highest coincidence degree

[0038] Furthermore, if the Δv of the target during the i - period i and Δf resp,i or the Δv of the target during the j - period j and Δf resp,j deviate from the motion - respiration curve by more than a preset value, then give an alarm

[0039] As can be seen from the above description, by calculating using the average value, occasional data errors are avoided, data is smoothed, and the overall monitoring effect is ensured. If the deviation from the motion - respiration curve is greater than the preset value, either the patient has an accident or there is a tracking error, and timely correction is required

[0040] Furthermore, the lengths of both the i - period and the j - period do not exceed 10 seconds

[0041] As can be seen from the above description, by setting the lengths of both the i - period and the j - period not to exceed 10 seconds, the engineering quantity of calculation caused by time is avoided, and generally, the overlap to departure does not exceed 5 seconds

[0042] Furthermore, the millimeter - wave radar also obtains the breathing frequency of each target

[0043] Furthermore, calculate the average value of the breathing frequencies, which is the average of the breathing frequency obtained by the infrared imaging device and the breathing frequency obtained by the millimeter - wave radar

[0044] As can be seen from the above description, by obtaining the breathing frequency through two methods, namely the infrared imaging device and the millimeter - wave radar, redundancy can be achieved to ensure the monitoring effect

[0045] Furthermore, if the coincidence time exceeds the preset value, directly give an alarm

[0046] As can be seen from the above description, by judging that the coincidence time exceeds the preset value, generally, the patients are in a separated state. Coincidence means that either there is a problem between the patients or there is a problem with the infrared sensing device

[0047] Furthermore, the infrared imaging device and the millimeter - wave radar are set on the ceiling

[0048] As can be seen from the above description, the infrared imaging device and the millimeter-wave radar are arranged on the ceiling to achieve a bird's-eye view. The biggest advantage of the bird's-eye view compared to monitoring at other angles is that it is difficult to have occlusion, and those who need health monitoring are all required to lie in bed. Therefore, it is easier to monitor when arranged on the ceiling.

[0049] Embodiment 1

[0050] A health monitoring method based on millimeter-wave radar and infrared imaging, comprising:

[0051] S1. Determine multiple targets in the video, the positions of the multiple targets, and the breathing frequency through the video of the infrared imaging device, and obtain the movement speed of each target through the millimeter-wave radar;

[0052] S2. Determine whether there is a situation where multiple targets overlap in the video of the infrared imaging. If not, do nothing. If so, establish a tracking algorithm. The period before the multiple targets move to overlap in the video of the infrared imaging is the i period, and the period after overlap and then separation is the j period. The tracking algorithm includes:

[0053]

[0054] Among them, C i，j represents the association confidence level of the target in the i period and the j period, Δv i is the average movement speed of target i, Δv j is the average movement speed of target j; Δf resp,i is the average breathing frequency of the target in the i period, Δf resp,j is the average breathing frequency of the target in the j period; σ v is the adjustable variance parameter of the movement speed v, σ f is the adjustable variance parameter of the breathing frequency f; ∥Δv i -Δv j ∥ represents the Euclidean norm of the vector, that is, the similarity of the movement states of the target in the i period and the j period;

[0055] S3. Obtain several targets with moving actions in the i period as the first set, and obtain several targets with moving actions in the j period as the second set. Each target in the first set is calculated with each target in the second set using the tracking algorithm one by one. Each target in the first set will obtain the same number of association confidence level values as the number of targets in the second set. Each target in the first set selects the target with the association confidence level value closest to 1 as the tracking target;

[0056] S4. If the target with the confidence value closest to 1 among the multiple targets in the first set is selected as the tracking target, determine whether there is a repetition when different targets in the first set select the tracking target. If not, end the process; if so, decrease σ f And re - execute S2 - S4.

[0057] Establish the motion - breathing curve of each target, with the horizontal axis being the motion speed and the vertical axis being the breathing frequency;

[0058] If σ is decreased f And after re - executing S2 - S4, there is still a repetition when different targets in the first set select the tracking target, then directly calculate Δv of the targets in the second set within the j period j And Δf resp,j And track after finding the motion - breathing curve with the highest coincidence degree.

[0059] If Δv of the target within the i period i And Δf resp,i Or Δv of the target within the j period j And Δf resp,j Deviate from the motion - breathing curve by more than the preset value, then give an alarm.

[0060] The lengths of both the i period and the j period do not exceed 10 seconds.

[0061] The millimeter - wave radar also obtains the breathing frequency of each target.

[0062] Calculate the average value of the breathing frequency, which is the average of the breathing frequency obtained by the infrared imaging device and the breathing frequency obtained by the millimeter - wave radar.

[0063] If the coincidence time exceeds the preset value (such as 5 seconds), then directly give an alarm.

[0064] The infrared imaging device and the millimeter - wave radar are set on the ceiling.

[0065] Embodiment 2

[0066] A health monitoring method based on millimeter - wave radar and infrared imaging, including:

[0067] S1. Determine multiple targets in the video, their respective positions and breathing frequencies through the video of the infrared imaging device, and obtain the motion speed of each target through the millimeter - wave radar;

[0068] S2. Determine whether there is a situation where multiple targets coincide in the video of the infrared imaging. If not, do nothing; if so, establish a tracking algorithm. The period before the multiple targets move to coincidence in the video of the infrared imaging is the i period, and the period after coincidence and then separation is the j period. The tracking algorithm includes:

[0069]

[0070] Among them, C i，j represents the association confidence of the target in the i-th period and the j-th period, and Δv i is the average motion speed of target i, and Δv j is the average motion speed of target j; Δf resp,i is the average breathing frequency of the target in the i-th period, and Δf resp,j is the average breathing frequency of the target in the j-th period; σ v is the adjustable variance parameter of the motion speed v, and σ f is the adjustable variance parameter of the breathing frequency f; ∥Δv i -Δv j ∥ represents the Euclidean norm of the vector, that is, the similarity of the motion states of the target in the i-th period and the j-th period;

[0071] S3. Obtain several targets with moving actions in the i-th period as the first set, and obtain several targets with moving actions in the j-th period as the second set. Each target in the first set is calculated with each target in the second set using a tracking algorithm. Each target in the first set will obtain an association confidence value with the same number as the number of targets in the second set. Each target in the first set selects the target with the association confidence value closest to 1 as the tracking target;

[0072] S4. If multiple targets in the first set select the target with the association confidence value closest to 1 as the tracking target, determine whether there is a repetition when different targets in the first set select the tracking target. If not, end the process. If so, decrease σ f and re-execute S2 - S4.

[0073] The lengths of the i-th period and the j-th period do not exceed 10 seconds.

[0074] The millimeter-wave radar also obtains the breathing frequency of each target.

[0075] If the coincidence time exceeds a preset value (such as 5 seconds), directly alarm.

[0076] The infrared imaging device and the millimeter-wave radar are installed on the ceiling.

[0077] Embodiment III

[0078] A computer, which includes a memory and a processor. A computer program is stored in the memory, and the processor executes the computer program to implement the health monitoring method based on millimeter-wave radar and infrared imaging according to any one of Embodiment I or Embodiment II.

[0079] Embodiment IV

[0080] A computer-readable storage medium has a computer program stored thereon, and a processor executes the computer program to implement the health monitoring method based on millimeter-wave radar and infrared imaging as described in any one of Embodiment 1 or Embodiment 2.

[0081] The above are only embodiments of the present invention, and thus do not limit the patent scope of the present invention. Any equivalent transformation made using the content of the specification of the present invention, directly or indirectly applied in the relevant technical fields, shall be similarly included in the patent protection scope of the present invention.

Claims

1. A health monitoring method based on millimeter-wave radar and infrared imaging, characterized in that, Including: S1. Judging multiple targets in the video by the infrared imaging device, the respective positions and breathing frequencies of the multiple targets, and obtaining the moving speed of each target by the millimeter-wave radar; S2. Judging whether there is a situation where multiple targets overlap in the infrared imaging video. If not, do nothing. If so, establish a tracking algorithm. The period before the multiple targets move to overlap in the infrared imaging video is the i period, and the period after overlapping and then separating is the j period. The tracking algorithm includes: Among them, C i，j represents the association confidence of the target in the i-th period and the j-th period, and Δv i is the average motion speed of target i, and Δv j is the average motion speed of target j; Δf resp,i is the average breathing frequency of the target in the i-th period, and Δf resp,j is the average breathing frequency of the target in the j-th period; σ v is the adjustable variance parameter of the motion speed v, and σ f is the adjustable variance parameter of the breathing frequency f; ∥Δv i -Δv j ∥ represents the Euclidean norm of the vector, that is, the similarity of the target's motion state in the i-th period and the j-th period; S3. Obtaining a number of targets with moving actions during the i period as the first set, and obtaining a number of targets with moving actions during the j period as the second set. Each target in the first set is calculated one by one with each target in the second set using the tracking algorithm. Each target in the first set will obtain an associated confidence value with the same number as the number of targets in the second set. Each target in the first set selects the target with the associated confidence value closest to 1 as the tracking target; S4. If the target with the correlation confidence value closest to 1 among the multiple targets in the first set is selected as the tracking target, determine whether there is a repetition when different targets in the first set select the tracking target. If not, end the process; if so, decrease σ f And re - execute S2 - S4.

2. The health monitoring method based on millimeter wave radar and infrared imaging according to claim 1, characterized in that, Establishing the motion and breathing curve of each target, with the horizontal axis being the moving speed and the vertical axis being the breathing frequency; If σ is lowered f and when repeating S2 - S4 and there is still repetition when selecting and tracking different targets within the first set, directly calculate the Δv of the targets in the second set during the j period j and Δf resp,j Then track after achieving the motion respiration curve with the highest coincidence degree.

3. The health monitoring method based on millimeter-wave radar and infrared imaging according to claim 2, wherein, If the Δv of the target within the i time period i and the Δf resp,i or the Δv of the target within the j time period j and the Δf resp,j deviate from the motion respiration curve by more than a preset value, an alarm is issued.

4. The health monitoring method based on millimeter-wave radar and infrared imaging according to claim 3, characterized in that, The lengths of both the i period and the j period do not exceed 10 seconds.

5. The health monitoring method based on millimeter-wave radar and infrared imaging according to claim 1, characterized in that The millimeter-wave radar also obtains the breathing frequency of each target.

6. The health monitoring method based on millimeter-wave radar and infrared imaging according to claim 5, wherein Calculating the average breathing frequency, which is the average of the breathing frequency obtained by the infrared imaging device and the breathing frequency obtained by the millimeter-wave radar.

7. The health monitoring method based on millimeter-wave radar and infrared imaging according to claim 1, characterized in that, If the overlapping time exceeds the preset value, directly give an alarm.

8. The health monitoring method based on millimeter wave radar and infrared imaging according to claim 1, characterized in that, The infrared imaging device and the millimeter-wave radar are arranged on the ceiling.

9. A computer, characterized in that, The computer includes a memory and a processor. A computer program is stored in the memory, and the processor executes the computer program to implement the health monitoring method based on the millimeter-wave radar and infrared imaging as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, A computer program is stored on the computer-readable storage medium, and the processor executes the computer program to implement the health monitoring method based on the millimeter-wave radar and infrared imaging as described in any one of claims 1-7.

Images