Human body sleep contour estimation method, device, equipment and medium

Abstract

The invention relates to a human body sleep contour estimation method, device and equipment and a medium, and belongs to the technical field of intelligent monitoring, and the human body sleep contour estimation method comprises the steps: obtaining an echo signal detected by a millimeter wave radar; performing three-dimensional transformation on the echo signal to obtain three-dimensional point cloud data; clustering and segmenting the three-dimensional point cloud data to obtain a human body target point cloud; and based on the human body target point cloud, fitting a human body contour by using the cuboid frame.

Description

Technical Field

[0001] The present invention belongs to the field of intelligent monitoring technology, and in particular relates to a method, device, equipment and medium for estimating a human sleep profile. Background Art

[0002] Radar can better output human vital signs information, such as body movement, respiratory rate and heart rate, etc. It is more suitable for sleep monitoring, and can give sleep reports and evaluate sleep quality, which greatly facilitates the monitoring of elderly patients.

[0003] However, in a multi-person scenario, the distance between two people on the bed may be relatively close, even smaller than the distance from the upper to lower body of a nearby human body. It is difficult for the radar to directly distinguish the body movements and vital signs of nearby and distant human bodies through the distance of the radar dynamic points. Therefore, there is an urgent need for a method that can distinguish radar signals of different human bodies in a multi-person scenario. Summary of the invention

[0004] In view of the above analysis, the embodiments of the present invention aim to provide a method, apparatus, device and medium for estimating a human sleep profile, so as to solve one or more of the above problems existing in the prior art.

[0005] The object of the present invention is achieved in that:

[0006] A first aspect of the present invention provides a method for estimating a human sleep profile, comprising:

[0007] Obtaining the echo signal detected by the millimeter-wave radar;

[0008] Performing a three-dimensional transformation on the echo signal to obtain three-dimensional point cloud data;

[0009] Clustering and segmenting the three-dimensional point cloud data to obtain a human target point cloud;

[0010] Based on the human target point cloud, the human body contour is fitted using a rectangular box.

[0011] Furthermore, the method of acquiring the echo signal detected by the millimeter wave radar includes: scanning the entire bed surface area with a millimeter wave radar sensor and collecting the echo signal in real time; and performing noise reduction processing on the echo signal to eliminate high-frequency noise and environmental interference.

[0012] Furthermore, the three-dimensional transformation of the echo signal to obtain three-dimensional point cloud data includes: transforming the echo signal after noise reduction processing in three dimensions of distance, speed and angle through three-dimensional fast Fourier transform to obtain three-dimensional spectrum data including distance image, speed image and angle information of each target; performing threshold detection on the three-dimensional spectrum data through a constant false alarm rate algorithm to extract peak points in the three-dimensional spectrum data; converting the index corresponding to the peak point into spatial coordinates to form three-dimensional point cloud data.

[0013] Furthermore, clustering the three-dimensional point cloud data includes: clustering adjacent points into independent target clusters according to the spatial density distribution of the three-dimensional point cloud data, so as to preliminarily distinguish potential human body point cloud areas.

[0014] Furthermore, the segmenting of the three-dimensional point cloud data includes: based on the potential human body point cloud area, using the RANSAC algorithm to eliminate outliers and segment the human body core point cloud.

[0015] Furthermore, the method of fitting a human body contour using a rectangular box based on the human body target point cloud includes: calculating the covariance matrix of the human body core point cloud, and eigendecomposing the covariance matrix to obtain three main directions; projecting the human body core point cloud onto the three main directions, and determining the extreme values ​​of the human body core point cloud along the three main directions respectively to determine the parameters of the rectangular box including the center, size and rotation angle, and generating a rectangular box surrounding the human body according to the parameters.

[0016] Furthermore, it also includes: predicting the position and posture of the human body through Kalman filtering according to the historical frame data, and dynamically calibrating the parameters of the rectangular box.

[0017] A second aspect of the present invention provides a human sleep profile estimation device, comprising:

[0018] A signal acquisition module is used to acquire the echo signal detected by the millimeter wave radar;

[0019] A signal processing module, used for performing a three-dimensional transformation on the echo signal to obtain three-dimensional point cloud data;

[0020] A point cloud processing module, used for clustering and segmenting the three-dimensional point cloud data to obtain a human target point cloud;

[0021] The contour fitting module is used to fit the human body contour using a rectangular box based on the human body target point cloud.

[0022] A third aspect of the present invention provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the method for estimating a human sleep profile as described in any embodiment is implemented.

[0023] A fourth aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the method for estimating a human sleep profile as described in any one of the embodiments is implemented.

[0024] Compared with the prior art, the present invention can achieve at least one of the following beneficial effects:

[0025] The human sleep contour estimation method provided by the present invention processes the echo signal detected by the radar into a three-dimensional point cloud, clusters the three-dimensional point cloud, segments each human target point cloud, and finally fits the human contour with a rectangular frame. This can effectively reduce the mutual influence of adjacent human bodies and ensure the accuracy of subsequent detection of vital signs information of different human bodies. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] In order to more clearly illustrate the embodiments of this specification or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the embodiments of this specification. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.

[0027] Figure 1 A flowchart of a method for estimating a human sleep profile provided in Example 1 of the present invention;

[0028] Figure 2 A schematic diagram of a human sleep profile estimation device provided in Embodiment 2 of the present invention;

[0029] Figure 3 This is a schematic diagram of the electronic device architecture provided in Example 3 of the present invention. DETAILED DESCRIPTION

[0030] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. It should be noted that, in the absence of conflict, the embodiments and features in the embodiments disclosed in this disclosure can be combined, separated, interchanged and / or rearranged with each other. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0031] Example 1

[0032] A specific embodiment of the present invention, as Figure 1 As shown, a method for estimating a human sleep profile is disclosed, comprising the following steps:

[0033] S1. Obtaining the echo signal detected by the millimeter wave radar;

[0034] In this embodiment, step S1 specifically includes:

[0035] S101, using a millimeter wave radar sensor to scan the entire bed surface area and collect echo signals in real time;

[0036] Exemplarily, the millimeter-wave radar uses the Texas Instruments AWR1843 chipset, which has an operating frequency of 77 GHz, a bandwidth of 4 GHz, and is equipped with a 3-transmit 4-receive MIMO antenna array. It is installed vertically 1.2 m above the bed surface, with a beam coverage range of 2 m × 1.5 m. The radar transmits frequency modulated continuous wave (FMCW), a sweep period of 2 ms, and samples 256 points in each sweep period.

[0037] S102: Perform noise reduction processing on the echo signal to eliminate high-frequency noise and environmental interference.

[0038] Specifically, a FIR low-pass filter is used with a cutoff frequency of 1kHz to filter out high-frequency noise in the radar signal caused by electronic equipment or environmental electromagnetic radiation. The background difference method is used: a reference signal in an empty bed state is collected, and the real-time echo signal is subtracted from the reference signal to eliminate static environmental reflections (such as wardrobes, walls, etc.).

[0039] S2, performing three-dimensional transformation on the echo signal to obtain three-dimensional point cloud data;

[0040] In this embodiment, step S2 specifically includes:

[0041] S201, transforming the echo signal after noise reduction processing in three dimensions of distance, speed and angle by three-dimensional fast Fourier transform, so as to obtain three-dimensional spectrum data including distance image, speed image and angle information of each target;

[0042] Specifically, the echo signal after noise reduction is subjected to distance dimension FFT, velocity dimension FFT and angle dimension FFT respectively to analyze the distance between the target and the radar, detect the micro-motion speed of the target and calculate the target azimuth, so as to obtain the three-dimensional spectrum data of each target.

[0043] S202, performing threshold detection on the three-dimensional spectrum data by using a constant false alarm rate algorithm to extract peak points in the three-dimensional spectrum data;

[0044] Specifically, an ordered statistical constant false alarm rate algorithm, such as OS-CFAR, is used to set corresponding parameters, extract the peak points of the distance image, velocity image and angle information in the above three-dimensional spectrum data, and return the distance dimension FFT peak index, velocity dimension FFT peak index and angle dimension FFT peak index to generate three-dimensional point cloud data.

[0045] S203: Convert the index corresponding to the peak point into spatial coordinates to form three-dimensional point cloud data.

[0046] Specifically, the index corresponding to the peak point is converted into spatial coordinates through the following coordinate conversion formula:

[0047]

[0048] Where n represents the FFT peak index of the distance dimension, m represents the FFT peak index of the velocity dimension, k represents the FFT peak index of the angle dimension, ΔR represents the distance resolution, ΔV represents the velocity resolution, and Δθ represents the angle resolution.

[0049] S3, clustering and segmenting the three-dimensional point cloud data to obtain a human target point cloud;

[0050] In this embodiment, step S3 specifically includes:

[0051] S301, clustering adjacent points into independent target clusters according to the spatial density distribution of the three-dimensional point cloud data, so as to preliminarily distinguish potential human body point cloud areas;

[0052] Exemplarily, the DBSCAN clustering algorithm is used to set parameters such as the neighborhood radius, the minimum number of neighborhood points, and the merged space density according to actual conditions, so as to cluster adjacent points into multiple independent target clusters. By setting point cloud density thresholds, size constraints, and other means, target clusters that may represent human bodies are screened out as potential human point cloud areas.

[0053] S302: Based on the potential human body point cloud area, the RANSAC algorithm is used to remove outliers and segment the human body core point cloud.

[0054] Specifically, after clustering in step S301, the potential human body point cloud area may contain real human body reflection points or outliers (such as bed surface vibration noise or environmental interference points). The RANSAC algorithm is used to randomly sample three points, determine a candidate plane equation based on the three points, calculate the distances from all points to the candidate plane, set a suitable threshold, and count the number of inliers whose distance is less than the threshold. After repeated iterations, the plane model with the largest number of inliers is retained, thereby screening out the core point cloud of the human body.

[0055] S4. Based on the human target point cloud, the human body contour is fitted using a rectangular box.

[0056] In this embodiment, step S4 specifically includes:

[0057] S401, calculating the covariance matrix of the human body core point cloud, and performing eigendecomposition on the covariance matrix to obtain three main directions;

[0058] Exemplarily, principal component analysis (PCA) is used to analyze the core point cloud of the human body, and three orthogonal directions are extracted therefrom, representing the main directions of the point cloud distribution, namely, the length, width and height directions.

[0059] S402, projecting the core point cloud of the human body onto three main directions, determining the extreme values ​​of the core point cloud of the human body along the three main directions respectively, so as to determine the parameters of the rectangular box including the center, size and rotation angle, and generating a rectangular box surrounding the human body according to the parameters, so as to reduce the influence of false reflection points in the point cloud data on the estimation of the human body contour.

[0060] In some embodiments, step S4 further includes: predicting the position and posture of the human body through Kalman filtering according to the historical frame data, and dynamically calibrating the parameters of the rectangular parallelepiped frame.

[0061] Specifically, the human body point clouds in multiple historical frames are recorded, and the Kalman filter is used to predict the position and posture of the human body in the next frame. The center point, size and angle of the rectangular box are adjusted according to the predicted value to make the estimation result smoother and more stable.

[0062] Compared with the prior art, the human sleep contour estimation method provided in this embodiment processes the echo signal detected by the radar into a three-dimensional point cloud, clusters the three-dimensional point cloud, segments each human target point cloud, and finally fits the human contour with a rectangular frame. This can effectively reduce the mutual influence of adjacent human bodies and ensure the accuracy of subsequent detection of vital signs information of different human bodies.

[0063] In some embodiments, the accuracy of human sleep profile estimation in multi-person scenarios is improved by using a variety of anti-interference strategies, including:

[0064] (1) In multi-person scenes, since the distance between adjacent human bodies may be relatively close, the point cloud data reflected by the millimeter-wave radar is prone to overlap, making it difficult to accurately distinguish each human target. The following steps are used to effectively separate different human bodies:

[0065] In the acquired three-dimensional point cloud data, the spatial positions of all human targets are calculated, and the distance between adjacent human bodies is estimated based on the three-dimensional spatial coordinates; when the distance between adjacent human bodies is less than the preset threshold, the millimeter-wave radar echo signal is subjected to spectral analysis in the velocity dimension through fast Fourier transform (FFT) to obtain the velocity spectrum diagram of each target; if the difference in the characteristic frequencies of two targets is detected to be greater than the threshold, the two targets are separated into independent human point cloud clusters.

[0066] (2) In order to stably track each human target in a dynamic scene, multi-frame tracking of human targets is achieved by matching target IDs in consecutive point cloud frames. The specific steps are as follows:

[0067] In each frame of point cloud data, a unique target ID is assigned to the detected human target to ensure that the movement trajectory of the same target can be tracked in multiple frames of data; the Hungarian algorithm is used to minimize the Euclidean distance of the target position in adjacent frames to achieve target ID matching; the default target moves at a constant speed between adjacent frames and predicts the next position; when the target fails to match in the current frame, the target is marked as "lost", and interpolation prediction is performed based on the motion trajectory of the previous frames to try to recover the lost target.

[0068] Example 2

[0069] This embodiment provides a human sleep profile estimation device, such as Figure 2 As shown, including:

[0070] A signal acquisition module is used to acquire the echo signal detected by the millimeter wave radar;

[0071] A signal processing module, used for performing a three-dimensional transformation on the echo signal to obtain three-dimensional point cloud data;

[0072] A point cloud processing module, used for clustering and segmenting the three-dimensional point cloud data to obtain a human target point cloud;

[0073] The contour fitting module is used to fit the human body contour using a rectangular box based on the human body target point cloud.

[0074] Example 3

[0075] This embodiment provides an electronic device, such as Figure 3 As shown, it includes a memory and a processor, the memory stores a computer program, and when the computer program is executed by the processor, the human sleep profile estimation method as described in any of the above embodiments is implemented.

[0076] Example 4

[0077] This embodiment provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the method for estimating a human sleep profile as described in any of the above embodiments is implemented.

[0078] Computer readable storage media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0079] The professionals should further realize that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented by electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to the function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0080] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be implemented using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0081] The specific implementation methods described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific implementation method of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for estimating a human sleep profile, characterized in that: include: Obtaining the echo signal detected by the millimeter-wave radar; Performing a three-dimensional transformation on the echo signal to obtain three-dimensional point cloud data; Clustering and segmenting the three-dimensional point cloud data to obtain a human target point cloud; Based on the human target point cloud, the human body contour is fitted using a rectangular box.

2. The method for estimating a human sleep profile according to claim 1, characterized in that: The obtaining of the echo signal detected by the millimeter wave radar includes: Use millimeter wave radar sensors to scan the entire bed surface area and collect echo signals in real time; The echo signal is subjected to noise reduction processing to eliminate high-frequency noise and environmental interference.

3. The method for estimating a human sleep profile according to claim 2, characterized in that: The three-dimensional transformation of the echo signal to obtain three-dimensional point cloud data includes: The denoised echo signal is transformed in three dimensions, namely, distance, velocity and angle, by means of three-dimensional fast Fourier transform to obtain three-dimensional spectrum data including distance image, velocity image and angle information of each target; Performing threshold detection on the three-dimensional spectrum data by a constant false alarm rate algorithm to extract peak points in the three-dimensional spectrum data; The index corresponding to the peak point is converted into spatial coordinates to form three-dimensional point cloud data.

4. The method for estimating a human sleep profile according to claim 1, characterized in that: The clustering of the three-dimensional point cloud data comprises: According to the spatial density distribution of the three-dimensional point cloud data, adjacent points are clustered into independent target clusters to preliminarily distinguish potential human body point cloud areas.

5. The method for estimating a human sleep profile according to claim 4, characterized in that: The segmenting of the three-dimensional point cloud data includes: Based on the potential human point cloud area, the RANSAC algorithm is used to remove outliers and segment the human core point cloud.

6. The method for estimating a human sleep profile according to claim 5, characterized in that: The method of fitting a human body contour using a rectangular box based on a human body target point cloud includes: Calculating the covariance matrix of the human body core point cloud, and performing eigendecomposition on the covariance matrix to obtain three main directions; The core point cloud of the human body is projected onto three main directions, and the extreme values ​​of the core point cloud of the human body are determined along the three main directions respectively to determine the parameters of the rectangular box including the center, size and rotation angle, and a rectangular box surrounding the human body is generated according to the parameters.

7. The method for estimating a human sleep profile according to claim 6, characterized in that: Also includes: According to the historical frame data, the human body position and posture are predicted through Kalman filtering, and the parameters of the rectangular box are dynamically calibrated.

8. A human sleep profile estimation device, characterized in that: The device comprises: A signal acquisition module is used to acquire the echo signal detected by the millimeter wave radar; A signal processing module, used for performing a three-dimensional transformation on the echo signal to obtain three-dimensional point cloud data; A point cloud processing module, used for clustering and segmenting the three-dimensional point cloud data to obtain a human target point cloud; The contour fitting module is used to fit the human body contour using a rectangular box based on the human body target point cloud.

9. An electronic device, characterized in that: The method comprises a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the method for estimating a human sleep profile as described in any one of claims 1 to 7 is implemented.

10. A storage medium, characterized in that: A computer program is stored thereon, and when the program is executed by a processor, the method for estimating a human sleep profile according to any one of claims 1 to 7 is implemented.

Images