A sleep monitoring method based on privacy protection camera and radar

Abstract

The present invention discloses a sleep monitoring method based on a privacy protection camera and a radar, which is applied to a sleep monitoring system based on the privacy protection camera and the radar, comprising: a radar for acquiring radar three-dimensional data and collecting human vital signs; a privacy protection camera, which is arranged in the same monitoring area of ​​the radar and is used to obtain a spatially blurred image; the method comprises: acquiring radar three-dimensional data of the current space by the radar, acquiring a spatially blurred image of the current space by the privacy protection camera, identifying the spatial position and range of a bed by the radar three-dimensional data and the blurred image; acquiring the current human position and posture by the radar three-dimensional data and the spatially blurred image, and judging whether the human body is in bed according to the current human body position and posture; if the human body is currently in bed, locating the human chest position according to the human body position and posture, and collecting human vital signs at the human chest position by the radar.

Description

Technical Field

[0001] The present invention relates to the technical field of sleep monitoring, and in particular to a sleep monitoring method based on a privacy protection camera and radar. Background Art

[0002] Sleep monitoring is an important means of understanding a user's sleep quality, behavioral habits, and potential health issues. By analyzing physiological signals (such as breathing, heart rate, and body movement), health risks such as sleep disorders and abnormal behavior can be promptly identified. Existing microwave radar-based sleep monitoring technology offers the advantages of being contactless and non-invasive. It can continuously monitor sleep status by detecting signals such as heart rate, breathing, and body movement.

[0003] However, in practical applications, single-use radar monitoring still has many limitations. First, radar primarily relies on body motion and micro-motion signals to determine human activity status. However, it struggles to accurately distinguish complex sleeping positions (such as supine, side, and prone), and cannot capture a person's specific spatial posture and posture changes. Furthermore, in home and medical environments, multiple objects and people are often present on or near the bed. Radar signals are susceptible to interference from environmental clutter, metal reflections, and pets, leading to signal aliasing and making it difficult to separate and accurately identify individual motion data. The accuracy and robustness of radar detection are particularly reduced in multi-person scenarios, when objects obstruct the view, or when individuals remain stationary for extended periods.

[0004] To compensate for radar's shortcomings in spatial structure and posture recognition, existing solutions often incorporate visual sensors (such as cameras) for assistance. Cameras can directly capture the spatial structure and posture characteristics of the human body, effectively improving the accuracy of sleeping posture recognition and behavioral identification. However, during the acquisition process, ordinary cameras capture recognizable image information of the user, including facial, biometric, and body shape, posing a high risk of privacy leakage. Even if the relevant image data is not stored, it may be intercepted or misused during transmission or processing, making it difficult to meet privacy compliance and user trust requirements in scenarios such as home, medical, and elderly care.

[0005] The purpose of this invention is to design a sleep monitoring method based on privacy protection camera and radar to address the above-mentioned problems in the existing technology. Summary of the Invention

[0006] In view of this, the purpose of the present invention is to propose a sleep monitoring method based on a privacy protection camera and radar, which can solve the above problems.

[0007] The present invention provides a sleep monitoring method based on a privacy protection camera and radar, which is applied to a sleep monitoring system based on a privacy protection camera and radar, comprising:

[0008] Radar, used to obtain radar three-dimensional data and collect human vital signs;

[0009] A privacy protection camera is installed in the same monitoring area as the radar to obtain spatially blurred images;

[0010] The method comprises:

[0011] The radar obtains three-dimensional data of the current space through the radar, obtains a spatial blurred image of the current space through the privacy protection camera, and identifies the spatial position and range of the bed through the radar three-dimensional data and the blurred image;

[0012] The current position and posture of the human body are obtained through the three-dimensional radar data and spatial fuzzy images, and whether the human body is in bed is determined based on the current position and posture of the human body;

[0013] If the current human body is in bed, the human chest position is located according to the human body position and posture, and the human body signs at the human chest position are collected by radar.

[0014] Furthermore, the privacy protection camera includes a camera and a blurring structure, wherein the blurring structure is arranged on a shooting path of the camera and is used to perform physical blurring processing on an image acquired by the camera;

[0015] The fuzzy structure is any one of a static fuzzy light-transmitting element, a modulated light-transmitting element, and a dynamic fuzzy structure.

[0016] Furthermore, the obtaining of radar three-dimensional data of the current space by radar, obtaining of a spatially blurred image of the current space by a privacy protection camera, and identifying of the spatial position and range of the bed by the radar three-dimensional data and the blurred image include:

[0017] Performing segmentation modeling of spatial objects and human bodies on the spatial fuzzy image, and outputting spatial distribution data of the first object and human body;

[0018] Extracting spatial distribution data of a second object and a human body through three-dimensional radar data;

[0019] Matching and fusing the first object and human body three-dimensional spatial data with the second object and human body three-dimensional spatial data to obtain the object and human body three-dimensional spatial distribution data;

[0020] The spatial coordinate position and range of the bed are marked and identified through the three-dimensional spatial distribution data of objects and human bodies.

[0021] Furthermore, the obtaining of the current human body position and posture through the radar three-dimensional data and the spatially blurred image, and determining whether the human body is in bed according to the current human body position and posture includes:

[0022] The system detects whether there is a human body in the current space through the three-dimensional spatial distribution data of objects and human bodies. If so, the system recognizes the human body's position coordinates and posture through spatial fuzzy images, and reconfirms the human body's position and posture through the three-dimensional radar data.

[0023] The coordinates of the key positions of the human body are calculated by the coordinates of the human body position, and it is determined whether the coordinates of the key positions of the human body fall within the spatial range of the bed. If the human body posture type is a lying type, it is determined that the human body is in bed.

[0024] Furthermore, the detecting whether there is a human body in the current space by using the three-dimensional spatial distribution data of objects and human bodies, identifying the human body position coordinates and posture by using the spatial fuzzy image, and secondarily confirming the human body position and posture by using the three-dimensional radar data includes:

[0025] The three-dimensional spatial distribution data of objects and human bodies is used to detect whether there is a human body in the current space. If so, a feature detection algorithm is used to extract the feature points of the spatial blurred image, and the feature points are matched with the feature points of the reference space to obtain the corresponding relationship between the feature points of the two;

[0026] According to the correspondence between the feature points, the spatial geometry algorithm is used to solve the image body position coordinates, and the current human posture type is identified through the image body position coordinates;

[0027] Obtain the coordinates of the radar 3D data at the same time point, convert the coordinates of the radar 3D data and the image human body position coordinates to the same coordinate system, calculate the mean square error between the coordinates of the radar 3D data and the image human body position coordinates. If the mean square error is less than the error threshold, the two are similar, and output the radar 3D data coordinates as the human body position coordinates and the current human body posture type.

[0028] Furthermore, the obtaining of the current human body position and posture through the radar three-dimensional data and the spatially blurred image, and determining whether the human body is in bed according to the current human body position and posture includes:

[0029] Focusing the radar beam within the spatial range of the bed, collecting radar echo signals within the spatial range of the bed, and analyzing whether the radar echo signals within the spatial range of the bed contain radar micro-motion signals;

[0030] The spatially blurred image is used to determine whether there is a human body posture within the spatial range of the bed. If there is a radar micro-motion signal and a human body posture within the spatial range of the bed at the same time point, it is determined that a person is in the bed. Otherwise, the privacy protection camera is turned off.

[0031] Furthermore, focusing the radar beam on the spatial range of the bed, collecting the radar echo signal within the spatial range of the bed, and analyzing whether the radar echo signal within the spatial range of the bed contains a radar micro-motion signal includes:

[0032] The radar echo signal within the spatial range of the bed is used to extract the radar body motion signal. If the radar body motion signal exists, the human body posture is judged based on the spatial fuzzy image.

[0033] If there is no radar body motion signal in the radar echo signal within the spatial range of the bed, radar micro-motion signal extraction is performed on the radar echo signal within the spatial range of the bed. If there is a radar micro-motion signal, human body posture judgment is performed on the spatial blurred image.

[0034] Furthermore, the steps of extracting the radar body motion signal are as follows:

[0035] The radar echo signal within the spatial range of the bed is subjected to multiple Fourier transform extractions to obtain the high-speed spectrum mean, which is used as the radar body motion signal.

[0036] Furthermore, the steps of extracting the radar micro-motion signal are as follows:

[0037] Performing one-dimensional Fourier transform on the radar echo signal within the spatial range of the bed and then performing frame accumulation to obtain frame accumulation data;

[0038] After performing two-dimensional Fourier transform on the frame-accumulated data, the low-speed spectrum and high-speed spectrum peaks are extracted respectively, and the ratio of the low-speed spectrum to the high-speed spectrum is calculated to obtain the human body micro-motion signal.

[0039] Furthermore, if the current human body is in bed, the human chest position is located according to the human body position and posture, and the human body signs of the human chest position are collected by radar, including:

[0040] Extract the 3D coordinates of the chest cavity through the human body position coordinates, and correct the 3D coordinates of the chest cavity according to the human body's sleeping posture;

[0041] Set a space radar monitoring area with the three-dimensional coordinates of the chest cavity as the center, and focus the radar beam on the space radar monitoring area;

[0042] Receive radar echo signals from the space radar monitoring area, and extract the respiratory frequency and heart rate of the radar echo signals after filtering and denoising.

[0043] Beneficial effects of the present invention:

[0044] First, the privacy protection camera is used to collect blurred physical images to ensure that no clear privacy images are generated from the source. Even if the algorithm is maliciously tampered with or attacked by hackers, the original picture cannot be reconstructed. It takes into account both artificial intelligence functions and user privacy protection. Radar and privacy protection cameras each have advantages and disadvantages in their perception of human posture and spatial objects. They can complement each other when combined. The combination of the two can support a variety of posture anomaly detection.

[0045] Secondly, by performing spatial object and human body segmentation modeling on the spatially blurred images collected by the privacy protection camera, the spatial distribution data of objects and human bodies in the image domain is obtained. The spatial distribution data of objects and human bodies obtained by the three-dimensional radar data is mapped and integrated into the spatial distribution data of objects and human bodies in the image domain. The three-dimensional distribution data of objects and human bodies can be obtained, which is used to mark the spatial position and range of the bed.

[0046] The third is to judge whether the human body is in bed in the current space by combining the corresponding human posture and type in the spatial fuzzy image and radar three-dimensional data or radar three-dimensional data and spatial fuzzy image, thereby reducing the probability of false alarm / missing alarm and laying the basis for subsequent human vital signs monitoring.

[0047] Fourth, the position of the human chest cavity is located by fusing spatial fuzzy images and radar three-dimensional data, and the radar focus area is adjusted to improve the separation quality and accuracy of vital sign signals. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0049] Figure 1 It is a system module diagram of embodiment 1.

[0050] Figure 2 This is a flow chart of the method of embodiment 2. DETAILED DESCRIPTION

[0051] To facilitate understanding by those skilled in the art, the structure of the present invention will now be further described in detail with reference to the embodiments and accompanying drawings. It should be understood that the steps mentioned in this embodiment, unless otherwise specified, can be adjusted in sequence according to actual needs, and can even be executed simultaneously or partially simultaneously.

[0052] Example 1

[0053] like Figure 1 As shown, embodiment 1 provides a sleep monitoring system based on a privacy protection camera and radar, including:

[0054] Radar, used to obtain radar three-dimensional data and collect human vital signs;

[0055] A privacy protection camera is installed in the same monitoring area as the radar to obtain spatially blurred images;

[0056] Specifically, the privacy protection camera includes a camera and a blur structure. The blur structure is set on the shooting path of the camera and is used to perform physical blur processing on the image captured by the camera.

[0057] Furthermore, the blur structure is any one of a static blur light-transmitting element, a modulated light-transmitting element, and a dynamic blur structure.

[0058] In this embodiment, traditional camera monitoring involves private images such as faces, body features, and environmental details. Once stored or transmitted, these images are at risk of being leaked or misused, creating privacy and security issues. Using a fuzzy structure to physically blur the image at the camera's front end ensures that no clear private images are generated at the source. Even if the algorithm is maliciously tampered with or hacked, the original image cannot be reconstructed. This balances AI functionality with user privacy protection, meeting privacy compliance requirements in multiple scenarios, including home, medical, and elderly care.

[0059] While radar-based monitoring can contactlessly capture physiological signals such as respiration and heart rate, its accuracy in identifying sleeping positions and behaviors (such as turning over, leaving the bed, unusual body positions, and distinguishing multiple people) is limited. In contrast, purely visual methods offer some accuracy, but privacy risks are difficult to manage. Blurred structures limit the image's representation of the general dynamics and contours of the monitored space, preventing privacy breaches. By combining radar with privacy-preserving cameras, "fuzzy vision" provides spatial structure and movement trends of the human body, supplemented by radar's three-dimensional body contours and physiological signals, enabling high-precision sleeping position recognition, dynamic monitoring, and bed exit detection.

[0060] The primary goal of static blurring translucent components is to allow light to pass through but scatter it sufficiently to obscure details from the camera's image. Examples include frosted glass, frosted acrylic, frosted polycarbonate, atomized PET film, and nano-transparent and scattering coatings. These components are fixed in the camera's path and blur the image through their inherent light-scattering properties.

[0061] Modulated light-transmitting elements include, but are not limited to, elements that achieve controllable light transmission or blurring through heating, electro-modulation, mechanical vibration, liquid flow, acoustic disturbance, and other methods. Examples include heated modulating elements (such as heated atomizing films), electrochromic or electro-scattering smart dimming films, mechanical vibration or micro-disturbance films, liquid-modulated light-transmitting elements, and acoustically modulated light-transmitting elements. For example, materials such as PET film and glass that can be heated on the surface can achieve dynamic blurring of images by inducing changes in the material's physical state (such as atomization, condensation, or optical disturbance) due to heating.

[0062] A dynamic blur structure, such as a high-speed mechanical wiper set on the camera shooting path, drives the wiper to reciprocate or rotate at a specific frequency, forming a continuous optical smear or occlusion effect through dynamic disturbance, making the captured image unrecognizable in time and space.

[0063] Example 2

[0064] This embodiment is based on a sleep monitoring system based on a privacy protection camera and radar provided in the first embodiment. Figure 2 As shown, embodiment 2 provides a sleep monitoring method based on a privacy protection camera and radar, including:

[0065] S1 obtains the current space's three-dimensional radar data through radar, obtains a spatially blurred image of the current space through a privacy protection camera, and identifies the bed's spatial position and range through the radar three-dimensional data and the blurred image;

[0066] S101 performs segmentation modeling of spatial objects and human bodies on the spatially blurred image, and outputs spatial distribution data of the first object and human body;

[0067] S102 extracts spatial distribution data of a second object and a human body through the radar three-dimensional data;

[0068] S103 matches and fuses the first object and human body three-dimensional spatial data with the second object and human body three-dimensional spatial data to obtain object and human body three-dimensional spatial distribution data;

[0069] S104 marks and identifies the spatial coordinate position and range of the bed through the three-dimensional spatial distribution data of objects and human bodies.

[0070] In this step, the 3D spatial distribution data of objects and people includes the location and bounding box of each object and person. Spatial object and person segmentation models can use methods such as SegFormer and Mask R-CNN. The second spatial distribution data of objects and people acquired by radar is mapped and fused to the spatially blurred image to obtain the first spatial distribution data of objects and people. This 3D distribution data of objects and people is then obtained, serving as the basis for subsequent in-bed detection. 3D radar data can be obtained by performing multiple Fourier transforms on radar echo data to obtain frequency domain features such as velocity spectrum, range spectrum, and angle spectrum, or by processing frequency domain features into 3D point cloud data.

[0071] S2 obtains the current human body position and posture through radar 3D data and spatial fuzzy image, and determines whether the human body is in bed based on the current human body position and posture;

[0072] S201 detects whether there is a human body in the current space through the three-dimensional spatial distribution data of objects and human bodies, and if so, identifies the human body's position coordinates and posture through the spatial fuzzy image, and reconfirms the human body's position and posture through the three-dimensional radar data;

[0073] S2011 detects whether there is a human body in the current space through the three-dimensional spatial distribution data of objects and human bodies, and if so, extracts feature points of the spatial blurred image using a feature detection algorithm, matches the feature points with feature points of the reference space, and obtains a corresponding relationship between the feature points of the two;

[0074] S2012 obtains the image human body position coordinates based on the correspondence between the feature points by using a spatial geometry algorithm, and identifies the current human body posture type through the image human body position coordinates;

[0075] S2013 obtains the coordinates of the radar three-dimensional data at the same time point, converts the coordinates of the radar three-dimensional data and the image human body position coordinates into the same coordinate system, calculates the mean square error between the coordinates of the radar three-dimensional data and the image human body position coordinates, and if the mean square error is less than the error threshold, the two are similar, and outputs the radar three-dimensional data coordinates as the human body position coordinates and the current human body posture type.

[0076] In this step, the three-dimensional spatial distribution data of objects and human bodies obtained in step S1 can roughly confirm whether there is a human body in the current space. If there is no human body, there is no need to proceed with the subsequent calculation process. If there is a human body, further refined position and posture calculation is required, and the specific position and posture of the human body are confirmed through spatial blurred images and radar three-dimensional data.

[0077] Even if the image is blurry, as long as there are key contours and edges, algorithms like SIFT and ORB can still identify key feature points, match them with feature points in the reference space, and establish a spatial mapping relationship. Leveraging feature point correspondences and geometric algorithms, the spatial position coordinates of the body in the image can be determined. Pre-defined posture recognition models can then be used for classification based on the spatial distribution of key points and principal axis angles. After unifying the 3D point cloud and image coordinate systems, calculating the mean squared error allows for detection consistency. A small error indicates high consistency between the two detection results, allowing accurate output of the body's position coordinates and posture type for subsequent in-bed assessment.

[0078] S202 calculates the coordinates of the key positions of the human body through the coordinates of the human body position, and determines whether the coordinates of the key positions of the human body fall within the spatial range of the bed. If the human body posture type is a lying type, it is determined that the human body is in bed.

[0079] In this step, the coordinates of each joint position can be directly selected through an algorithm (such as OpenPose), and the key position coordinates for lying down can be selected for subsequent judgment, with a focus on the joint position coordinates of the head, neck, shoulders, hips / hips, knees, ankles, etc. In addition, the coordinates of multiple joint positions can be combined to calculate the coordinates of the hips, hips, and the midpoint position / average point position of the hips as the key position coordinates of the human body, representing the position of the main weight-bearing parts of the human body, to ensure that the head and feet outside the bed will not affect the judgment based on the "trunk main point". When the coordinates of the key position of the human body fall within the spatial range coordinates of the bed obtained in step S1, and the posture type obtained in step S201 is a lying type, it is confirmed that the human body is in bed. This can avoid the situation where a person is standing / sitting at the edge of the bed and is judged as being in bed.

[0080] S3: If the current human body is in bed, the human chest position is located according to the human body position and posture, and the human body signs at the human chest position are collected by radar.

[0081] S301 extracts the three-dimensional coordinates of the chest cavity through the human body position coordinates, and corrects the three-dimensional coordinates of the human chest cavity according to the human body sleeping posture;

[0082] S302 sets a space radar monitoring area with the three-dimensional coordinates of the chest cavity as the center, and focuses the radar beam on the space radar monitoring area;

[0083] S303 receives the radar echo signal of the space radar monitoring area, and extracts the respiratory frequency and heart rate of the radar echo signal after filtering and denoising.

[0084] In this step, the chest cavity is the most direct and prominent location for vital signs like breathing and heartbeat. By using the body's overall spatial coordinates and combining them with sleep posture semantics to correct the specific chest cavity location, we can precisely lock onto the monitoring area, avoiding signal interference from the pelvis and limbs.

[0085] The actual position and orientation of the chest in three-dimensional space will change depending on the sleeping posture. Dynamic correction of the chest coordinates ensures that accurate data can be collected regardless of body posture or turning over after falling asleep. For example, when lying on your back, the chest point = center of gravity is offset upward (Z-axis) by a certain distance. When lying on your side, the chest point = center of gravity is offset toward the side of the main axis of the body. When lying on your stomach, the chest point = center of gravity is offset downward (Z-axis) by a certain distance. The specific distance can be obtained through actual measurement or training data fitting.

[0086] By setting a region of interest (ROI) centered on the chest coordinates and adjusting the radar beam, monitoring resources and data processing capabilities are focused on the most valuable area, background clutter is effectively filtered, and the separation quality and accuracy of vital sign signals are improved. Extracting human respiratory rate and heart rate through radar echo signals is an existing technology and will not be described in detail in this application.

[0087] Example 3

[0088] This embodiment is based on the sleep monitoring system based on the privacy protection camera and radar provided in the first embodiment. The difference between this embodiment and the second embodiment lies in the different process of determining whether a person is in bed. Specifically:

[0089] Focus the radar beam on the spatial range of the bed, collect the radar echo signal within the spatial range of the bed, and analyze whether the radar echo signal within the spatial range of the bed contains radar micro-motion signals. Specifically:

[0090] The radar echo signal within the spatial range of the bed is used to extract the radar body motion signal. If the radar body motion signal exists, the human body posture is judged based on the spatial fuzzy image.

[0091] If there is no radar body motion signal in the radar echo signal within the spatial range of the bed, radar micro-motion signal extraction is performed on the radar echo signal within the spatial range of the bed. If there is a radar micro-motion signal, human body posture judgment is performed on the spatial blurred image.

[0092] The steps of extracting the radar body motion signal are as follows:

[0093] The radar echo signal within the spatial range of the bed is subjected to multiple Fourier transform extractions to obtain the high-speed spectrum mean, which is used as the radar body motion signal.

[0094] The steps of extracting the radar micro-motion signal are as follows:

[0095] Performing one-dimensional Fourier transform on the radar echo signal within the spatial range of the bed and then performing frame accumulation to obtain frame accumulation data;

[0096] After performing two-dimensional Fourier transform on the frame-accumulated data, the low-speed spectrum and high-speed spectrum peaks are extracted respectively, and the ratio of the low-speed spectrum to the high-speed spectrum is calculated to obtain the human body micro-motion signal.

[0097] In this step, body motion signals have much higher energy, making them easy to detect with a low false positive rate. First, assessing body motion can quickly screen for occupants (most movements like turning over and getting in and out of bed are obvious). If no body motion is detected, then micro-motion is assessed. Micro-motion signals can sensitively reflect subtle movements, such as those associated with resting or breathing, helping to avoid missing individuals who are quietly lying down.

[0098] The spatially blurred image is used to determine whether there is a human body posture within the spatial range of the bed. If there is a radar micro-motion signal and a human body posture within the spatial range of the bed at the same time point, it is determined that a person is in the bed. Otherwise, the privacy protection camera is turned off.

[0099] In this step, although radar (especially millimeter-wave radar) has high sensitivity in detecting body motion and micro-movement, it can be affected by environmental interference (such as metal, fans, pets, etc.), clutter, and artificial body movements, leading to false positives. The micro-movement signal of a person who has been stationary for a long time is extremely weak, and may be mistakenly interpreted as an absence. If there are other moving objects near the bed (such as a caregiver or pet), the radar may not be able to accurately distinguish them. Therefore, spatially blurred images captured by the privacy protection camera are needed for auxiliary judgment. If a "human shape" is detected within the spatial range of the bed in the spatially blurred image, the presence of a person in the bed can be confirmed, reducing the probability of false positives and false negatives, and laying the foundation for subsequent human vital sign monitoring.

[0100] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Thus, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0101] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowcharts and / or block diagrams, as well as combinations of processes and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowcharts and / or block diagrams. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0102] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0103] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0104] It should be noted that in the claims, any reference signs placed between parentheses shall not be construed as limiting the claims. The word "comprising" does not exclude the presence of components or steps not listed in the claim. The word "a" or "an" preceding a component does not exclude the presence of a plurality of such components. The invention can be implemented by means of hardware comprising several distinct components and by means of a suitably programmed computer. In a unit claim enumerating several means, several of these means may be embodied by one and the same item of hardware. The use of the words first, second, third etc. does not indicate any order. These words may be interpreted as names.

[0105] Although the preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0106] Obviously, those skilled in the art may make various modifications and variations to the present invention without departing from the spirit and scope of the present invention. Thus, if such modifications and variations fall within the scope of the claims and their equivalents, the present invention is intended to include such modifications and variations.

[0107] In the present invention, unless otherwise expressly specified or limited, the terms "mounted," "connected," "connect," "fixed," etc. should be understood broadly. For example, they may refer to fixed connection, detachable connection, or integration; mechanical connection or electrical connection; direct connection or indirect connection through an intermediate medium; internal communication between two components or interaction between two components. Those skilled in the art will understand the specific meanings of the above terms in the present invention based on specific circumstances.

[0108] In the description of this specification, the reference terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" mean that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic expressions of the above terms should not be understood as necessarily referring to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art can combine and combine different embodiments or examples described in this specification and features of different embodiments or examples without contradiction.

Claims

1. A sleep monitoring method based on privacy protection camera and radar, characterized in that: Applied to a sleep monitoring system based on privacy-preserving cameras and radar, including: Radar, used to obtain radar three-dimensional data and collect human vital signs; A privacy protection camera is installed in the same monitoring area as the radar to obtain spatially blurred images; The privacy protection camera includes a camera and a blurring structure, wherein the blurring structure is arranged on a shooting path of the camera and is used to perform physical blurring processing on an image acquired by the camera; The fuzzy structure is any one of a static fuzzy light-transmitting element, a modulated light-transmitting element, and a dynamic fuzzy structure; The method comprises: The radar obtains the current space's three-dimensional radar data, and the privacy protection camera obtains the current space's spatial blurred image. The spatial position and range of the bed are identified by the radar three-dimensional data and the blurred image. Specifically: Performing segmentation modeling of spatial objects and human bodies on the spatial fuzzy image, and outputting spatial distribution data of the first object and human body; Extracting spatial distribution data of a second object and a human body through three-dimensional radar data; Matching and fusing the first object and human body three-dimensional spatial data with the second object and human body three-dimensional spatial data to obtain the object and human body three-dimensional spatial distribution data; Mark and identify the spatial coordinate position and range of the bed through the three-dimensional spatial distribution data of objects and human bodies; The current position and posture of the human body are obtained through the three-dimensional radar data and the spatial fuzzy image. Based on the current position and posture of the human body, it is determined whether the human body is in bed. Specifically: The system detects whether there is a human body in the current space through the three-dimensional spatial distribution data of objects and human bodies. If so, the system recognizes the human body's position coordinates and posture through spatial fuzzy images, and reconfirms the human body's position and posture through the three-dimensional radar data. Calculate the coordinates of the key positions of the human body through the coordinates of the human body position, and determine whether the coordinates of the key positions of the human body fall within the spatial range of the bed. If the human body posture type is lying down, it is determined that the human body is in bed; If the current human body is in bed, the human chest position is located according to the human body position and posture, and the human body signs at the human chest position are collected by radar.

2. The sleep monitoring method based on privacy protection camera and radar according to claim 1, characterized in that: The method of detecting whether there is a human body in the current space by using the three-dimensional spatial distribution data of objects and human bodies, and recognizing the position coordinates and posture of the human body by using the spatial fuzzy image, and confirming the position and posture of the human body by using the three-dimensional radar data includes: The three-dimensional spatial distribution data of objects and human bodies is used to detect whether there is a human body in the current space. If so, a feature detection algorithm is used to extract the feature points of the spatial blurred image, and the feature points are matched with the feature points of the reference space to obtain the corresponding relationship between the feature points of the two; According to the correspondence between the feature points, the spatial geometry algorithm is used to solve the image body position coordinates, and the current human posture type is identified through the image body position coordinates; Obtain the coordinates of the radar 3D data at the same time point, convert the coordinates of the radar 3D data and the image human body position coordinates to the same coordinate system, calculate the mean square error between the coordinates of the radar 3D data and the image human body position coordinates. If the mean square error is less than the error threshold, the two are similar, and output the radar 3D data coordinates as the human body position coordinates and the current human body posture type.

3. The sleep monitoring method based on privacy protection camera and radar according to claim 1, characterized in that: The method of obtaining the current position and posture of the human body through the radar three-dimensional data and the spatial fuzzy image, and determining whether the human body is in bed according to the current position and posture of the human body includes: Focusing the radar beam within the spatial range of the bed, collecting radar echo signals within the spatial range of the bed, and analyzing whether the radar echo signals within the spatial range of the bed contain radar micro-motion signals; The spatially blurred image is used to determine whether there is a human body posture within the spatial range of the bed. If there is a radar micro-motion signal and a human body posture within the spatial range of the bed at the same time point, it is determined that a person is in the bed. Otherwise, the privacy protection camera is turned off.

4. The sleep monitoring method based on privacy protection camera and radar according to claim 3, characterized in that: Focusing the radar beam on the spatial range of the bed, collecting the radar echo signal within the spatial range of the bed, and analyzing whether the radar echo signal within the spatial range of the bed contains a radar micro-motion signal includes: The radar echo signal within the spatial range of the bed is used to extract the radar body motion signal. If the radar body motion signal exists, the human body posture is judged based on the spatial fuzzy image. If there is no radar body motion signal in the radar echo signal within the spatial range of the bed, radar micro-motion signal extraction is performed on the radar echo signal within the spatial range of the bed. If there is a radar micro-motion signal, human body posture judgment is performed on the spatial blurred image.

5. The sleep monitoring method based on privacy protection camera and radar according to claim 4, characterized in that: The steps for extracting the radar body motion signal are as follows: The radar echo signal within the spatial range of the bed is subjected to multiple Fourier transform extractions to obtain the high-speed spectrum mean, which is used as the radar body motion signal.

6. The sleep monitoring method based on privacy protection camera and radar according to claim 4, characterized in that: The steps of extracting the radar micro-motion signal are as follows: Performing one-dimensional Fourier transform on the radar echo signal within the spatial range of the bed and then performing frame accumulation to obtain frame accumulation data; After performing two-dimensional Fourier transform on the frame-accumulated data, the low-speed spectrum and high-speed spectrum peaks are extracted respectively, and the ratio of the low-speed spectrum to the high-speed spectrum is calculated to obtain the human body micro-motion signal.

7. The sleep monitoring method based on privacy protection camera and radar according to claim 1, characterized in that: If the current human body is in bed, the human chest position is located according to the human body position and posture, and the human body signs of the human chest position collected by radar include: Extract the 3D coordinates of the chest cavity through the human body position coordinates, and correct the 3D coordinates of the chest cavity according to the human body's sleeping posture; Set a space radar monitoring area with the three-dimensional coordinates of the chest cavity as the center, and focus the radar beam on the space radar monitoring area; Receive radar echo signals from the space radar monitoring area, and extract the respiratory frequency and heart rate of the radar echo signals after filtering and denoising.

Images