Sleep monitoring method based on privacy protection camera and radar

Abstract

The invention discloses a sleep monitoring method based on a privacy protection camera and a radar, which is applied to a sleep monitoring system based on the privacy protection camera and the radar, and comprises the radar used for obtaining radar three-dimensional data and collecting human body signs; the privacy protection camera is arranged in the same monitoring area of the radar and is used for acquiring a space blurred image; the method comprises the following steps: acquiring radar three-dimensional data of a current space through a radar, acquiring a spatial blurred image of the current space through a privacy protection camera, and identifying the spatial position and range of a bed through the radar three-dimensional data and the blurred image; obtaining the current human body position and posture through the radar three-dimensional data and the spatial blurred image, and judging whether the human body is in the bed or not according to the current human body position and posture; if the current human body is in the bed, positioning the chest position of the human body according to the human body position and posture, and collecting human body signs of the chest position of the human body through a radar.

Description

Technical Field

[0001] The present invention relates to the technical field of sleep monitoring, and in particular to a sleep monitoring method based on a privacy protection camera and a radar. Background Art

[0002] Sleep monitoring is an important means of obtaining information about a user's sleep quality, behavioral habits, and potential health problems. By analyzing physiological signals (such as breathing, heart rate, and body movement), health risks such as sleep disorders and abnormal behaviors can be discovered in a timely manner. Existing sleep monitoring technology based on microwave radar has the advantages of being non-contact and non-invasive, and can continuously monitor sleep status by detecting signals such as the human heart rate, breathing, and body movement.

[0003] However, in practical applications, single radar monitoring still has many limitations. First, radar mainly relies on body motion and micro-motion signals to determine the activity status of the human body, but it is difficult to accurately distinguish complex human sleeping positions (such as supine, side, prone, etc.), and it is impossible to obtain the specific spatial posture and posture changes of the human body. In addition, in home and medical environments, there are often multiple objects and people on or beside the bed. Radar signals are easily interfered by environmental clutter, metal reflections, pets, etc., resulting in signal aliasing, making it difficult to separate and accurately identify individual dynamic data. Especially in multi-person scenes, when objects are blocked or the static time is long, the accuracy and robustness of radar detection are further reduced.

[0004] To make up for the shortcomings of radar in spatial structure and posture recognition, existing solutions usually introduce visual sensors (such as cameras) for assistance. Cameras can directly capture the spatial structure and posture characteristics of the human body, effectively improving the accuracy of sleeping posture recognition and behavior discrimination. However, ordinary cameras will obtain recognizable image information of users during the collection process, including facial, biometric features, body shape, etc., which poses a high risk of privacy leakage. Even if the relevant image data is not stored, it may be intercepted or abused during transmission or processing, making it difficult to meet privacy compliance and user trust requirements in scenarios such as home, medical, and elderly care.

[0005] In view of the above-mentioned problems in the prior art, the purpose of the present invention is to design a sleep monitoring method based on a privacy protection camera and radar. Summary of the invention

[0006] In view of this, the purpose of the present invention is to propose a sleep monitoring method based on a privacy protection camera and radar, which can solve the above-mentioned problems.

[0007] The present invention provides a sleep monitoring method based on a privacy protection camera and a radar, which is applied to a sleep monitoring system based on a privacy protection camera and a radar, comprising: Radar, used to obtain radar three-dimensional data and collect human vital signs; A privacy protection camera, which is set in the same monitoring area as the radar and is used to obtain a spatially blurred image; The method includes: Obtain the radar three-dimensional data of the current space through the radar, obtain the spatially blurred image of the current space through the privacy protection camera, and identify the spatial position and range of the bed through the radar three-dimensional data and the blurred image; Obtain the current human body position and posture through the radar three-dimensional data and the spatially blurred image, and judge whether the human body is in bed according to the current human body position and posture; If the current human body is in bed, then locate the position of the human chest according to the human body position and posture, and collect the human body signs at the position of the human chest through the radar.

[0008] Further, the privacy protection camera includes a camera and a blurring structure, and the blurring structure is arranged on the shooting path of the camera and is used to perform physical blurring processing on the image obtained by the camera; The blurring structure is any one of a static blurring light-transmitting member, a modulating light-transmitting element, and a dynamic blurring structure.

[0009] Further, the obtaining the radar three-dimensional data of the current space through the radar, obtaining the spatially blurred image of the current space through the privacy protection camera, and identifying the spatial position and range of the bed through the radar three-dimensional data and the blurred image includes: Perform segmentation modeling of spatial objects and the human body on the spatially blurred image, and output the spatial distribution data of the first object and the human body; Extract the spatial distribution data of the second object and the human body through the radar three-dimensional data; Match and fuse the three-dimensional spatial data of the first object and the human body and the three-dimensional spatial data of the second object and the human body to obtain the three-dimensional spatial distribution data of the object and the human body; Mark and identify the spatial coordinate position and range of the bed through the three-dimensional spatial distribution data of the object and the human body.

[0010] Further, the obtaining the current human body position and posture through the radar three-dimensional data and the spatially blurred image, and judging whether the human body is in bed according to the current human body position and posture includes: Detect whether there is a human body in the current space through the three-dimensional spatial distribution data of the object and the human body. If so, identify the human body position coordinates and posture through the spatially blurred image, and reconfirm the human body position and posture through the radar three-dimensional data; Calculate the key position coordinates of the human body through the human body position coordinates, judge whether the key position coordinates of the human body fall within the spatial range of the bed, and if the human body posture type is the lying type, it is determined that the human body is in bed.

[0011] Further, detecting whether there is a human body in the current space through the three-dimensional spatial distribution data of the object and the human body, and if so, identifying the position coordinates and posture of the human body through the spatially blurred image, and reconfirming the human body position and posture through the radar three-dimensional data includes: Detecting whether there is a human body in the current space through the three-dimensional spatial distribution data of the object and the human body. If so, using a feature detection algorithm to extract the feature points of the spatially blurred image, and matching the feature points with the feature points of the reference space to obtain the corresponding relationship of the feature points between the two; According to the corresponding relationship of the feature points, solving through a spatial geometric algorithm to obtain the image human body position coordinates, and identifying the current human body posture type through the image human body position coordinates; Obtaining the coordinates of the radar three-dimensional data at the same time point, converting the coordinates of the radar three-dimensional data and the image human body position coordinates to the same coordinate system, calculating the mean square error between the coordinates of the radar three-dimensional data and the image human body position coordinates. If the mean square error is less than the error threshold, the two are similar, and the radar three-dimensional data coordinates are output as the human body position coordinates and the current human body posture type.

[0012] Further, obtaining the current human body position and posture through the radar three-dimensional data and the spatially blurred image, and judging whether the human body is in bed according to the current human body position and posture includes: Focusing the radar beam on the spatial range of the bed, collecting the radar echo signals within the spatial range of the bed, and analyzing whether there is a radar micro-motion signal in the radar echo signals within the spatial range of the bed; Judging whether there is a human body posture within the spatial range of the bed through the spatially blurred image. If there is a radar micro-motion signal and a human body posture within the spatial range of the bed at the same time point, it is determined that the human body is in bed, otherwise the privacy protection camera is turned off.

[0013] Further, the step of focusing the radar beam on the spatial range of the bed, collecting the radar echo signals within the spatial range of the bed, and analyzing whether there is a radar micro-motion signal in the radar echo signals within the spatial range of the bed includes: Extracting the radar body motion signal from the radar echo signals within the spatial range of the bed. If there is a radar body motion signal, then judging the human body posture of the spatially blurred image; If there is no radar body motion signal in the radar echo signals within the spatial range of the bed, then extracting the radar micro-motion signal from the radar echo signals within the spatial range of the bed. If there is a radar micro-motion signal, then judging the human body posture of the spatially blurred image.

[0014] Further, the extraction steps of the radar body motion signal are as follows: Performing multiple Fourier transforms on the radar echo signals within the spatial range of the bed to extract the high-speed spectrum mean value, and using the high-speed spectrum mean value as the radar body motion signal.

[0015] Further, the steps for extracting the radar micro-motion signal are as follows: Perform one-dimensional Fourier transform on the radar echo signal within the spatial range of the bed and then perform frame extraction and accumulation to obtain frame-extracted and accumulated data; Perform two-dimensional Fourier transform on the frame-extracted and accumulated data, then extract the peak values of the low-speed spectrum and the high-speed spectrum respectively, calculate the ratio of the low-speed spectrum to the high-speed spectrum, and obtain the human micro-motion signal.

[0016] Further, if the current human is in bed, the position of the human chest is located according to the human position and posture, and the human body signs collected by the radar at the position of the human chest include: Extract the three-dimensional coordinates of the chest through the human position coordinates and correct the three-dimensional coordinates of the human chest according to the human sleeping posture; Taking the three-dimensional coordinates of the chest as the center, set up a spatial radar monitoring area and focus the radar beam on the spatial radar monitoring area; Receive the radar echo signal in the spatial radar monitoring area and extract the breathing frequency and heart rate of the radar echo signal after filtering and denoising.

[0017] Advantages of the present invention: First, physical image blurring is performed through the privacy protection camera for acquisition, ensuring that no clear privacy images will be generated from the source. Even if the algorithm is maliciously tampered with or hacked, the original image cannot be reconstructed. It takes into account both the artificial intelligence function and user privacy protection. The radar and the privacy protection camera have their own advantages and disadvantages in perceiving human postures and spatial objects. When combined, they can complement each other, and the combination of the two can support diverse abnormal posture detections.

[0018] Second, by performing segmentation and modeling of spatial objects and humans on the spatially blurred images collected by the privacy protection camera to obtain the spatial distribution data of objects and humans in the image domain, and mapping and fusing the spatial distribution data of objects and humans obtained from the radar to the spatial distribution data of objects and humans in the image domain, three-dimensional spatial distribution data of objects and humans can be obtained, which is used to mark the spatial position and range of the bed.

[0019] Third, by combining the corresponding human postures and types in the spatially blurred images and the radar three-dimensional data in sequence, or the radar three-dimensional data and the spatially blurred images, it is determined whether the current human in the space is in bed, reducing the false alarm / miss rate and serving as the monitoring basis for subsequent human body sign monitoring.

[0020] Fourth, the position of the human chest is located by fusing the spatially blurred images and the radar three-dimensional data, and at the same time, the radar focusing area is adjusted to improve the separation quality and accuracy of the sign signals. Description of the Drawings

[0021] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0022] Figure 1 is the system module diagram of Embodiment 1.

[0023] Figure 2 is the method flow chart of Embodiment 2. Detailed implementation manners

[0024] For the convenience of those skilled in the art to understand, the embodiments will now be further described in detail in conjunction with the drawings for the structure of the present invention. It should be understood that the steps mentioned in this embodiment, unless specifically stating their order, can be adjusted in their front and rear order according to actual needs, and even can be executed simultaneously or partially simultaneously.

[0025] Embodiment 1 As Figure 1 shown, Embodiment 1 provides a sleep monitoring system based on a privacy - protected camera and radar, including: A radar, used to obtain radar three - dimensional data and collect human body signs; A privacy - protected camera, arranged in the same monitoring area as the radar, used to obtain spatially blurred images; Specifically, the privacy - protected camera includes a camera and a blurring structure. The blurring structure is arranged on the shooting path of the camera and is used to perform physical blurring processing on the images obtained by the camera.

[0026] Further, the blurring structure is any one of a static blurring light - transmitting member, a modulating light - transmitting element, and a dynamic blurring structure.

[0027] In this embodiment, traditional camera monitoring involves privacy images such as human faces, body features, and environmental details. Once the images are stored or transmitted, there is a risk of leakage or abuse, resulting in privacy and security problems. Using the blurring structure to perform physical image blurring at the front end of the camera ensures that no clear privacy images are generated from the source. Even if the algorithm is maliciously tampered with or hacked, the original picture cannot be reconstructed, taking into account both the artificial intelligence function and user privacy protection, and meeting the privacy compliance requirements in multiple scenarios such as home / medical / elderly care.

[0028] Although single-radar monitoring can non-contactedly obtain physiological signals such as respiration and heart rate, its accuracy in judging sleeping postures and identifying human behaviors (such as turning over, getting out of bed, abnormal postures, differentiating multiple people) is limited. In contrast, pure visual methods have a certain recognition accuracy, but it is difficult to control privacy risks. The blurred structure only allows the image to present the general dynamics and contours of the monitored space, preventing the leakage of personal privacy. By combining radar with a privacy-protecting camera, "blurred vision" provides the human body's spatial structure / action trends, and radar supplements the three-dimensional contours and physiological signals of the body, enabling functions such as high-precision sleeping posture recognition, dynamic monitoring, and out-of-bed detection.

[0029] The main objective of static blurred light-transmitting components is to allow light to pass through while scattering it sufficiently so that the camera cannot capture clear details. Such as frosted glass, frosted acrylic plates, frosted polycarbonate plates, atomized PET films, nano-transmissive scattering coatings, etc. These components are fixedly set on the camera's shooting path and achieve image blurring through their own light-transmitting and scattering properties.

[0030] Modulated light-transmitting components include, but are not limited to, components whose light-transmitting state or degree of blurring can be controlled by means such as heating, electro-optic modulation, mechanical vibration, liquid flow, acoustic perturbation, etc. For example: heating-type modulation components (such as heating atomizing films), electrochromic or electro-scattering intelligent dimming films, mechanical vibration or micro-perturbation films, liquid-modulated light-transmitting components, acoustic-modulated light-transmitting components, etc. For example, materials such as PET films and glass that can be surface-heated utilize physical state changes (such as atomization, water vapor condensation, optical perturbation, etc.) caused by heating to achieve dynamic blurring of images.

[0031] Dynamic blurred structures, such as high-speed mechanical brush pieces set on the camera's shooting path, drive the brush pieces to reciprocate or rotate at a specific frequency, forming continuous optical trailing or occlusion effects through dynamic perturbations, making the captured images unrecognizable both in time and space.

[0032] Embodiment 2 This embodiment provides a sleep monitoring system based on a privacy-protecting camera and radar based on Embodiment 1. As Figure 2 shown, Embodiment 2 provides a sleep monitoring method based on a privacy-protecting camera and radar, including: S1 Obtain the radar three-dimensional data of the current space through the radar, obtain the space blurred image of the current space through the privacy-protecting camera, and identify the spatial position and range of the bed through the radar three-dimensional data and the blurred image; S101 Perform segmentation modeling of spatial objects and the human body on the space blurred image, and output the first object and human body spatial distribution data; S102 Extract the second object and human body spatial distribution data through the radar three-dimensional data; S103 Matches and fuses the three-dimensional spatial data of the first object and the human body and the three-dimensional spatial data of the second object and the human body to obtain the three-dimensional spatial distribution data of the object and the human body; S104 Marks and identifies the spatial coordinate position and range of the bed through the three-dimensional spatial distribution data of the object and the human body.

[0033] In this step, the three-dimensional spatial distribution data of the object and the human body includes the distribution positions and their bounding boxes of each object and the human body. Spatial object-human segmentation models such as SegFormer and Mask R-CNN can be used to map and fuse the three-dimensional spatial distribution data of the second object and the human body obtained by the radar into the spatial blurred image to obtain the three-dimensional spatial distribution data of the first object and the human body, and the three-dimensional distribution data of the object and the human body can be obtained, which serves as the basis for subsequent detection of whether the human body is in bed. The radar three-dimensional data can be the frequency domain features such as velocity spectrum features, distance spectrum features, and angle spectrum features obtained by performing multiple Fourier transforms on the radar echo data, or the three-dimensional point cloud data after processing the frequency domain features.

[0034] S2 Obtains the current human body position and posture through the radar three-dimensional data and the spatial blurred image, and determines whether the human body is in bed according to the current human body position and posture; S201 Detects whether there is a human body in the current space through the three-dimensional spatial distribution data of the object and the human body. If so, it identifies the human body position coordinates and posture through the spatial blurred image, and reconfirms the human body position and posture through the radar three-dimensional data; S2011 Detects whether there is a human body in the current space through the three-dimensional spatial distribution data of the object and the human body. If so, it uses the feature detection algorithm to extract the feature points of the spatial blurred image, matches the feature points with the feature points of the reference space, and obtains the corresponding relationship of the feature points between the two; S2012 Solves through the spatial geometric algorithm according to the corresponding relationship of the feature points to obtain the image human body position coordinates, and identifies the current human body posture type through the image human body position coordinates; S2013 Obtains the coordinates of the radar three-dimensional data at the same time point, converts the coordinates of the radar three-dimensional data and the image human body position coordinates to the same coordinate system, calculates the mean square error between the coordinates of the radar three-dimensional data and the image human body position coordinates. If the mean square error is less than the error threshold, the two are similar, and the coordinates of the radar three-dimensional data are output as the human body position coordinates and the current human body posture type.

[0035] In this step, through the three-dimensional spatial distribution data of the object and the human body obtained in step S1, it can be roughly confirmed whether there is a human body in the current space. If there is no human body, there is no need to perform the subsequent calculation process. If there is a human body, further refined position and posture calculations are required, and the specific position and posture of the human body are confirmed through the spatial blurred image and the radar three-dimensional data.

[0036] Even if the image is blurred, as long as there are main contours, edges, etc., algorithms such as SIFT / ORB can still find key feature points, match them with reference space feature points, establish a spatial mapping relationship. Using the corresponding relationship of feature points and geometric algorithms, the spatial position coordinates of the human body in the image can be calculated. A pre - established pose recognition model can be used to classify according to the spatial distribution of key points, the angle of the main axis, etc. After unifying the three - dimensional point cloud and the image coordinate system, calculating the mean square error can be used to judge the detection consistency. A small error indicates that the two detection results are highly consistent, and the human body position coordinates and pose type can be accurately output for subsequent judgment of whether the human body is in bed.

[0037] S202 calculates the key position coordinates of the human body through the human body position coordinates, and judges whether the key position coordinates of the human body fall within the spatial range of the bed. If the human body pose type is the lying type, it is determined that the human body is in bed.

[0038] In this step, the position coordinates of each joint can be directly selected through an algorithm (such as OpenPose), and the key position coordinates for lying are selected for subsequent judgment, focusing on the position coordinates of joints such as the head, neck, shoulders, hips / knees, knees, and ankles. In addition, the position coordinates of the mid - point / mean point of the buttocks, hips, and buttocks can be calculated by combining the position coordinates of multiple joints as the key position coordinates of the human body, representing the position of the main load - bearing parts of the human body, ensuring that the head and feet outside the bed do not affect the judgment based on the "main point of the torso". When the key position coordinates of the human body fall within the spatial range coordinates of the bed obtained in step S1, and the pose type obtained in step S201 is the lying type, it is confirmed that the human body is in bed, which can avoid the situation where a person standing / sitting beside the bed is misjudged as being in bed.

[0039] S3 If the current human body is in bed, locate the thoracic cavity position of the human body according to the human body position and pose, and collect the human body signs at the thoracic cavity position through the radar.

[0040] S301 extracts the three - dimensional coordinates of the thoracic cavity through the human body position coordinates, and corrects the three - dimensional coordinates of the human body thoracic cavity according to the human body sleeping posture; S302 sets a spatial radar monitoring area centered on the three - dimensional coordinates of the thoracic cavity, and focuses the radar beam on the spatial radar monitoring area; S303 receives the radar echo signal in the spatial radar monitoring area, and extracts the breathing frequency and heart rate of the radar echo signal after filtering and denoising.

[0041] In this step, the thoracic cavity is the most direct and significant position for vital sign signals such as breathing and heartbeat. By combining the overall spatial coordinates of the human body and the semantic correction of the sleeping posture to further determine the specific position of the thoracic cavity, the monitoring object area can be accurately locked, avoiding interference from the pelvis, limbs, etc.

[0042] When the sleeping posture is different, the actual position and direction of the chest cavity in the three-dimensional space will change. Dynamically correcting the chest cavity coordinates ensures continuous and accurate acquisition in different postures / turning over after falling asleep, etc. For example, when lying on the back, the chest cavity point = the center of gravity point shifted upward (Z-axis) by a certain distance; when lying on the side, the chest cavity point = the center of gravity point shifted to the positive side of the main axis of the body by a certain distance; when lying prone, the chest cavity point = the center of gravity point shifted downward (Z-axis) by a certain distance. The specific distance can be obtained through actual measurement or fitting of training data.

[0043] By setting an area of interest (ROI) centered on the chest cavity coordinates and adjusting the radar beam, the monitoring resources and data processing capabilities are focused on the most valuable area, effectively filtering background clutter, and improving the separation quality and accuracy of the vital sign signals. Extracting the human respiratory rate and heart rate from the radar echo signal is an existing technology, which will not be elaborated in this application.

[0044] Embodiment III This embodiment is based on the sleep monitoring system based on a privacy-protected camera and radar provided in Embodiment I. The difference between this embodiment and Embodiment II is the judgment process of whether there is a human body in the bed. Specifically: Focus the radar beam on the space range of the bed, collect the radar echo signals within the space range of the bed, and analyze whether there are radar micro-motion signals in the radar echo signals within the space range of the bed. Specifically: Extract the radar body motion signals from the radar echo signals within the space range of the bed. If there are radar body motion signals, then perform human body posture judgment on the spatially blurred image; If there are no radar body motion signals in the radar echo signals within the space range of the bed, then extract the radar micro-motion signals from the radar echo signals within the space range of the bed. If there are radar micro-motion signals, then perform human body posture judgment on the spatially blurred image.

[0045] The extraction steps of the radar body motion signals are as follows: Perform multiple Fourier transforms on the radar echo signals within the space range of the bed to extract the high-speed spectrum mean value, and use the high-speed spectrum mean value as the radar body motion signal.

[0046] The extraction steps of the radar micro-motion signals are as follows: Perform one-dimensional Fourier transform on the radar echo signals within the space range of the bed and then perform frame extraction and accumulation to obtain the frame extraction and accumulation data; Perform two-dimensional Fourier transform on the frame extraction and accumulation data, then extract the low-speed spectrum and high-speed spectrum peak values respectively, and calculate the ratio of the low-speed spectrum to the high-speed spectrum to obtain the human micro-motion signal.

[0047] In this step, the body movement signal energy is much higher, making it easy to detect with a low false positive rate. First, quickly screen for the presence of a person based on body movement energy (most turning over and getting in and out of bed movements are obvious). When there is no body movement, then judge micro-movements. Micro-movement signals can sensitively reflect subtle movements such as lying still / breathing, avoiding missing people lying quietly.

[0048] Judge whether there is a human posture within the space range of the bed through the spatially blurred image. If there is a radar micro-movement signal and a human posture within the space range of the bed at the same time point, it is determined that there is a human in the bed; otherwise, turn off the privacy protection camera.

[0049] In this step, although radar (especially millimeter-wave radar) has high sensitivity in detecting body movements and micro-movements, the radar may be affected by environmental interference (such as metal, fans, pets, etc.), clutter, pseudo body movements, etc., resulting in false judgments. For a human body that remains stationary for a long time, the micro-movement signal is extremely weak and may be misjudged as no one. If there are other moving objects (such as escorts, pets) beside the bed, the radar may not be able to accurately distinguish. Therefore, it is necessary to use the spatially blurred image collected by the privacy protection camera for auxiliary judgment. If there is a "human shape" within the corresponding space range of the bed in the spatially blurred image, it is confirmed that there is a human in the bed, reducing the probability of false alarms / missing reports and serving as a monitoring basis for subsequent human vital sign monitoring.

[0050] Those skilled in the art should understand that the embodiments of the present invention can be provided as a method, a system, or a computer program product. Therefore, the present invention can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present invention can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0051] The present invention is described with reference to the flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to the embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, such that the instructions executed by the processor of the computer or other programmable data processing devices generate means for implementing the functions specified in Figure 1 one process or multiple processes and / or blocks Figure 1 one block or multiple blocks.

[0052] These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means that implement the function specified in one or more of the flows Figure 1 and / or boxes Figure 1 specified in one or more of the flows and / or boxes.

[0053] These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process, so that the instructions executed on the computer or other programmable apparatus provide steps for implementing the function specified in one or more of the flows Figure 1 and / or boxes Figure 1 specified in one or more of the boxes or boxes.

[0054] It should be noted that, in the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. The word "comprising" does not exclude the presence of elements or steps not listed in a claim. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The present invention may be implemented by means of hardware comprising several distinct elements and by means of a suitably programmed computer. In the unit claims listing several means, several of these means may be embodied by one and the same item of hardware. The use of the words first, second, and third, etc. does not denote any order. These words may be interpreted as names.

[0055] Although the preferred embodiments of the present invention have been described, additional changes and modifications can be made by those skilled in the art once they learn of the basic inventive concept. Therefore, the appended claims are intended to be construed to cover the preferred embodiments and all changes and modifications falling within the scope of the present invention.

[0056] It is obvious that those skilled in the art can make various changes and deformations to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and deformations of the present invention fall within the scope of the claims of the present invention and their equivalent technologies, the present invention is also intended to include these modifications and deformations.

[0057] In the present invention, unless otherwise clearly defined and limited, terms such as "installed", "connected", "coupled", "fixed", etc. shall be construed in a broad sense. For example, it may be a fixed connection, a detachable connection, or integrated; it may be a mechanical connection or an electrical connection; it may be directly connected or indirectly connected through an intermediate medium, and it may be the internal communication of two components or the interaction relationship between two components. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0058] In the description of this specification, the description with reference to terms such as "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials, or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms should not be understood as necessarily referring to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described can be combined in a suitable manner in any one or more embodiments or examples. In addition, without contradiction, those skilled in the art can combine and combine the different embodiments or examples described in this specification and the features of different embodiments or examples.

Claims

1. A sleep monitoring method based on a privacy - protected camera and radar, characterized in that, Applied to a sleep monitoring system based on a privacy - protected camera and radar, including: A radar, which is used to obtain radar three - dimensional data and collect human body signs; A privacy - protected camera, which is arranged in the same monitoring area as the radar and is used to obtain a spatially blurred image; The method includes: Obtaining the radar three - dimensional data of the current space through the radar, obtaining the spatially blurred image of the current space through the privacy - protected camera, and identifying the spatial position and range of the bed through the radar three - dimensional data and the blurred image; Obtaining the current human body position and posture through the radar three - dimensional data and the spatially blurred image, and judging whether the human body is in bed according to the current human body position and posture; If the current human body is in bed, then locate the position of the human chest according to the human body position and posture, and collect the human body signs at the position of the human chest through the radar.

2. The sleep monitoring method based on a privacy-protected camera and radar according to claim 1, characterized in that The privacy - protected camera includes a camera and a blur structure, and the blur structure is arranged on the shooting path of the camera and is used to perform physical blur processing on the image obtained by the camera; The blur structure is any one of a static blur light - transmissive member, a modulated light - transmissive element, and a dynamic blur structure.

3. A sleep monitoring method based on a privacy-protected camera and radar according to claim 1, characterized in that, The step of obtaining the radar three - dimensional data of the current space through the radar, obtaining the spatially blurred image of the current space through the privacy - protected camera, and identifying the spatial position and range of the bed through the radar three - dimensional data and the blurred image includes: Performing segmentation and modeling of spatial objects and the human body on the spatially blurred image, and outputting the first object and human body spatial distribution data; Extracting the second object and human body spatial distribution data through the radar three - dimensional data; Matching and fusing the first object and human body three - dimensional space data and the second object and human body three - dimensional space data to obtain the object and human body three - dimensional space distribution data; Marking and identifying the spatial coordinate position and range of the bed through the object and human body three - dimensional space distribution data.

4. The sleep monitoring method based on a privacy-protected camera and radar according to claim 1, characterized in that, The step of obtaining the current human body position and posture through the radar three - dimensional data and the spatially blurred image, and judging whether the human body is in bed according to the current human body position and posture includes: Detecting whether there is a human body in the current space through the object and human body three - dimensional space distribution data. If so, identifying the human body position coordinates and posture through the spatially blurred image, and re - confirming the human body position and posture through the radar three - dimensional data; Calculating the key position coordinates of the human body through the human body position coordinates, and judging whether the key position coordinates of the human body fall within the spatial range of the bed and the human body posture type is the lying type, then determining that the human body is in bed.

5. The sleep monitoring method based on a privacy-protected camera and radar according to claim 4, characterized in that, The step of detecting whether there is a human body in the current space through the object and human body three - dimensional space distribution data. If so, identifying the human body position coordinates and posture through the spatially blurred image, and re - confirming the human body position and posture through the radar three - dimensional data includes: Detecting whether there is a human body in the current space through the object and human body three - dimensional space distribution data. If so, using a feature detection algorithm to extract the feature points of the spatially blurred image, and matching the feature points with the feature points of the reference space to obtain the corresponding relationship of the feature points between the two; According to the corresponding relationship of the feature points, solving through a spatial geometric algorithm to obtain the image human body position coordinates, and identifying the current human body posture type through the image human body position coordinates; Obtain the coordinates of the radar three-dimensional data at the same time point, convert the coordinates of the radar three-dimensional data and the coordinates of the human body position in the image to the same coordinate system, calculate the mean square error between the coordinates of the radar three-dimensional data and the coordinates of the human body position in the image. If the mean square error is less than the error threshold, then the two are similar, and output the coordinates of the radar three-dimensional data as the human body position coordinates and the current human body posture type.

6. The sleep monitoring method based on a privacy-protected camera and radar according to claim 1, characterized in that, The method for obtaining the current human body position and posture through the radar three-dimensional data and the spatial blurred image, and judging whether the human body is in bed according to the current human body position and posture includes: Focus the radar beam on the space range of the bed, collect the radar echo signals within the space range of the bed, and analyze whether there are radar micro-motion signals in the radar echo signals within the space range of the bed; Judge whether there is a human body posture within the space range of the bed through the spatial blurred image. If there are radar micro-motion signals and human body postures within the space range of the bed at the same time point, it is determined that the human body is in bed, otherwise the privacy protection camera is turned off.

7. The sleep monitoring method based on a privacy-protected camera and radar according to claim 6, characterized in that The step of focusing the radar beam on the space range of the bed, collecting the radar echo signals within the space range of the bed, and analyzing whether there are radar micro-motion signals in the radar echo signals within the space range of the bed includes: Extract the radar body motion signals from the radar echo signals within the space range of the bed. If there are radar body motion signals, then judge the human body posture of the spatial blurred image; If there are no radar body motion signals in the radar echo signals within the space range of the bed, then extract the radar micro-motion signals from the radar echo signals within the space range of the bed. If there are radar micro-motion signals, then judge the human body posture of the spatial blurred image.

8. A sleep monitoring method based on a privacy-protected camera and radar according to claim 7, characterized in that, The extraction steps of the radar body motion signals are as follows: Perform multiple Fourier transforms on the radar echo signals within the space range of the bed to extract the high-speed spectrum mean value, and use the high-speed spectrum mean value as the radar body motion signal.

9. The sleep monitoring method based on a privacy-protected camera and radar according to claim 7, characterized in that, The extraction steps of the radar micro-motion signals are as follows: Perform one-dimensional Fourier transform on the radar echo signals within the space range of the bed and then perform frame extraction and accumulation to obtain the frame extraction and accumulation data; Perform two-dimensional Fourier transform on the frame extraction and accumulation data, then extract the low-speed spectrum and high-speed spectrum peaks respectively, calculate the ratio of the low-speed spectrum and the high-speed spectrum to obtain the human body micro-motion signal.

10. A sleep monitoring method based on a privacy-protected camera and radar according to claim 1, characterized in that, If the current human body is in bed, then locate the human chest position according to the human body position and posture, and collect the human body signs at the human chest position through the radar, including: Extract the three-dimensional chest coordinates through the human body position coordinates, and correct the three-dimensional chest coordinates of the human body according to the human body sleeping posture; Set a spatial radar monitoring area centered on the three-dimensional chest coordinates, and focus the radar beam on the spatial radar monitoring area; Receive the radar echo signals in the spatial radar monitoring area, and extract the breathing frequency and heart rate of the radar echo signals after filtering and denoising.

Images