A spatial object perception method based on privacy-preserving camera and radar

Abstract

The present invention discloses a spatial object perception method based on a privacy protection camera and a radar, comprising: obtaining three-dimensional spatial features of a current space through a radar, judging whether a human body exists in the current space through the three-dimensional spatial features, and if so, starting a privacy camera to collect a global image; obtaining two-dimensional spatial features of the current space through the global image, fusing the two-dimensional spatial features and the three-dimensional spatial features of the current space to obtain a fused spatial feature of the current space; inputting the fused feature of the current space into a pre-trained object recognition model to obtain the type of object in the current space; marking interference source labels and general furniture labels according to the type of object in the current space, filtering the current space objects with interference source labels to obtain the distribution of general furniture in the current space; adjusting radar detection parameters according to the distribution of general furniture in the current space by detection area to obtain detection thresholds for different detection areas.

Description

Technical Field

[0001] The present invention relates to the field of space object perception technology, and in particular to a space object perception method based on a privacy protection camera and radar. Background Art

[0002] With the growing demand for intelligent security, smart offices, and unmanned management, technologies for automatically sensing, identifying, and modeling people and objects within a space are gaining increasing attention. Existing spatial perception systems primarily use a variety of sensing methods, including cameras, millimeter-wave radar, infrared, and ultrasonic sensors, to detect, identify, and track objects.

[0003] Radar transmits electromagnetic waves and detects echoes, acquiring information about the distance, velocity, and angle of objects in space. It also possesses a certain level of 3D point cloud modeling capability, enabling it to perceive the depth and dynamic changes of spatial objects. This makes it particularly well-suited for detecting dynamic targets (such as people and moving objects). However, single radars have significant limitations in spatial perception: their point cloud resolution is low, making it difficult to distinguish specific object types. They can only capture object outlines or rough spatial distribution information, making it difficult to achieve detailed 3D modeling and accurate identification of various objects in complex scenes.

[0004] To improve the accuracy of spatial perception, existing technologies typically combine radar with other sensors (such as cameras), using visual image information captured by the camera to identify and model spatial targets. Cameras can complement radar's shortcomings in object classification and detail capture, improving the system's ability to recognize static structures and dynamic targets. However, when capturing spatial images, cameras also capture visible image information of the user, including facial features, biometric features, body posture, and environmental details. Once this sensitive data is stored, transmitted, or processed, it poses a risk of interception, leakage, or misuse, making it difficult to meet the needs of scenarios with high privacy requirements, such as homes, healthcare, and offices.

[0005] Furthermore, existing spatial perception methods still suffer from the following deficiencies in practical applications: First, the systems have limited ability to distinguish dynamic objects (such as people and pets) from interference sources (such as reflective objects like mirrors, glass, and televisions) and static objects (such as furniture and spatial structures) in the environment, making them prone to false alarms, false alerts, and missed detections. Second, the differences in the data types and spatiotemporal resolutions collected by cameras and radars make simple data fusion approaches difficult to achieve high-confidence spatial perception and target recognition. This is especially true in complex environments where privacy protection and high-precision recognition requirements coexist. The adaptability and robustness of existing solutions need to be improved.

[0006] The purpose of this invention is to design a space object perception method based on privacy protection cameras and radars to address the above-mentioned problems in the existing technology. Summary of the Invention

[0007] In view of this, the purpose of the present invention is to propose a spatial object perception method based on privacy protection cameras and radars, which can solve the above problems.

[0008] The present invention provides a space object perception method based on a privacy protection camera and a radar, which is applied to a space object perception system based on a privacy protection camera and a radar, comprising:

[0009] Radar, used to obtain the three-dimensional features of the current space;

[0010] The privacy protection camera is set in the same monitoring area as the radar to obtain the two-dimensional features of the current space;

[0011] The method comprises:

[0012] The radar obtains the three-dimensional spatial features of the current space, and uses the three-dimensional spatial features to determine whether there is a human body in the current space. If so, the privacy camera is activated to collect a global image;

[0013] Obtaining the two-dimensional spatial features of the current space through the global image, fusing the two-dimensional spatial features of the current space with the three-dimensional spatial features to obtain the fused spatial features of the current space;

[0014] Input the fusion features of the current space into the pre-trained object recognition model to obtain the object type in the current space;

[0015] According to the object type in the current space, the interference source label and the general furniture label are marked, and the objects in the current space with the interference source label are filtered to obtain the general furniture distribution in the current space;

[0016] According to the general furniture distribution in the current space, the radar detection parameters are adjusted according to the detection area to obtain the detection thresholds of different detection areas.

[0017] Furthermore, the privacy protection camera includes a camera and a blur structure, and the blur structure is set on the shooting path of the camera and is used to perform physical blur processing on the image obtained by the camera.

[0018] Furthermore, the blur structure is any one of a static blur light-transmitting element, a modulated light-transmitting element, and a dynamic blur structure.

[0019] Furthermore, the acquiring of three-dimensional spatial features of the current space by the radar, determining whether a human body exists in the current space by the three-dimensional spatial features, and activating the privacy camera to collect a global image if a human body exists in the current space includes:

[0020] The radar digital signal of the reflected signal in the current space is obtained by the radar, and the radar digital signal is subjected to a two-dimensional Fourier transform to obtain the speed, distance and angle information of the target point;

[0021] Calculate the three-dimensional spatial coordinates of the target point based on the speed, distance and angle information of the target point to form a spatial point cloud;

[0022] Perform clustering and feature extraction analysis on spatial point clouds to identify whether there are point cloud clusters that meet the preset human body characteristics;

[0023] If a human body is detected, the privacy protection camera is triggered to collect a global image of the current space.

[0024] Furthermore, the two-dimensional spatial features of the current space are obtained through the global image, and the two-dimensional spatial features of the current space and the three-dimensional spatial features are fused to obtain the fused spatial features of the current space, including:

[0025] Performing channel transformation on the global image to obtain images with at least two channel configurations;

[0026] The images with different channel configurations are superimposed and input into the target detection model to detect different objects in the current space and their detection frames;

[0027] Based on the spatial calibration parameters between the privacy camera and the radar, the detection frames of different objects in the current space are aligned and fused with the spatial point cloud to obtain the fused spatial features of the current space.

[0028] Furthermore, based on the spatial calibration parameters between the privacy camera and the radar, the detection frames of different objects in the current space are aligned and fused with the spatial point cloud to obtain the fused spatial features of the current space, including:

[0029] Project the detection frames of different objects in the current space into the three-dimensional space coordinate system of the spatial point cloud according to the spatial calibration parameters;

[0030] In the spatial point cloud, point cloud subsets that intersect with the detection frames of different objects are located, and the image features and point cloud features in the point cloud subsets are fused to obtain the fused spatial features of the current space.

[0031] Furthermore, the two-dimensional spatial features of the current space are obtained through the global image, and the two-dimensional spatial features of the current space and the three-dimensional spatial features are fused to obtain the fused spatial features of the current space, including:

[0032] Performing channel transformation on the global image to obtain images with at least two channel configurations;

[0033] The frequency domain features of images with different channel configurations are extracted through Fourier transform, and the frequency domain features of images with different channel configurations are superimposed and input into the object classification model to detect different objects in the current space and their detection frames;

[0034] Based on the spatial calibration parameters between the privacy camera and the radar, the detection frames of different objects in the current space are aligned and fused with the spatial point cloud to obtain the fused spatial features of the current space.

[0035] Furthermore, the object recognition model is trained through the following steps:

[0036] Collect historical spatial global images and their corresponding spatial point clouds, fuse image features with point cloud features through spatial registration and feature alignment to obtain fused spatial features, and annotate each group of samples with real object category labels;

[0037] A neural network model is used to build the infrastructure of the object recognition model, with fused spatial features as input and object category labels as output. The model parameters are trained through forward reasoning and backpropagation, and the cross-entropy loss is used to optimize the object recognition model.

[0038] Furthermore, the radar detection parameters are adjusted based on the current distribution of general furniture in the space, and the detection thresholds for different detection areas are obtained, including:

[0039] Divide the current space into regions based on the three-dimensional spatial positions of objects in the current space with universal furniture labels, and obtain detection areas corresponding to different furniture;

[0040] Set corresponding radar detection parameters for the detection areas corresponding to different furniture to obtain the detection thresholds for different areas of the current space.

[0041] Furthermore, the radar detection parameters corresponding to the detection areas of different furniture are set to obtain the detection thresholds of different areas in the current space, including:

[0042] The radar detection parameters for different time periods are set for the detection areas corresponding to different furniture to obtain the detection thresholds for different areas of the current space.

[0043] Beneficial effects of the present invention:

[0044] The first is to blur the camera through physical means to ensure user privacy and security, cut off the potential risks to user privacy and security from the root, and combine radar and privacy protection camera. The privacy protection camera obtains blurred global images to extract two-dimensional spatial information, and the radar extracts three-dimensional spatial information. The spatial perception ability is improved through the complementarity of two-dimensional information and three-dimensional information.

[0045] Secondly, the purpose of using radar to pre-monitor whether there are people in the space is to minimize privacy exposure and unnecessary data collection. Only when a human body is detected in the space will the privacy protection camera be activated for global image acquisition. When performing human body detection, the energy consumption, data volume and computational complexity of the radar are much lower than those of the privacy protection camera, and it is not affected by light and occlusion, and can provide continuous real-time monitoring. The camera is only activated when it is judged to be "necessary", which significantly reduces the overall energy consumption and computing burden of the system. In addition, it can also avoid monitoring and data storage of unmanned environments or non-essential scenarios to the greatest extent.

[0046] Third, the fusion of two-dimensional features obtained by global images and three-dimensional features obtained by radar can make up for the defects that global images can only provide the relative position and size of objects in the image, and cannot accurately know the actual spatial distance and size. When a single radar performs spatial perception, the radar's recognition ability is limited, and it is difficult to distinguish specific object categories. It can only obtain the outline or point cloud of the object, and it is difficult to achieve refined three-dimensional space reconstruction and object identification. It can not only obtain the position and size of objects in the current space, but also guide the depth of objects in the current space.

[0047] Fourth, by performing object type recognition based on the fusion features of the current space, relevant object information can be extracted from the raw perception data for subsequent differentiation between interference sources and static furniture. Radar parameter adjustments based on the label type of the current spatial object focus perception and recognition accuracy on the corresponding three-dimensional spatial structure, ignoring interference sources and avoiding wasting resources in irrelevant areas. This improves overall system efficiency, enhances spatial object recognition accuracy, and builds a precise spatial layout. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0049] Figure 1 It is a system module diagram of embodiment 1.

[0050] Figure 2 This is a flow chart of the method of embodiment 2. DETAILED DESCRIPTION

[0051] To facilitate understanding by those skilled in the art, the structure of the present invention will now be further described in detail with reference to the embodiments and accompanying drawings. It should be understood that the steps mentioned in this embodiment, unless otherwise specified, can be adjusted in sequence according to actual needs, and can even be executed simultaneously or partially simultaneously.

[0052] Example 1

[0053] like Figure 1 As shown, embodiment 1 provides a space object perception system based on a privacy protection camera and a radar, including:

[0054] Radar, used to obtain the three-dimensional features of the current space;

[0055] The privacy protection camera is set in the same monitoring area as the radar to obtain the two-dimensional features of the current space;

[0056] Specifically, the privacy protection camera includes a camera and a blur structure. The blur structure is set on the shooting path of the camera and is used to perform physical blur processing on the image captured by the camera.

[0057] Furthermore, the blur structure is any one of a static blur light-transmitting element, a modulated light-transmitting element, and a dynamic blur structure.

[0058] In this embodiment, traditional camera monitoring captures private images such as faces, body features, and environmental details. Once stored or transmitted, these images pose a risk of leakage or misuse, creating privacy and security issues. The fuzzy structure allows images to only show the general dynamics and outlines of the monitored space, preventing personal privacy leaks and ensuring that no clear private images are generated in the first place. Even malicious algorithm manipulation or hacking cannot reconstruct the original image. This balances AI functionality with user privacy protection, meeting privacy compliance requirements in various scenarios, including home, healthcare, and senior care.

[0059] When using radar alone for spatial perception, its recognition capabilities are limited, making it difficult to distinguish specific object categories. Only outlines or point clouds can be obtained, making it difficult to achieve detailed 3D spatial reconstruction and object identification. Combining radar with a privacy-preserving camera, where the privacy-preserving camera extracts 2D spatial information and the radar extracts 3D spatial information, improves spatial perception capabilities through the complementary use of 2D and 3D information.

[0060] The primary goal of static blurring translucent components is to allow light to pass through but scatter it sufficiently to obscure details from the camera's image. Examples include frosted glass, frosted acrylic, frosted polycarbonate, atomized PET film, and nano-transparent and scattering coatings. These components are fixed in the camera's path and blur the image through their inherent light-scattering properties.

[0061] Modulated light-transmitting elements include, but are not limited to, elements that achieve controllable light transmission or blurring through heating, electro-modulation, mechanical vibration, liquid flow, acoustic disturbance, and other methods. Examples include heated modulating elements (such as heated atomizing films), electrochromic or electro-scattering smart dimming films, mechanical vibration or micro-disturbance films, liquid-modulated light-transmitting elements, and acoustically modulated light-transmitting elements. For example, materials such as PET film and glass that can be heated on the surface can achieve dynamic blurring of images by inducing changes in the material's physical state (such as atomization, condensation, or optical disturbance) due to heating.

[0062] A dynamic blur structure, such as a high-speed mechanical wiper installed on the camera shooting path, drives the wiper to perform reciprocating or rotational motion at a specific frequency, forming a continuous optical smear or occlusion effect through dynamic disturbance, making the captured image unrecognizable in both time and space.

[0063] Example 2

[0064] This embodiment provides a space object perception system based on privacy protection camera and radar based on embodiment 1. Figure 2 As shown, embodiment 2 provides a method for sensing spatial objects based on a privacy protection camera and radar, including:

[0065] S1 uses radar to obtain the three-dimensional spatial features of the current space, and uses the three-dimensional spatial features to determine whether there is a human body in the current space. If so, it activates the privacy camera to collect a global image;

[0066] S101 obtains the radar digital signal of the reflected signal in the current space through the radar, performs a two-dimensional Fourier transform on the radar digital signal, and obtains the speed, distance and angle information of the target point;

[0067] S102 calculates the three-dimensional spatial coordinates of the target point based on the speed, distance and angle information of the target point to form a spatial point cloud;

[0068] S103 performs clustering and feature extraction analysis on the spatial point cloud to identify whether there is a point cloud cluster that meets the preset human body characteristics;

[0069] If a human body is detected in S104, the privacy protection camera is triggered to collect a global image of the current space.

[0070] In this step, the purpose of using radar to pre-monitor whether there are people in the space is to minimize privacy exposure and unnecessary data collection. Only when a human body is detected in the space will the privacy protection camera be activated for global image acquisition. When performing human body detection, the energy consumption, data volume and computational complexity of the radar are much lower than those of the privacy protection camera, and it is not affected by light and occlusion, and can provide continuous real-time monitoring. The camera is only activated when it is judged to be "necessary", which significantly reduces the overall energy consumption and computing burden of the system. In addition, it can also avoid monitoring and data storage of unmanned environments or unnecessary scenarios to the greatest extent.

[0071] S2 obtains the two-dimensional spatial features of the current space through the global image, and fuses the two-dimensional spatial features and the three-dimensional spatial features of the current space to obtain the fused spatial features of the current space;

[0072] S201 performs channel transformation processing on the global image to obtain images with at least two channel configurations;

[0073] S202: Superimpose the images with different channel configurations and input them into the target detection model to detect different objects in the current space and their detection frames;

[0074] In this step, the channel transformation includes, but is not limited to, adjusting the channel order, switching channel combinations, or mapping channels. Assuming the global image is an RGB image, a BGR image can be obtained after channel transformation. The RGB and BGR images can be input into a target detection model (e.g., a YOLO model, a regional convolutional neural network, etc.) to detect objects in the current space and their detection frames. Because the global image captured by the privacy protection camera is a blurred image, using only a single channel configuration image as input may result in the model being unresponsive to specific features. Superimposing different channel configuration images for input helps capture residual signals and reduce feature information loss.

[0075] S203 aligns and fuses the detection frames of different objects in the current space with the spatial point cloud based on the spatial calibration parameters between the privacy camera and the radar to obtain the fused spatial features of the current space.

[0076] S2031 projects the detection frames of different objects in the current space into the three-dimensional space coordinate system of the space point cloud according to the space calibration parameters;

[0077] S2032 locates point cloud subsets that intersect with detection frames of different objects in the spatial point cloud, fuses image features and point cloud features in the point cloud subsets, and obtains fused spatial features of the current space.

[0078] In this step, the privacy camera collects blurred images, and it is difficult to perform high-confidence detection and positioning based on data from one side alone. The global image can only provide the relative position and size of the object in the image, and cannot accurately know the actual spatial distance and size. The point cloud data obtained by the fusion radar can make up for this shortcoming. It can not only obtain the position and size of the current spatial object, but also guide the depth of the current spatial object.

[0079] S3 inputs the fusion features of the current space into the pre-trained object recognition model to obtain the object type in the current space;

[0080] The object recognition model is trained through the following steps:

[0081] Collect historical spatial global images and their corresponding spatial point clouds, fuse image features with point cloud features through spatial registration and feature alignment to obtain fused spatial features, and annotate each group of samples with real object category labels;

[0082] A neural network model is used to build the infrastructure of the object recognition model, with fused spatial features as input and object category labels as output. The model parameters are trained through forward reasoning and backpropagation, and the cross-entropy loss is used to optimize the object recognition model.

[0083] In this step, by performing object type recognition on the fusion features of the current space, relevant object information can be extracted from the original perception data for subsequent differentiation between interference sources and static furniture.

[0084] S4 tags the interference source and general furniture according to the object type in the current space, filters the objects in the current space with the interference source tag, and obtains the general furniture distribution in the current space;

[0085] In this step, the objects identified in the space are distinguished by labeling. The essence of interference sources refers to those objects that are prone to produce false alarms, clutter, misidentification, false triggering and other negative effects on the perception system. Filtering out these targets helps improve the system's recognition accuracy, reduce false alarms, and increase stability and reliability. Objects such as mirrors, glass, televisions, air conditioners, radiators, and highly reflective surfaces may produce abnormally strong echoes, virtual images, multipaths, reflections, or light spots in radar or camera images, affecting subsequent normal target recognition. If these objects are not eliminated, the system may easily misjudge the interference source as a key monitoring target such as a human body or a pet, resulting in false alarms - such as mistaking the image in the mirror for a "real person" and misjudge the moving pixels in the TV screen as "moving objects."

[0086] S5 adjusts the radar detection parameters according to the current general furniture distribution in the space and obtains the detection thresholds of different detection areas.

[0087] S501 divides the current space into regions according to the three-dimensional spatial positions of the objects in the current space with the universal furniture tags, and obtains detection regions corresponding to different furniture;

[0088] S502 sets corresponding radar detection parameters for detection areas corresponding to different furniture to obtain detection thresholds for different areas in the current space.

[0089] In this step, various furniture items (such as beds, sofas, and cabinets) can block, reflect, and absorb radar signals, causing radar sensitivity and echo characteristics in certain areas to differ significantly from those in open areas. For example, near a bed, human motion and fall signals can easily be masked by the bed or become false signals. Near a sofa, prolonged sitting and falls can be difficult to distinguish. Furniture often encourages unusual activities (such as resting in bed or playing on the floor). Applying the same threshold across different areas can lead to misjudgments. For example, a quick sit-down on a bed can easily be misjudged as a fall. Parameters are adjusted based on the actual furniture distribution, ensuring the radar uses the most appropriate detection logic for typical areas such as beds, bedside areas, and open spaces, reducing false alarms and missed alerts overall.

[0090] In this step S502, radar detection parameters corresponding to different time periods may be set for detection areas corresponding to different furniture to obtain detection thresholds for different areas of the current space.

[0091] The use and activity patterns of furniture areas often change over time. For example, the bed area is more occupied by sleeping at night and getting out of bed and getting ready during the day. The sofa area is more occupied by activities and resting during the day and is often empty or occupied for short periods at night. Setting differentiated radar detection parameters for different furniture areas based on different time periods can dynamically adapt to changes in space use and behavior patterns over time, significantly improving the accuracy and relevance of human activity detection and reducing false alarm rates and energy consumption.

[0092] Example 3

[0093] This embodiment provides a spatial object perception system based on a privacy protection camera and radar based on the first embodiment. The difference between the third embodiment and the second embodiment is that the third embodiment extracts the frequency domain features of the global image as the two-dimensional features of the global image. The specific steps are as follows:

[0094] Performing channel transformation on the global image to obtain images with at least two channel configurations;

[0095] The frequency domain features of images with different channel configurations are extracted through Fourier transform, and the frequency domain features of images with different channel configurations are superimposed and input into the object classification model to detect different objects in the current space and their detection frames;

[0096] Based on the spatial calibration parameters between the privacy camera and the radar, the detection frames of different objects in the current space are aligned and fused with the spatial point cloud to obtain the fused spatial features of the current space.

[0097] In this embodiment, since the privacy protection camera obtains a blurred global image, image blur essentially weakens the high-frequency components of the original image (loss of edge and texture information). Although the Fourier spectra before and after blurring are different, the mid- and low-frequency components can still basically reflect the overall structure, contour patterns, and periodic regions of the object. Compared to pixel-domain features, frequency-domain features can intuitively reflect residual structural information (even with blurring, low-frequency features such as the object's main frequency distribution and shape axis are not necessarily completely lost). Frequency-domain features are highly robust to image "degradation" (such as blurring, resolution reduction, and compression damage). In noisy and blurry scenes, using frequency-domain features for target detection and classification can achieve more accurate results.

[0098] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Thus, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0099] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowcharts and / or block diagrams, as well as combinations of processes and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowcharts and / or block diagrams. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0100] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0101] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0102] It should be noted that in the claims, any reference signs placed between parentheses shall not be construed as limiting the claims. The word "comprising" does not exclude the presence of components or steps not listed in the claim. The word "a" or "an" preceding a component does not exclude the presence of a plurality of such components. The invention can be implemented by means of hardware comprising several distinct components and by means of a suitably programmed computer. In a unit claim enumerating several means, several of these means may be embodied by one and the same item of hardware. The use of the words first, second, third etc. does not indicate any order. These words may be interpreted as names.

[0103] Although the preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0104] Obviously, those skilled in the art may make various modifications and variations to the present invention without departing from the spirit and scope of the present invention. Thus, if such modifications and variations fall within the scope of the claims and their equivalents, the present invention is intended to include such modifications and variations.

[0105] In the present invention, unless otherwise expressly specified or limited, the terms "mounted," "connected," "connect," "fixed," etc. should be understood broadly. For example, they may refer to fixed connection, detachable connection, or integration; mechanical connection or electrical connection; direct connection or indirect connection through an intermediate medium; internal communication between two components or interaction between two components. Those skilled in the art will understand the specific meanings of the above terms in the present invention based on specific circumstances.

[0106] In the description of this specification, the reference terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" mean that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic expressions of the above terms should not be understood as necessarily referring to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art can combine and combine different embodiments or examples described in this specification and features of different embodiments or examples without contradiction.

Claims

1. A method for spatial object perception based on privacy protection camera and radar, characterized in that: Applied to a space object perception system based on privacy-preserving cameras and radar, including: Radar, used to obtain the three-dimensional features of the current space; The privacy protection camera is set in the same monitoring area as the radar to obtain the two-dimensional features of the current space; The privacy protection camera includes a camera and a blurring structure, wherein the blurring structure is arranged on a shooting path of the camera and is used to perform physical blurring processing on an image acquired by the camera; The method comprises: The radar obtains the three-dimensional spatial features of the current space, and uses the three-dimensional spatial features to determine whether there is a human body in the current space. If so, the privacy camera is activated to collect a global image; Obtaining the two-dimensional spatial features of the current space through the global image, fusing the two-dimensional spatial features of the current space with the three-dimensional spatial features to obtain the fused spatial features of the current space; Input the fusion features of the current space into the pre-trained object recognition model to obtain the object type in the current space; According to the object type in the current space, the interference source label and the general furniture label are marked, and the objects in the current space with the interference source label are filtered to obtain the general furniture distribution in the current space; According to the current general furniture distribution in the space, the radar detection parameters are adjusted by detection area to obtain the detection thresholds for different detection areas. Specifically: Divide the current space into regions based on the three-dimensional spatial positions of objects in the current space with universal furniture labels, and obtain detection areas corresponding to different furniture; Set corresponding radar detection parameters for the detection areas corresponding to different furniture to obtain the detection thresholds for different areas of the current space.

2. The method for spatial object perception based on privacy protection camera and radar according to claim 1, characterized in that: The fuzzy structure is any one of a static fuzzy light-transmitting element, a modulated light-transmitting element, and a dynamic fuzzy structure.

3. The method for spatial object perception based on privacy protection camera and radar according to claim 1, characterized in that: The method of acquiring the three-dimensional spatial features of the current space by using the radar, determining whether a human body exists in the current space by using the three-dimensional spatial features, and activating the privacy camera to collect a global image if a human body exists in the current space includes: The radar digital signal of the reflected signal in the current space is obtained by the radar, and the radar digital signal is subjected to a two-dimensional Fourier transform to obtain the speed, distance and angle information of the target point; Calculate the three-dimensional spatial coordinates of the target point based on the speed, distance and angle information of the target point to form a spatial point cloud; Perform clustering and feature extraction analysis on spatial point clouds to identify whether there are point cloud clusters that meet the preset human body characteristics; If a human body is detected, the privacy protection camera is triggered to collect a global image of the current space.

4. The method for sensing space objects based on privacy protection cameras and radars according to claim 3, characterized in that: The two-dimensional spatial features of the current space are obtained through the global image, and the two-dimensional spatial features and the three-dimensional spatial features of the current space are fused to obtain the fused spatial features of the current space. The fused spatial features of the current space include: Performing channel transformation on the global image to obtain images with at least two channel configurations; The images with different channel configurations are superimposed and input into the target detection model to detect different objects in the current space and their detection frames; Based on the spatial calibration parameters between the privacy camera and the radar, the detection frames of different objects in the current space are aligned and fused with the spatial point cloud to obtain the fused spatial features of the current space.

5. The method for spatial object perception based on privacy protection camera and radar according to claim 4, characterized in that: Based on the spatial calibration parameters between the privacy camera and the radar, the detection frames of different objects in the current space are aligned and fused with the spatial point cloud to obtain the fused spatial features of the current space, including: Project the detection frames of different objects in the current space into the three-dimensional space coordinate system of the spatial point cloud according to the spatial calibration parameters; In the spatial point cloud, point cloud subsets that intersect with the detection frames of different objects are located, and the image features and point cloud features in the point cloud subsets are fused to obtain the fused spatial features of the current space.

6. The method for spatial object perception based on privacy protection camera and radar according to claim 3, characterized in that: The two-dimensional spatial features of the current space are obtained through the global image, and the two-dimensional spatial features and the three-dimensional spatial features of the current space are fused to obtain the fused spatial features of the current space. The fused spatial features of the current space include: Performing channel transformation on the global image to obtain images with at least two channel configurations; The frequency domain features of images with different channel configurations are extracted through Fourier transform, and the frequency domain features of images with different channel configurations are superimposed and input into the object classification model to detect different objects in the current space and their detection frames; Based on the spatial calibration parameters between the privacy camera and the radar, the detection frames of different objects in the current space are aligned and fused with the spatial point cloud to obtain the fused spatial features of the current space.

7. The method for spatial object perception based on privacy protection camera and radar according to claim 3, characterized in that: The object recognition model is trained through the following steps: Collect historical spatial global images and their corresponding spatial point clouds, fuse image features with point cloud features through spatial registration and feature alignment to obtain fused spatial features, and annotate each group of samples with real object category labels; A neural network model is used to build the infrastructure of the object recognition model, with fused spatial features as input and object category labels as output. The model parameters are trained through forward reasoning and backpropagation, and the cross-entropy loss is used to optimize the object recognition model.

8. The method for spatial object perception based on privacy protection camera and radar according to claim 1, characterized in that: The radar detection parameters corresponding to the detection areas of different furniture are set to obtain the detection thresholds of different areas in the current space, including: The radar detection parameters for different time periods are set for the detection areas corresponding to different furniture to obtain the detection thresholds for different areas of the current space.

Images