Space object sensing method based on privacy protection camera and radar

Abstract

The invention discloses a space object sensing method based on a privacy protection camera and a radar, and the method comprises the steps: obtaining the three-dimensional space characteristics of a current space through the radar, judging whether a human body exists in the current space or not through the three-dimensional space characteristics, and starting the privacy camera to collect a global image if the human body exists in the current space; obtaining two-dimensional space features of the current space through the global image, and fusing the two-dimensional space features and the three-dimensional space features of the current space to obtain fused space features of the current space; inputting the fusion feature of the current space into a pre-trained object recognition model to obtain an object type of the current space; interference source labels and universal furniture labels are marked according to the object types of the current space, the current space objects with the interference source labels are filtered, and universal furniture distribution of the current space is obtained; and according to the current space general furniture distribution, radar detection parameters are adjusted according to detection areas, and detection thresholds of different detection areas are obtained.

Description

Technical Field

[0001] The present invention relates to the technical field of spatial object perception, and particularly to a method for spatial object perception based on a privacy-protected camera and radar. Background Art

[0002] With the increasing demand for scenarios such as intelligent security, intelligent office, and unmanned management, the technologies for automatic perception, recognition, and modeling of people and objects in space have been increasingly emphasized. Existing spatial perception systems mainly use a variety of sensing means such as cameras, millimeter-wave radars, infrared, and ultrasonic to achieve the detection, recognition, and tracking of target objects.

[0003] The radar detects echoes by emitting electromagnetic waves, can obtain the distance, speed, and angle information of objects in space, and has a certain three-dimensional point cloud modeling ability, capable of perceiving the depth and dynamic changes of spatial objects, and is particularly suitable for the detection of dynamic targets (such as people and moving objects). However, a single radar has obvious limitations in spatial perception: its point cloud resolution is low, it is difficult to distinguish the specific categories of objects, and only the contour or rough spatial distribution information of the objects can be obtained, making it difficult to achieve refined three-dimensional modeling and accurate recognition of various objects in complex scenarios.

[0004] To improve the accuracy of spatial perception, existing technologies usually combine the radar with other sensors (such as cameras), and use the visual image information obtained by the camera to identify and model spatial targets. The camera can make up for the deficiencies of the radar in object category discrimination and detail capture, and improve the system's recognition ability for static structures and dynamic targets. However, when the camera captures spatial images, it will capture the visible image information of users, including faces, biometric features, body postures, and environmental details. Once these sensitive data are stored, transmitted, or processed, there is a risk of being intercepted, leaked, or misused, and it is difficult to meet the scenario requirements with high privacy protection requirements such as home, medical, and office.

[0005] In addition, the existing spatial perception methods also have the following deficiencies in practical applications: on the one hand, the system has limited ability to distinguish dynamic targets (such as people and pets), interference sources (such as reflective objects like mirrors, glass, and TVs), and static targets (such as furniture and spatial structures) in the environment, and false alarms, false positives, or missed detections are likely to occur. On the other hand, there are differences in the data types and spatio-temporal resolutions collected by the camera and the radar, and simple data fusion methods are difficult to achieve high-confidence spatial perception and target recognition. Especially in complex environments where privacy protection and high-precision recognition requirements coexist, the adaptability and robustness of existing solutions need to be improved.

[0006] Designing a method for spatial object perception based on a privacy-protected camera and radar to address the above problems of the existing technology is the purpose of the research of the present invention. Summary of the Invention

[0007] In view of this, the object of the present invention is to propose a method for perceiving spatial objects based on a privacy - protected camera and radar, which can solve the above problems.

[0008] The present invention provides a method for perceiving spatial objects based on a privacy - protected camera and radar, which is applied to a spatial object perception system based on a privacy - protected camera and radar, and includes: A radar for obtaining three - dimensional features of the current space; A privacy - protected camera, which is arranged in the same monitoring area as the radar, for obtaining two - dimensional features of the current space; The method includes: Obtaining three - dimensional spatial features of the current space through the radar, and judging whether there is a human body in the current space through the three - dimensional spatial features. If so, starting the privacy camera to collect a global image; Obtaining two - dimensional spatial features of the current space through the global image, and fusing the two - dimensional spatial features and three - dimensional spatial features of the current space to obtain fused spatial features of the current space; Inputting the fused features of the current space into a pre - trained object recognition model to obtain the object types in the current space; Marking interference source labels and general furniture labels according to the object types in the current space, filtering the current space objects with interference source labels to obtain the distribution of general furniture in the current space; Adjusting the radar detection parameters for different detection areas according to the distribution of general furniture in the current space to obtain detection thresholds for different detection areas.

[0009] Furthermore, the privacy - protected camera includes a camera and a blurring structure, and the blurring structure is arranged on the shooting path of the camera for physically blurring the image obtained by the camera.

[0010] Furthermore, the blurring structure is any one of a static blurring light - transmitting member, a modulating light - transmitting element, and a dynamic blurring structure.

[0011] Furthermore, the step of obtaining three - dimensional spatial features of the current space through the radar, and judging whether there is a human body in the current space through the three - dimensional spatial features. If so, starting the privacy camera to collect a global image includes: Obtaining the radar digital signal of the reflected signal of the current space through the radar, performing two - dimensional Fourier transform on the radar digital signal to obtain the velocity, distance, and angle information of the target point; Calculating the three - dimensional spatial coordinates of the target point according to the velocity, distance, and angle information of the target point to form a spatial point cloud; Performing clustering and feature extraction analysis on the spatial point cloud to identify whether there is a point cloud cluster that conforms to the preset human body characteristics; If a human body is detected, the privacy protection camera is triggered to collect the global image of the current space.

[0012] Further, obtaining the two-dimensional space features of the current space through the global image, and fusing the two-dimensional space features and three-dimensional space features of the current space to obtain the fused space features of the current space includes: Perform channel transformation processing on the global image to obtain images with at least two channel configurations; Overlay and input the images with different channel configurations into the target detection model to detect different objects and their detection frames in the current space; Based on the space calibration parameters between the privacy camera and the radar, perform data alignment and fusion on the detection frames of different objects in the current space and the space point cloud to obtain the fused space features of the current space.

[0013] Further, based on the space calibration parameters between the privacy camera and the radar, performing data alignment and fusion on the detection frames of different objects in the current space and the space point cloud to obtain the fused space features of the current space includes: Project the detection frames of different objects in the current space into the three-dimensional space coordinate system of the space point cloud according to the space calibration parameters; Locate the point cloud subset that intersects with the detection frames of different objects in the space point cloud, and fuse the image features and point cloud features in the point cloud subset to obtain the fused space features of the current space.

[0014] Further, obtaining the two-dimensional space features of the current space through the global image, and fusing the two-dimensional space features and three-dimensional space features of the current space to obtain the fused space features of the current space includes: Perform channel transformation processing on the global image to obtain images with at least two channel configurations; Extract the frequency domain features of the images with different channel configurations through Fourier transform, overlay and input the frequency domain features of the images with different channel configurations into the object classification model to detect different objects and their detection frames in the current space; Based on the space calibration parameters between the privacy camera and the radar, perform data alignment and fusion on the detection frames of different objects in the current space and the space point cloud to obtain the fused space features of the current space.

[0015] Further, the object recognition model is trained through the following steps: Collect historical space global images and their corresponding space point clouds, fuse the image features and point cloud features through space registration and feature alignment to obtain fused space features, and label the true object category labels for each group of samples; Build the infrastructure of the object recognition model using a neural network model, with the fused spatial features as the input and the object category labels as the output. Train the model parameters through forward inference and backpropagation, and optimize the object recognition model using cross-entropy loss.

[0016] Furthermore, the adjustment of the radar detection parameters for different detection regions according to the current general furniture distribution in the space to obtain the detection thresholds for different detection regions includes: Divide the current space into regions according to the three-dimensional spatial positions of the current space objects with general furniture labels to obtain detection regions corresponding to different furniture; Set corresponding radar detection parameters for the detection regions corresponding to different furniture to obtain the detection thresholds for different regions in the current space.

[0017] Furthermore, the setting of corresponding radar detection parameters for the detection regions corresponding to different furniture to obtain the detection thresholds for different regions in the current space includes: Set corresponding radar detection parameters for different time periods for the detection regions corresponding to different furniture to obtain the detection thresholds for different regions in the current space.

[0018] Advantages of the present invention: First, the camera is blurred physically to ensure user privacy and security, cutting off the potential privacy risks at the source. The radar and the privacy protection camera are combined. The privacy protection camera obtains a blurred global image to extract two-dimensional spatial information, and the radar extracts three-dimensional spatial information. The spatial perception ability is improved through the complementarity of the two-dimensional and three-dimensional information.

[0019] Second, the purpose of pre-monitoring whether there is anyone in the space by the radar is to minimize privacy exposure and unnecessary data collection. Only when it is detected that there is a human body in the space, the privacy protection camera is activated to collect the global image. When the radar performs human detection, the energy consumption, data volume, and computational complexity are much lower than those of the privacy protection camera, and it is not affected by light and occlusion, and can continuously monitor in real time. Only when it is judged that it is "necessary" is the camera activated, significantly reducing the overall energy consumption and computational burden of the system. In addition, it can also avoid monitoring and data storage in unoccupied environments or non-essential scenarios to the greatest extent.

[0020] Third, the two-dimensional features obtained from the global image and the three-dimensional features obtained from the radar can make up for the defects that the global image can only provide the relative position and size of the object in the image and cannot accurately know the actual spatial distance and size, as well as the limited recognition ability of a single radar in spatial perception, making it difficult to distinguish specific object categories, and only being able to obtain the contour or point cloud of the object, and it is difficult to achieve refined three-dimensional space reconstruction and item identification. It can not only obtain the position and size of the current space objects, but also guide the depth of the current space objects.

[0021] Fourth, by performing object type recognition on the fusion features of the current space, relevant object information can be extracted from the original perception data for subsequent differentiation between interference sources and static furniture. According to the label types of objects in the current space, the radar parameters are adjusted to focus the sensing and recognition accuracy in the corresponding three-dimensional space structure, ignoring interference sources, avoiding wasting resources in irrelevant areas, improving the overall efficiency of the system, enhancing the accuracy of space object recognition, and constructing an accurate space layout. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0023] Figure 1 It is a system module diagram of Embodiment 1.

[0024] Figure 2 It is a method flowchart of Embodiment 2. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0025] For the convenience of those skilled in the art to understand, the structure of the present invention will be further described in detail below in combination with the drawings for the embodiments. It should be understood that the steps mentioned in this embodiment, unless specifically stating their order, can be adjusted according to actual needs for their front and back order, and can even be executed simultaneously or partially simultaneously.

[0026] Embodiment 1 As Figure 1 shown, Embodiment 1 provides a space object perception system based on a privacy protection camera and a radar, including: A radar for obtaining the three-dimensional features of the current space; A privacy protection camera disposed in the same monitoring area as the radar for obtaining the two-dimensional features of the current space; Specifically, the privacy protection camera includes a camera and a blurring structure, and the blurring structure is disposed on the shooting path of the camera for physically blurring the images obtained by the camera.

[0027] Further, the blurring structure is any one of a static blurring light-transmitting member, a modulating light-transmitting element, and a dynamic blurring structure.

[0028] In this embodiment, traditional camera monitoring involves privacy images such as faces, body features, and environmental details. Once the images are stored or transmitted, there is a risk of leakage or abuse, resulting in privacy and security issues. The blurred structure enables the image to only present the general dynamics and contours of the monitored space, preventing the leakage of personal privacy, ensuring that clear privacy images will not be generated from the source, and even if the malicious tampering algorithm or being hacked, the original picture cannot be reconstructed, taking into account the artificial intelligence function and user privacy protection, and meeting the privacy compliance requirements of multiple scenarios such as home / medical / elderly care, etc.

[0029] When using a single radar for spatial perception, the radar's recognition ability is limited, it is difficult to distinguish specific object categories, and only the contours or point clouds of the objects can be obtained, making it difficult to achieve refined three-dimensional space reconstruction and item identification. By combining the radar and the privacy protection camera, the privacy protection camera extracts two-dimensional spatial information, and the radar extracts three-dimensional spatial information, improving the spatial perception ability through the complementarity of the two-dimensional and three-dimensional information.

[0030] The main objective of the static blurred light-transmitting component is to allow light to pass through but scatter sufficiently so that the camera cannot image clear details. Such as frosted glass, frosted acrylic sheet, frosted polycarbonate sheet, atomized PET film, nano-transmissive scattering coating, etc. These components are fixedly arranged on the shooting path of the camera and achieve image blurring through their own light-transmitting and scattering properties.

[0031] The modulated light-transmitting component includes, but is not limited to, components that can achieve controllable light-transmitting states or blurring degrees through methods such as heating, electro-optic modulation, mechanical vibration, liquid flow, acoustic disturbance, etc. For example: heating-type modulation components (such as heating atomizing film), electrochromic or electro-scattering intelligent dimming film, mechanical vibration or micro-perturbation film, liquid-modulated light-transmitting component, acoustic-modulated light-transmitting component, etc. For example, materials such as PET film and glass that can be surface-heated are used to utilize the physical state changes (such as atomization, water vapor condensation, optical disturbance, etc.) caused by heating to achieve dynamic blurring of the image.

[0032] The dynamic blurred structure, such as a high-speed mechanical brush set on the shooting path of the camera, drives the brush to reciprocate or rotate at a specific frequency, forming a continuous optical smear or occlusion effect through dynamic perturbation, making the captured image unrecognizable both in time and space.

[0033] Embodiment Two Based on Embodiment One, this embodiment provides a spatial object perception system based on a privacy protection camera and a radar, as Figure 2 shown. Embodiment Two provides a spatial object perception method based on a privacy protection camera and a radar, including: S1 obtains the three-dimensional spatial features of the current space through radar, and determines whether there is a human body in the current space based on the three-dimensional spatial features. If so, it activates the privacy camera to collect the global image; S101 obtains the radar digital signal of the reflected signal in the current space through radar, performs two-dimensional Fourier transform on the radar digital signal, and obtains the velocity, distance, and angle information of the target point; S102 calculates the three-dimensional spatial coordinates of the target point based on the velocity, distance, and angle information of the target point to form a spatial point cloud; S103 performs clustering and feature extraction analysis on the spatial point cloud to identify whether there is a point cloud cluster that conforms to the preset human body characteristics; S104 If a human body is detected, it triggers the privacy protection camera to collect the global image of the current space.

[0034] In this step, the purpose of pre-monitoring whether there is someone in the space through radar is to minimize privacy exposure and unnecessary data collection. Only when it is detected that there is indeed a human body in the space, the privacy protection camera is activated to collect the global image. When the radar is performing human detection, the energy consumption, data volume, and computational complexity are much lower than those of the privacy protection camera, and it is not affected by light and occlusion, and can continuously monitor in real time. Only when it is judged that it is "necessary" is the camera activated, which significantly reduces the overall energy consumption and computational burden of the system. In addition, it can also maximize the avoidance of monitoring and data storage in unoccupied environments or non-essential scenarios.

[0035] S2 obtains the two-dimensional spatial features of the current space through the global image, and fuses the two-dimensional spatial features and three-dimensional spatial features of the current space to obtain the fused spatial features of the current space; S201 performs channel transformation processing on the global image to obtain images with at least two channel configurations; S202 superimposes and inputs the images with different channel configurations into the target detection model to detect different objects and their detection frames in the current space; In this step, the channel transformation includes, but is not limited to, channel order adjustment, channel combination switching, or channel mapping. Assuming that the global image is an RGB image, a BGR image can be obtained after channel transformation processing. The RGB image and the BGR image can be input into the target detection model (such as the YOLO model, region convolutional neural network, etc.) to detect the objects and their detection frames in the current space. Since the global image obtained by the privacy protection camera is a blurred image, when using only a single-channel configuration image as the input, the model may be insensitive to specific features. Superimposing and inputting images with different channel configurations helps to capture residual signals and reduce feature information loss.

[0036] Based on the spatial calibration parameters between the privacy camera and the radar, S203 aligns and fuses the detection frames of different objects in the current space with the spatial point cloud to obtain the fused spatial features of the current space.

[0037] S2031 projects the detection frames of different objects in the current space into the three-dimensional space coordinate system of the spatial point cloud according to the spatial calibration parameters; S2032 locates the point cloud subset intersecting with the detection frames of different objects in the spatial point cloud, and fuses the image features and point cloud features in the point cloud subset to obtain the fused spatial features of the current space.

[0038] In this step, the privacy camera captures blurred images. It is difficult to perform high-confidence detection and positioning relying solely on one-side data. The global image can only provide the relative position and size of the object in the image, and it is impossible to accurately know the actual spatial distance and size. Fusing the point cloud data obtained by the radar can make up for this defect. It can not only obtain the position and size of the objects in the current space, but also guide the depth of the objects in the current space.

[0039] S3 inputs the fused features of the current space into a pre-trained object recognition model to obtain the object types in the current space; The object recognition model is trained through the following steps: Collect historical space global images and their corresponding spatial point clouds. Through spatial registration and feature alignment, fuse the image features and point cloud features to obtain the fused spatial features, and label the true object category labels for each group of samples; Use a neural network model to construct the basic architecture of the object recognition model. With the fused spatial features as the input and the object category labels as the output, train the model parameters through forward inference and backpropagation, and optimize the object recognition model using cross-entropy loss.

[0040] In this step, by performing object type recognition on the fused features of the current space, relevant object information can be extracted from the original perception data for subsequent distinction between interference sources and static furniture.

[0041] S4 marks the interference source labels and general furniture labels according to the object types in the current space, filters the current space objects with interference source labels, and obtains the distribution of general furniture in the current space; In this step, the objects identified in the space are distinguished by labeling. The essence of interference sources refers to those objects that are prone to false alarms, clutter, misidentification, false triggering and other negative effects on the perception system. Filtering out these targets helps to improve the recognition accuracy of the system, reduce false alarms, and increase stability and reliability. Objects such as mirrors, glass, televisions, air conditioners, radiators, and strong reflective surfaces may produce abnormally strong echoes, virtual images, multipaths, reflections or spots in radar or camera images, affecting subsequent normal target recognition. If these objects are not eliminated, the system is likely to misjudge the interference source as a key monitoring object such as a human body or a pet, resulting in a false alarm - such as mistaking the image in the mirror for a "real person" and misjudging the moving pixels in the TV screen as "moving objects."

[0042] S5 adjusts the radar detection parameters according to the current general furniture distribution in the space and divides the detection areas into detection areas to obtain detection thresholds for different detection areas.

[0043] S501 divides the current space into regions according to the three-dimensional spatial positions of the current space objects of the universal furniture tags to obtain detection regions corresponding to different furniture; S502 sets corresponding radar detection parameters for detection areas corresponding to different furniture to obtain detection thresholds for different areas of the current space.

[0044] In this step, different furniture (such as beds, sofas, and cabinets) will block, reflect, and absorb radar signals, making the radar perception sensitivity and echo characteristics in certain spatial areas significantly different from those in open areas. For example: near the bed, human movements and fall signals are easily "covered by the bed" or form "false signals"; next to the sofa, it is difficult to distinguish between sitting for a long time and falling. In places with furniture, there are often abnormal activities (such as resting on the bed and playing on the ground). Such behaviors are likely to be misjudged using the same threshold in different areas. For example, "quickly sitting down" on the bed is easily misjudged as a fall. According to the actual distribution of furniture, the parameters are adjusted in a targeted manner so that the radar can use the most appropriate detection logic for typical areas such as "on the bed", "next to the bed", and "open space", thereby reducing false alarms and missed alarms overall.

[0045] In this step S502, corresponding radar detection parameters of different time periods may also be set for detection areas corresponding to different furniture to obtain detection thresholds for different areas of the current space.

[0046] The use and activity patterns of furniture areas often change over time, for example: Bed area: mostly used for sleeping at night, mostly for getting out of bed / making up during the day. Sofa area: mostly used for activities and rest during the day, mostly for no one or short stays at night. Setting differentiated radar detection parameters for different furniture areas according to different time periods can dynamically adapt to the time changes in space use and behavior patterns, significantly improve the accuracy and relevance of human behavior detection, and reduce false alarm rates and energy consumption.

[0047] Embodiment III Based on Embodiment I, this embodiment provides a spatial object perception system based on a privacy-protected camera and radar. The difference between Embodiment III and Embodiment II is that Embodiment III extracts the frequency-domain features of the global image as the two-dimensional features of the global image. The specific steps are as follows: Perform channel transformation processing on the global image to obtain images with at least two channel configurations; Extract the frequency-domain features of the images with different channel configurations through Fourier transform, and superimpose the frequency-domain features of the images with different channel configurations and input them into the object classification model to detect different objects and their detection frames in the current space; Based on the spatial calibration parameters between the privacy camera and the radar, align and fuse the detection frames of different objects in the current space with the spatial point cloud to obtain the fused spatial features of the current space.

[0048] In this embodiment, since the privacy-protected camera obtains a blurred global image, image blurring is essentially the weakening of the high-frequency components of the original image (loss of edge and texture information). Although the Fourier spectra before and after blurring are different, the mid-low frequency parts can still basically reflect the overall structure, contour law, periodic regions, etc. of the object. Compared with pixel-domain features, frequency-domain features can intuitively reflect the residual structure information (even if blurred, the low-frequency features such as the main frequency distribution and shape principal axis of the object may not be completely lost). Frequency-domain features show greater robustness to image "degradation" (such as blurring, resolution reduction, compression damage). In scenarios with noise and blurring, more accurate results can be obtained by performing object detection and classification through frequency-domain features.

[0049] Those skilled in the art should understand that the embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present invention can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk memory, CD-ROM, optical memory, etc.) containing computer-usable program code.

[0050] The present invention is described with reference to the flowcharts and / or block diagrams of methods, apparatuses (systems), and computer program products according to the embodiments of the present invention. It should be understood that each flow and / or block in the flowchart and / or block diagram, and the combination of flows and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing devices generate for implementation in the processFigure 1 one process or multiple processes and / or blocks Figure 1 a device for the functions specified in one block or multiple blocks.

[0051] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured article including an instruction device, and the instruction device implements the processes Figure 1 one process or multiple processes and / or blocks Figure 1 the functions specified in one block or multiple blocks.

[0052] These computer program instructions can also be loaded onto a computer or other programmable data processing device, so that a series of operation steps are executed on the computer or other programmable device to generate a computer-implemented process. Thus, the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in Figure 1 one process or multiple processes and / or blocks Figure 1 one block or multiple blocks.

[0053] It should be noted that in the claims, any reference signs placed between parentheses shall not be construed as limiting the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The present invention can be implemented by means of hardware including several different elements and by means of a suitably programmed computer. In a unit claim listing several devices, several of these devices can be embodied by the same item of hardware. The use of the words first, second, and third, etc. does not denote any order. These words can be interpreted as names.

[0054] Although the preferred embodiments of the present invention have been described, those skilled in the art can make additional changes and modifications once they learn the basic creative concept. Therefore, the appended claims are intended to be construed to include the preferred embodiments as well as all changes and modifications falling within the scope of the present invention.

[0055] Obviously, those skilled in the art can make various changes and deformations to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and deformations of the present invention fall within the scope of the claims of the present invention and their equivalent technologies, the present invention is also intended to include these modifications and deformations.

[0056] In the present invention, unless otherwise clearly defined or limited, terms such as "installed", "connected", "coupled", "fixed", etc. shall be understood in a broad sense. For example, it may be a fixed connection, a detachable connection, or integrated; it may be a mechanical connection or an electrical connection; it may be directly connected or indirectly connected through an intermediate medium, and it may be the internal communication of two components or the interaction relationship between two components. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0057] In the description of this specification, the description with reference to terms such as "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials, or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms should not be understood as necessarily referring to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described can be combined in a suitable manner in any one or more embodiments or examples. In addition, without contradiction, those skilled in the art can combine and combine the different embodiments or examples described in this specification and the features of different embodiments or examples.

Claims

1. A method for spatial object perception based on a privacy-protected camera and radar, characterized in that, Applied to a spatial object perception system based on a privacy - protected camera and radar, including: A radar for obtaining three - dimensional features of the current space; A privacy - protected camera set in the same monitoring area as the radar for obtaining two - dimensional features of the current space; The method includes: Obtaining three - dimensional space features of the current space through the radar, and judging whether there is a human body in the current space through the three - dimensional space features. If so, start the privacy camera to collect a global image; Obtaining two - dimensional space features of the current space through the global image, and fusing the two - dimensional space features and three - dimensional space features of the current space to obtain fused space features of the current space; Inputting the fused features of the current space into a pre - trained object recognition model to obtain the object types in the current space; Marking interference source labels and general furniture labels according to the object types in the current space, filtering the current space objects with interference source labels to obtain the distribution of general furniture in the current space; Adjusting the radar detection parameters for different detection areas according to the distribution of general furniture in the current space to obtain detection thresholds for different detection areas.

2. The spatial object perception method based on a privacy-protected camera and radar according to claim 1, characterized in that, The privacy - protected camera includes a camera and a blurring structure. The blurring structure is set on the shooting path of the camera for physically blurring the image obtained by the camera.

3. A method for spatial object perception based on a privacy-protected camera and radar according to claim 2, characterized in that, The blurring structure is any one of a static blurring light - transmitting member, a modulated light - transmitting element, and a dynamic blurring structure.

4. A method for spatial object perception based on a privacy-protected camera and radar according to claim 1, characterized in that, The step of obtaining three - dimensional space features of the current space through the radar, judging whether there is a human body in the current space through the three - dimensional space features, and if so, starting the privacy camera to collect a global image includes: Obtaining the radar digital signal of the reflection signal in the current space through the radar, performing a two - dimensional Fourier transform on the radar digital signal to obtain the velocity, distance, and angle information of the target point; Calculating the three - dimensional space coordinates of the target point according to the velocity, distance, and angle information of the target point to form a spatial point cloud; Performing clustering and feature extraction analysis on the spatial point cloud to identify whether there is a point cloud cluster that conforms to the preset human body characteristics; If a human body is detected, trigger the privacy - protected camera to collect a global image of the current space.

5. A method for spatial object perception based on a privacy-protected camera and radar according to claim 4, characterized in that, The step of obtaining two - dimensional space features of the current space through the global image, fusing the two - dimensional space features and three - dimensional space features of the current space to obtain fused space features of the current space includes: Performing channel transformation processing on the global image to obtain images with at least two channel configurations; Stacking and inputting the images with different channel configurations into a target detection model to detect different objects and their detection frames in the current space; Based on the spatial calibration parameters between the privacy camera and the radar, aligning and fusing the detection frames of different objects in the current space with the spatial point cloud to obtain the fused space features of the current space.

6. The spatial object perception method based on a privacy protection camera and radar according to claim 5, characterized in that, The step of based on the spatial calibration parameters between the privacy camera and the radar, aligning and fusing the detection frames of different objects in the current space with the spatial point cloud to obtain the fused space features of the current space includes: Projecting the detection frames of different objects in the current space into the three - dimensional space coordinate system of the spatial point cloud according to the spatial calibration parameters; Locate the point cloud subset that intersects the detection boxes of different objects in the spatial point cloud, fuse the image features and point cloud features in the point cloud subset to obtain the fused spatial features of the current space.

7. A method for spatial object perception based on a privacy - protected camera and radar according to claim 4, characterized in that, The method of obtaining the fused spatial features of the current space by fusing the two-dimensional spatial features of the current space obtained from the global image and the three-dimensional spatial features includes: Perform channel transformation processing on the global image to obtain images with at least two channel configurations; Extract the frequency domain features of the images with different channel configurations through Fourier transform, superimpose the frequency domain features of the images with different channel configurations and input them into the object classification model to detect different objects and their detection boxes in the current space; Based on the spatial calibration parameters between the privacy camera and the radar, align and fuse the detection boxes of different objects in the current space with the spatial point cloud to obtain the fused spatial features of the current space.

8. A method for spatial object perception based on a privacy-protected camera and radar according to claim 4, characterized in that, The object recognition model is trained through the following steps: Collect historical space global images and their corresponding spatial point clouds, fuse the image features and point cloud features through spatial registration and feature alignment to obtain fused spatial features, and label each group of samples with the true object category labels; Use a neural network model to construct the basic architecture of the object recognition model, use the fused spatial features as the input and the object category labels as the output, train the model parameters through forward inference and backpropagation, and optimize the object recognition model using cross-entropy loss.

9. A method for spatial object perception based on a privacy-protected camera and radar according to claim 1, characterized in that, The method of adjusting the radar detection parameters for different detection regions according to the current space general furniture distribution to obtain the detection thresholds for different detection regions includes: Divide the current space into regions according to the three-dimensional spatial positions of the current space objects with general furniture labels to obtain detection regions corresponding to different furniture; Set corresponding radar detection parameters for the detection regions corresponding to different furniture to obtain the detection thresholds for different regions in the current space.

10. A method for spatial object perception based on a privacy-protected camera and radar according to claim 9, characterized in that, The method of setting corresponding radar detection parameters for the detection regions corresponding to different furniture to obtain the detection thresholds for different regions in the current space includes: Set the radar detection parameters for different time periods corresponding to the detection regions corresponding to different furniture to obtain the detection thresholds for different regions in the current space.

Images