A radar point cloud preprocessing method, system, device and storage medium

Abstract

The present invention discloses a radar point cloud preprocessing method, system, device and storage medium, including: placing a radar and a camera in the same plane in a spatial coordinate system, both facing the target, and their orientation is parallel to the x-axis of the spatial coordinate system, and determining the coordinates of the radar center and the camera center; obtaining the camera external parameter matrix and internal parameter matrix under the current position of the camera; according to the external parameter matrix, converting the radar point cloud coordinates from the spatial coordinate system to the camera coordinate system; according to the internal parameter matrix, converting the radar point cloud coordinates from the camera coordinate system to the pixel coordinate system; adjusting the radar point cloud coordinates in the pixel coordinate system according to the set offset and scaling parameters; normalizing the other dimensional features of the radar as the channel dimension of the radar image; filling each radar pixel point into an n*n square, and finally outputting the radar image. The radar image generated by the present invention can be used as an effective input for the AI ​​model, and can be used for vision-based AI model training and data analysis.

Description

Technical Field

[0001] The present invention relates to radar signal processing technology, and in particular to a radar point cloud preprocessing method, system, device and storage medium. Background Art

[0002] With the continuous breakthroughs in chip technology, millimeter wave chips can be miniaturized and mass-produced at low cost, which enables radar technology to be applied to modern intelligent services and improve user experience. The rapid development of AI technology in the field of vision has gradually changed people's production and lifestyle. AI vision technology has been applied to medical, security, education, home and other fields. As AI vision technology continues to develop and mature, AI vision models can not only be used for media such as images and videos, but also for processing other types of information. There have been a lot of research and applications, trying to use AI vision models in signal processing, data analysis, event prediction and other fields, and have achieved remarkable results. Similarly, AI vision models can also process radar point cloud data. It only needs to convert the radar point cloud into image information first for graphic input. The effective combination of radar technology and AI vision technology makes radar perception a current and future development focus. Radar perception can be applied in industrial production and people's daily life, especially in the fields of medical care and health care. Radar perception technology can detect people's behavior, perceive the number of people, whether they exist, breathing and heartbeat. Unlike using cameras, there is no light restriction and no privacy issues, so it is easier for people to accept.

[0003] Although there are some studies combining radar point clouds with AI vision models, the existing point cloud to image conversion methods have poor generalization and are difficult to perform data analysis. For example, radar point clouds are converted into heat maps, but analyzing data based on heat maps is not intuitive. Although very high classification accuracy can be obtained when training data, the accuracy drops significantly on the test set. Although it is sometimes possible to analyze the correspondence between some features of the heat map and the labels, it is difficult when the sample size is huge and the number of classification types increases significantly, especially in practical applications. The actual data is more complex than the experimental data, so the AI ​​vision model is likely to infer abnormal results. Despite this, detection technology based on radar point clouds still has huge potential for development, and there are more technologies to be explored.

[0004] Insufficient existing technology:

[0005] 1) The existing radar point cloud to image conversion method cannot explicitly express the information of the detected target, and thus has poor generalization performance in AI model reasoning.

[0006] 2) The existing radar point cloud to image conversion methods make it difficult to analyze data and problems directly from the generated images.

[0007] 3) Existing radar point cloud image conversion methods are difficult to accurately calibrate with label data. Summary of the invention

[0008] Purpose of the invention: The purpose of the present invention is to provide a radar point cloud preprocessing method, system, device and storage medium.

[0009] Technical solution: A radar point cloud preprocessing method of the present invention converts the features of the radar point cloud in the spatial coordinate system into a radar image in the pixel coordinate system, and different features of the radar point cloud correspond to images of different channels of the radar image; specifically, the following steps are included:

[0010] S1. Place the radar and camera in the spatial coordinate system (o g , x g ,y g , z g ) are placed on the same plane facing the target, and the coordinates of the radar center in the space coordinate system are (x g,r ,y g,r , z g,r ), the coordinates of the camera center in the space coordinate system are (x g,c ,y g,c , z g,c ); establish a radar coordinate system and a camera coordinate system with the radar center and the camera center respectively, wherein the radar coordinate system and the camera coordinate system are in the same direction as the x, y, and z axes of the space coordinate system;

[0011] S2, according to the coordinates of the radar center in the space coordinate system (x g,r ,y g,r , z g,r ), converting the radar point cloud coordinates from the radar coordinate system to the space coordinate system;

[0012] S3, measure the current position of the camera (x g,c ,y g,c , z g,c ) under the camera external parameter matrix and internal parameter matrix;

[0013] S4, converting the radar point cloud coordinates from the space coordinate system to the camera coordinate system according to the external parameter matrix;

[0014] S5. Convert the radar point cloud coordinates from the camera coordinate system to the pixel coordinate system according to the intrinsic parameter matrix;

[0015] S6. According to the radar point cloud coordinates in the pixel coordinate system described in S5, the offsets of the radar point cloud in different coordinate axes of the pixel coordinate system and the scaling parameters of the radar point cloud in the pixel coordinate system are set, and the pixel coordinates of the radar point cloud in the pixel coordinate system are adjusted according to the set offsets and scaling parameters;

[0016] S7, normalizing the speed, intensity, and depth features of the radar point cloud, and using them as pixel values ​​of three channel images of the radar image respectively;

[0017] S8. Fill the pixel coordinates of the radar point on the radar image corresponding to each point in the radar point cloud into an n*n plane, and finally output the radar image.

[0018] Furthermore, the camera is a distortion-free fixed-focus camera.

[0019] Furthermore, the internal parameter matrix in step S3 is:

[0020]

[0021] The external parameter matrix is:

[0022]

[0023] Among them, u0 and v0 represent the pixel coordinates of the camera image in the pixel coordinate system; dx and dy represent the pixel size, that is, the actual length corresponding to u0 and v0; w represents the distance between the camera coordinate system and the pixel coordinate system;

[0024] Represents the rotation matrix of the camera coordinate system relative to the space coordinate system, r ij Represents the elements in the rotation matrix, i=1,2,3, j=1,2,3;

[0025] Represents the translation vector of the camera coordinate system relative to the space coordinate system, where t x , t y , t z Represents different translations relative to each coordinate axis.

[0026] Furthermore, in step S4, the formula for converting the radar point cloud coordinates from the space coordinate system to the camera coordinate system is:

[0027]

[0028] in, is the inverse of the external parameter matrix, x c,p y c,p z c,p Respectively represent the coordinates of the pth radar point in the camera coordinate system, x g,p y g,p z g,p They respectively represent the coordinates of the p-th radar point in the spatial coordinate system.

[0029] Furthermore, in step S5, the formula for converting the radar point cloud coordinates from the camera coordinate system to the pixel coordinate system is:

[0030]

[0031] Among them, Z C Indicates the z-axis coordinate of the radar point cloud in the camera coordinate system, u p , v p Represents the pixel coordinates of the pth point in the radar point cloud in the pixel coordinate system.

[0032] Furthermore, in step S7, the speed, intensity, and depth features of the radar point cloud are normalized into integer variables ranging from 0 to 255.

[0033] Furthermore, the feature of each point of the radar point cloud in step S7 corresponds to a pixel value of a radar point on the radar image.

[0034] A radar point cloud preprocessing system of the present invention is used to convert the features of the radar point cloud in the spatial coordinate system into a radar image in the pixel coordinate system, wherein different features of the radar point cloud correspond to images of different channels of the radar image; specifically, the system comprises:

[0035] The radar and camera coordinate determination unit is used to place the radar and camera in the spatial coordinate system (o g , x g ,y g , z g ) are placed on the same plane facing the target, and the coordinates of the radar center in the space coordinate system are (x g,r ,y g,r , z g,r ), the coordinates of the camera center in the space coordinate system are (x g,c ,y g,c , z g,c ); establish a radar coordinate system and a camera coordinate system with the radar center and the camera center respectively, wherein the radar coordinate system and the camera coordinate system are in the same direction as the x, y, and z axes of the space coordinate system;

[0036] The first coordinate conversion unit is used to convert the coordinates (x g,r ,y g,r , z g,r ), converting the radar point cloud coordinates from the radar coordinate system to the space coordinate system;

[0037] The external and internal parameter matrix determination unit is used to measure the current position of the camera (x g,c ,y g,c , z g,c ) under the camera external parameter matrix and internal parameter matrix;

[0038] A second coordinate conversion unit is used to convert the radar point cloud coordinates from the space coordinate system to the camera coordinate system according to the external parameter matrix;

[0039] A third coordinate conversion unit, used for converting the radar point cloud coordinates from the camera coordinate system to the pixel coordinate system according to the intrinsic parameter matrix;

[0040] A radar point cloud coordinate adjustment unit is used to set the offset of the radar point cloud in different coordinate axes of the pixel coordinate system and the scaling parameters of the radar point cloud in the pixel coordinate system according to the radar point cloud coordinates in the pixel coordinate system, and adjust the pixel coordinates of the radar point cloud in the pixel coordinate system according to the set offset and scaling parameters;

[0041] An image channel determination unit is used to normalize the speed, intensity, and depth features of the radar point cloud and use them as pixel values ​​of three channel images of the radar image respectively;

[0042] The radar image output unit is used to fill the pixel coordinates of the radar point on the radar image corresponding to each point in the radar point cloud into an n*n plane, and finally output the radar image.

[0043] An electronic device of the present invention comprises:

[0044] A memory storing executable program code;

[0045] a processor coupled to the memory;

[0046] The processor calls the executable program code stored in the memory to execute the steps of the radar point cloud preprocessing method as described above.

[0047] A computer-readable storage medium of the present invention stores computer instructions, and when the computer instructions are called, they are used to execute the steps of the radar point cloud preprocessing method as described above.

[0048] Beneficial effects: Compared with the prior art, the advantages of the present invention are: According to the imaging principle of the camera, the present invention maps the radar point cloud into an image format to facilitate processing using an AI visual model. Compared with other point cloud processing methods, the method of the present invention can explicitly capture the local motion information of the target and can strictly correspond to the image information, so it is also more convenient to do data preprocessing, calibration, and data analysis. This method can be widely used in radar-based intelligent target perception and detection technology. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 is a flow chart of the method of the present invention;

[0050] Figure 2 This is a schematic diagram of the placement of the camera and radar;

[0051] Figure 3 is the camera image;

[0052] Figure 4 yes Figure 3 Schematic diagram of radar point cloud coordinates obtained by the corresponding radar;

[0053] Figure 5 is the transformed radar image. DETAILED DESCRIPTION

[0054] The present invention is described in detail below with reference to the accompanying drawings and specific embodiments.

[0055] Commonly used CNN-based AI models are designed for image input. When these AI models process data from other fields, they usually need to convert the new data into image format, and the effective conversion method determines the knowledge transfer effect of the AI ​​model. The present invention proposes a radar point cloud preprocessing method, that is, a method for converting radar point clouds into images. The radar image generated by this method is closer to the camera imaging law than the existing method, which facilitates the effective use of AI visual models.

[0056] The radar point cloud is a collection of multiple points formed by radar detecting moving targets, and each point includes the three-dimensional coordinates of the point in the radar coordinate system and characteristic information such as the speed, intensity, and depth of the point.

[0057] like Figure 1 As shown, a radar point cloud preprocessing method of the present invention converts the features of the radar point cloud in the spatial coordinate system into a radar image in the pixel coordinate system, and different features of the radar point cloud correspond to images of different channels of the radar image; the method comprises the following steps:

[0058] S1. Place the radar and camera in the spatial coordinate system (o g , x g ,y g , z g ) are placed on the same plane facing the target, and the coordinates of the radar center in the space coordinate system are (x g,r ,y g,r , z g,r ), the coordinates of the camera center in the space coordinate system are (x g,c ,y g,c , z g,c );Use the radar center and camera center to establish the radar coordinate system and camera coordinate system respectively, where the radar coordinate system and camera coordinate system are in the same direction as the x, y, and z axes of the space coordinate system; the camera and radar are placed as shown in Figure 2 shown.

[0059] In this embodiment, a 4-transmitter, 4-receiver 60GHz millimeter-wave radar is used, and a 100-degree FOV distortion-free, fixed-focus industrial camera is used for data calibration. The two are placed close to each other and 1 meter high, and the radar and camera normal phases are both horizontally facing the target position.

[0060] S2, according to the coordinates of the radar center in the space coordinate system (x g,r ,y g,r , z g,r ), converting the radar point cloud coordinates acquired by the radar from the radar coordinate system to the space coordinate system;

[0061] S3, use a camera calibration method to measure the current position of the camera (x g,c ,y g,c , z g,c ) under the camera external parameter matrix and internal parameter matrix, such as:

[0062] Internal parameter matrix:

[0063] External parameter matrix:

[0064] Among them, u0 and v0 represent the pixel coordinates of the camera image in the pixel coordinate system, with the unit of pixel; dx and dy represent the pixel size, that is, the actual length corresponding to u0 and v0, with the unit of mm / pixel; w represents the distance between the camera coordinate system and the pixel coordinate system.

[0065] Represents the rotation matrix of the camera coordinate system relative to the space coordinate system, r ij Represents the elements in the rotation matrix, i=1,2,3, j=1,2,3;

[0066] Represents the translation vector of the camera coordinate system relative to the space coordinate system, where t x , t y , t z Represents different translations relative to each coordinate axis.

[0067] S4. According to the external parameter matrix, the radar point cloud coordinates are converted from the space coordinate system to the camera coordinate system, that is:

[0068]

[0069] in, is the inverse of the external parameter matrix, x c,p y c,p z c,p Respectively represent the coordinates of the pth radar point in the camera coordinate system, x g,p y g,p z g,p They respectively represent the coordinates of the p-th radar point in the spatial coordinate system.

[0070] S5. According to the intrinsic parameter matrix, the radar point cloud coordinates are converted from the camera coordinate system to the pixel coordinate system, that is:

[0071]

[0072] Among them, Z C Indicates the z-axis coordinate of the radar point cloud in the camera coordinate system, u p , v p Represents the pixel coordinates of the pth point in the radar point cloud in the pixel coordinate system.

[0073] S6. According to the radar point cloud coordinates in the pixel coordinate system described in step S5, the offset of the radar point cloud in different coordinate axes of the pixel coordinate system and the scaling parameters of the radar point cloud in the pixel coordinate system can be set; the pixel coordinates of the radar point cloud in the pixel coordinate system are adjusted according to the set offset and scaling parameters.

[0074] S7. Normalize the speed, intensity, and depth features of the radar point cloud and use them as pixel values ​​of the three channel images of the radar image.

[0075] In this embodiment, the speed, intensity, and depth features of the radar point cloud are normalized into integer variables of 0 to 255, and the radar point cloud data is 5*N floating-point data, where 5 represents the five dimensions of x, y, z axes and speed and intensity, and N represents the number of point clouds in the current frame.

[0076] S8. In the radar image described in step S7, the feature of each point of the radar point cloud corresponds to the pixel value of a radar point on the radar image, and the pixel coordinates of each radar point are filled into an n*n plane, where the value of n is determined according to the size of the radar image and the density of the radar point cloud, and finally the radar image is output.

[0077] like Figure 3 As shown, this is the image taken by the camera. Figure 4 for Figure 3 The corresponding radar point cloud coordinates obtained by the radar, Figure 5 is the transformed radar image obtained according to the above method; Figure 5 It can be seen that the present invention utilizes the camera imaging principle to map the radar point cloud into an image format. The formed radar point cloud image is similar to a visual image, which can intuitively observe the state and position of human body movement and can strictly correspond to the camera image information.

[0078] A radar point cloud preprocessing system of the present invention is used to convert the features of the radar point cloud in the spatial coordinate system into a radar image in the pixel coordinate system, wherein different features of the radar point cloud correspond to images of different channels of the radar image; specifically, the system comprises:

[0079] The radar and camera coordinate determination unit is used to place the radar and camera in the spatial coordinate system (o g , x g ,y g , z g) are placed on the same plane facing the target, and the coordinates of the radar center in the space coordinate system are (x g,r ,y g,r , z g,r ), the coordinates of the camera center in the space coordinate system are (x g,c ,y g,c , z g,c ); establish a radar coordinate system and a camera coordinate system with the radar center and the camera center respectively, wherein the radar coordinate system and the camera coordinate system are in the same direction as the x, y, and z axes of the space coordinate system;

[0080] The first coordinate conversion unit is used to convert the coordinates (x g,r ,y g,r , z g,r ), converting the radar point cloud coordinates from the radar coordinate system to the space coordinate system;

[0081] The external and internal parameter matrix determination unit is used to measure the current position of the camera (x g,c ,y g,c , z g,c ) under the camera external parameter matrix and internal parameter matrix;

[0082] A second coordinate conversion unit is used to convert the radar point cloud coordinates from the space coordinate system to the camera coordinate system according to the external parameter matrix;

[0083] A third coordinate conversion unit, used for converting the radar point cloud coordinates from the camera coordinate system to the pixel coordinate system according to the intrinsic parameter matrix;

[0084] A radar point cloud coordinate adjustment unit is used to set the offset of the radar point cloud in different coordinate axes of the pixel coordinate system and the scaling parameters of the radar point cloud in the pixel coordinate system according to the radar point cloud coordinates in the pixel coordinate system, and adjust the pixel coordinates of the radar point cloud in the pixel coordinate system according to the set offset and scaling parameters;

[0085] An image channel determination unit is used to normalize the speed, intensity, and depth features of the radar point cloud and use them as pixel values ​​of three channel images of the radar image respectively;

[0086] The radar image output unit is used to fill the pixel coordinates of the radar point on the radar image corresponding to each point in the radar point cloud into an n*n plane, and finally output the radar image.

[0087] An electronic device of the present invention may include: a memory storing executable program code; a processor coupled to the memory; the processor calls the executable program code stored in the memory to execute the steps in the above-mentioned radar point cloud preprocessing method.

[0088] The memory may include computer system readable media in the form of volatile memory, such as random access memory (RAM) and / or cache memory. The device may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, the memory may be used to read and write non-removable, non-volatile magnetic media (commonly referred to as a "hard drive"). A program / utility having a set (at least one) of program modules may be stored in, for example, the memory, such program modules including but not limited to an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include the implementation of a network environment. The program modules typically perform the functions and / or methods in the embodiments described herein.

[0089] The processor executes various functional applications and data processing by running the program stored in the memory, for example, implementing a radar point cloud preprocessing method provided by the present invention.

[0090] A computer-readable storage medium of the present invention contains computer-executable instructions. When the computer-executable instructions are executed by a computer processor, they are used to execute the radar point cloud preprocessing method of the present invention.

[0091] The computer storage medium of the embodiment of the present invention can adopt any combination of one or more computer-readable media. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium can be, for example, but not limited to, a system, device or device of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples (non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this document, a computer-readable storage medium can be any tangible medium containing or storing a program, which can be used by an instruction execution system, a device or a device or used in combination with it.

[0092] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, which carry computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0093] The program code embodied on the computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0094] Computer program code for performing the operation of the present invention may be written in one or more programming languages ​​or combinations thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0095] Of course, the computer executable instructions of a storage medium including computer executable instructions provided by an embodiment of the present invention are not limited to the above method operations, and can also execute related operations in the method provided by any embodiment of the present invention.

[0096] An embodiment of the present invention discloses a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program, and the computer program is operable to enable a computer to execute the steps in radar point cloud preprocessing described in the present invention.

[0097] The system embodiments described above are only illustrative, wherein the modules described as separate components may or may not be physically separated, and the components displayed as modules may or may not be physical modules, i.e., they may be located in one place, or they may be distributed on multiple network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Those of ordinary skill in the art may understand and implement it without creative labor.

[0098] Through the specific description of the above embodiments, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on such an understanding, the above technical solution can be essentially or partly contributed to the prior art in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, and the storage medium includes a read-only memory (ROM), a random access memory (RAM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), a one-time programmable read-only memory (OTPROM), an electronically erasable rewritable read-only memory (EEPROM), a compact disc (CD-ROM) or other optical disc storage, magnetic disk storage, magnetic tape storage, or any other computer-readable medium that can be used to carry or store data.

Claims

1. A radar point cloud preprocessing method, characterized in that: The features of the radar point cloud in spatial coordinates are converted into radar images in pixel coordinates. Different features of the radar point cloud correspond to images of different channels of the radar image. Specifically, it includes: S1. Place the radar and camera in the spatial coordinate system (o g , x g ,y g , z g ) are placed on the same plane facing the target, and the coordinates of the radar center in the space coordinate system are (x g,r ,y g,r , z g,r ), the coordinates of the camera center in the space coordinate system are (x g,c ,y g,c , z g,c ); establish a radar coordinate system and a camera coordinate system with the radar center and the camera center respectively, wherein the radar coordinate system and the camera coordinate system are in the same direction as the x, y, and z axes of the space coordinate system; S2, according to the coordinates of the radar center in the space coordinate system (x g,r ,y g,r , z g,r ), converting the radar point cloud coordinates from the radar coordinate system to the space coordinate system; S3, measure the current position of the camera (x g,c ,y g,c , z g,c ) under the camera external parameter matrix and internal parameter matrix; S4, converting the radar point cloud coordinates from the space coordinate system to the camera coordinate system according to the external parameter matrix; S5. Convert the radar point cloud coordinates from the camera coordinate system to the pixel coordinate system according to the intrinsic parameter matrix; S6. According to the radar point cloud coordinates in the pixel coordinate system described in S5, the offsets of the radar point cloud in different coordinate axes of the pixel coordinate system and the scaling parameters of the radar point cloud in the pixel coordinate system are set, and the pixel coordinates of the radar point cloud in the pixel coordinate system are adjusted according to the set offsets and scaling parameters; S7, normalizing the speed, intensity, and depth features of the radar point cloud, and using them as pixel values ​​of three channel images of the radar image respectively; S8. Fill the pixel coordinates of the radar point on the radar image corresponding to each point in the radar point cloud into an n*n plane, and finally output the radar image.

2. A radar point cloud preprocessing method according to claim 1, characterized in that: The camera is a distortion-free fixed-focus camera.

3. A radar point cloud preprocessing method according to claim 1, characterized in that: The internal parameter matrix in step S3 is: The external parameter matrix is: Among them, u0 and v0 represent the pixel coordinates of the camera image in the pixel coordinate system; dx and dy represent the pixel size, that is, the actual length corresponding to u0 and v0; w represents the distance between the camera coordinate system and the pixel coordinate system; Represents the rotation matrix of the camera coordinate system relative to the space coordinate system, r ij Represents the elements in the rotation matrix, i=1,2,3, j=1,2,3; Represents the translation vector of the camera coordinate system relative to the space coordinate system, where t x , t y , t z Represents different translations relative to each coordinate axis.

4. A radar point cloud preprocessing method according to claim 1, characterized in that: The formula for converting the radar point cloud coordinates from the space coordinate system to the camera coordinate system in step S4 is: in, is the inverse of the external parameter matrix, x c,p y c,p z c,p Respectively represent the coordinates of the pth radar point in the camera coordinate system, x g,p y g,p z g,p They respectively represent the coordinates of the p-th radar point in the spatial coordinate system.

5. The radar point cloud preprocessing method according to claim 1, characterized in that: The formula for converting the radar point cloud coordinates from the camera coordinate system to the pixel coordinate system in step S5 is: Among them, Z C Indicates the z-axis coordinate of the radar point cloud in the camera coordinate system, u p , v p Represents the pixel coordinates of the pth point in the radar point cloud in the pixel coordinate system.

6. A radar point cloud preprocessing method according to claim 1, characterized in that: In step S7, the speed, intensity, and depth features of the radar point cloud are normalized into integer variables ranging from 0 to 255.

7. A radar point cloud preprocessing method according to claim 1, characterized in that: The feature of each point of the radar point cloud in step S7 corresponds to the pixel value of a radar point on the radar image.

8. A radar point cloud preprocessing system, characterized in that: It is used to convert the features of radar point cloud in spatial coordinates into radar images in pixel coordinate system. Different features of radar point cloud correspond to images of different channels of radar image. Specifically, it includes: The radar and camera coordinate determination unit is used to place the radar and camera in the spatial coordinate system (o g , x g ,y g , z g ) are placed on the same plane facing the target, and the coordinates of the radar center in the space coordinate system are (x g,r ,y g,r , z g,r ), the coordinates of the camera center in the space coordinate system are (x g,c ,y g,c , z g,c ); establish a radar coordinate system and a camera coordinate system with the radar center and the camera center respectively, wherein the radar coordinate system and the camera coordinate system are in the same direction as the x, y, and z axes of the space coordinate system; The first coordinate conversion unit is used to convert the coordinates (x g,r ,y g,r , z g,r ), converting the radar point cloud coordinates from the radar coordinate system to the space coordinate system; The external and internal parameter matrix determination unit is used to measure the current position of the camera (x g,c ,y g,c , z g,c ) under the camera external parameter matrix and internal parameter matrix; A second coordinate conversion unit is used to convert the radar point cloud coordinates from the space coordinate system to the camera coordinate system according to the external parameter matrix; A third coordinate conversion unit, used for converting the radar point cloud coordinates from the camera coordinate system to the pixel coordinate system according to the intrinsic parameter matrix; A radar point cloud coordinate adjustment unit is used to set the offset of the radar point cloud in different coordinate axes of the pixel coordinate system and the scaling parameters of the radar point cloud in the pixel coordinate system according to the radar point cloud coordinates in the pixel coordinate system, and adjust the pixel coordinates of the radar point cloud in the pixel coordinate system according to the set offset and scaling parameters; An image channel determination unit is used to normalize the speed, intensity, and depth features of the radar point cloud and use them as pixel values ​​of three channel images of the radar image respectively; The radar image output unit is used to fill the pixel coordinates of the radar point on the radar image corresponding to each point in the radar point cloud into an n*n plane, and finally output the radar image.

9. An electronic device, characterized in that: The device comprises: A memory storing executable program code; a processor coupled to the memory; The processor calls the executable program code stored in the memory to execute the steps of the radar point cloud preprocessing method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, which, when called, are used to execute the steps of the radar point cloud preprocessing method according to any one of claims 1 to 7.

Images