A radar data labeling method and related device

Abstract

The present application discloses a radar data annotation method and related devices, which relate to the field of data annotation technology, including: obtaining radar data and image data; wherein the radar data is radar point data of a target scene collected by a radar device, and the image data is an image of a target scene collected by a camera; according to the external parameter matrix between the radar device and the camera, the radar data is projected to the camera coordinate system, and according to the internal parameter matrix of the camera, the radar data on the camera coordinate system is projected to the pixel coordinate system to obtain the point coordinates of the radar data on the pixel coordinate system; a target frame containing the point coordinates is generated; the category of the radar point in the target frame is obtained to obtain the labeled radar data. The present application can achieve the purpose of improving the accuracy of data annotation.

Description

Technical Field

[0001] The present application relates to the field of data labeling technology, and in particular to a radar data labeling method and related devices. Background Art

[0002] In autonomous driving, millimeter-wave radar plays an important role in detecting obstacles in extreme weather conditions. Before running the millimeter-wave radar detection algorithm, an accurate millimeter-wave radar dataset needs to be constructed. However, due to the low resolution of millimeter-wave radar, it is not possible to directly label the data by the human eye after visualization. Therefore, how to implement millimeter-wave radar data labeling is very important for improving the data accuracy of the millimeter-wave radar dataset. Summary of the invention

[0003] In view of the above problems, the present application provides a radar data labeling method and related devices to achieve the purpose of improving the accuracy of data labeling. The specific scheme is as follows:

[0004] The first aspect of the present application provides a radar data labeling method, comprising:

[0005] Acquire radar data and image data; wherein the radar data is radar point data of a target scene collected by a radar device, and the image data is an image of the target scene collected by a camera;

[0006] Projecting the radar data to a camera coordinate system according to an external parameter matrix between the radar device and the camera, and projecting the radar data on the camera coordinate system to a pixel coordinate system according to an internal parameter matrix of the camera to obtain point coordinates of the radar data on the pixel coordinate system;

[0007] Generate a target frame containing the coordinates of the point;

[0008] The categories of the radar points in the target frame are obtained to obtain the labeled radar data.

[0009] In a possible implementation, after acquiring the radar data, the method further includes:

[0010] Obtaining radar point velocity and radar point acceleration from the radar data;

[0011] Searching for radar points whose radar point speed is 0 and whose radar point acceleration is not 0, and radar points whose radar point speed changes and whose radar point acceleration is 0, from the radar data;

[0012] The radar point data corresponding to the found radar point is deleted from the radar data to obtain filtered radar data.

[0013] In a possible implementation, the method further includes:

[0014] Acquire a relative speed of a target measured by the radar device relative to the radar device, a speed of the radar device, and an installation angle of the radar device;

[0015] Calculating the product of the speed of the radar device and the cosine value of the installation angle, adding the product to the relative speed, and obtaining the absolute speed of the target;

[0016] When the absolute speed of the target is greater than a speed threshold, determining that the target is a dynamic target;

[0017] When the absolute speed of the target is not greater than the speed threshold, determining that the target is a static target;

[0018] The generating a target frame including the point coordinates includes:

[0019] When the radar point represented by the point coordinates is a dynamic target, generating a first target frame including the point coordinates;

[0020] When the radar point represented by the point coordinates is a static target, a second target frame including the point coordinates is generated.

[0021] In a possible implementation, generating a target frame including the point coordinates further includes:

[0022] According to the longitudinal distance of the radar point represented by the point coordinates, target frames of different brightness are generated.

[0023] In a possible implementation, after generating a target frame including the point coordinates, the method further includes:

[0024] The image containing the target frame is subjected to a frame switching operation to display the motion trajectory of the radar point in the target frame; wherein the motion trajectory is used to identify the category of the radar point in the target frame.

[0025] In a possible implementation, after obtaining the category of the radar point in the target frame, the method further includes:

[0026] When the number of the target frame is one, saving the category and attribute data of the radar point in the target frame;

[0027] When there are multiple target frames and the multiple target frames overlap, the longitudinal distances between the multiple target frames are determined, and the categories and attribute data of the radar points in the target frame with the shortest longitudinal distance are saved.

[0028] A second aspect of the present application provides a radar data annotation system, comprising:

[0029] A data acquisition module, used to acquire radar data and image data; wherein the radar data is radar point data of a target scene acquired by a radar device, and the image data is an image of the target scene acquired by a camera;

[0030] A coordinate system conversion module, used to project the radar data to the camera coordinate system according to the external parameter matrix between the radar device and the camera; project the radar data on the camera coordinate system to the pixel coordinate system according to the internal parameter matrix of the camera to obtain the point coordinates of the radar data on the pixel coordinate system;

[0031] A target frame generation module, used to generate a target frame containing the point coordinates;

[0032] The labeling module is used to obtain the category of the radar point in the target frame and obtain the labeled radar data.

[0033] A third aspect of the present application provides a computer program product, including computer-readable instructions. When the computer-readable instructions are executed on an electronic device, the electronic device implements the radar data labeling method of the first aspect or any implementation manner of the first aspect.

[0034] A fourth aspect of the present application provides an electronic device, comprising at least one processor and a memory connected to the processor, wherein:

[0035] The memory is used to store computer programs;

[0036] The processor is used to execute the computer program so that the electronic device can implement the radar data labeling method of the above-mentioned first aspect or any implementation manner of the first aspect.

[0037] A fifth aspect of the present application provides a computer storage medium, which carries one or more computer programs. When the one or more computer programs are executed by an electronic device, the electronic device can implement the radar data labeling method of the above-mentioned first aspect or any implementation method of the first aspect.

[0038] By means of the above technical scheme, the radar data labeling method and related devices provided in the present application obtain radar point data of the target scene collected by the radar device and the image of the target scene collected by the camera. Through the external parameter matrix and the internal parameter matrix, the radar points can be projected onto the image, and a target frame containing the radar points can be generated on the image to facilitate obtaining the category of the radar points in the target frame, thereby realizing radar data labeling. The method adopted in the present application can avoid the problem that radar data cannot be directly used for labeling when detecting obstacles in extreme weather due to the low resolution of the millimeter-wave radar, and can achieve the purpose of improving the accuracy of radar data labeling. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the accompanying drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and the originals and elements are not necessarily drawn to scale.

[0040] Figure 1 A flow chart of a radar data annotation method provided in this application;

[0041] Figure 2 A schematic diagram of an installation location provided for this application;

[0042] FIG3 (a) is a schematic diagram of a radar coordinate system provided by the present application;

[0043] FIG3 (b) is a schematic diagram of a camera coordinate system provided in the present application;

[0044] Figure 4 A schematic diagram of a labeled data set provided for this application;

[0045] Figure 5 A schematic diagram of the structure of a radar data annotation system provided for this application;

[0046] Figure 6 A schematic diagram of the structure of an electronic device provided in this application. DETAILED DESCRIPTION

[0047] The following describes the embodiments of the present application in conjunction with the drawings in the embodiments of the present application. The terms used in the implementation method section of the present application are only used to explain the specific embodiments of the present application, and are not intended to limit the present application.

[0048] The embodiments of the present application are described below in conjunction with the accompanying drawings. Those skilled in the art will appreciate that, with the development of technology and the emergence of new scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0049] The terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and need not be used to describe a specific order or sequential order. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances, which is only to describe the distinction mode adopted by the objects of the same attributes when describing in the embodiments of the present application. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, so that the process, method, system, product or equipment comprising a series of units need not be limited to those units, but may include other units that are not clearly listed or inherent to these processes, methods, products or equipment.

[0050] The embodiment of the present application provides a radar data labeling method. The radar data labeling method of the embodiment of the present application is described in detail below with reference to the accompanying drawings.

[0051] Reference Figure 1 , Figure 1 A schematic diagram of a radar data labeling method provided in an embodiment of the present application is shown in FIG. Figure 1 As shown, a radar data labeling method provided in an embodiment of the present application may include steps 101 to 104, and these steps are described in detail below.

[0052] Step 101: Acquire radar data and image data; wherein the radar data is radar point data of a target scene collected by a radar device, and the image data is an image of a target scene collected by a camera.

[0053] The radar device can be a millimeter wave radar. In practical applications, a 3D millimeter wave radar can be selected. Millimeter wave radars usually have two acquisition modes: cluster point cloud and object target. In order to facilitate labeling and reduce preprocessing, this application uses the object target mode to collect millimeter wave radar data and collects point cloud data sets in the object mode. The data in the object mode is more intuitive. The collected radar data includes point sets with 13 dimensional feature attributes such as coordinates, reflection cross section, velocity, acceleration, etc. The camera can be an RGB camera. The RGB camera collects image data of continuous frames. The image data collected by the RGB camera is a color scene image.

[0054] The target scene data is collected simultaneously using 3D millimeter wave radar and RGB camera. When collecting scene data, the two sensors need to be installed on the same plane, and the longitudinal center lines of the sensors are aligned and installed up and down. The purpose of this is to simplify the external parameter matrix between the millimeter wave radar and the camera, making projection simpler and more convenient. Figure 2 As shown, Figure 2 This is a schematic diagram of the installation position of the 3D millimeter-wave radar and the RGB camera. The millimeter-wave radar is installed directly above the RGB camera along the vertical center line of the same plane. This installation can reduce the projection complexity when the millimeter-wave radar data is projected onto the pixel plane.

[0055] The collected radar data contains interference targets such as empty targets, false targets, and false alarm targets. Therefore, after obtaining the radar data, the radar data can be filtered to remove interference targets to the greatest extent while retaining the real targets.

[0056] In a possible implementation, after acquiring the radar data, the method further includes:

[0057] Obtain radar point velocity and radar point acceleration from radar data;

[0058] Find radar points whose radar point velocity is 0 and whose radar point acceleration is not 0, and radar points whose radar point velocity changes and whose radar point acceleration is 0 from radar data;

[0059] The radar point data corresponding to the found radar point is deleted from the radar data to obtain filtered radar data.

[0060] Since millimeter wave radar relies on the Doppler effect for target detection, it is susceptible to the influence of multipath effect and clutter, resulting in many false alarm targets. These false alarm targets will cause the radar data collected to contain a large number of empty targets and false targets, affecting the calibration of real targets. Therefore, after obtaining radar data, radar data can be subjected to target filtering, and interference targets can be removed to the greatest extent under the premise of retaining real targets, so as to reduce the calibration interference of false alarm targets to the greatest extent. When radar data is filtered, it can be filtered for the characteristics that the change of radar point speed and acceleration does not conform to the laws of physics, and radar points whose longitudinal speed and lateral speed of radar point are 0 but acceleration is not 0, or radar points whose acceleration is 0 but speed changes all the time can be deleted. The present application filters false alarm targets according to radar characteristics, provides a basis for data annotation, and does not need to go deep into the Doppler spectrum of the bottom layer for complex signal analysis.

[0061] In a possible implementation, the radar data labeling method provided by the present application further includes:

[0062] Acquire the relative speed of the target measured by the radar device relative to the radar device, the speed of the radar device and the installation angle of the radar device;

[0063] Calculate the product of the velocity of the radar device and the cosine value of the installation angle, add the product to the relative velocity, and obtain the absolute velocity of the target;

[0064] When the absolute speed of the target is greater than the speed threshold, the target is determined to be a dynamic target;

[0065] When the absolute speed of the target is not greater than the speed threshold, the target is determined to be a static target.

[0066] When the millimeter-wave radar itself moves, the collected radar data contains targets that were originally stationary with the ground and targets that were originally moving with the ground. In order to improve the accuracy of subsequent annotations, it is necessary to distinguish between targets that were originally stationary with the ground and targets that were originally moving with the ground in the millimeter-wave radar data, and screen out dynamic targets to provide accuracy support for subsequent annotations. Since the static target value considers that the speed of dynamic targets is different from that of static targets, speed is used as a benchmark for screening. Since the millimeter-wave radar measures relative speed, this application calculates the absolute speed of the radar target based on the speed at which the millimeter-wave radar moves with the vehicle. The formula for the absolute speed of the target is as follows:

[0067]

[0068] In the formula, v abs is the absolute velocity of the target, v rel v is the relative speed of the target relative to the radar device measured by the radar device. car is the speed of the radar device. Since the radar device is installed on the vehicle, v car Also the vehicle speed, v car Obtained through the vehicle-mounted GPS or other sensors, θ is the installation angle of the radar device, that is, the installation angle of the radar device relative to the vehicle. If the radar is installed directly in front of the vehicle, the angle θ is 0 degrees.

[0069] After obtaining the absolute speed of the target, the dynamic properties of the target can be screened out. The absolute speed of a static target is close to zero, while the absolute speed of a dynamic target is greater than zero. By setting a speed threshold v thresh , dynamic targets can be screened out, namely v abs >v thresh This application can distinguish between real millimeter-wave radar data with dynamic and static attributes by adding a dynamic target screening mechanism based on false alarm target filtering.

[0070] Step 102: According to the external parameter matrix between the radar device and the camera, the radar data is projected to the camera coordinate system, and according to the internal parameter matrix of the camera, the radar data on the camera coordinate system is projected to the pixel coordinate system to obtain the point coordinates of the radar data on the pixel coordinate system.

[0071] Taking the camera coordinate system as the reference coordinate system, the radar data is projected to the camera coordinate system using the external parameter matrix between the millimeter-wave radar and the camera, and then the radar data on the camera coordinate system is projected to the pixel coordinate system using the camera's internal parameter matrix to obtain the point coordinates of the radar data on the pixel coordinate system. The point coordinates of the radar data on the pixel coordinate system are the coordinates of the radar point on the image.

[0072] Through the installation position relationship between the millimeter-wave radar and the RGB camera, the external parameter matrix between the millimeter-wave radar and the camera can be calibrated. According to the external parameter matrix between the radar device and the camera, the radar data is projected to the camera coordinate system, that is, the millimeter-wave radar coordinate system and the RGB camera coordinate system are converted. The centers of the millimeter-wave radar and the RGB camera are installed on the same vertical line. The conversion formula is as follows:

[0073]

[0074] Where R is the rotation matrix, which is used to represent the rotation of the radar coordinate system relative to the camera coordinate system. Figure 3 (a) is a schematic diagram of the radar coordinate system. From the inside out, the radar coordinate system is a two-dimensional coordinate system with the x axis forward and the y axis to the left. Figure 3 (b) is a schematic diagram of the camera coordinate system. The RGB camera coordinate system is a three-dimensional coordinate system with the x axis to the right, the y axis downward, and the z axis forward. The millimeter-wave radar and the RGB camera are installed on the same vertical line, so the rotation matrix uses an axis exchange matrix to achieve the conversion of the coordinate system by rearranging and changing the sign on the basis of the unit matrix. T is the translation matrix, (x c ,y c ,z c ) is the spatial coordinate of the radar point in the camera coordinate system, (x r ,y r ,z r ) is the coordinate of the radar point data point. In actual use, the rotation matrix R and the translation matrix T can be adjusted according to different coordinate systems.

[0075] After projecting the radar data to the camera coordinate system, the conversion formula for projecting the millimeter-wave radar data from the camera coordinate system to the pixel coordinate system is as follows:

[0076]

[0077]

[0078] In the formula, (x c ,y c ,z c ) is the coordinate value of the millimeter-wave radar point projected onto the camera coordinate system, which is obtained by dividing the coordinate value of the camera coordinate system in each direction by z c Normalize to get the normalized coordinates (x c / z c ,y c / z c ,1), and then the camera internal parameters (f x ,f y ,c x ,c y), convert the normalized coordinates to the pixel coordinate system, and obtain the coordinates of the millimeter wave radar point in the pixel coordinate system (u, v, 1), where u and v represent the horizontal and vertical positions of the pixel point in the image respectively. (u, v, 1) is the point coordinate of the radar data in the pixel coordinate system, that is, the spatial coordinates of the pixel point of the target in the color image in the pixel coordinate system, z c is the depth value corresponding to the pixel in the depth map. The internal parameter f of the camera x and f y are the scaling ratios of the camera focal length in the x and y directions, c x and c y are the offsets of the camera optical axis in the x and y directions respectively. These internal parameters f x 、f y 、c x and c y The corresponding value can be obtained through the camera calibration method. Through the conversion formula, the millimeter wave radar data can be accurately projected onto the camera image.

[0079] Step 103: Generate a target box containing the point coordinates.

[0080] After obtaining the point coordinates of the radar data in the pixel coordinate system, the coordinates of the radar point on the image can be obtained, and a target frame containing the point coordinates can be generated. The target frame contains the radar point. In practical applications, opencv can be used to generate a target frame with the point coordinates as the center point.

[0081] Optionally, generate a target box containing the point coordinates, including:

[0082] When the radar point represented by the point coordinates is a dynamic target, generating a first target frame including the point coordinates;

[0083] When the radar point represented by the point coordinates is a static target, a second target frame including the point coordinates is generated.

[0084] Different target frames can be generated for dynamic targets and static targets, and the first target frame can be distinguished from the second target frame by color, line type, etc., so as to distinguish dynamic targets from static targets. For example, the actual moving target is a red frame, and the stationary target is a green frame.

[0085] Optionally, generate a target box containing the point coordinates, and also include:

[0086] According to the longitudinal distance of the radar point represented by the point coordinates, target frames of different brightness are generated.

[0087] The brightness of the target frame color is set according to the longitudinal distance of the radar target. In practical applications, the color of the target frame closest to the radar target is the brightest, and vice versa.

[0088] In a possible implementation, after generating a target frame containing point coordinates, the following steps are further included:

[0089] The image containing the target frame is switched between the previous and next frames to display the motion trajectory of the radar point in the target frame; wherein the motion trajectory is used to identify the category of the radar point in the target frame.

[0090] By using the ROSbag tool to control the start and stop of millimeter-wave radar data and image data and the switching of previous and next frames, the motion trajectory of the radar point in the target frame can be displayed. The category of the radar point in the target frame can be identified through the changes in the visualized position correspondence on the image containing the target frame and the motion trajectory of the radar points in the previous and next frames.

[0091] In practical applications, by starting ROS Master (the main control program of ROS) and writing Python scripts, the loading, playback and interactive control of ROSbag data (including radar data and image data) are realized. The script uses the ROSbag API (an interface for recording and replaying ROS messages) to load the specified ROSbag file and store all messages in a list. Then, the script detects the playback status in a loop and publishes messages in chronological order. The user can control the playback by listening to the up and down keys on the keyboard. Press the up key to return to the previous frame and pause, and press the key to continue playing the next frame, ensuring the flexibility and real-time operation. Through the correspondence of image visualization and the continuous playback of the previous and next frames controlled by the up and down keys on the keyboard, the motion trajectory of the radar point target can be viewed and the category attributes of the target can be identified.

[0092] Step 104: Obtain the categories of radar points in the target frame and obtain labeled radar data.

[0093] During the labeling process, ROSbag play is used to run the scene data collected by the millimeter-wave radar and camera to control the playback rate. The space bar on the keyboard can be used to pause the playback. For the paused frame of radar data and image data, an image containing the target frame is obtained and annotated on the image containing the target frame.

[0094] The user uses the keyboard to send a data playback pause signal. In response to the pause signal, the message receiving time is updated every second through the timer. Through the timer callback function, the value of a storage variable last_msg_time is increased in real time, by 1 second each time, to indicate the time elapsed since the last message was received. Compare the current last_msg_time with the preset threshold (such as 4 seconds). If the last_msg_time value exceeds the threshold, the state variable bag_paused is set to "pause", indicating that the message playback is considered to be in a paused state; otherwise, it is set to the "continue" state, indicating that the message playback is proceeding normally. When it is determined that the message playback is in a paused state, the corresponding processing function (such as prev_image_callback()) is triggered to facilitate the user to mark, which can ensure the accuracy and efficiency of the marking work.

[0095] Optionally, after obtaining the category of the radar point in the target frame, it also includes:

[0096] When the number of target frames is one, the category and attribute data of the radar points in the target frame are saved;

[0097] When there are multiple target frames and the multiple target frames overlap, the longitudinal distances between the multiple target frames are determined, and the categories and attribute data of the radar points in the target frame with the shortest longitudinal distance are saved.

[0098] According to the visualization relationship of the target in the image, the millimeter-wave radar data is interactively annotated with the mouse. According to different situations of the target frame, the category and attribute data of the radar point can be saved by clicking the target frame with the mouse. For a single target on the image corresponding to a single target frame of the millimeter-wave radar, by clicking the target frame with the mouse, the radar attribute data corresponding to the target frame can be saved, including the radar point speed, acceleration, coordinates, frame number, etc., and the category of the radar point corresponding to the target frame can be set. The dynamic attribute of the radar point can also be set according to the color of the target frame, that is, whether the radar point is a dynamic target or a static target. For the same target with multiple completely overlapping frames, the longitudinal distance of multiple target frames is determined, and the attribute data and radar point category of the radar point corresponding to the target frame with the closest longitudinal distance are saved. The dynamic attributes of the radar point can also be saved. The category and attribute data of the radar point can be saved in a csv file, and the annotation is completed.

[0099] Of course, if there are multiple partially overlapping frames for the same target on the image, since different color brightness of the target frame is set according to the longitudinal distance of the radar target, it is possible to identify which center point of the overlapping frame corresponds to the target more accurately. Click the identified target frame with the mouse to mark the nearest target frame, save the radar attribute data corresponding to the target frame, and the category and dynamic attributes of the radar point into a csv file to complete the calibration.

[0100] The labeled dataset is as follows Figure 4 As shown, Figure 4 In addition to the radar point category class, it also includes 13 types of radar attribute data, frame represents the frame number, id represents the sequence number, dist_long represents the longitudinal distance, dist_lat represents the lateral distance, vrel_long represents the longitudinal velocity, vrel_lat represents the lateral velocity, rcs represents the reflection cross section, dyn_prop represents the dynamic property, arel_long represents the longitudinal acceleration, tag represents the built-in category attribute, arel_lat represents the lateral acceleration, orientation represents the azimuth, length represents the length, and width represents the width.

[0101] This application only uses an ordinary RGB camera to capture images, and projects the millimeter-wave radar points onto the image through the external parameter matrix and the internal parameter matrix. Only by setting the ROSbag tool and interacting with the mouse, the visual position correspondence on the image and the target trajectory changes of the previous and next frames can be observed, thereby realizing data annotation. No specific RGB-D depth camera is required for depth information collection, the hardware requirements are low, and the annotation effect is more intuitive.

[0102] A radar data labeling method provided in an embodiment of the present application is introduced above, and a system for executing the above radar data labeling method will be introduced below.

[0103] See also Figure 5 , Figure 5 This is a schematic diagram of the structure of a radar data annotation system provided in an embodiment of the present application. Figure 5 As shown, the radar data annotation system includes:

[0104] The data acquisition module 501 is used to acquire radar data and image data; wherein the radar data is radar point data of the target scene collected by the radar device, and the image data is an image of the target scene collected by the camera.

[0105] The coordinate system conversion module 502 is used to project the radar data to the camera coordinate system according to the external parameter matrix between the radar device and the camera; and to project the radar data on the camera coordinate system to the pixel coordinate system according to the internal parameter matrix of the camera to obtain the point coordinates of the radar data on the pixel coordinate system.

[0106] The target frame generation module 503 is used to generate a target frame containing point coordinates.

[0107] The labeling module 504 is used to obtain the categories of radar points in the target frame and obtain labeled radar data.

[0108] In a possible implementation, the radar data annotation system further includes:

[0109] The filtering module is used to obtain the radar point velocity and the radar point acceleration from the radar data after acquiring the radar data; search for radar points whose radar point velocity is 0 and whose radar point acceleration is not 0, and radar points whose radar point velocity changes and whose radar point acceleration is 0 from the radar data; delete the radar point data corresponding to the found radar points from the radar data to obtain filtered radar data.

[0110] In a possible implementation, the radar data annotation system further includes:

[0111] The dynamic attribute determination module is used to obtain the relative speed of the target measured by the radar device relative to the radar device, the speed of the radar device and the installation angle of the radar device; calculate the product of the speed of the radar device and the cosine value of the installation angle, add the product to the relative speed, and obtain the absolute speed of the target; when the absolute speed of the target is greater than the speed threshold, determine the target as a dynamic target; when the absolute speed of the target is not greater than the speed threshold, determine the target as a static target.

[0112] In a possible implementation, the target frame generation module 503 is specifically configured to:

[0113] When the radar point represented by the point coordinates is a dynamic target, generating a first target frame including the point coordinates;

[0114] When the radar point represented by the point coordinates is a static target, a second target frame including the point coordinates is generated.

[0115] In a possible implementation, the target frame generating module 503 is further configured to:

[0116] According to the longitudinal distance of the radar point represented by the point coordinates, target frames of different brightness are generated.

[0117] In a possible implementation, the radar data annotation system further includes:

[0118] The image operation module is used to switch the previous and next frames of the image containing the target frame to display the motion trajectory of the radar point in the target frame; wherein the motion trajectory is used to identify the category of the radar point in the target frame.

[0119] In a possible implementation, the radar data annotation system further includes:

[0120] The saving module is used to save the category and attribute data of the radar point in the target frame after obtaining the category of the radar point in the target frame when the number of target frames is one; when the number of target frames is multiple and multiple target frames overlap, determine the longitudinal distance of multiple target frames, and save the category and attribute data of the radar point in the target frame with the closest longitudinal distance.

[0121] The present application also provides an electronic device in an embodiment. Figure 6 As shown, it shows a schematic diagram of the structure of an electronic device suitable for implementing the embodiment of the present application. The electronic device in the embodiment of the present application may include but is not limited to fixed terminals such as mobile phones, laptops, PDAs (personal digital assistants), PADs (tablet computers), desktop computers, etc. Figure 6 The electronic device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0122] like Figure 6 As shown, the electronic device may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 601, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 602 or a program loaded from a storage device 608 to a random access memory (RAM) 603. When the electronic device is powered on, various programs and data required for the operation of the electronic device are also stored in the RAM 603. The processing device 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0123] Typically, the following devices may be connected to the I / O interface 605: an input device 606 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 607 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 608 including, for example, a memory card, a hard disk, etc.; and a communication device 609. The communication device 609 may allow the electronic device to communicate with other devices wirelessly or by wire to exchange data. Although Figure 6 An electronic device having various devices is shown, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead.

[0124] The electronic device can implement the above-mentioned radar data labeling method.

[0125] An embodiment of the present application also provides a computer program product, including computer-readable instructions. When the computer-readable instructions are executed on an electronic device, the electronic device implements any radar data labeling method provided in the embodiment of the present application.

[0126] A computer-readable storage medium is also provided in an embodiment of the present application. The storage medium carries one or more computer programs. When the one or more computer programs are executed by an electronic device, the electronic device can implement any radar data labeling method provided in the embodiment of the present application.

[0127] It should also be noted that the device embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed over multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. In addition, in the drawings of the device embodiments provided by the present application, the connection relationship between the modules indicates that there is a communication connection between them, which may be specifically implemented as one or more communication buses or signal lines.

[0128] Through the description of the above implementation mode, the technicians in the field can clearly understand that the present application can be implemented by means of software plus necessary general hardware, and of course, it can also be implemented by special hardware including special integrated circuits, special CPUs, special memories, special components, etc. In general, all functions completed by computer programs can be easily implemented by corresponding hardware, and the specific hardware structure used to implement the same function can also be various, such as analog circuits, digital circuits or special circuits. However, for the present application, software program implementation is a better implementation mode in more cases. Based on such an understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a readable storage medium, such as a computer floppy disk, a U disk, a mobile hard disk, a ROM, a RAM, a disk or an optical disk, etc., including a number of instructions to enable a computer device (which can be a personal computer, a training device, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0129] In the above embodiments, all or part of the embodiments may be implemented by software, hardware, firmware or any combination thereof. When implemented by software, all or part of the embodiments may be implemented in the form of a computer program product.

[0130] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from a website site, a computer, a training device, or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, training device, or data center. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a training device, a data center, etc. that includes one or more available media integrations. The available medium may be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive (SSD)), etc.

Claims

1. A radar data labeling method, characterized in that: include: Acquire radar data and image data; wherein the radar data is radar point data of a target scene collected by a radar device, and the image data is an image of the target scene collected by a camera; Projecting the radar data to a camera coordinate system according to an external parameter matrix between the radar device and the camera, and projecting the radar data on the camera coordinate system to a pixel coordinate system according to an internal parameter matrix of the camera to obtain point coordinates of the radar data on the pixel coordinate system; Generate a target frame containing the coordinates of the point; The categories of the radar points in the target frame are obtained to obtain the labeled radar data.

2. The radar data labeling method according to claim 1, characterized in that: After acquiring the radar data, the method further includes: Obtaining radar point velocity and radar point acceleration from the radar data; Searching for radar points whose radar point speed is 0 and whose radar point acceleration is not 0, and radar points whose radar point speed changes and whose radar point acceleration is 0, from the radar data; The radar point data corresponding to the found radar point is deleted from the radar data to obtain filtered radar data.

3. The radar data labeling method according to claim 1, characterized in that: Also includes: Acquire a relative speed of a target measured by the radar device relative to the radar device, a speed of the radar device, and an installation angle of the radar device; Calculating the product of the speed of the radar device and the cosine value of the installation angle, adding the product to the relative speed, and obtaining the absolute speed of the target; When the absolute speed of the target is greater than a speed threshold, determining that the target is a dynamic target; When the absolute speed of the target is not greater than the speed threshold, determining that the target is a static target; The generating a target frame including the point coordinates includes: When the radar point represented by the point coordinates is a dynamic target, generating a first target frame including the point coordinates; When the radar point represented by the point coordinates is a static target, a second target frame including the point coordinates is generated.

4. The radar data labeling method according to any one of claims 1 to 3, characterized in that: The generating of the target frame including the point coordinates further includes: According to the longitudinal distance of the radar point represented by the point coordinates, target frames of different brightness are generated.

5. The radar data labeling method according to any one of claims 1 to 3, characterized in that: After generating the target frame including the point coordinates, the method further includes: The image containing the target frame is subjected to a frame switching operation to display the motion trajectory of the radar point in the target frame; wherein the motion trajectory is used to identify the category of the radar point in the target frame.

6. The radar data labeling method according to any one of claims 1 to 3, characterized in that: After obtaining the category of the radar point in the target frame, the method further includes: When the number of the target frame is one, saving the category and attribute data of the radar point in the target frame; When there are multiple target frames and the multiple target frames overlap, the longitudinal distances between the multiple target frames are determined, and the categories and attribute data of the radar points in the target frame with the shortest longitudinal distance are saved.

7. A radar data annotation system, characterized in that: include: A data acquisition module, used to acquire radar data and image data; wherein the radar data is radar point data of a target scene acquired by a radar device, and the image data is an image of the target scene acquired by a camera; A coordinate system conversion module, used to project the radar data to the camera coordinate system according to the external parameter matrix between the radar device and the camera; project the radar data on the camera coordinate system to the pixel coordinate system according to the internal parameter matrix of the camera to obtain the point coordinates of the radar data on the pixel coordinate system; A target frame generation module, used to generate a target frame containing the point coordinates; The labeling module is used to obtain the category of the radar point in the target frame and obtain the labeled radar data.

8. A computer program product, characterized in that It includes computer-readable instructions, and when the computer-readable instructions are executed on an electronic device, the electronic device implements the radar data labeling method as described in any one of claims 1 to 6.

9. An electronic device, characterized in that: The method comprises at least one processor and a memory connected to the processor, wherein: The memory is used to store computer programs; The processor is used to execute the computer program so that the electronic device can implement the radar data labeling method as described in any one of claims 1 to 6.

10. A computer storage medium, characterized in that: The storage medium carries one or more computer programs, and when the one or more computer programs are executed by an electronic device, the electronic device can implement the radar data labeling method as described in any one of claims 1 to 6.

Images