Simulation method of 4D millimeter wave radar sensor based on Ray-tracing technology

Abstract

The invention relates to the technical field of automatic driving simulation, in particular to a simulation method of a 4D millimeter-wave radar sensor based on a Ray-tracing technology, which comprises the following steps of: sensing a sensor model by virtue of a truth value of a simulation platform, configuring configuration parameters of a 4D millimeter-wave radar, then loading an executable program of the 4D millimeter-wave radar in the simulation platform, and executing the 4D millimeter-wave radar by virtue of the executable program. Configuring UDP (User Datagram Protocol) data release address and wiring harness angular density information; according to the method, the 4D millimeter-wave radar model for automatic driving of the vehicle can be constructed, the model can be compatible with various existing virtual simulation platforms on the market, and detailed target object information lists and radar point cloud information can be provided for users in a JSON format through a UDP (User Datagram Protocol).

Description

Technical Field

[0001] The present invention relates to the technical field of autonomous driving simulation, and in particular to a simulation method for a 4D millimeter-wave radar sensor based on Ray-tracing technology. Background Art

[0002] In vehicle autonomous driving, the application of sensors such as cameras, millimeter-wave radars, and lidars is the core content of accurate perception and reconstruction of the environment, but each has its own shortcomings and deficiencies. 4D millimeter-wave radar uses a large multi-input (MIMO) antenna array for echo positioning. Compared with the traditional millimeter-wave radar's detection of distance, direction, and speed in three dimensions, it can also obtain target information in the vertical direction, and has the potential to replace lidar to a certain extent.

[0003] By improving algorithms and increasing radar aperture, 4D millimeter-wave radar can output dense point cloud data like lidar, with a resolution of up to 1°, greatly enhancing recognition accuracy. At the same time, it has better perception capabilities in severe rainy and foggy weather conditions, and has high potential in performance, anti-interference ability, and cost reduction. The market size is growing steadily and the technology prospects are excellent.

[0004] At present, the simulation technologies for cameras and lidars in various vehicle autonomous driving simulation platforms on the market are already very advanced, but there is still a huge gap in the 4D millimeter-wave radar part. For example, the Carla simulation platform includes this type of sensor, but the point cloud is only calculated to the surface of the ideal rectangular frame instead of the actual 3D model, which is inconsistent with the actual point cloud distribution. Summary of the invention

[0005] The purpose of the present invention is to provide a simulation method of a 4D millimeter wave radar sensor based on Ray-tracing technology to solve the problems raised in the above background technology.

[0006] To achieve the above object, the present invention provides the following technical solution: a simulation method of a 4D millimeter wave radar sensor based on Ray-tracing technology, the method comprising the following steps: Step 1: Use the true value perception sensor model of the simulation platform to configure the configuration parameters of the 4D millimeter wave radar, then load the executable program of the 4D millimeter wave radar in the simulation platform, and configure the UDP data publishing address and beam angle density information; Step 2: Run the simulation after deploying the vehicle, sensors, static terrain, and dynamic scene, obtain static target information from the static terrain and scene description files, and use the simulation platform API to obtain the vehicle information and dynamic target information; Step 3: Use the platform sensor to sense the targets within the detection range, which are part of the static targets and dynamic targets in step 2, search for 3D model files, parse the model surface data, and establish a Ray-tracing scene; Step 4: Assume that starting from the position of the 4D millimeter-wave radar, a number of beams are emitted to the ray-tracing scene within the sensor field of view, and the intersection of the rays and the outer surface of the target object is calculated using the ray-tracing technology; Step 5: Calculate the millimeter-wave radar point cloud and other true value information of the target object according to the position of the vehicle where the sensor is installed and the intersection information of the ray and the target object; select the intersection points where the incident angle between the ray and the outer surface is less than a certain value as the radar point cloud; Step 6: Organize the collected point cloud data and true value data of various targets at each simulation moment, configure the target IP address and port number for the 4D millimeter wave radar executable file, and send the collected and sorted information to the specified address in JSON format through the UDP protocol.

[0007] In the step 1, according to the simulation platform used, a true value perception sensor model that meets the requirements is deployed on the vehicle so that its installation position, field of view angle, harness shape, detection distance, and detectable target type meet the target 4D millimeter wave radar.

[0008] In step 2, the static target object information specifically includes: target object name, type, 3D model file name and storage path, position and angle in the global coordinate system; the vehicle information includes: vehicle position, direction, speed and acceleration; the dynamic target object information includes: target object name, ID, type, 3D model file name and storage path, position, angle, speed and acceleration in the global coordinate system.

[0009] In step 3, according to the read target object 3D model file name, the corresponding 3D model file is searched in the file system, and the vertex normal information of the model assembly is read, which specifically includes: S1: Use to load 3D model file; S2: Obtain all triangle vertices and normal information describing the outer contour of the object in the 3D model file; S3: Organizational data, including all triangle vertices and normal information that make up the outer contour of each object, as well as the object type, ID, position, and direction; S5: Construct a Ray-tracing scenario based on the organized data.

[0010] In step 4, the angular density of emitting a number of beams into the Ray-tracing scene is selected according to the user configuration, the emission quantity is calculated according to the angular density and the field of view angle, and a random error is added to the actual emission angle of the beam.

[0011] The intersection of the wire harness and the Ray-tracing scene is calculated using Ray-tracing technology, which includes the following steps: S1: Import the above Ray-tracing scene; S2: Calculate the direction of the ray after adding random errors, and calculate the intersection of each ray with the Ray-tracing scene, considering only the first intersection; S3: Record the azimuth, altitude, distance, incident angle, and material information of the intersection point.

[0012] In step 5, the millimeter wave radar point cloud data is calculated to include: azimuth, altitude, distance, reflection intensity, and target ID corresponding to the ray; The other true value information of the calculated target object includes: target object ID; type; corresponding speed and acceleration in the sensor coordinate system, vehicle coordinate system and world coordinate system; visible ratio; and time information of entering the sensor field of view.

[0013] In step 6, the point cloud data collected at each simulation moment and the true value data of various target objects specifically include: data acquisition time, target ID, target type, sensor coordinate system, vehicle coordinate system, target position, speed, acceleration information in the world coordinate system, and the coordinates, reflection intensity, distance, azimuth and altitude of the radar point cloud in the sensor coordinate system.

[0014] Compared with the prior art, the present invention has the following beneficial effects: The present invention can construct a 4D millimeter-wave radar model for vehicle autonomous driving, which is compatible with various virtual simulation platforms currently available on the market, and can provide users with a detailed list of target information and radar point cloud information in JSON format via UDP; The 4D millimeter-wave radar model simulation method of the present invention can detect the outline of the target object and form a dense radar point cloud, thus making up for the deficiencies of the traditional millimeter-wave radar model in the simulation field in terms of detection dimension and data volume. Compared with traditional technologies, the present invention can transfer the amount of calculation to the GPU to the greatest extent by introducing Ray-tracing technology and matching it with appropriate computing libraries and hardware for point cloud calculation, giving full play to the advantages of GPU parallel computing and realizing real-time calculation of large-scale accurate radar point cloud data. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 The figure is a flow chart of a simulation method of a 4D millimeter wave radar sensor based on Ray-tracing technology according to the present invention.

[0016] Figure 2 Schematic diagram of the target object true value data information output structure of the present invention.

[0017] Figure 3 It is a schematic diagram of the target object point cloud data information output structure of the present invention. DETAILED DESCRIPTION

[0018] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0019] See also Figures 1 to 3 , the present invention provides a technical solution: The present invention first constructs a 4D millimeter-wave radar model that can adapt to multiple simulation platforms. With the help of the Ray-tracing method of GPU parallel computing, the 3D model is dynamically loaded, the radar point cloud is calculated in real time, and the target object information list can be presented to the user in JSON format through the UDP protocol.

[0020] The core of the present invention is to use Ray-tracing technology for point cloud calculation, give full play to the advantages of GPU parallel computing, and realize real-time calculation of large-scale accurate radar point cloud data.

[0021] Ray-tracing is a technology used in GPUs under the current technical background. It tracks light rays that interact with optical surfaces to obtain a model of the path that the light rays take.

[0022] like Figure 1 As shown in , the present invention first uses the true value perception sensor model of the simulation platform to simulate the configuration of the 4D millimeter wave radar: specifically including: 1. The installation position of the sensor in the vehicle coordinate system; 2. The installation angle of the sensor in the vehicle coordinate system, including pitch angle, heading angle, and yaw angle; 3. Sensor horizontal and vertical field of view angles; 4. Sensor harness shape; 5. Sensor detection range; 6. The sensor can detect the target type; Then load the executable program of the 4D millimeter-wave radar on the simulation platform, and configure the UDP data publishing address and beam angle density information to receive the IP address, port and radar beam density of the target list.

[0023] After deploying the vehicle, sensors, static terrain, and dynamic scene, the simulation is run. The executable program of the 4D millimeter wave radar first obtains the information of static targets, such as triangle cones, water barriers, traffic lights, signs, etc., from the static terrain and dynamic scene description files. The information includes: 1. Target name, type, 3D model name and path; 2. The coordinate position of the target object in the world coordinate system; During operation, the 4D millimeter wave radar executable program will continue to obtain the vehicle information and the real value data of other targets through the simulation scene API, including: 1. The real-time position of the vehicle in the world coordinate system; 2. The real-time posture of the vehicle in the world coordinate system; 3. The real-time speed and acceleration of the vehicle; 4. The sensor can detect target information within its field of view, including the following: 4.1, ID, name, 3D model file name, type; 4.2. The relative position, relative velocity, relative acceleration and angle of the target in the sensor coordinate system; 4.3. The relative position, relative speed, relative acceleration and angle of the target in the vehicle coordinate system; 4.4. The absolute position, absolute velocity, absolute acceleration and angle of the target object in the world coordinate system.

[0024] Use the platform sensor to sense the target objects within the detection range. All targets should be within the above target range. Search the 3D model file according to the file name and path, parse the model surface data, and establish the Ray-tracing scene, including: 1. The triangle vertices and normal information that constitute the outer surface of each target object; 1.1. Use to load 3D model files; 1.2. Obtain all triangle vertices and normal information describing the outer contour of the object in the 3D model file; 1.3. Organize the data, including all triangle vertices and normal information that make up the outer contour of each target, as well as the target type, ID, position, and direction; 2. Model material; 3. Combined with the absolute position and angle of each target in front in the world coordinate system, the position and angle of each target in the Ray-tracing scene can be determined.

[0025] Based on the above-mentioned scene, the executable program of the 4D millimeter-wave radar uses the Ray-tracing technology to shoot several beams of light into the Ray-tracing scene within the sensor's field of view, starting from the position of the 4D millimeter-wave radar, and calculates the information related to the intersection of the rays and the model in the Ray-tracing scene.

[0026] 1. The angular density of the rays is determined by the model settings, and the number of rays is determined by the angular density and the field of view. Each ray adds a tiny random error. 2. Only consider the first intersection between the ray and the target; 3. The relevant information of the intersection point includes: 3.1、Data acquisition time; 3.2. The position of the intersection in the sensor coordinate system, including distance, horizontal angle, and vertical angle; 3.3, the reflection angle formed by the ray and the triangle on the model surface; 3.4. The material of the intersection; 3.5. Calculate the reflection intensity based on the intersection distance, reflection angle and material.

[0027] The executable program of the 4D millimeter-wave radar calculates the millimeter-wave radar point cloud and other true value information of the target object based on the position of the vehicle on which the sensor is installed, the intersection of the ray and the target object, and selects the intersection points where the incident angle between the ray and the outer surface is less than a certain value as the radar point cloud.

[0028] 1. The point cloud information specifically includes: 1.1、Data acquisition time; 1.2. ID and type of the target object where the intersection is located; 1.3. Distance, horizontal angle, and vertical angle of the point cloud in the sensor coordinate system; 1.3. The three-dimensional coordinates of the point cloud in the sensor coordinate system; 1.4. Point cloud reflection intensity.

[0029] 2. Other true value information of the target object includes: 2.1、Data acquisition time; 2.2、Target ID; 2.3. Target type; 2.4. The relative position, relative velocity, relative acceleration and angle of the target in the sensor coordinate system; 2.5. The relative position, relative speed, relative acceleration and angle of the target in the vehicle coordinate system; 2.6. The absolute position, absolute velocity, absolute acceleration and angle of the target in the world coordinate system; 2.7, visible proportion; 2.8. Information about the duration of entering the sensor's field of view.

[0030] When the position of the vehicle and the position of the dynamic target in the scene change in each frame, the position and state of each target in the Ray-tracing scene are modified accordingly, and the Ray-tracing calculation is performed again.

[0031] Organize the collected point cloud data and true value data of various targets at each simulation moment, configure the target IP address and port number for the 4D millimeter-wave radar executable file, and send the collected and sorted information to the specified address in JSON format through the UDP protocol; the collected point cloud data and true value data of various targets at each simulation moment include: data acquisition time, target ID, target type, sensor coordinate system, vehicle coordinate system, target position, speed, acceleration information in the world coordinate system, as well as the coordinates, reflection intensity, distance, azimuth and altitude of the radar point cloud in the sensor coordinate system.

[0032] The user can stop the simulation at any time, or stop it according to the simulation end conditions set for the test case.

[0033] The 4D radar sensor model of the present invention can support the completion of the above-mentioned simulation method. The model includes a radar parameter configuration module, a simulation interface module, a data acquisition module, a Ray-tracing module, a data extraction and calculation module, and a simulation control module.

[0034] Radar parameter configuration module: This module is responsible for obtaining and storing various configuration parameters of the 4D millimeter-wave radar sensor to ensure that the radar settings meet the actual application requirements. The installation position and field of view parameters of the 4D millimeter-wave radar sensor are the same as those of the ordinary millimeter-wave radar configured in the simulation vehicle, and can be obtained by reading the sensor configuration file.

[0035] Simulation interface module: This module ensures the adaptation and compatibility between the 4D millimeter wave radar model and different simulation platforms, and provides corresponding API interfaces for different virtual simulation platforms. It realizes communication and synchronization with the simulation platform during simulation. It also provides an error capture and processing mechanism, which can provide corresponding feedback for users to actively handle when data transmission or interface call fails.

[0036] Data acquisition module: This module extracts the information of the ego vehicle and the target object from the simulation environment. The position and posture of the vehicle in the environment are determined by obtaining the ego vehicle information, and the scene elements involved in Ray-tracing are determined by the target object information perceived by the true value sensor, providing data for subsequent calculations.

[0037] During the simulation, the status information of the vehicle and the target object is updated in real time to ensure simulation synchronization.

[0038] Ray-tracing module: implements the framework of Ray-tracing radar point cloud computing. On the basis of establishing the model scene, it uses the parallel acceleration capability of GPU to calculate the point cloud in real time and give full play to the hardware performance.

[0039] Data extraction and calculation module: responsible for extracting key information from the data acquisition module and the Ray-tracing module and performing calculations. It includes two parts: point cloud data and true value data. In addition to the parts that can be directly obtained, it is necessary to calculate the reflection intensity based on the intersection distance, reflection angle, and material, and calculate the actual position of the target object based on the relative position of the vehicle and the target object.

[0040] Simulation control module: This system has simulation control functions, which can start and terminate the simulation process and allow users to set the end time of the simulation. In addition, the system also supports the pause operation of the simulation, so that users can interrupt the simulation when necessary. During the simulation process, the system will monitor the working status of the 4D radar, record any abnormal conditions, and trigger the alarm mechanism when necessary. At the same time, the system will save the key information of the simulation, including the start and end time of the simulation and the target data during the operation, for subsequent analysis and evaluation. The simulation control module is equipped with a start-stop control module to realize the start, pause and termination operations of the simulation. The simulation control module is equipped with a log recording module, which can save key information of the simulation process including the start time, end time and target data during operation.

[0041] The present invention is a simulation method for a physical-level 4D millimeter-wave radar sensor model designed for autonomous driving simulation and sensor simulation. Its greatest feature is that, unlike the traditional millimeter-wave radar simulation model that only obtains the features of the nearest point of the target, this model can obtain large-area radar point cloud data of the target, which has greatly improved the data integrity and accuracy. At the same time, by taking advantage of Ray-tracing's suitability for parallel computing and giving full play to GPU performance, it has a higher potential in generating and processing dense point cloud problems.

[0042] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A simulation method for a 4D millimeter wave radar sensor based on Ray-tracing technology, characterized in that: The method comprises the following steps: Step 1: Use the true value perception sensor model of the simulation platform to configure the configuration parameters of the 4D millimeter wave radar, then load the executable program of the 4D millimeter wave radar in the simulation platform, and configure the UDP data publishing address and beam angle density information; Step 2: Run the simulation after deploying the vehicle, sensors, static terrain, and dynamic scene, obtain static target information from the static terrain and scene description files, and use the simulation platform API to obtain the vehicle information and dynamic target information; Step 3: Use the platform sensor to sense the targets within the detection range, which are part of the static targets and dynamic targets in step 2, search for 3D model files, parse the model surface data, and establish a Ray-tracing scene; Step 4: Assume that starting from the position of the 4D millimeter-wave radar, a number of beams are emitted to the ray-tracing scene within the sensor field of view, and the intersection of the rays and the outer surface of the target object is calculated using the ray-tracing technology; Step 5: Calculate the millimeter-wave radar point cloud and other true value information of the target object according to the position of the vehicle where the sensor is installed and the intersection information of the ray and the target object; select the intersection points where the incident angle between the ray and the outer surface is less than a certain value as the radar point cloud; Step 6: Organize the collected point cloud data and true value data of various targets at each simulation moment, configure the target IP address and port number for the 4D millimeter wave radar executable file, and send the collected and sorted information to the specified address in JSON format through the UDP protocol.

2. The simulation method of a 4D millimeter wave radar sensor based on Ray-tracing technology according to claim 1, characterized in that: In the step 1, according to the simulation platform used, a true value perception sensor model that meets the requirements is deployed on the vehicle so that its installation position, field of view angle, harness shape, detection distance, and detectable target type meet the target 4D millimeter wave radar.

3. The simulation method of a 4D millimeter wave radar sensor based on Ray-tracing technology according to claim 1, characterized in that: In step 2, the static target object information specifically includes: target object name, type, 3D model file name and storage path, position and angle in the global coordinate system; the vehicle information includes: vehicle position, direction, speed and acceleration; the dynamic target object information includes: target object name, ID, type, 3D model file name and storage path, position, angle, speed and acceleration in the global coordinate system.

4. The simulation method of a 4D millimeter wave radar sensor based on Ray-tracing technology according to claim 1, characterized in that: In step 3, according to the read target object 3D model file name, the corresponding 3D model file is searched in the file system, and the vertex normal information of the model assembly is read, which specifically includes: S1: Use to load 3D model file; S2: Obtain all triangle vertices and normal information describing the outer contour of the object in the 3D model file; S3: Organizational data, including all triangle vertices and normal information that make up the outer contour of each object, as well as the object type, ID, position, and direction; S5: Construct a Ray-tracing scenario based on the organized data.

5. The simulation method of a 4D millimeter wave radar sensor based on Ray-tracing technology according to claim 1, characterized in that: In step 4, the angular density of emitting a number of beams into the Ray-tracing scene is selected according to the user configuration, the emission quantity is calculated according to the angular density and the field of view angle, and a random error is added to the actual emission angle of the beam.

6. The simulation method of a 4D millimeter wave radar sensor based on Ray-tracing technology according to claim 5, characterized in that: Use Ray-tracing technology to calculate the intersection of the wire harness and the Ray-tracing scene. The following steps are involved: S1: Import the above Ray-tracing scene; S2: Calculate the direction of the ray after adding random errors, and calculate the intersection of each ray with the Ray-tracing scene, considering only the first intersection; S3: Record the azimuth, altitude, distance, incident angle, and material information of the intersection point.

7. The simulation method of a 4D millimeter wave radar sensor based on Ray-tracing technology according to claim 1, characterized in that: In step 5, the millimeter wave radar point cloud data is calculated to include: azimuth, altitude, distance, reflection intensity, and target ID corresponding to the ray; The other true value information of the calculated target object includes: target object ID; type; corresponding speed and acceleration in the sensor coordinate system, vehicle coordinate system and world coordinate system; visible ratio; and time information of entering the sensor field of view.

8. The simulation method of a 4D millimeter wave radar sensor based on Ray-tracing technology according to claim 1, characterized in that: In step 6, the point cloud data collected at each simulation moment and the true value data of various target objects specifically include: data acquisition time, target ID, target type, sensor coordinate system, vehicle coordinate system, target position, speed, acceleration information in the world coordinate system, and the coordinates, reflection intensity, distance, azimuth and altitude of the radar point cloud in the sensor coordinate system.

Images