Method for storing and reading radar reflectivity of automatic driving simulation 3D model

Abstract

The invention relates to the technical field of automatic driving, and discloses an automatic driving simulation 3D model radar reflectivity storage and reading method, and the method comprises the steps: collecting and calculating the real reflectivity of an object in an automatic driving related scene; embedding the real reflectivity of an automatic driving related scene object into a texture map on the surface of the 3D model; and calculating and reading radar reflection intensity feedback data. According to the method, the radar reflectivity field is embedded in the 3D model texture, the radar reflection characteristics of different objects are accurately simulated, the influence of environmental factors on radar reflection is considered, and efficient storage and reading of data are realized, so that the support capability of an automatic driving simulation system on an end-to-end test is improved, and the authenticity and effectiveness of simulation are enhanced.

Description

Technical Field

[0001] The present application relates to the field of autonomous driving technology, and in particular to a method for storing and reading radar reflectivity of an autonomous driving simulation 3D model. Background Art

[0002] With the rapid development of autonomous driving technology, simulation testing has become an important means of developing and verifying autonomous driving systems. Simulation testing can provide a flexible and safe environment to verify the performance of autonomous driving systems in complex scenarios. However, current simulation technology still faces many challenges, especially in the simulation of real perception data, especially radar (including lidar and millimeter wave radar) data simulation.

[0003] Currently, simulation systems typically rely on three main testing methodologies:

[0004] 1. Drive testing and structured testing in real scenarios:

[0005] This method verifies the performance of the autonomous driving system through actual road testing, which can provide the most realistic environmental data. However, it is costly and difficult to test in certain dangerous or difficult-to-reproduce scenarios. In addition, real-scene testing consumes a huge amount of time, and environmental changes (such as weather, road conditions, and traffic conditions) are difficult to control, which poses certain safety risks.

[0006] 2. Test based on real data playback:

[0007] This method simulates the scene by replaying the collected road data. It ensures the authenticity of the test environment, but lacks interactivity and flexibility. For example, the vehicle may face environmental changes that are inconsistent with the preset during the data playback process, and the test cannot reflect the environmental response of the autonomous driving vehicle after autonomous decision-making. This method also cannot simulate complex physical effects (such as reflection, occlusion, etc.).

[0008] 3. Test based on simulator simulation:

[0009] Simulators can flexibly construct various traffic scenarios, reduce testing costs, and simulate extreme or rare scenarios. However, existing simulators usually rely on simplified physical models, such as vehicle dynamics simulation and game engine technology, which makes it difficult to accurately simulate the reflection characteristics of radar sensors, resulting in a large gap between simulation results and the real world. Insufficient simulation of radar data, especially in terms of reflectivity, propagation characteristics, and multipath effects, often affects the reliability of simulation results and cannot meet the end-to-end verification requirements of autonomous driving systems.

[0010] Therefore, improving the simulation accuracy of radar sensor data, especially the simulation of radar reflection characteristics, has become a key technical problem in the current autonomous driving simulation research. Radar sensors (lidar and millimeter wave radar) are an important part of the autonomous driving perception system, and accurate radar data is crucial for target detection, environment modeling, and path planning. How to accurately and realistically simulate radar data in a simulation environment, especially the interaction between simulated radar reflectivity and actual objects, has become the key to improving simulation accuracy and test efficiency. Summary of the invention

[0011] In order to solve the above problems, this application provides a method for storing and reading radar reflectivity of a 3D model for autonomous driving simulation, so as to solve the problem of insufficient radar reflectivity simulation in current autonomous driving simulation tests. Specifically, this application can solve the following core problems:

[0012] 1. How to accurately simulate the radar reflection characteristics of different objects in the simulator.

[0013] The reflection characteristics of objects in current simulation systems usually use a unified reflectivity parameter, ignoring the effects of different materials, surface roughness and geometric shapes on radar reflection characteristics.

[0014] This application accurately embeds different radar reflectivity fields in the 3D model of each object and dynamically adjusts the reflectivity according to the material type, surface structure and geometric shape of the object, so that each object in the simulation environment can exhibit radar reflection characteristics consistent with the real scene.

[0015] 2. How to comprehensively consider the impact of environmental factors on radar reflection.

[0016] The intensity of radar reflection is not only related to the reflectivity of the object itself, but also affected by factors such as distance, environmental attenuation, weather conditions (such as rain, fog, snow, etc.) and multipath effects. Existing simulation methods are difficult to fully simulate these effects.

[0017] The present application proposes a comprehensive calculation model that combines distance attenuation, object reflectivity, environmental attenuation and multipath effects, through which radar feedback data that conforms to the actual situation is dynamically generated during the simulation process.

[0018] 3. How to achieve efficient storage, reading and portability of radar reflectivity data.

[0019] Traditional simulation methods usually store radar reflectivity and 3D models separately, making data synchronization and management difficult.

[0020] This application innovatively embeds radar reflectivity information directly into the texture map of the 3D model (such as a JPEG file), and stores reflectivity data by embedding the "reflection" field in the JPEG metadata area. This method not only improves storage efficiency, but also enables rapid reading and parsing of reflectivity information during the simulation process, ensuring efficient use and real-time performance of data.

[0021] 4. How to improve the simulation system’s support capabilities for end-to-end autonomous driving testing.

[0022] When conducting end-to-end tests on complex autonomous driving systems, current simulation systems often lead to inconsistent test results due to the lack of accurate simulation of radar reflection characteristics.

[0023] This application enables the simulation system to better support end-to-end testing of autonomous driving systems, especially comprehensive verification of perception modules (such as object detection, path planning, and environmental modeling), by accurately simulating changes in radar reflectivity and combining dynamic changes in scenes with real-time feedback.

[0024] By solving the above core problems, this application can significantly improve the authenticity of radar data in the autonomous driving simulation system, enhance the simulation environment's ability to simulate complex scenarios and dynamic changes, and provide more efficient and accurate support for the research and development and verification of autonomous driving technology.

[0025] In order to achieve the above purpose, the technical solution adopted in this application is as follows:

[0026] In a first aspect, the present application provides a method for storing and reading radar reflectivity of a 3D model for autonomous driving simulation, the method comprising:

[0027] Collect and calculate the true reflectivity of objects in scenes related to autonomous driving;

[0028] Embed the real reflectivity of objects in autonomous driving scenes into the texture map of the 3D model surface;

[0029] Calculate and read radar reflection intensity feedback data.

[0030] Furthermore, the real reflectivity of objects in scenes related to autonomous driving is collected and calculated, including:

[0031] According to the objects in the autonomous driving scene, the objects are classified and the physical properties of the object surfaces are analyzed; wherein the objects in the autonomous driving scene include vehicles, windshields, buildings, road surfaces, and lamp posts; the physical properties of the object surfaces include material properties, surface roughness, and geometric features; the material properties include metal, plastic, glass, concrete, and asphalt; and the geometric properties include the shape of the object; based on the object classification and physical property analysis, a basic reflectivity parameter range is set for each type of object to provide a reference for subsequent acquisition and modeling;

[0032] The data are calibrated using a correction method that combines theoretical methods with actual collected data;

[0033] Get real scene data and perform denoising and normalization processing; the normalization formula is:

[0034]

[0035] In the formula, R 归一化 is the normalized reflectance value; R max and R min are the minimum and maximum reflectivity values ​​of different objects respectively; R is the reflectivity value of the object;

[0036] Construct the reflection intensity calculation formula, expressed as:

[0037] ρ=f(material, roughness, angle, distance, wavelength)

[0038] Where ρ is the reflection intensity; in function f, the material determines the ratio of the object's absorption and reflection of electromagnetic waves; the roughness affects the distribution characteristics of the reflection direction; the angle and distance change the intensity distribution of the received signal; the wavelength determines the penetration and scattering characteristics of the wave;

[0039] The reflection intensity characteristics of different object categories are classified, distribution-fitted and stored in a standardized manner.

[0040] Furthermore, a correction method combining theoretical methods and actual collected data is used to calibrate the data, including:

[0041] If it is a laser radar, the reflectivity and laser echo energy are positively correlated, and the formula for modeling the laser radar is:

[0042]

[0043] Where P r is the laser receiving power; P s is the laser emission power; D r is the size of the radar receiving aperture; η atm is the atmospheric effect constant; η sysis the optical transmission coefficient; R is the distance; β is the laser beam width; σ is the effective backscattering cross-sectional area.

[0044] Furthermore, the reflection intensity characteristics of different object categories are classified, distributed and stored in a standardized manner, including:

[0045] Based on the characteristic distribution curve, for each type of object, the collected reflectivity data is classified according to material, surface roughness and geometric shape;

[0046] Use the distribution fitting algorithm to generate the reflectivity characteristic curve of the corresponding category, which shows the law of how the reflectivity of this type of object changes with the incident angle and wavelength;

[0047] Establish a standardized radar reflectivity database and store the reflectivity characteristics of each type of object as a parameterized model, including the numerical or functional form of reflectivity changes with variables such as distance, angle, wavelength, etc.

[0048] Stores the reflectivity properties of each material in JSON format.

[0049] Furthermore, the real reflectivity of the scene objects related to autonomous driving is embedded into the texture map of the 3D model surface, including:

[0050] The 3D model of the target object in the simulation scene is established with the modeling requirements that the surface material of the target object matches the actual scene and the proportion, texture, and details of the target object match the real scene;.

[0051] Mapping the surface of the 3D model to a two-dimensional image space, generating a texture map, and exporting the texture map in a JPEG format, wherein the exported texture map in the JPEG format includes an APP1 segment for storing reflectivity data, and the APP1 segment is used for EXIF ​​metadata;

[0052] Use EXIF ​​metadata to store reflectivity, and use the EXIF ​​UserComment field to embed the reflectivity of each object;

[0053] Represent reflectance values ​​as colors on a texture map, map the reflectance values ​​to a heat map, and overlay the reflectance heat map onto the original texture map.

[0054] Furthermore, the radar reflection intensity feedback data is calculated and read, including:

[0055] In the simulation system, the direction and angle range of the current radar scanning beam are calculated according to the installation position and design parameters of the radar. The direction and angle range of the current radar scanning beam determine the object area that each radar beam may touch in the simulation scene. The simulator solves the transmission path of the radar point at the current angle one by one, and forms complete radar point cloud data through step-by-step scanning.

[0056] Based on the radar point cloud data, a ray tracing algorithm is used to determine whether the propagation path of each radar point collides with the surface of an object. When a collision occurs, the simulator records the 3D object, collision position and corresponding surface normal information hit by the radar point; the target object in the complex scene is located by combining the scene tree or acceleration structure of the simulation environment;

[0057] The surface of the 3D object based on the radar point collision contains a pre-embedded radar reflectivity field. The simulation system obtains the radar reflectivity parameters of the current object by reading the set field in the texture map. If the reflectivity field is empty or undefined, the default parameter library is called to assign a reasonable reflectivity value to the unknown material. The direction and intensity of the radar echo are calculated based on the read reflectivity value combined with the surface normal and the incident angle.

[0058] A physical simulation model is established according to the radar reflectivity, distance attenuation and environmental conditions of the object surface, and the radar reflection intensity is comprehensively calculated to generate radar reflection intensity feedback data; wherein the radar reflection intensity feedback data includes the position coordinates and reflection intensity values ​​of each radar point;

[0059] Based on the position coordinates and reflection intensity value of each radar point, point cloud data is output.

[0060] Furthermore, after outputting the point cloud data based on the position coordinates and reflection intensity values ​​of each radar point, the method further includes:

[0061] If the current radar scanning beam has not completed the preset angle range, the solution and reflectivity calculation of the next radar beam will continue until all scans are completed. After the scan is completed, the complete point cloud data will be stored and output as the simulation result.

[0062] In a second aspect, the present application provides a storage and reading device for radar reflectivity of an autonomous driving simulation 3D model, the device comprising:

[0063] A reflectivity calculation module, configured to collect and calculate the real reflectivity of objects in scenes related to autonomous driving;

[0064] The reflectivity embedding module embeds the real reflectivity of objects in autonomous driving scenes into the texture map of the 3D model surface;

[0065] The feedback data calculation module is configured to calculate and read radar reflection intensity feedback data.

[0066] In a third aspect, the present application provides a storage and reading system for radar reflectivity of a 3D model for autonomous driving simulation, the system comprising: a memory for storing a computer program; and a processor for executing the computer program to implement the method as described above.

[0067] In a fourth aspect, the present application provides a non-transitory computer-readable storage medium storing instructions, which, when executed by a processor, executes the method described above.

[0068] This application has at least the following beneficial effects:

[0069] This application embeds the storage and reading of radar reflectivity in the simulation environment, realizes high-precision simulation of real radar signals, and provides strong support for the testing of autonomous driving systems. Its beneficial effects are reflected in the following two points:

[0070] 1. This application can improve the accuracy of radar reflection characteristic simulation.

[0071] Refined reflectivity modeling: This application can dynamically reflect the impact of the object's material, surface roughness, and geometry on radar reflection by embedding the radar reflectivity field into the texture map of the 3D model.

[0072] Multi-material support: Different materials (such as metal, glass, plastic, concrete, etc.) have different reflective properties in the simulation environment. This application can define exclusive reflectivity parameters for each material to ensure that the simulation data is highly consistent with the actual scene.

[0073] Enhanced realism: By accurately depicting the reflective characteristics of each object, it breaks through the limitations of traditional simulation methods that use uniform reflectivity parameters, provides simulation results that are closer to the real world, and improves the overall authenticity and credibility of the simulation environment.

[0074] 2. Provide efficient storage and reading solutions.

[0075] Embedded storage design: This application innovatively embeds radar reflectivity data directly into the metadata area of ​​the 3D model's texture map (such as a JPEG file) by adding a "reflection" field to the JPEG file to store the reflectivity information.

[0076] Simplified data synchronization and management: The reflectivity information is tightly coupled with the 3D model data, avoiding the difficulties in data synchronization and management caused by traditional separate storage, and improving the convenience and consistency of data management.

[0077] Fast reading and real-time analysis: The simulator automatically analyzes the reflectivity field in JPEG when loading the 3D model through the plug-in module, realizing efficient reading and real-time application of reflectivity data, significantly improving the operating efficiency and response speed of the simulation system.

[0078] Scalability and compatibility: This application is compatible with existing 3D modeling and image storage formats, and can be seamlessly integrated into mainstream simulation engines (such as Unity, Unreal Engine). It also supports the expanded storage of other physical properties (such as absorptivity, refractive index) in the future, providing good scalability and adaptability. BRIEF DESCRIPTION OF THE DRAWINGS

[0079] Figure 1 A flowchart of a method for storing and reading radar reflectivity of a 3D model for autonomous driving simulation according to an embodiment of the present application is shown.

[0080] Figure 2 A comparison diagram of a simulated traffic scene and a real traffic scene according to an embodiment of the present application is shown.

[0081] Figure 3 A schematic diagram of a testing device according to an embodiment of the present application is shown.

[0082] Figure 4 A laser radar point cloud image according to an embodiment of the present application is shown.

[0083] Figure 5 A flow chart of radar reflection intensity acquisition in a simulation environment according to an embodiment of the present application is shown.

[0084] Figure 6 A structural diagram of a storage and reading device for radar reflectivity of a 3D model for autonomous driving simulation according to an embodiment of the present application is shown. DETAILED DESCRIPTION

[0085] The following describes the embodiments of the present application through specific examples, and those skilled in the art can easily understand other advantages and effects of the present application from the contents disclosed in this specification. The present application can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present application. It should be noted that the following embodiments and features in the embodiments can be combined with each other without conflict.

[0086] The specific implementation of the present application is further described in detail below in conjunction with the drawings and examples.

[0087] An embodiment of the present application provides a method for storing and reading radar reflectivity of a 3D model for autonomous driving simulation. By embedding a radar reflectivity field in a 3D model texture, the radar reflection characteristics of different objects are accurately simulated, the impact of environmental factors on radar reflection is considered, and efficient storage and reading of data are achieved, so as to improve the autonomous driving simulation system's support capabilities for end-to-end testing and enhance the authenticity and effectiveness of the simulation.

[0088] Specifically, see Figure 1 , which is a flow chart of a method for storing and reading the radar reflectivity of a 3D model for autonomous driving simulation. The method for storing and reading the radar reflectivity of a 3D model for autonomous driving simulation includes the following steps S1 to S3, which are described in detail as follows.

[0089] S1: Collection and calculation of the true reflectivity of objects in scenes related to autonomous driving.

[0090] In order to accurately simulate radar reflectivity in real scenes, the first step of this method is to collect and calculate real reflectivity data for various typical objects. The following are the specific implementation details and technical descriptions of step S1:

[0091] S11: Object classification and reflectivity characteristics analysis.

[0092] According to common objects in autonomous driving scenarios (such as vehicles, windshields, buildings, road surfaces, lamp posts, etc.), these objects are classified and their surface physical properties are analyzed: Material properties: Including metal, plastic, glass, concrete, asphalt, etc., different materials have significant differences in the reflection characteristics of electromagnetic waves. Surface roughness: The smoothness or roughness of the surface directly affects the reflection direction and intensity of electromagnetic waves. Geometric properties: The shape of the object (such as plane, curve, edges and corners) will lead to different multipath effects and scattering characteristics.

[0093] By analyzing these properties, a basic reflectivity parameter range can be defined for each type of object, providing a reference for subsequent acquisition and modeling.

[0094] S12: Data calibration.

[0095] Early studies generally used intensity information directly without any processing, which had a great impact on the reflection intensity received by the later radar. Therefore, the embodiment of the present application performs multi-step calibration on the laser radar used to ensure the accuracy and consistency of the reflectivity data. In general, intensity correction is mainly divided into two categories: theoretical correction and empirical correction. Theoretical correction focuses on understanding the factors that affect intensity changes under ideal conditions, such as atmospheric attenuation, scanner characteristics, target surface characteristics, and the geometric relationship of data acquisition. In contrast, empirical correction relies on establishing the actual functional relationship between intensity influencing factors in field data, such as distance, incident angle, surface roughness, and color, and intensity. In terms of accuracy, the theoretical correction method is higher than the empirical correction method. The embodiment of the present application adopts a correction method that combines theoretical methods with actual collected data to further improve the accuracy of radar reflection intensity.

[0096] Taking laser radar as an example, the reflectivity is positively correlated with the laser echo energy, and the formula for modeling it is:

[0097]

[0098] Where: P r is the laser receiving power; P s is the laser emission power; D r , is the size of the radar receiving aperture; η atm is the atmospheric effect constant; η sys is the optical transmission coefficient; R is the distance; β is the laser beam width; σ is the effective backscattering cross-sectional area.

[0099] In the actual intensity calibration process, due to the deviation between the theoretical value and the actual situation, in order to improve the accuracy of the calibration, several common parameter estimation techniques can be used: first, set the combination of certain parameters to remain constant; second, the specification parameters provided by the equipment manufacturer can be used; finally, the specific values ​​of the parameters can be determined by analyzing statistical data. These methods help reduce the errors in the model and thus improve the accuracy of the calibration.

[0100] S13: Real scene data collection and preprocessing.

[0101] like Figure 2 As shown in FIG. 1 , a comparison chart of a simulated traffic scene and a real traffic scene is shown. The “real scene data” described in this embodiment is collected in a real traffic scene using a test device. Figure 3 As shown in FIG. 1 , a schematic diagram of a test device is shown. In this embodiment, a car is used as the test device. The car carries a sensor device to collect real scene data, and the car chip carried by the car can be used to implement preprocessing of the collected data. The specific data collection and preprocessing methods will be described in detail later.

[0102] Using sensor equipment such as millimeter-wave radar and lidar, reflectivity data of typical objects are collected in experimental environments and actual roads:

[0103] Equipment selection:

[0104] The millimeter wave radar uses ARS 408-21 produced by Continental of Germany. The ARS 408-21 millimeter wave radar has the following technical parameters: working frequency band 77GHz, detection distance 0.20m to 250m, azimuth angle field of view ±60° to ±9.0°, elevation angle field of view 14° to 20°, distance resolution 1.79m to 0.39m, speed range -400km / h to +200km / h, speed accuracy ±0.1km / h, and supports CAN bus interface. It is mainly used to measure the large-area reflection intensity of objects and is sensitive to material and distance.

[0105] The laser radar used is RS-Ruby-80 produced by RoboSense. RS-Ruby-80 is an 80-line laser radar with a vertical angular resolution of 0.1° and a ranging capability of 180 meters @ 10% NIST. It outputs about 1.44 million points per second (single echo) and 2.88 million points per second (dual echo) in balanced mode, and about 2.88 million points per second (single echo) and 5.76 million points per second (dual echo) in high-performance mode. The operating voltage is 9-32V, the operating temperature range is -40℃ to +60℃, the weight is about 1.85kg, and the protection level is IP67 and IP6K9K. The 80-line laser radar has a higher angular resolution and is suitable for accurately measuring small objects and edge reflection characteristics.

[0106] Collection process:

[0107] The radar sensor is mounted on a fixed platform, and the surface of the object is scanned at different angles and distances, and the intensity of the radar return signal is recorded. Environmental conditions such as light, humidity, rain and fog are controlled in the experiment to ensure the comprehensiveness and diversity of data collection. The schematic diagram of the collected lidar point cloud is shown in the figure below. Figure 4 shown.

[0108] The reflectivity values ​​of different materials and angles obtained by measurement are shown in Table 1:

[0109] Table 1 Schematic table of reflectivity values ​​under different materials and angles

[0110]

[0111] Denoising: Apply filtering algorithms (median filtering) to remove random noise and improve data smoothness. For abnormal points (such as sudden high or low reflection values), use anomaly detection technology to eliminate or correct them.

[0112] Normalization and standardization: Based on the calibration results, the data collected by different devices are normalized and standardized to a unified scale range (such as between 0 and 1) for subsequent processing and storage.

[0113] Normalization formula:

[0114]

[0115] In the formula, R 归一化 is the normalized reflectance value; R max and R min are the minimum and maximum reflectivity values ​​of different objects respectively; R is the reflectivity value of the object.

[0116] Group objects by material and scene, and record the normalized values ​​as shown in the table:

[0117] Table 2 Schematic diagram of normalized reflectivity of different materials

[0118]

[0119] S14: Construction of reflection intensity calculation formula

[0120] The reflection intensity depends not only on the properties of the object, but also on the measurement conditions (such as distance, incident angle, wavelength, etc.). Therefore, it is necessary to build a mathematical model to summarize and calculate the collected data:

[0121] Basic reflection intensity formula:

[0122] ρ=f(material, roughness, angle, distance, wavelength)

[0123] Among them: ρ is the reflection intensity; in the function f, the material determines the object's absorption and reflection ratio of electromagnetic waves; the roughness affects the distribution characteristics of the reflection direction; the angle and distance will change the intensity distribution of the received signal; the wavelength (millimeter wave and laser wavelength) determines the penetration and scattering characteristics of the wave.

[0124] S15: Distribution generation and classification storage of reflectivity data.

[0125] After completing the collection and calibration of the basic reflection intensity parameters, the reflection intensity characteristics of different object categories need to be classified, distributed and stored in a standardized manner so as to be called in subsequent simulation modeling. The specific implementation steps of step S15 are as follows:

[0126] S151: Generate a characteristic distribution curve, and for each type of object (such as metal vehicles, glass windows, asphalt roads, etc.), classify the collected reflectivity data according to material, surface roughness and geometric shape.

[0127] S152: Use a distribution fitting algorithm (Gaussian distribution) to generate a reflectivity characteristic curve for this category, which indicates how its reflectivity changes with incident angle and wavelength.

[0128] Example: A vehicle surface may have peak points of high reflectivity, whereas a concrete pavement may exhibit a more uniform distribution of low reflectivity.

[0129] S153: Classification storage: Establish a standardized radar reflectivity database and store the reflectivity characteristics of each type of object as a parameterized model, including the numerical or functional form of the reflectivity changing with variables such as distance, angle, and wavelength.

[0130] S154: The data format needs to be compatible with the simulator requirements and store the reflectivity characteristics of each material in JSON format for subsequent parsing.

[0131] S2: Generation of 3D model surface images and reflectivity embedding.

[0132] The measured reflectivity data is embedded into the texture map (JPEG file) of the 3D model surface, providing a convenient format for the simulation engine to load and read data.

[0133] In some embodiments, the specific implementation steps of step S2 are as follows:

[0134] S21: Generate a surface texture image of the 3D model.

[0135] The specific implementation steps of step S21 are:

[0136] S211: Create a 3D model of the target object.

[0137] Use professional modeling software (such as Blender, 3ds Max, Maya) to create target objects in the simulation scene.

[0138] Modeling requirements: The surface material of the object should match the actual scene (such as smooth metal, rough concrete). Ensure that the proportion, texture, and details of the object are as realistic as possible.

[0139] S212: Generate surface texture map.

[0140] Use UV unwrapping technology to map the 3D model surface to a 2D image space to generate a texture map. Export the texture map in JPEG format.

[0141] S22: Add reflectance field to JPEG metadata.

[0142] It should be noted that JPEG files are composed of multiple segments, and common segments include: SOI (Start of Image): file start mark. APPn (Application-specific segments): used to store user-defined data. DQT / DHT / DNL: image data such as quantization table and Huffman table. The APP1 segment of JPEG (usually used for EXIF ​​metadata) supports extended fields and is very suitable for storing reflectivity data.

[0143] Therefore, in some embodiments, the reflectivity field is embedded by the following steps:

[0144] S221: Use EXIF ​​metadata to store reflectivity.

[0145] S222: Use the UserComment field of EXIF ​​to embed the reflectivity of each object.

[0146] For example:

[0147] Millimeter wave radar reflectivity: "mmwave_reflectivity":0.85;

[0148] Lidar reflectivity: "lidar_reflectivity": 0.9.

[0149] Example Chinese pseudo code:

[0150] from PIL import Image

[0151] from PIL.ExifTags import TAGS,GPSTAGS

[0152] #Open JPEG image

[0153] img=Image.open("texture.jpg")

[0154] #Add reflectivity metadata

[0155] exif_data = img.getexif()

[0156] exif_data

[37510] ="mmwave_reflectivity:0.85;lidar_reflectivity:0.9"#

[0157] UserComment Field

[0158] #Save the image

[0159] img.save("texture_with_reflection.jpg",exif=exif_data)

[0160] S23: Reflectivity heat map generation and manual discrimination.

[0161] Represent reflectivity values ​​with colors on the texture map: Specific settings are: Low reflectivity: dark blue. High reflectivity: red. Use Python or OpenCV to map reflectivity values ​​to heat maps. Overlay the reflectivity heat map on the original texture map and manually confirm whether its distribution is reasonable.

[0162] S3: Calculation and reading of radar reflection intensity feedback data.

[0163] This step S3 will describe in detail how the simulation system combines the radar reflectivity field and environmental parameters, calculates radar reflection intensity feedback data and reads related information to generate highly realistic simulated radar data.

[0164] The flow chart of radar reflection intensity acquisition in simulation environment is as follows Figure 5 , step S3 can be specifically implemented by the following steps:

[0165] S31: Calculate the angle of the radar scanning beam:

[0166] In the simulation system, the direction and angle range of the current radar scanning beam are calculated according to the installation position and design parameters of the radar. The scanning angle determines the area of ​​objects that each radar beam may touch in the simulation scene. The simulator needs to solve the transmission path of the radar point at the current angle one by one. Through step-by-step scanning, a complete radar point cloud data is formed, laying the foundation for subsequent reflectivity calculation.

[0167] S32: Determine the 3D object surface corresponding to the radar point:

[0168] During the propagation process, radar points will interact with the 3D model in the scene. Ray tracing algorithm is used to determine whether the propagation path of each radar point collides with the surface of an object. When a collision occurs, the simulator records the 3D object hit by the radar point, the collision position and the corresponding surface normal information.

[0169] In addition, the scene tree or acceleration structure (such as BVH tree) of the simulation environment can be combined to quickly locate target objects in complex scenes, ensuring the real-time and high efficiency of radar calculations.

[0170] S33: Read the radar reflectivity field of the 3D object surface map:

[0171] The surface of the 3D object that the radar point hits contains a pre-embedded radar reflectivity field (stored in a texture map). The simulation system obtains the radar reflectivity parameters of the current object by reading the "reflection" field in the texture map. If the reflectivity field is empty or undefined, the system calls the default parameter library to assign a reasonable reflectivity value to the unknown material to ensure the integrity of the calculation. Based on the read reflectivity value, the direction and intensity of the radar echo are calculated in combination with the surface normal and the angle of incidence.

[0172] S34: Comprehensive calculation of radar reflection intensity feedback data:

[0173] Factors affecting reflection intensity: Radar reflection intensity depends not only on the radar reflectivity of the surface of the object, but also on the following factors: Distance attenuation: As the propagation distance of the radar point increases, the echo intensity decays exponentially. Environmental conditions: Weather factors such as rain, fog, and snow will further reduce the echo intensity.

[0174] Calculation method:

[0175] The system establishes a physical simulation model based on the above factors, comprehensively calculates the radar reflection intensity, and generates feedback data that is highly consistent with the real scene.

[0176] S35: Output radar point position and reflection intensity:

[0177] After the calculation of each radar point is completed, the simulator writes its position coordinates (XYZ) and reflection intensity value (Intensity) into the output point cloud data format (such as .pcd, .las). The generated point cloud data can be directly used for the perception module test of the autonomous driving system to verify the reliability of core algorithms such as target detection and path planning.

[0178] S36: Determine whether the radar scan is completed:

[0179] If the current radar scanning beam has not completed the preset angle range, the solution and reflectivity calculation of the next radar beam will continue until all scans are completed. After the scan is completed, the complete point cloud data is stored and output as the simulation result.

[0180] The present application also provides a storage and reading device for the radar reflectivity of a 3D model for autonomous driving simulation, such as Figure 6 As shown, the device comprises:

[0181] A reflectivity calculation module 601 is configured to collect and calculate the real reflectivity of objects in scenes related to autonomous driving;

[0182] A reflectivity embedding module 602 embeds the real reflectivity of the scene objects related to autonomous driving into the texture map of the surface of the 3D model;

[0183] The feedback data calculation module 603 is configured to calculate and read radar reflection intensity feedback data.

[0184] In some embodiments, the reflectivity calculation module is further configured to:

[0185] According to the objects in the autonomous driving scene, the objects are classified and the physical properties of the object surfaces are analyzed; wherein the objects in the autonomous driving scene include vehicles, windshields, buildings, road surfaces, and lamp posts; the physical properties of the object surfaces include material properties, surface roughness, and geometric features; the material properties include metal, plastic, glass, concrete, and asphalt; and the geometric properties include the shape of the object; based on the object classification and physical property analysis, a basic reflectivity parameter range is set for each type of object to provide a reference for subsequent acquisition and modeling;

[0186] The data are calibrated using a correction method that combines theoretical methods with actual collected data;

[0187] Get real scene data and perform denoising and normalization processing; the normalization formula is:

[0188]

[0189] In the formula, R 归一化 is the normalized reflectance value; R max and R min are the minimum and maximum reflectivity values ​​of different objects respectively; R is the reflectivity value of the object;

[0190] Construct the reflection intensity calculation formula, expressed as:

[0191] ρ=f(material, roughness, angle, distance, wavelength)

[0192] Where ρ is the reflection intensity; in function f, the material determines the ratio of the object's absorption and reflection of electromagnetic waves; the roughness affects the distribution characteristics of the reflection direction; the angle and distance change the intensity distribution of the received signal; the wavelength determines the penetration and scattering characteristics of the wave;

[0193] The reflection intensity characteristics of different object categories are classified, distribution-fitted and stored in a standardized manner.

[0194] In some embodiments, the reflectivity calculation module is further configured to:

[0195] If it is a laser radar, the reflectivity and laser echo energy are positively correlated, and the formula for modeling the laser radar is:

[0196]

[0197] Where P r is the laser receiving power; P s is the laser emission power; D r is the size of the radar receiving aperture; η atm is the atmospheric effect constant; η sys is the optical transmission coefficient; R is the distance; β is the laser beam width; σ is the effective backscattering cross-sectional area.

[0198] In some embodiments, the reflectivity calculation module is further configured to:

[0199] Based on the characteristic distribution curve, for each type of object, the collected reflectivity data is classified according to material, surface roughness and geometric shape;

[0200] Use the distribution fitting algorithm to generate the reflectivity characteristic curve of the corresponding category, which shows the law of how the reflectivity of this type of object changes with the incident angle and wavelength;

[0201] Establish a standardized radar reflectivity database and store the reflectivity characteristics of each type of object as a parameterized model, including the numerical or functional form of reflectivity changes with variables such as distance, angle, wavelength, etc.

[0202] Stores the reflectivity properties of each material in JSON format.

[0203] In some embodiments, the reflectivity embedding module is further configured to:

[0204] The 3D model of the target object in the simulation scene is established with the modeling requirements that the surface material of the target object matches the actual scene and the proportion, texture, and details of the target object match the real scene;.

[0205] Mapping the surface of the 3D model to a two-dimensional image space, generating a texture map, and exporting the texture map in a JPEG format, wherein the exported texture map in the JPEG format includes an APP1 segment for storing reflectivity data, and the APP1 segment is used for EXIF ​​metadata;

[0206] Use EXIF ​​metadata to store reflectivity, and use the EXIF ​​UserComment field to embed the reflectivity of each object;

[0207] Represent reflectance values ​​as colors on a texture map, map the reflectance values ​​to a heat map, and overlay the reflectance heat map onto the original texture map.

[0208] In some embodiments, the feedback data calculation module is further configured to:

[0209] In the simulation system, the direction and angle range of the current radar scanning beam are calculated according to the installation position and design parameters of the radar. The direction and angle range of the current radar scanning beam determine the object area that each radar beam may touch in the simulation scene. The simulator solves the transmission path of the radar point at the current angle one by one, and forms complete radar point cloud data through step-by-step scanning.

[0210] Based on the radar point cloud data, a ray tracing algorithm is used to determine whether the propagation path of each radar point collides with the surface of an object. When a collision occurs, the simulator records the 3D object, collision position and corresponding surface normal information hit by the radar point; the target object in the complex scene is located by combining the scene tree or acceleration structure of the simulation environment;

[0211] The surface of the 3D object based on the radar point collision contains a pre-embedded radar reflectivity field. The simulation system obtains the radar reflectivity parameters of the current object by reading the set field in the texture map. If the reflectivity field is empty or undefined, the default parameter library is called to assign a reasonable reflectivity value to the unknown material. The direction and intensity of the radar echo are calculated based on the read reflectivity value combined with the surface normal and the incident angle.

[0212] A physical simulation model is established according to the radar reflectivity, distance attenuation and environmental conditions of the object surface, and the radar reflection intensity is comprehensively calculated to generate radar reflection intensity feedback data; wherein the radar reflection intensity feedback data includes the position coordinates and reflection intensity values ​​of each radar point;

[0213] Based on the position coordinates and reflection intensity value of each radar point, point cloud data is output.

[0214] In some embodiments, the feedback data calculation module is further configured to:

[0215] If the current radar scanning beam has not completed the preset angle range, the solution and reflectivity calculation of the next radar beam will continue until all scans are completed. After the scan is completed, the complete point cloud data will be stored and output as the simulation result.

[0216] It should be noted that the device described in this embodiment and the method described previously belong to the same technical concept, have the same technical principles, and can achieve the same beneficial effects, so they will not be repeated here.

[0217] An embodiment of the present application also provides a storage and reading system for radar reflectivity of an autonomous driving simulation 3D model, the system comprising: a memory for storing a computer program; and a processor for executing the computer program to implement a method as described in any of the above embodiments.

[0218] An embodiment of the present application further provides a non-temporary computer-readable storage medium storing instructions, and when the instructions are executed by a processor, the method described in any of the above embodiments is executed.

[0219] The above implementation modes are only used to illustrate the present application, and are not intended to limit the present application. Ordinary technicians in the relevant technical field may make various changes and modifications without departing from the spirit and scope of the present application. Therefore, all equivalent technical solutions also belong to the scope of the present application, and the scope of patent protection of the present application shall be limited by the claims.

Claims

1. A method for storing and reading radar reflectivity of a 3D model for autonomous driving simulation, characterized in that: The method comprises: Collect and calculate the true reflectivity of objects in scenes related to autonomous driving; Embed the real reflectivity of objects in autonomous driving scenes into the texture map of the 3D model surface; Calculate and read radar reflection intensity feedback data.

2. The method for storing and reading radar reflectivity of an autonomous driving simulation 3D model according to claim 1, characterized in that: Collect and calculate the true reflectivity of objects in scenes related to autonomous driving, including: According to the objects in the autonomous driving scene, the objects are classified and the physical properties of the object surfaces are analyzed; wherein the objects in the autonomous driving scene include vehicles, windshields, buildings, road surfaces, and lamp posts; the physical properties of the object surfaces include material properties, surface roughness, and geometric features; the material properties include metal, plastic, glass, concrete, and asphalt; and the geometric properties include the shape of the object; based on the object classification and physical property analysis, a basic reflectivity parameter range is set for each type of object to provide a reference for subsequent acquisition and modeling; The data are calibrated using a correction method that combines theoretical methods with actual collected data; Get real scene data and perform denoising and normalization processing; the normalization formula is: In the formula, R 归一化 is the normalized reflectance value; R max and R min are the minimum and maximum reflectivity values ​​of different objects respectively; R is the reflectivity value of the object; Construct the reflection intensity calculation formula, expressed as: ρ=f(material, roughness, angle, distance, wavelength) Where ρ is the reflection intensity; in function f, the material determines the ratio of the object's absorption and reflection of electromagnetic waves; the roughness affects the distribution characteristics of the reflection direction; the angle and distance change the intensity distribution of the received signal; the wavelength determines the penetration and scattering characteristics of the wave; The reflection intensity characteristics of different object categories are classified, distribution-fitted and stored in a standardized manner.

3. The method for storing and reading radar reflectivity of an autonomous driving simulation 3D model according to claim 2, characterized in that: The data is calibrated using a correction method that combines theoretical methods with actual collected data, including: If it is a laser radar, the reflectivity and laser echo energy are positively correlated, and the formula for modeling the laser radar is: Where P r is the laser receiving power; P s is the laser emission power; D r is the size of the radar receiving aperture; η atm is the atmospheric effect constant; η sys is the optical transmission coefficient; R is the distance; β is the laser beam width; σ is the effective backscattering cross-sectional area.

4. The method for storing and reading radar reflectivity of an autonomous driving simulation 3D model according to claim 2, characterized in that: Classify, distribute and store reflection intensity characteristics of different object categories, including: Based on the characteristic distribution curve, for each type of object, the collected reflectivity data is classified according to material, surface roughness and geometric shape; Use the distribution fitting algorithm to generate the reflectivity characteristic curve of the corresponding category, which shows the law of how the reflectivity of this type of object changes with the incident angle and wavelength; Establish a standardized radar reflectivity database and store the reflectivity characteristics of each type of object as a parameterized model, including the numerical or functional form of reflectivity changes with variables such as distance, angle, wavelength, etc. Stores the reflectivity properties of each material in JSON format.

5. The method for storing and reading radar reflectivity of an autonomous driving simulation 3D model according to claim 4, characterized in that: Embed the real reflectivity of autonomous driving related scene objects into the texture map of the 3D model surface, including: The 3D model of the target object in the simulation scene is established with the modeling requirements that the surface material of the target object matches the actual scene and the proportion, texture, and details of the target object match the real scene;. Mapping the surface of the 3D model to a two-dimensional image space, generating a texture map, and exporting the texture map in a JPEG format, wherein the exported texture map in the JPEG format includes an APP1 segment for storing reflectivity data, and the APP1 segment is used for EXIF ​​metadata; Use EXIF ​​metadata to store reflectivity, and use the EXIF ​​UserComment field to embed the reflectivity of each object; Represent reflectance values ​​as colors on a texture map, map the reflectance values ​​to a heat map, and overlay the reflectance heat map onto the original texture map.

6. The method for storing and reading radar reflectivity of an autonomous driving simulation 3D model according to claim 1, characterized in that: Calculate and read radar reflection intensity feedback data, including: In the simulation system, the direction and angle range of the current radar scanning beam are calculated according to the installation position and design parameters of the radar. The direction and angle range of the current radar scanning beam determine the object area that each radar beam may touch in the simulation scene. The simulator solves the transmission path of the radar point at the current angle one by one, and forms complete radar point cloud data through step-by-step scanning. Based on the radar point cloud data, a ray tracing algorithm is used to determine whether the propagation path of each radar point collides with the surface of an object. When a collision occurs, the simulator records the 3D object, collision position and corresponding surface normal information hit by the radar point; the target object in the complex scene is located by combining the scene tree or acceleration structure of the simulation environment; The surface of the 3D object based on the radar point collision contains a pre-embedded radar reflectivity field. The simulation system obtains the radar reflectivity parameters of the current object by reading the set field in the texture map. If the reflectivity field is empty or undefined, the default parameter library is called to assign a reasonable reflectivity value to the unknown material. The direction and intensity of the radar echo are calculated based on the read reflectivity value combined with the surface normal and the incident angle. A physical simulation model is established according to the radar reflectivity, distance attenuation and environmental conditions of the object surface, and the radar reflection intensity is comprehensively calculated to generate radar reflection intensity feedback data; wherein the radar reflection intensity feedback data includes the position coordinates and reflection intensity values ​​of each radar point; Based on the position coordinates and reflection intensity value of each radar point, point cloud data is output.

7. The method for storing and reading radar reflectivity of an autonomous driving simulation 3D model according to claim 1, characterized in that: After outputting the point cloud data based on the position coordinates and reflection intensity values ​​of each radar point, the method further includes: If the current radar scanning beam has not completed the preset angle range, the solution and reflectivity calculation of the next radar beam will continue until all scans are completed. After the scan is completed, the complete point cloud data will be stored and output as the simulation result.

8. A storage and reading device for radar reflectivity of a 3D model for autonomous driving simulation, characterized in that: The device comprises: A reflectivity calculation module, configured to collect and calculate the real reflectivity of objects in scenes related to autonomous driving; The reflectivity embedding module embeds the real reflectivity of objects in autonomous driving scenes into the texture map of the 3D model surface; The feedback data calculation module is configured to calculate and read radar reflection intensity feedback data.

9. A storage and reading system for radar reflectivity of a 3D model for autonomous driving simulation, characterized in that: The system comprises: Memory for storing computer programs; A processor, configured to execute the computer program to implement the method according to any one of claims 1 to 7. 10 . A non-transitory computer-readable storage medium storing instructions, which, when executed by a processor, perform the method according to claim 1 .

Images