Virtual-real combination automobile automatic driving test method

Abstract

The invention belongs to the field of data analysis and processing, and discloses a virtuality and reality combined automobile automatic driving test method. Comprising the following steps: connecting a sensor suite and an automatic driving system of an automobile with a virtual twin center by using a CAN bus or an Ethernet interface, operating the automatic driving system, and generating a performance report; the method comprises the following steps: initializing raindrop attributes, setting the transparency of raindrops based on real environment data, introducing a complex wind field model and a surface ponding dynamic evolution mechanism, carrying out collision detection on all raindrop pairs by using a space segmentation algorithm, and simulating dynamic behaviors of the raindrops in the air based on a discrete element method and a smoothed particle fluid dynamics method. Visual rendering is carried out on the water spray effect; under the influence of raindrop physical characteristic modeling, output of laser radar point cloud scattering modeling, camera image fuzzy modeling, millimeter wave radar signal attenuation and scattering modeling and ultrasonic sensor interference modeling is iteratively updated, and automatic driving testing of an automobile under virtual-real combination is achieved.

Description

Technical Field

[0001] The present invention relates to the field of data analysis and processing, and more specifically, to a virtual-reality combined automobile automatic driving testing method. Background Art

[0002] Rainy days are the most common weather in life and also the most common weather scenario encountered during autonomous driving. If the test can be passed on a rainy day, more than half of the entire test content has been completed. Therefore, this application mainly focuses on the testing of autonomous driving in rainy days.

[0003] Rainy days not only change the physical environment, but also directly affect the performance of sensors. Rainy days can cause blurry cameras and sparse lidar point clouds, while the simulation environment may simply simulate the effect of rainfall, but fail to accurately simulate the interference of raindrops on the lidar point cloud or the occlusion effect on the camera image. This "double simplification" causes the system to perform well in simulation, but may completely fail in the actual rainy environment.

[0004] Therefore, the interaction between the over-idealization of the sensor model in the simulation and the simplification of the physical environment causes a significant deviation between the performance of the autonomous driving system in the simulation test and the actual performance. This deviation not only increases the difficulty of development and testing, but may also lead to safety hazards of the system in reality. For example, the system may pass all tests in the simulation, but cause an accident due to perception errors or decision-making errors in actual rainy days. Summary of the invention

[0005] In order to overcome the above-mentioned defects of the prior art and to achieve the above-mentioned purpose, the present invention provides the following technical solutions: A virtual-real combined automobile autonomous driving test method, comprising: deploying a sensor kit and an autonomous driving system on a test vehicle, connecting the sensor kit and the autonomous driving system of the vehicle to a virtual twin center using a CAN bus or an Ethernet interface, running the autonomous driving system, and generating a performance report; The virtual twin center is extended and designed based on the simulation platform, including modeling of raindrop physical characteristics and physical impact of sensors. Twin virtual test scenarios are preset according to the real environment, and the modeling of raindrop physical characteristics and physical impact of sensors are imported into the twin virtual test scenarios and pre-trained. Among them, the modeling of raindrop physical characteristics is used for the generation of raindrops, including: initializing raindrop properties, setting the transparency of raindrops based on real environment data, introducing complex wind field models and the dynamic evolution mechanism of ground water, using space segmentation algorithms to perform collision detection on all raindrop pairs, simulating the dynamic behavior of raindrops in the air based on discrete element method and smooth particle fluid dynamics method, and visually rendering the water splash effect; The physical impact modeling of the sensor includes the sensor dynamic interference unit, which iteratively updates the outputs of the lidar point cloud scattering modeling, the camera image blur modeling, the millimeter wave radar signal attenuation and scattering modeling, and the ultrasonic sensor interference modeling under the influence of the raindrop physical property modeling.

[0006] Preferably, the raindrop physical property modeling includes: Provide a parameterized interface, where users can set evolution parameters through input boxes. Evolution parameters include rainfall intensity, wind speed, wind direction, and real environment data for testing. The Marshall-Palmer distribution is sampled using the Monte Carlo method to generate the initial diameter of raindrops, with a sampling range of 0.1 mm to 10 mm; Based on the rainfall intensity set by the user, the volumetric microscopic density of raindrops is calculated using Python; If the initial diameter of the raindrop is less than 1 mm, Stokes' law is used to calculate the terminal velocity. If the initial diameter of the raindrop is ≥ 1 mm, the Beard model is used to calculate the terminal velocity. The terminal velocity is stored as a property of the raindrop. Set the transparency of raindrops based on real environment data; Assign an initial position and initial velocity to each raindrop; A complex wind field model is designed according to the actual test environment and added to the modeling of the physical characteristics of raindrops. In each time step, the gravity, air resistance and wind force of raindrops are calculated. Afterwards, the speed and position of raindrops are updated using the numerical integration method. If the raindrops reach the ground or other surfaces, the raindrops are removed and the splash effect is triggered. Otherwise, the update continues and the operations within the iterative time step are repeated until the preset number of iterations or time is reached. At each time step, a spatial segmentation algorithm is used to perform collision detection on all raindrop pairs, and the dynamic behavior of raindrops in the air is simulated based on the discrete element method and smoothed particle fluid dynamics method, including: defining a collision detection range for each raindrop, and judging a collision when the center distance between two raindrops is less than the sum of the two raindrop radii. The merger or splitting is determined based on the size and direction of the relative speed. If the relative speed of the two raindrops is less than the preset threshold, they are merged into a larger raindrop, and the new raindrop diameter satisfies the volume conservation and the speed satisfies the momentum conservation. If the relative speed of the two raindrops is greater than or equal to the preset threshold, they are randomly split into several small raindrops. The size and number of the raindrops after the split conform to the Marshall-Palmer distribution, and the total volume of the raindrops is conserved. The properties of the raindrops are updated, and the newly generated raindrops are added to the modeling of the physical properties of the raindrops.

[0007] Preferably, the method for calculating the gravity, air resistance and wind force of raindrops comprises: For gravity, a random number ξ~(0, 1) is generated at each time step, where ξ represents the chance of adsorption. , then adsorption occurs, and the adsorption mass is Δm, otherwise no adsorption occurs. is the adsorption probability, , is the probability adjustment factor, Indicates the concentration of particulate matter in the air. is the surface area of ​​the raindrop; the mass of the raindrop is iteratively updated as the sum of the mass of the previous time step and the adsorption mass, and the gravity of the raindrop is calculated based on the updated mass; For air resistance, preset the standard density of air based on historical data or literature , Standard dynamic viscosity And the standard drag coefficient , adjusted air density , is the coefficient of influence of particles on air density, and the adjusted dynamic viscosity , is the coefficient of influence of particles on viscosity, and the adjusted resistance coefficient , is the influence coefficient of particles on the drag coefficient. If the initial diameter of the raindrop is less than 1 mm, the Stokes law is used to calculate the air resistance. If the initial diameter of the raindrop is ≥ 1 mm, the Beard model is used to calculate the air resistance. For wind force, CFD software is used to generate three-dimensional wind field data, including: defining the geometric model of the simulation scene according to the actual test environment, presetting the wind speed boundary conditions, generating the wind speed vector field by solving the Navier-Stokes equation, and importing it into the modeling of the physical characteristics of raindrops, interpolating and calculating the wind speed at each raindrop position, and then adding wind force during the movement of raindrops.

[0008] Preferably, the transparency of raindrops is set based on real environment data, including the influence of particle adsorption and the influence of ambient light interaction, wherein the particle adsorption influence is set as , , [PM] and The linear representation of is the transparency adjustment coefficient, [PM] is the local PM concentration in the area where the raindrops are located, represents the number of particles in the raindrops at each time step, is the surface area of ​​the raindrop; the particle adsorption effect follows the exponential law and affects the transparency attenuation; The transparency under the influence of ambient light intersection is modulated by the light transmittance γ of the obstruction, and the light source is introduced to correct the influence of ambient light intersection; The transparency under the influence of corrected ambient light intersection and the transparency under the influence of particle adsorption are combined to obtain the transparency of raindrops in each time step.

[0009] Preferably, the visual rendering method of the water splash effect includes: Set the position of the raindrop when it reaches the ground or other surface as the impact point, the size as the initial bonding size, the mass as the original mass, the terminal velocity as the initial splash velocity, and preset the life cycle of the raindrop; The materials reaching the ground or other object surfaces are preset as smooth hard materials, non-smooth medium hard materials and liquid materials; the material correlation coefficient is preset for each material; Calculate the impact energy based on the original mass of the raindrop and the initial splash velocity , and record the normal direction of the collision point; according to the impact energy Generate water splash particle clusters , is the number of particles per cluster, represents the material correlation coefficient; The mass attenuation of water splash particles is simulated by evaporation effect according to temperature and humidity; Set the splash direction of water particles along the ground normal direction as the main velocity component, and form a tangential direction by random horizontal deflection angle. Get the splash direction by combining the main velocity component with the tangential direction. Add a scene where a car drives over the water surface, set the directional spray to be triggered in this scene, and the initial splash velocity is positively correlated with the vehicle speed. Use the same method as raindrops to iteratively calculate the gravity, air resistance and wind force of water droplets, and update the speed and position; At the time step, the speed, direction and position of the water splash particles are iteratively updated to generate the water splash effect.

[0010] Preferably, the dynamic evolution mechanism of ground water is introduced into the modeling of the physical characteristics of raindrops, and the method includes: For each material reaching the ground or other surface, a humidity saturation limit is preset. At each time step, the number of raindrops on the unit material is updated in combination with the evaporation effect. The specular reflection of the material is set to be positively correlated with the number of raindrops. When the number of raindrops on the unit material reaches the humidity saturation limit, the specular reflection of the material is set to reach the limit and no longer change. The drainage capacity of the rainwater well is preset according to the real environment used for testing, and the depression water level capacity curve method is used to simulate the depression water accumulation process according to the number of raindrops reaching the ground at each time step; Starting from the depression water depth of 0, iteratively update the water depth of the depression at each time step, and obtain the water surface area of ​​each depression, and use the mirror reflection of the corresponding material of the water surface on the water surface.

[0011] Preferably, the physical impact modeling of the sensor comprises: Preset optical, electromagnetic and acoustic properties of raindrop materials; At each time step, the intersection point between the laser beam and the raindrop is iteratively calculated, and the noise points and intensity attenuation in the point cloud are updated; Iteratively update the position and shape of water droplets on the camera, recalculate occlusion, blur and optical effects, and generate dynamic image sequences; Iteratively updates the intersection point of the millimeter-wave beam and raindrops, recalculates the attenuation and scattering effects, and generates dynamic radar signals; The intersection point between the ultrasonic beam and the raindrops is iteratively updated, the occlusion and scattering effects are recalculated, and a dynamic echo signal is generated.

[0012] Preferably, the sensor dynamic interference unit includes: Modeling of LiDAR point cloud scattering; Using ray tracing technology, the path of each laser beam is traced and its intersection with the raindrop is calculated. The scattering probability at the intersection is determined by the Mie scattering cross section, and the scattering direction is determined by the Mie scattering phase function. If the laser beam is scattered, a noise point is added to the point cloud. The position of the noise point is determined by the scattering direction and intensity. If the laser beam is not completely scattered, its intensity decays exponentially. Modeling camera image blur; Project raindrops onto the camera lens and obtain the transparency of the raindrops. Apply a Gaussian blur filter to the area blocked by raindrops in the image. The blur radius is proportional to the size of the raindrops to obtain the occlusion and blurring effect of raindrops on the lens. The relationship between the gravity of raindrops, wind force, surface friction of the lens, and the dynamic movement of raindrops on the lens is expressed through linear formulas. At each time step, the position and shape of raindrops are updated, and the occlusion and blur effects are recalculated. Modeling of millimeter wave radar signal attenuation and scattering; For each mmWave beam, ray tracing techniques are used to calculate its intersection with raindrops to determine attenuation and scattering effects; The attenuation coefficient and scattering cross section were obtained by fitting the rainfall intensity using the ITU-R model; The scattering direction of millimeter waves is determined by the Mie scattering phase function; The scattered millimeter-wave signal will appear as noise or false targets in the radar receiver, thereby obtaining an affected millimeter-wave radar signal; Modeling interference with ultrasonic sensors; For each ultrasonic beam, calculate its intersection with the raindrop. If the beam is completely blocked, the echo signal strength is 0. The blocking probability is determined by the projection area of ​​the raindrop and the beam cross-sectional area. If the initial diameter of the raindrop is less than 1mm, Rayleigh scattering is used to calculate the scattering intensity of the raindrop to the ultrasonic wave; if the initial diameter of the raindrop is ≥1mm, Born is used to calculate the scattering intensity of the raindrop to the ultrasonic wave; the scattered ultrasonic signal appears as noise or false echo in the receiver; and then the affected ultrasonic signal is obtained.

[0013] Preferably, optical effect and slippery effect are introduced into the camera image blur modeling; For raindrops on the camera, an incident ray is generated from the light source, the intersection point between the ray and the raindrop surface is calculated, the intersection point position and normal vector are determined using the geometric optics method, the direction and intensity of the refracted and reflected rays are calculated based on Snell's law and Fresnel's equations, and the ray path is recursively traced at each time step until the ray leaves the raindrop or the intensity decays below a threshold. The incident white light is decomposed into multiple monochromatic lights, and the refraction angle of each wavelength of the incident white light inside the raindrop is calculated based on the dispersion effect. The refraction and reflection path of each wavelength of light inside the raindrop is calculated using ray tracing technology, and the rainbow effect is synthesized based on the angle and intensity of the outgoing light. For the slippery-wet effect, the specular reflection of the material is modeled using the Bidirectional Reflectance Distribution Function model.

[0014] Preferably, the method of connecting the sensor suite and the autonomous driving system of the vehicle to the virtual twin center using a CAN bus or Ethernet interface, running the autonomous driving system, and generating a performance report includes: Preset quantitative indicators for performance reports and real environments for testing; Use digital twin technology to virtually construct the real environment used for testing, produce a virtual twin center, use real environment data to pre-train the modeling of raindrop physical characteristics, and stop when the similarity between the modeling of raindrop physical characteristics and the real environment data reaches a threshold. Use the sensor kit data in autonomous driving to pre-train the modeling of the physical impact of the sensor until it reaches the expected level, thus forming a trained virtual twin center. Use the CAN bus or Ethernet interface to connect the car's sensor kit and autonomous driving system to the virtual twin center, run the autonomous driving system, record data on quantitative indicators, and form a performance report.

[0015] The technical effects and advantages of the virtual-real combined vehicle automatic driving test method of the present invention are as follows: 1. Virtual and real combined testing method By combining the real test site environment with the cloud twin simulation system, the rapid generation and switching of virtual scenes can be achieved, which greatly shortens the test cycle and realizes accelerated testing, large-scale testing and enhanced testing. It reduces the dangerous situations that may occur in real vehicle testing, improves test efficiency and diversified traffic scene simulation, and reduces test costs and risks.

[0016] 2. Modeling the physical properties of raindrops Through detailed modeling of the physical characteristics of raindrops, including the generation, dynamic behavior and visual effects of raindrops, the impact of rainy weather on autonomous driving sensors is simulated more realistically. The authenticity of the test scene is improved through complex physical models and optical effect simulations. The physical characteristics of raindrops can be dynamically adjusted according to different environmental conditions (such as rainfall intensity and wind speed). The adaptability of the model is improved. A complex wind field model is introduced to simulate the movement of raindrops under different wind speeds and wind directions. Combined with the dynamic evolution mechanism of ground water accumulation, the process of water accumulation after raindrops reach the ground is simulated. The spatial segmentation algorithm is used to perform collision detection on all raindrop pairs, which improves the computational efficiency. Combined with the discrete element method and smoothed particle fluid dynamics method, the dynamic behavior of raindrops in the air is simulated. The splash effect is visually rendered to improve the visualization of the simulation results and achieve a comprehensive description of the physical characteristics of raindrops.

[0017] 3. Modeling the physical impact of sensors The accuracy of the test results is improved by accurately simulating the dynamic interference of sensors in rainy environments. Comprehensive sensor physical impact modeling is provided. The physical impact of sensors is updated in real time to improve the real-time and response speed of the test. Ray tracing technology is used to track the path of each laser beam, calculate its intersection with raindrops, and simulate the scattering effect. Optical effects and slippery effects are introduced to simulate the impact of raindrops on camera images. The ITU-R model is used to fit the rainfall intensity to simulate the attenuation and scattering of millimeter-wave radar signals. The interference of ultrasonic sensors in rainy environments is simulated to improve the accuracy of the model. The physical impact of sensors is updated in real time through dynamic interference units to improve the adaptability of the model. By simulating the dynamic interference of sensors in rainy environments, sensor performance can be evaluated more accurately. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 A schematic diagram of the steps of a virtual-real combined vehicle automatic driving test method of the present invention; Figure 2 It is a structural schematic diagram of a virtual-reality combined automobile automatic driving testing method of the present invention. DETAILED DESCRIPTION

[0019] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0020] Example 1 See also Figure 1 As shown, the virtual-real combined vehicle automatic driving test method described in this embodiment includes: The difference between simulation and reality is one of the core challenges in the virtual-reality test of autonomous driving. This difference directly affects the reliability of the test results and the performance of the autonomous driving system in the real world. The weather, light, road conditions, etc. in the real world are highly diverse and dynamically changing, while the simulation environment is usually based on limited assumptions and models, that is, the virtual simulation environment cannot fully reproduce the complexity of the real world, and the sensor models in the simulation (such as lidar, camera) may be too idealized and cannot accurately simulate the errors or interference in reality. Specifically, the autonomous driving system relies on a variety of sensors (such as lidar, camera, millimeter wave radar, ultrasonic sensor, etc.) to perceive the environment. In virtual simulation, the behavior of these sensors is usually based on theoretical models, ignoring the various errors and interferences in reality.

[0021] Rainy days are the most common weather in life and also the most common weather scenario encountered during autonomous driving. If the test can be passed on a rainy day, more than half of the entire test content has been completed. Therefore, this application mainly focuses on the testing of autonomous driving in rainy days.

[0022] In rainy days, not only the physical environment (such as road friction coefficient and light conditions) will be changed, but also the performance of the sensor will be directly affected. Rainy days will cause the camera to be blurred and the lidar point cloud to be sparse, while the simulation environment may simply simulate the effect of rainfall (such as adding raindrops in the rendering), but it does not accurately simulate the interference of raindrops on the lidar point cloud or the occlusion effect on the camera image. This "double simplification" (physical environment simplification + sensor model idealization) causes the system to perform well in simulation, but may completely fail in the actual rainy environment.

[0023] LiDAR may generate erroneous data under mirror reflection (such as slippery roads). In reality, multi-sensor fusion needs to take into account changes in the physical environment and sensor performance. For example, on rainy days, the perception range of LiDAR may be significantly reduced, cameras may fail, and millimeter-wave radar may become the main reliance. However, the simulation environment usually assumes that all sensor data is perfectly synchronized, and does not accurately simulate the specific impact of fog on each sensor data. As a result, the system can easily achieve multi-sensor fusion in simulation, but it may fail in the actual environment due to inconsistent or missing data.

[0024] Therefore, the interaction between the over-idealization of the sensor model in the simulation and the simplification of the physical environment causes a significant deviation between the performance of the autonomous driving system in the simulation test and the actual performance. This deviation not only increases the difficulty of development and testing, but may also lead to safety hazards of the system in reality. For example, the system may pass all tests in the simulation, but cause an accident due to perception errors or decision-making errors in actual rainy days.

[0025] A virtual-real combination vehicle autonomous driving test method, comprising: deploying a sensor kit (e.g., laser radar, camera, millimeter wave radar, ultrasonic sensor, IMU, GPS, etc.) and an autonomous driving system on a test vehicle, connecting the vehicle's sensor kit and autonomous driving system to a virtual twin center using a CAN bus or Ethernet interface, running the autonomous driving system, and generating a performance report; The vehicle's sensor kit and autonomous driving system are connected to the virtual twin center via the CAN bus or Ethernet interface, enabling integrated and synchronous operation of the vehicle, cloud and field.

[0026] The virtual twin center is designed based on simulation platforms (such as CARLA, AirSim, LGSVL Simulator, etc., which have integrated some sensor and environment modeling functions and can be used as a basis for expansion), including raindrop physical property modeling and sensor physical impact modeling. Twin virtual test scenarios are preset according to the real environment, and raindrop physical property modeling and sensor physical impact modeling are imported into the twin virtual test scenarios and pre-trained. Generate detailed performance reports through preset quantitative indicators and real environment data to provide data support for the optimization of the autonomous driving system.

[0027] Among them, the modeling of raindrop physical characteristics is used for the generation of raindrops, including: initializing raindrop properties, setting the transparency of raindrops based on real environment data, introducing complex wind field models and the dynamic evolution mechanism of ground water, using space segmentation algorithms to perform collision detection on all raindrop pairs, simulating the dynamic behavior of raindrops in the air based on discrete element method and smooth particle fluid dynamics method, and visually rendering the water splash effect; The physical impact modeling of the sensor includes the sensor dynamic interference unit, which iteratively updates the outputs of the lidar point cloud scattering modeling, the camera image blur modeling, the millimeter wave radar signal attenuation and scattering modeling, and the ultrasonic sensor interference modeling under the influence of the raindrop physical property modeling.

[0028] Modeling of raindrop physics includes: Provide a parameterized interface (for example, develop a GUI interface in the simulation platform, such as Unreal Engine's UMG or Unity's UI system), where users set evolution parameters through input boxes. Evolution parameters include rainfall intensity, wind speed, wind direction, and real environment data for testing; The Marshall-Palmer distribution is sampled using the Monte Carlo method to generate the initial diameter of raindrops, with a sampling range of 0.1 mm to 10 mm (covering the range from fine raindrops to large raindrops); In the present invention, it is preset; Light rain (1 mm / h): Raindrop diameters are mainly concentrated below 0.5 mm.

[0029] Moderate rain (10 mm / h): The diameter of raindrops is mainly concentrated around 1 mm.

[0030] Heavy rain (50 mm / h): The diameter of raindrops ranges widely, and there may be many large raindrops larger than 3 mm.

[0031] Based on the rainfall intensity set by the user, use Python to calculate the volumetric microdensity of raindrops (number of raindrops per cubic meter); the total number of raindrops , is the rainfall intensity, is the volume of the test scene, is a constant representing the reference raindrop density.

[0032] If the initial diameter of the raindrop is less than 1 mm, Stokes' law is used to calculate the terminal velocity. If the initial diameter of the raindrop is ≥ 1 mm, the Beard model is used to calculate the terminal velocity. The terminal velocity is stored as a property of the raindrop for subsequent dynamic simulation.

[0033] Use the numerical calculation module of MATLAB or Python to implement speed calculation, and specifically use NumPy for vectorized calculation.

[0034] Set the transparency of raindrops based on real environment data; Assign an initial position and initial velocity to each raindrop. For example, randomly generate a position (x_0, y_0, z_0) at the upper boundary (height H) of the simulation scene, where (z_0 = H) and (x_0, y_0) are uniformly distributed in the horizontal plane. The initial velocity is usually set to zero or initialized to downward motion according to the terminal velocity.

[0035] A complex wind field model is designed according to the actual test environment and added to the modeling of the physical characteristics of raindrops. In each time step, the gravity, air resistance and wind force of raindrops are calculated. Afterwards, the speed and position of raindrops are updated using the numerical integration method. If the raindrops reach the ground or other surfaces, the raindrops are removed and the splash effect is triggered. Otherwise, the update continues and the operations within the iterative time step are repeated until the preset number of iterations or time is reached. Use Python's NumPy and SciPy libraries for numerical integration, or use physics engines such as PhysX or BulletPhysics to accelerate motion simulations.

[0036] At each time step, a spatial segmentation algorithm (such as octree or grid division) is used to perform collision detection on all raindrop pairs, and the dynamic behavior of raindrops in the air is simulated based on the discrete element method and smooth particle fluid dynamics method, including: defining a collision detection range for each raindrop, that is, a sphere with a raindrop diameter D as the diameter. When the distance between the centers of two raindrops is less than the sum of the radii of the two raindrops (that is, when the spheres of the two raindrops intersect), it is judged that a collision has occurred. According to the size and direction of the relative speed, it is determined whether to merge or split. If the relative speed of the two raindrops is less than the preset threshold (the speed difference is small), they are merged into a larger raindrop, and the new raindrop diameter satisfies the conservation of volume and the speed satisfies the conservation of momentum. If the relative speed of the two raindrops is greater than or equal to the preset threshold (the speed difference between the two raindrops is large), they are randomly split into several small raindrops; the size and number of the raindrops after splitting conform to the Marshall-Palmer distribution, and the total volume of the raindrops is conserved; the properties of the raindrops (size, speed, position) are updated, and the newly generated raindrops are added to the modeling of the physical characteristics of the raindrops; Methods for calculating the weight, air resistance, and wind force on raindrops include: Raindrops will randomly absorb particulate impurities in the air during their fall, causing the mass of the raindrops to increase, thereby changing gravity. The mass of the absorbed particles depends on the concentration of particulate matter in the air. , the surface area of ​​raindrops (because adsorption mainly occurs on the surface of raindrops), the time raindrops are exposed to the air, and the adsorption efficiency (which indicates the ability of raindrops to adsorb particles, which is related to the size of raindrops, the type of particles, etc.).

[0037] For gravity, since the adsorption process is random, randomness can be introduced to simulate the mass of adsorbed particles. The specific method is: in each time step, a random number ξ~(0, 1) is generated, where ξ represents the chance of adsorption. If ξ< , then adsorption occurs, and the adsorption mass is Δm, otherwise no adsorption occurs. is the adsorption probability, , is the probability adjustment coefficient, which requires experimental calibration, Indicates the concentration of particulate matter in the air. is the surface area of ​​the raindrop; the mass of the raindrop is iteratively updated as the sum of the mass of the previous time step and the adsorption mass, and the gravity of the raindrop is calculated based on the updated mass; Air pollution (such as the concentration of particulate matter such as PM2.5 and PM10) may change the density, dynamic viscosity and drag coefficient of the air, thereby affecting air resistance.

[0038] For air resistance, preset the standard density of air based on historical data or literature , Standard dynamic viscosity And the standard drag coefficient , adjusted air density , is the coefficient of influence of particulate matter on air density, in kg / μg, and the adjusted dynamic viscosity , is the coefficient of influence of particles on viscosity, and the adjusted resistance coefficient , is the influence coefficient of particles on the drag coefficient. If the initial diameter of the raindrop is less than 1 mm, the Stokes law is used to calculate the air resistance. If the initial diameter of the raindrop is ≥ 1 mm, the Beard model is used to calculate the air resistance. For wind force, CFD software (such as ANSYS Fluent or OpenFOAM) is used to generate three-dimensional wind field data, including: defining the geometric model of the simulation scene according to the actual test environment (such as urban streets, open terrain), presetting wind speed boundary conditions (such as inlet wind speed, wind direction, turbulence intensity), generating wind speed vector field by solving Navier-Stokes equations, and importing it into the modeling of raindrop physical characteristics, interpolating and calculating the wind speed at each raindrop position, and then adding wind force to the movement of raindrops; introducing complex wind field models to use ANSYS Fluent and OpenFOAM to generate three-dimensional wind field data, and simulating the impact of wind on raindrop trajectories in the simulation platform; The transparency of raindrops is set based on real environment data. The transparency of raindrops is mainly determined by the optical interaction between the particles (such as PM2.5 and dust) adsorbed by them and the background environment. It includes the influence of particle adsorption and the influence of ambient light interaction. The particle adsorption influence is set as , , [PM] and Linear representation of, for example, transparency under the influence of particle adsorption , is the transparency adjustment coefficient, [PM] is the local PM concentration in the area where the raindrops are located, represents the number of particles in the raindrops at each time step, is the surface area of ​​the raindrop; the particle adsorption effect follows an exponential law (e.g., Lambert-Beer law expansion) affecting transparency decay; When raindrops overlap with background objects (such as buildings and vehicles), the transparency under the influence of ambient light intersection is modulated by the transmittance γ of the occluder. For example, the transparency under the influence of ambient light intersection G=1-(1-γ)⋅Occlusion, Occlusion is a binary occlusion factor (1 means occluded, 0 means no occlusion), γ is the transmittance of the occluder (such as glass γ≈0.9, concrete γ≈0.2), and moving light sources (such as car lights) will locally increase the transmittance of raindrops. The introduction of light sources to correct the influence of ambient light intersection, for example, corrected ambient light intersection influence = ambient light intersection influence + β×I light / r 2 , I light is the light source intensity (unit: candela), r is the distance from the raindrop to the light source, and β is the light source-raindrop interaction coefficient (experimental calibration); The transparency under the influence of corrected ambient light intersection is combined with the transparency under the influence of particle adsorption (e.g., weighted summation or linear accumulation) to obtain the transparency of raindrops in each time step.

[0039] The splash effect produced when raindrops hit different surfaces (such as asphalt, soil, and water) will have a certain blurring effect on the sensor. Specifically, when raindrops hit the surface, they will produce different forms of splashes, including crown-shaped splashes perpendicular to the water surface and drop-shaped splashes that quickly break away from the water surface. These splash shapes will affect the sensor's field of view, resulting in irregular blurred areas in the image. Secondly, the splash height of the splash depends on the impact speed, liquid film thickness, and surface characteristics. A higher splash height may cause water droplets to splash onto the sensor lens, forming residual water droplets, further affecting image quality.

[0040] In addition, different material surfaces will also have different effects. For example, when raindrops hit the asphalt surface, they will produce a higher splash height and larger water splashes, which may lead to more severe optical blur and increased noise. The soil surface has strong water absorption, which may reduce the splash height of the water splash, but increase the diffusion area of ​​the water splash, resulting in more extensive blur. When raindrops hit the water surface, they will produce complex water splash shapes, including crown-shaped water splashes and water drop-shaped water splashes. These water splashes may splash onto the sensor lens, forming water drop residues and affecting image quality. Therefore, water splashes are also an important factor affecting sensor quality. In virtual simulation environments, these factors are generally not considered, resulting in sensor data that is too ideal, which in turn creates a gap with the real sensor.

[0041] The visual rendering method of the water splash effect comprises: Set the position of the raindrop when it reaches the ground or other surface as the impact point, the size as the initial bonding size, and the mass as the original mass. Use Raycast to determine whether the raindrop collides with the ground or other surface. Use the terminal velocity as the initial splash velocity. Preset the life cycle of the raindrop (e.g., 0.2-1 second). The materials reaching the ground or other object surfaces are preset as smooth and hard materials (such as asphalt), non-smooth medium-hard materials (such as soil) and liquid materials (such as water surface); the material correlation coefficient is preset for each material; Calculate the impact energy based on the original mass of the raindrop and the initial splash velocity , and record the normal direction of the collision point (for subsequent splash direction calculation); according to the impact energy Generate water splash particle clusters , is the number of particles per cluster, It represents the material correlation coefficient, which is used to describe the particle generation efficiency of different material surfaces. The material correlation coefficients of different material surfaces are determined based on experimental data or literature. For example, the material correlation coefficient of asphalt is 3; the material correlation coefficient of soil is 5, and the material correlation coefficient of water surface is 8; The mass of water splash particles is decayed based on temperature and humidity to simulate evaporation effects; for example, a linear formula is created to calculate the decay update of the mass of water splash particles by combining temperature and humidity.

[0042] Set the splash direction of water particles along the ground normal direction as the main velocity component, and form a tangential direction by random horizontal deflection angle. Get the splash direction by combining the main velocity component with the tangential direction. Add a scene where a car drives over the water surface, set the directional spray to be triggered in this scene, and the initial splash velocity is positively correlated with the vehicle speed. Use the same method as raindrops to iteratively calculate the gravity, air resistance and wind force of water droplets, and update the speed and position; At the time step, the speed, direction and position of the water splash particles are iteratively updated to generate the water splash effect.

[0043] Rainfall will change the reflective properties of the ground, and the reflection parameters of the ground will be updated synchronously in the material modeling module. For example, asphalt road surface will increase specular reflection after it becomes wet.

[0044] The dynamic evolution mechanism of ground water is introduced into the modeling of raindrop physical characteristics. The methods include: For each material that reaches the ground or other surface, a preset humidity saturation limit is set (i.e., the material stops absorbing water after reaching the humidity saturation limit and begins to accumulate water). At each time step, the number of raindrops on the unit material is updated in combination with the evaporation effect. The mirror reflection of the material is set to be positively correlated with the number of raindrops. When the number of raindrops on the unit material reaches the humidity saturation limit, the mirror reflection of the material is set to reach the limit and no longer change. The drainage capacity of the rainwater well is preset according to the actual environment used for testing. The depression water level capacity curve method is used to simulate the depression water accumulation process according to the number of raindrops reaching the ground at each time step. When the rainstorm runoff intensity exceeds the drainage capacity of the rainwater well, the runoff overflows from the rainwater well of the pipeline network to the low-lying areas of the urban surface to form water accumulation.

[0045] Starting from the depression water depth of 0, the water depth of the depression at each time step is iteratively updated, and the water surface area of ​​each depression is obtained. The mirror reflection of the corresponding material of the water surface is used for the water area. Based on the relationship between the different water levels and the corresponding water volume of each depression, the water level capacity curve of each depression is obtained by interpolation or fitting, and stored as a depression attribute.

[0046] The goal of modeling the physical impact of raindrops on sensors is to simulate the impact of raindrops on lidar point clouds and camera images, that is, to simulate the impact of raindrops on multiple sensors (lidar, camera, millimeter wave radar, ultrasonic sensor) in the autonomous driving system and generate realistic sensor data for testing and verifying the robustness of autonomous driving algorithms. Specific goals include: 1. Simulate the scattering and noise interference of raindrops on lidar point cloud.

[0047] 2. Simulate the occlusion, blurring and optical effects (such as refraction, reflection and rainbow effect) of raindrops on camera images.

[0048] 3. Simulate the effect of raindrops on the ground reflection characteristics.

[0049] 4. Simulate the attenuation and scattering of millimeter-wave radar signals by raindrops.

[0050] 5. Simulate the blocking and scattering of ultrasonic sensor signals by raindrops.

[0051] The specific design contents include: Preset the optical, electromagnetic, and acoustic properties of the raindrop material; in the optical properties, the refractive index of the raindrop can be set to 1.33 (the refractive index of water), and the absorption coefficient can be approximately 0 (assuming pure water); in the electromagnetic properties, the dielectric constant of the raindrop can be the dielectric constant of water (about 80) for millimeter-wave radar modeling; in the acoustic properties, the acoustic impedance of the raindrop can be close to the acoustic impedance of water for ultrasonic sensor modeling.

[0052] The sensor dynamic interference unit includes lidar point cloud scattering modeling, camera image blur modeling, millimeter wave radar signal attenuation and scattering modeling, and ultrasonic sensor interference modeling; Specifically: At each time step, the intersection point between the laser beam and the raindrop is iteratively calculated, and the noise points and intensity attenuation in the point cloud are updated; Iteratively update the position and shape of water droplets on the camera, recalculate occlusion, blur and optical effects, and generate dynamic image sequences; Iteratively updates the intersection point of the millimeter-wave beam and raindrops, recalculates the attenuation and scattering effects, and generates dynamic radar signals; The intersection point between the ultrasonic beam and the raindrops is iteratively updated, the occlusion and scattering effects are recalculated, and a dynamic echo signal is generated.

[0053] LiDAR point cloud scattering modeling; Mie scattering theory is used to calculate the scattering effect of laser beams and raindrops. Mie scattering is applicable to the case where the size of raindrops is close to the laser wavelength (about 905 nm or 1550 nm).

[0054] In the LiDAR point cloud generation module, a raindrop scattering model is added. For each laser beam, its intersection with the raindrop is calculated, and the scattering probability and intensity are calculated according to the Mie scattering theory. If the laser beam is scattered, a noise point is added to the point cloud or the intensity of the reflection point is reduced.

[0055] Use ray tracing technology to trace the path of each laser beam and calculate its intersection with raindrops. To improve efficiency, the stereo microscope (Voxel Grid) method can be used to divide the space into voxels. Scattering is calculated only in the voxels containing raindrops. The scattering probability at the intersection is determined by the Mie scattering cross section, and the scattering direction is determined by the Mie scattering phase function. The phase function can be calculated by numerical methods (such as the T-matrix method) or using a pre-calculated lookup table; If the laser beam is scattered, a noise point is added to the point cloud. The position of the noise point is determined by the scattering direction and intensity. If the laser beam is not completely scattered, its intensity decays exponentially. Camera image blur modeling; The raindrops are projected onto the camera lens. The projection shape is the same as the raindrop shape, for example, a circle or an ellipse. The area is related to the size of the raindrop and the focal length of the lens. The transparency of the raindrops is obtained. A Gaussian blur filter is applied to the area blocked by the raindrops in the image. The blur radius is proportional to the size of the raindrops. The occlusion and blurring effect of the raindrops on the lens is obtained. The relationship between the gravity of raindrops, wind force, surface friction of the lens, and the dynamic movement of raindrops on the lens is expressed through linear formulas. At each time step, the position and shape of raindrops are updated, and the occlusion and blur effects are recalculated. Millimeter wave radar signal attenuation and scattering modeling; When millimeter waves (24 GHz or 77 GHz) propagate in raindrops, absorption and scattering occur. The attenuation coefficient is related to the size, density and wavelength of the raindrops and can be calculated using Rayleigh scattering or Mie scattering theory.

[0056] For each mmWave beam, ray tracing techniques are used to calculate its intersection with raindrops to determine attenuation and scattering effects; The attenuation coefficient and scattering cross section were obtained by fitting the rainfall intensity using the ITU-R model; The scattering direction of millimeter waves is determined by the Mie scattering phase function; since the wavelength of millimeter waves (about 4mm or 12.5mm) is close to the size of raindrops, the scattering effect is significant, and the scattering direction can be calculated by numerical methods (such as the T-matrix method) or using a pre-calculated lookup table.

[0057] The scattered millimeter wave signal will appear as noise or false targets in the radar receiver, thus obtaining the affected millimeter wave radar signal. The position of the false target is determined by the scattering direction and intensity, and the intensity is determined by the scattering cross section.

[0058] Ultrasonic sensor interference modeling; Ultrasonic waves (usually with a frequency of 40kHz) are blocked and scattered by raindrops during propagation. The blocking effect is determined by the geometric blocking area of ​​the raindrops, and the scattering effect is determined by the acoustic impedance and size of the raindrops.

[0059] For each ultrasonic beam, calculate its intersection with the raindrop. If the beam is completely blocked, the echo signal strength is 0. The blocking probability is determined by the projection area of ​​the raindrop and the beam cross-sectional area. If the initial diameter of the raindrop is less than 1mm, Rayleigh scattering is used to calculate the scattering intensity of the raindrop on the ultrasonic wave; if the initial diameter of the raindrop is ≥1mm, Born is used to calculate the scattering intensity of the raindrop on the ultrasonic wave; the scattered ultrasonic signal appears as noise or false echo in the receiver; then the affected ultrasonic signal is obtained, and the distance of the false echo is determined by the position of the scattering point, and the intensity is determined by the scattering cross section; In camera image blur modeling, optical effects and slippery effects are introduced; Ray tracing technology generates images by simulating the propagation of light in a scene. It can accurately simulate the refraction and reflection of light. These optical effects directly affect the clarity and quality of camera images. The rainbow effect is caused by the difference in the refractive index of raindrops for light of different wavelengths. The refraction and reflection of incident white light inside the raindrops are calculated by spectral decomposition. The rainbow effect may cause colored halos or blurred areas in the image, especially under strong light.

[0060] Ray tracing technology is used to simulate the refraction and reflection of light by raindrops. Refraction follows Snell's law, and reflection follows Fresnel's equation.

[0061] Follows Snell's law, which describes the refraction of light when it passes from one medium into another. Refraction may cause light to bend at the interface between different media, resulting in blurred or distorted areas in the image.

[0062] Following the Fresnel equation, the reflection and transmission coefficients of light at the interface between two media are described. Reflection may cause unwanted bright spots or reflected light in the image, affecting the contrast and clarity of the image.

[0063] For raindrops on the camera, generate incident light from the light source (or ambient light in the scene), calculate the intersection of the light and the raindrop surface, use geometric optics to determine the intersection position and normal vector, calculate the direction and intensity of the refracted and reflected light according to Snell's law and Fresnel equations, and recursively trace the light path at each time step until the light leaves the raindrop or the intensity decays below the threshold; The rainbow effect is caused by the difference in the refractive index of raindrops for light of different wavelengths. The secondary refraction and primary reflection inside the raindrops will decompose white light into different colors of light, forming a rainbow. Through spectral decomposition, the incident white light is decomposed into multiple monochromatic lights (wavelength range 400nm to 700nm), that is, the refractive index of water changes with wavelength. The refraction angle of each wavelength of incident white light inside the raindrop is calculated based on the dispersion effect. The refraction and reflection path of each wavelength of light inside the raindrop is calculated using ray tracing technology. According to the angle and intensity of the outgoing light, the rainbow effect is synthesized; For the slippery effect, use a bidirectional reflectance distribution function model (such as the Phong model or the Cook-Torrance model) to model the specular reflection of the material.

[0064] Methods for connecting the vehicle's sensor suite and autonomous driving system to the virtual twin center using a CAN bus or Ethernet interface, running the autonomous driving system, and generating performance reports include: Quantitative indicators for pre-set performance reports (e.g., key performance indicators, perception accuracy, decision success rate, and safety) and a real environment for testing; Use digital twin technology to virtually construct the real environment used for testing, produce a virtual twin center, use real environment data to pre-train the modeling of raindrop physical characteristics, and stop when the similarity between the modeling of raindrop physical characteristics and the real environment data reaches a threshold. Use the sensor kit data in autonomous driving to pre-train the modeling of the physical impact of the sensor until it reaches the expected level, thus forming a trained virtual twin center. Use the CAN bus or Ethernet interface to connect the car's sensor kit and autonomous driving system to the virtual twin center, run the autonomous driving system, record the data of quantitative indicators, form a performance report, and complete the evaluation of autonomous driving vehicles under the combination of virtual and real. It can not only set up a variety of environmental scenarios, but also save test space, thereby saving the cost expenditure of automobile companies and improving economic benefits.

[0065] Example 2 See also Figure 2 As shown, the part not described in detail in this embodiment is described in Example 1, which provides a virtual-real combined automobile automatic driving test system, including: Pre-preparation module: used to deploy the sensor kit and autonomous driving system on the test car, and connect the car's sensor kit and autonomous driving system to the virtual twin center using the CAN bus or Ethernet interface; Virtual Twin Center Design Module: Based on the simulation platform, it expands the design of raindrop physical characteristics modeling and sensor physical impact modeling; Pre-training module: Preset the twin virtual test scene according to the real environment, import the modeling of raindrop physical characteristics and the physical impact modeling of sensors into the twin virtual test scene and perform pre-training; Evaluation output module: Run the autonomous driving system in the virtual twin center, record the data of quantitative indicators, and output performance reports.

[0066] The virtual-reality combined automobile autonomous driving test method realizes efficient, safe and comprehensive autonomous driving testing by integrating the real test site environment with the cloud twin simulation system. The authenticity of the test scene and the accuracy of sensor performance evaluation are improved through detailed modeling of raindrop physical characteristics and sensor physical impact. By connecting the vehicle and the virtual twin center through the CAN bus or Ethernet interface, the integrated synchronous operation of the vehicle-cloud-field is realized, which improves the real-time and response speed of the test. Finally, by generating detailed performance reports, data support is provided for the optimization of the autonomous driving system. It not only improves the test efficiency and safety, but also provides strong support for the development of autonomous driving technology.

[0067] Example 3 This embodiment discloses an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the operation mode of the above-mentioned virtual-reality combined automobile automatic driving test method is implemented.

[0068] Since the electronic device introduced in this embodiment is an electronic device used to implement a virtual-reality combined vehicle automatic driving test method in the embodiment of this application, based on the virtual-reality combined vehicle automatic driving test method introduced in the embodiment of this application, the technical personnel of this field can understand the specific implementation of the electronic device of this embodiment and its various variations, so how the electronic device implements the method in the embodiment of this application is not introduced in detail here. As long as the technical personnel of this field implement the electronic device used in the virtual-reality combined vehicle automatic driving test method in the embodiment of this application, it belongs to the scope of protection of this application.

[0069] The above formulas are all dimensionless and numerical calculations. The formula is a formula for the most recent real situation obtained by collecting a large amount of data and performing software simulation. The preset parameters and thresholds in the formula are set by technicians in this field according to actual conditions.

[0070] The above is only a preferred embodiment of the present invention, and the protection scope of the present invention is not limited to the above embodiments. All technical solutions under the concept of the present invention belong to the protection scope of the present invention. It should be pointed out that for ordinary technical users in this technical field, some improvements and modifications without departing from the principle of the present invention should also be regarded as the protection scope of the present invention.

Claims

1. A virtual and real combined vehicle automatic driving test method, characterized in that: include: Deploy the sensor kit and autonomous driving system on the test car, connect the car's sensor kit and autonomous driving system to the virtual twin center using the CAN bus or Ethernet interface, run the autonomous driving system, and generate performance reports; The virtual twin center is extended and designed based on the simulation platform, including modeling of raindrop physical characteristics and physical impact of sensors. Twin virtual test scenarios are preset according to the real environment, and the modeling of raindrop physical characteristics and physical impact of sensors are imported into the twin virtual test scenarios and pre-trained. Among them, the modeling of raindrop physical characteristics is used for the generation of raindrops, including: initializing raindrop properties, setting the transparency of raindrops based on real environment data, introducing complex wind field models and the dynamic evolution mechanism of ground water, using space segmentation algorithms to perform collision detection on all raindrop pairs, simulating the dynamic behavior of raindrops in the air based on discrete element method and smooth particle fluid dynamics method, and visually rendering the water splash effect; The physical impact modeling of the sensor includes the sensor dynamic interference unit, which iteratively updates the outputs of the lidar point cloud scattering modeling, the camera image blur modeling, the millimeter wave radar signal attenuation and scattering modeling, and the ultrasonic sensor interference modeling under the influence of the raindrop physical property modeling.

2. The method for testing automatic driving of a virtual and real vehicle according to claim 1, characterized in that: The raindrop physical property modeling includes: Provide a parameterized interface, where users can set evolution parameters through input boxes. Evolution parameters include rainfall intensity, wind speed, wind direction, and real environment data for testing. The Marshall-Palmer distribution is sampled using the Monte Carlo method to generate the initial diameter of raindrops, with a sampling range of 0.1 mm to 10 mm; Based on the rainfall intensity set by the user, the volumetric microscopic density of raindrops is calculated using Python; If the initial diameter of the raindrop is less than 1 mm, Stokes' law is used to calculate the terminal velocity. If the initial diameter of the raindrop is ≥ 1 mm, the Beard model is used to calculate the terminal velocity. The terminal velocity is stored as a property of the raindrop. Set the transparency of raindrops based on real environment data; Assign an initial position and initial velocity to each raindrop; A complex wind field model is designed according to the actual test environment and added to the modeling of the physical characteristics of raindrops. In each time step, the gravity, air resistance and wind force of raindrops are calculated. Afterwards, the speed and position of raindrops are updated using the numerical integration method. If the raindrops reach the ground or other surfaces, the raindrops are removed and the splash effect is triggered. Otherwise, the update continues and the operations within the iterative time step are repeated until the preset number of iterations or time is reached. At each time step, a spatial segmentation algorithm is used to perform collision detection on all raindrop pairs, and the dynamic behavior of raindrops in the air is simulated based on the discrete element method and smoothed particle fluid dynamics method, including: defining a collision detection range for each raindrop, and judging a collision when the center distance between two raindrops is less than the sum of the two raindrop radii. The merger or splitting is determined based on the size and direction of the relative speed. If the relative speed of the two raindrops is less than the preset threshold, they are merged into a larger raindrop, and the new raindrop diameter satisfies the volume conservation and the speed satisfies the momentum conservation. If the relative speed of the two raindrops is greater than or equal to the preset threshold, they are randomly split into several small raindrops. The size and number of the raindrops after the split conform to the Marshall-Palmer distribution, and the total volume of the raindrops is conserved. The properties of the raindrops are updated, and the newly generated raindrops are added to the modeling of the physical properties of the raindrops.

3. The method for testing automatic driving of a virtual and real vehicle according to claim 2, characterized in that: The method for calculating the gravity, air resistance and wind force of raindrops comprises: For gravity, a random number ξ~(0, 1) is generated at each time step, where ξ represents the chance of adsorption. , then adsorption occurs, and the adsorption mass is Δm, otherwise no adsorption occurs. is the adsorption probability, , is the probability adjustment factor, Indicates the concentration of particulate matter in the air. is the surface area of ​​the raindrop; the mass of the raindrop is iteratively updated as the sum of the mass of the previous time step and the adsorption mass, and the gravity of the raindrop is calculated based on the updated mass; For air resistance, preset the standard density of air based on historical data or literature , Standard dynamic viscosity And the standard drag coefficient , adjusted air density , is the coefficient of influence of particles on air density, and the adjusted dynamic viscosity , is the coefficient of influence of particles on viscosity, and the adjusted resistance coefficient , is the influence coefficient of particles on the drag coefficient. If the initial diameter of the raindrop is less than 1 mm, the Stokes law is used to calculate the air resistance. If the initial diameter of the raindrop is ≥ 1 mm, the Beard model is used to calculate the air resistance. For wind force, CFD software is used to generate three-dimensional wind field data, including: defining the geometric model of the simulation scene according to the actual test environment, presetting the wind speed boundary conditions, generating the wind speed vector field by solving the Navier-Stokes equation, and importing it into the modeling of the physical characteristics of raindrops, interpolating and calculating the wind speed at each raindrop position, and then adding wind force during the movement of raindrops.

4. The method for testing automatic driving of a virtual and real vehicle according to claim 3, characterized in that: The transparency of raindrops is set based on real environment data, including the influence of particle adsorption and the influence of ambient light intersection, wherein the particle adsorption influence is set as , , [PM] and The linear representation of is the transparency adjustment coefficient, [PM] is the local PM concentration in the area where the raindrops are located, represents the number of particles in the raindrops at each time step, is the surface area of ​​the raindrop; the particle adsorption effect follows the exponential law and affects the transparency attenuation; The transparency under the influence of ambient light intersection is modulated by the light transmittance γ of the obstruction, and the light source is introduced to correct the influence of ambient light intersection; The transparency under the influence of corrected ambient light intersection and the transparency under the influence of particle adsorption are combined to obtain the transparency of raindrops in each time step.

5. The method for testing automatic driving of a virtual and real vehicle according to claim 4, characterized in that: The visual rendering method of the water splash effect comprises: Set the position of the raindrop when it reaches the ground or other surface as the impact point, the size as the initial bonding size, the mass as the original mass, the terminal velocity as the initial splash velocity, and preset the life cycle of the raindrop; The materials reaching the ground or other object surfaces are preset as smooth hard materials, non-smooth medium hard materials and liquid materials; the material correlation coefficient is preset for each material; Calculate the impact energy based on the original mass of the raindrop and the initial splash velocity , and record the normal direction of the collision point; according to the impact energy Generate water splash particle clusters , is the number of particles per cluster, represents the material correlation coefficient; The mass attenuation of water splash particles is simulated by evaporation effect according to temperature and humidity; Set the splash direction of water particles along the ground normal direction as the main velocity component, and form a tangential direction by random horizontal deflection angle. Get the splash direction by combining the main velocity component with the tangential direction. Add a scene where a car drives over the water surface, set the directional spray to be triggered in this scene, and the initial splash velocity is positively correlated with the vehicle speed. Use the same method as raindrops to iteratively calculate the gravity, air resistance and wind force of water droplets, and update the speed and position; At the time step, the speed, direction and position of the water splash particles are iteratively updated to generate the water splash effect.

6. The method for testing automatic driving of a virtual and real vehicle according to claim 5, characterized in that: The dynamic evolution mechanism of ground water is introduced into the modeling of raindrop physical characteristics, including: For each material reaching the ground or other surface, a humidity saturation limit is preset. At each time step, the number of raindrops on the unit material is updated in combination with the evaporation effect. The specular reflection of the material is set to be positively correlated with the number of raindrops. When the number of raindrops on the unit material reaches the humidity saturation limit, the specular reflection of the material is set to reach the limit and no longer change. The drainage capacity of the rainwater well is preset according to the real environment used for testing, and the depression water level capacity curve method is used to simulate the depression water accumulation process according to the number of raindrops reaching the ground at each time step; Starting from the depression water depth of 0, iteratively update the water depth of the depression at each time step, and obtain the water surface area of ​​each depression, and use the mirror reflection of the corresponding material of the water surface on the water surface.

7. The method for testing automatic driving of a virtual and real vehicle according to claim 6, characterized in that: Modeling the physical impact of the sensor, the method comprising: Preset optical, electromagnetic and acoustic properties of raindrop materials; At each time step, the intersection point between the laser beam and the raindrop is iteratively calculated, and the noise points and intensity attenuation in the point cloud are updated; Iteratively update the position and shape of water droplets on the camera, recalculate occlusion, blur and optical effects, and generate dynamic image sequences; Iteratively updates the intersection point of the millimeter-wave beam and raindrops, recalculates the attenuation and scattering effects, and generates dynamic radar signals; The intersection point between the ultrasonic beam and the raindrops is iteratively updated, the occlusion and scattering effects are recalculated, and a dynamic echo signal is generated.

8. The method for testing automatic driving of a virtual and real vehicle according to claim 7, characterized in that: The sensor dynamic interference unit comprises: Modeling of LiDAR point cloud scattering; Using ray tracing technology, the path of each laser beam is traced and its intersection with the raindrop is calculated. The scattering probability at the intersection is determined by the Mie scattering cross section, and the scattering direction is determined by the Mie scattering phase function. If the laser beam is scattered, a noise point is added to the point cloud. The position of the noise point is determined by the scattering direction and intensity. If the laser beam is not completely scattered, its intensity decays exponentially. Modeling camera image blur; Project raindrops onto the camera lens and obtain the transparency of the raindrops. Apply a Gaussian blur filter to the area blocked by raindrops in the image. The blur radius is proportional to the size of the raindrops to obtain the occlusion and blurring effect of raindrops on the lens. The relationship between the gravity of raindrops, wind force, surface friction of the lens, and the dynamic movement of raindrops on the lens is expressed through linear formulas. At each time step, the position and shape of raindrops are updated, and the occlusion and blur effects are recalculated. Modeling of millimeter wave radar signal attenuation and scattering; For each mmWave beam, ray tracing techniques are used to calculate its intersection with raindrops to determine attenuation and scattering effects; The attenuation coefficient and scattering cross section were obtained by fitting the rainfall intensity using the ITU-R model; The scattering direction of millimeter waves is determined by the Mie scattering phase function; The scattered millimeter-wave signal will appear as noise or false targets in the radar receiver, thereby obtaining an affected millimeter-wave radar signal; Modeling interference with ultrasonic sensors; For each ultrasonic beam, calculate its intersection with the raindrop. If the beam is completely blocked, the echo signal strength is 0. The blocking probability is determined by the projection area of ​​the raindrop and the beam cross-sectional area. If the initial diameter of the raindrop is less than 1mm, Rayleigh scattering is used to calculate the scattering intensity of the raindrop to the ultrasonic wave; if the initial diameter of the raindrop is ≥1mm, Born is used to calculate the scattering intensity of the raindrop to the ultrasonic wave; the scattered ultrasonic signal appears as noise or false echo in the receiver; and then the affected ultrasonic signal is obtained.

9. The method for testing automatic driving of a virtual and real vehicle according to claim 8, characterized in that: In the camera image blur modeling, optical effect and slippery effect are introduced; For raindrops on the camera, an incident ray is generated from the light source, the intersection point between the ray and the raindrop surface is calculated, the intersection point position and normal vector are determined using the geometric optics method, the direction and intensity of the refracted and reflected rays are calculated based on Snell's law and Fresnel's equations, and the ray path is recursively traced at each time step until the ray leaves the raindrop or the intensity decays below a threshold. The incident white light is decomposed into multiple monochromatic lights, and the refraction angle of each wavelength of the incident white light inside the raindrop is calculated based on the dispersion effect. The refraction and reflection path of each wavelength of light inside the raindrop is calculated using ray tracing technology, and the rainbow effect is synthesized based on the angle and intensity of the outgoing light. For the slippery-wet effect, the specular reflection of the material is modeled using the Bidirectional Reflectance Distribution Function model.

10. The method for testing automatic driving of a virtual and real vehicle according to claim 9, characterized in that: The method of connecting the sensor kit and the autonomous driving system of the vehicle to the virtual twin center using the CAN bus or Ethernet interface, running the autonomous driving system, and generating a performance report includes: Preset quantitative indicators for performance reports and real environments for testing; Use digital twin technology to virtually construct the real environment used for testing, produce a virtual twin center, use real environment data to pre-train the modeling of raindrop physical characteristics, and stop when the similarity between the modeling of raindrop physical characteristics and the real environment data reaches a threshold. Use the sensor kit data in autonomous driving to pre-train the modeling of the physical impact of the sensor until it reaches the expected level, thus forming a trained virtual twin center. Use the CAN bus or Ethernet interface to connect the car's sensor kit and autonomous driving system to the virtual twin center, run the autonomous driving system, record data on quantitative indicators, and form a performance report.

Images