A virtual-real combined vehicle autonomous driving testing method

Abstract

The present invention belongs to the field of data analysis and processing, and discloses a method for testing automobile autonomous driving in a virtual-reality combination; the method comprises: connecting a sensor kit and an autonomous driving system of an automobile to a virtual twin center using a CAN bus or an Ethernet interface, running the autonomous driving system, and generating a performance report; initializing raindrop properties, setting the transparency of raindrops based on real environmental data, introducing a complex wind field model and a dynamic evolution mechanism of surface water on the ground, performing collision detection on all raindrop pairs using a spatial segmentation algorithm, simulating the dynamic behavior of raindrops in the air based on a discrete element method and a smoothed particle fluid dynamics method, and visually rendering the water splash effect; under the influence of raindrop physical property modeling, iteratively updating the outputs of lidar point cloud scattering modeling, camera image blur modeling, millimeter-wave radar signal attenuation and scattering modeling, and ultrasonic sensor interference modeling, to achieve autonomous driving testing of automobiles in a virtual-reality combination.

Description

Technical Field

[0001] The present invention relates to the field of data analysis and processing, and more specifically, to a virtual-reality combined automobile automatic driving testing method. Background Art

[0002] Rainy days are the most common weather conditions in life and are also the most common weather scenarios encountered during autonomous driving. If the test can be passed on rainy days, more than half of the entire test content has been completed. Therefore, this application mainly focuses on the test of autonomous driving in rainy days.

[0003] Rain not only alters the physical environment but also directly impacts sensor performance. Rain can cause blurry camera images and sparse LiDAR point clouds. Simulations may simply simulate the effects of rainfall, but fail to accurately simulate the raindrops' interference with the LiDAR point cloud or their occlusion of the camera image. This "double simplification" can lead to systems that perform well in simulation but completely fail in real-world rainy conditions.

[0004] Therefore, the interaction between the overly idealized sensor models in simulation and the simplified physical environment can lead to significant discrepancies between the performance of autonomous driving systems in simulation and in real-world conditions. This discrepancy not only increases the difficulty of development and testing but can also lead to safety hazards in the real world. For example, a system may pass all tests in simulation but cause an accident in real-world rainy conditions due to perception errors or poor decision-making. Summary of the Invention

[0005] In order to overcome the above-mentioned shortcomings of the prior art and to achieve the above-mentioned objectives, the present invention provides the following technical solutions: a virtual-reality combined vehicle autonomous driving testing method, comprising: deploying a sensor suite and an autonomous driving system on a test vehicle, connecting the vehicle's sensor suite and autonomous driving system to a virtual twin center using a CAN bus or Ethernet interface, running the autonomous driving system, and generating a performance report;

[0006] The virtual twin center is designed based on an extended simulation platform, including modeling of raindrop physical properties and sensor physical impacts. Twin virtual test scenarios are preset based on real environments, and these models are imported into the twin virtual test scenarios for pre-training.

[0007] The physical characteristics of raindrops are modeled for raindrop generation, including initializing raindrop properties, setting raindrop transparency based on real-world environmental data, introducing complex wind field models and the dynamic evolution of surface water, using a spatial segmentation algorithm to perform collision detection on all raindrop pairs, simulating the dynamic behavior of raindrops in the air based on discrete element methods and smoothed particle fluid dynamics, and visually rendering the water splash effect.

[0008] The physical impact modeling of the sensor includes the sensor dynamic interference unit, which iteratively updates the outputs of the lidar point cloud scattering modeling, camera image blur modeling, millimeter wave radar signal attenuation and scattering modeling, and ultrasonic sensor interference modeling under the influence of raindrop physical characteristics modeling.

[0009] Preferably, the raindrop physical property modeling includes:

[0010] Provides a parameterized interface where users can set evolution parameters through input boxes. The evolution parameters include rainfall intensity, wind speed, wind direction, and real environment data used for testing.

[0011] The Monte Carlo method is used to sample the Marshall-Palmer distribution to generate the initial diameter of raindrops, with a sampling range of 0.1 mm to 10 mm;

[0012] Based on the rainfall intensity set by the user, the volumetric microscopic density of raindrops is calculated using Python;

[0013] If the initial diameter of the raindrop is less than 1 mm, Stokes' law is used to calculate the terminal velocity. If the initial diameter of the raindrop is ≥ 1 mm, the Beard model is used to calculate the terminal velocity. The terminal velocity is stored as a property of the raindrop.

[0014] Set the transparency of raindrops based on real environment data;

[0015] Assign an initial position and initial velocity to each raindrop;

[0016] A complex wind field model is designed based on the actual test environment and added to the raindrop physical property modeling. In each time step, the gravity, air resistance, and wind force of the raindrop are calculated. Subsequently, the velocity and position of the raindrop are updated using a numerical integration method. If the raindrop reaches the ground or other surface, the raindrop is removed and the splash effect is triggered. Otherwise, the update continues, and the operation within the iterative time step is repeated until the preset number of iterations or time is reached.

[0017] At each time step, a spatial segmentation algorithm is used to perform collision detection on all raindrop pairs. The dynamic behavior of raindrops in the air is simulated based on the discrete element method and smoothed particle fluid dynamics method, including: defining a collision detection range for each raindrop. When the distance between the centers of two raindrops is less than the sum of the two raindrop radii, it is judged that a collision has occurred. The size and direction of the relative velocity are used to determine whether to merge or split. If the relative velocity of the two raindrops is less than a preset threshold, they are merged into a larger raindrop, and the new raindrop diameter satisfies the conservation of volume and the velocity satisfies the conservation of momentum. If the relative velocity of the two raindrops is greater than or equal to the preset threshold, they are randomly split into several small raindrops. The size and number of the raindrops after splitting conform to the Marshall-Palmer distribution, and the total volume of the raindrops is conserved. The properties of the raindrops are updated, and the newly generated raindrops are added to the raindrop physical property modeling.

[0018] Preferably, the method for calculating the gravity, air resistance and wind force of raindrops includes:

[0019] For gravity, a random number ξ~(0, 1) is generated in each time step, where ξ represents the chance of adsorption. , then adsorption occurs, and the adsorption mass is Δm, otherwise no adsorption occurs. is the adsorption probability, , is the probability adjustment coefficient, Indicates the concentration of particulate matter in the air. is the surface area of ​​the raindrop; the mass of the raindrop is iteratively updated as the sum of the mass of the previous time step and the adsorption mass, and the gravity of the raindrop is calculated based on the updated mass;

[0020] For air resistance, the standard density of air is preset based on historical data or literature. , standard dynamic viscosity and the standard drag coefficient , adjusted air density , is the coefficient of influence of particles on air density, and the adjusted dynamic viscosity , is the coefficient of influence of particles on viscosity, and the adjusted resistance coefficient , is the influence coefficient of particulate matter on the drag coefficient. If the initial diameter of the raindrop is less than 1 mm, the Stokes law is used to calculate the air resistance; if the initial diameter of the raindrop is ≥ 1 mm, the Beard model is used to calculate the air resistance.

[0021] For wind forces, CFD software is used to generate three-dimensional wind field data, including: defining the geometric model of the simulation scene according to the actual test environment, presetting the wind speed boundary conditions, generating the wind speed vector field by solving the Navier-Stokes equation, and importing it into the raindrop physical characteristics modeling. The wind speed at each raindrop position is interpolated and calculated, and the wind force is added during the raindrop movement.

[0022] Preferably, the transparency of raindrops is set based on real environment data, including the effect of particle adsorption and the effect of ambient light interaction, wherein the particle adsorption effect is set to 、 , [PM] and The linear representation of is the transparency adjustment coefficient, [PM] is the local PM concentration in the area where the raindrops are located, represents the number of particles in the raindrops at each time step, is the surface area of ​​the raindrop; the effect of particle adsorption follows an exponential law affecting transparency attenuation;

[0023] The transparency under the influence of ambient light is modulated by the transmittance γ of the obstruction, and the introduction of light source is used to correct the influence of ambient light.

[0024] The transparency under the influence of corrected ambient light intersection is combined with the transparency under the influence of particle adsorption to obtain the transparency of raindrops in each time step.

[0025] Preferably, the visual rendering method of the water splash effect includes:

[0026] Set the position of the raindrop when it reaches the ground or other surface as the impact point, the size as the initial bonding size, the mass as the original mass, the terminal velocity as the initial splash velocity, and preset the life cycle of the raindrop;

[0027] The preset materials reaching the ground or other object surfaces are smooth hard material, non-smooth medium hard material and liquid material; the material correlation coefficient is preset for each material;

[0028] Calculate the impact energy based on the original mass of the raindrop and the initial splash velocity , and record the normal direction of the collision point; according to the impact energy Generate water splash particle clusters , is the number of particles per cluster, represents the material correlation coefficient;

[0029] The mass attenuation of water droplets is simulated by evaporation effect according to temperature and humidity;

[0030] The splash direction of water particles is set to be along the ground normal direction as the main velocity component, and the random horizontal deflection angle forms the tangential direction. The splash direction is obtained by combining the main velocity component and the tangential direction. A scene is added in which a car drives over the water surface, and a directional spray is triggered in this scene. The initial splash velocity is positively correlated with the vehicle speed.

[0031] Use the same method as raindrops to iteratively calculate the gravity, air resistance, and wind force of the water droplets, and update the speed and position;

[0032] At the time step, the speed, direction and position of the water splash particles are iteratively updated to generate the water splash effect.

[0033] Preferably, the dynamic evolution mechanism of ground water is introduced into the modeling of the physical characteristics of raindrops, and the method includes:

[0034] A humidity saturation limit is set for each material reaching the ground or other surface. At each time step, the number of raindrops per unit material is updated in combination with the evaporation effect. The specular reflection of the material is set to be positively correlated with the number of raindrops. When the number of raindrops per unit material reaches the humidity saturation limit, the specular reflection of the material is set to reach its limit and no longer change.

[0035] The drainage capacity of the rainwater well is preset according to the actual environment used for testing. The depression water level capacity curve method is used to simulate the depression water accumulation process based on the number of raindrops reaching the ground at each time step.

[0036] Starting from the depression water depth of 0, iteratively update the water depth of the depression at each time step, and obtain the surface area of ​​water in each depression. The mirror reflection of the corresponding material of the water surface is used on the water area.

[0037] Preferably, the physical impact modeling of the sensor comprises:

[0038] Preset optical, electromagnetic, and acoustic properties of raindrop materials;

[0039] At each time step, the intersection point between the laser beam and the raindrop is iteratively calculated, and the noise points and intensity attenuation in the point cloud are updated;

[0040] Iteratively updates the position and shape of water droplets on the camera, recalculates occlusion, blur, and optical effects, and generates dynamic image sequences;

[0041] Iteratively updates the intersection point of the millimeter-wave beam and raindrops, recalculates attenuation and scattering effects, and generates dynamic radar signals;

[0042] The intersection point between the ultrasonic beam and the raindrops is iteratively updated, the occlusion and scattering effects are recalculated, and a dynamic echo signal is generated.

[0043] Preferably, the sensor dynamic interference unit includes:

[0044] Modeling of LiDAR point cloud scattering;

[0045] Using ray tracing technology, the path of each laser beam is traced and its intersection with the raindrop is calculated. The scattering probability at the intersection is determined by the Mie scattering cross section, and the scattering direction is determined by the Mie scattering phase function.

[0046] If the laser beam is scattered, a noise point is added to the point cloud. The position of the noise point is determined by the scattering direction and intensity. If the laser beam is not completely scattered, its intensity decays exponentially.

[0047] Modeling camera image blur;

[0048] Project raindrops onto the camera lens and obtain the transparency of the raindrops. Apply a Gaussian blur filter to the areas of the image obscured by raindrops, with a blur radius proportional to the size of the raindrops, to obtain the occlusion and blurring effect of the raindrops on the lens.

[0049] A linear formula is used to express the relationship between the gravity of raindrops, wind force, surface friction of the lens, and the dynamic movement of raindrops on the lens. At each time step, the position and shape of the raindrops are updated, and the occlusion and blur effects are recalculated.

[0050] Modeling of millimeter-wave radar signal attenuation and scattering;

[0051] For each mmWave beam, ray tracing techniques are used to calculate its intersection with raindrops to determine attenuation and scattering effects.

[0052] The attenuation coefficient and scattering cross section were obtained by fitting the rainfall intensity using the ITU-R model;

[0053] The scattering direction of millimeter waves is determined by the Mie scattering phase function;

[0054] The scattered millimeter-wave signal will appear as noise or false targets in the radar receiver, thereby obtaining an affected millimeter-wave radar signal;

[0055] Modeling interference with ultrasonic sensors;

[0056] For each ultrasonic beam, calculate its intersection with the raindrop. If the beam is completely blocked, the echo signal strength is 0. The blocking probability is determined by the projected area of ​​the raindrop and the beam cross-sectional area.

[0057] If the initial diameter of the raindrop is less than 1mm, Rayleigh scattering is used to calculate the scattering intensity of the raindrop on the ultrasonic wave; if the initial diameter of the raindrop is ≥1mm, Born scattering is used to calculate the scattering intensity of the raindrop on the ultrasonic wave; the scattered ultrasonic signal appears as noise or false echo in the receiver; and the affected ultrasonic signal is then obtained.

[0058] Preferably, optical effect and slippery effect are introduced into the camera image blur modeling;

[0059] For raindrops on the camera, an incident ray is generated from the light source. The intersection point of the ray with the raindrop surface is calculated. The intersection position and normal vector are determined using geometric optics methods. The direction and intensity of the refracted and reflected rays are calculated based on Snell's law and Fresnel equations. The ray path is recursively traced at each time step until the ray leaves the raindrop or the intensity decays below a threshold.

[0060] The incident white light is decomposed into multiple monochromatic lights. The refraction angle of each wavelength of the incident white light inside the raindrop is calculated based on the dispersion effect. Ray tracing technology is used to calculate the refraction and reflection path of each wavelength of light inside the raindrop. The rainbow effect is synthesized based on the angle and intensity of the outgoing light.

[0061] For the slippery-wet effect, the specular reflection of the material is modeled using the Bidirectional Reflectance Distribution Function model.

[0062] Preferably, the method of connecting the vehicle's sensor suite and autonomous driving system to the virtual twin center using a CAN bus or Ethernet interface, running the autonomous driving system, and generating a performance report includes:

[0063] Preset quantitative indicators for performance reports and real-world environments for testing;

[0064] Use digital twin technology to virtually construct the real environment used for testing to produce a virtual twin center. Use real-world data to pre-train the raindrop physical property modeling until the similarity between the raindrop physical property modeling and the real-world data reaches a threshold. Use data from the sensor suite used in autonomous driving to pre-train the sensor physical impact modeling until the expected level is achieved, thus forming a fully trained virtual twin center.

[0065] Use the CAN bus or Ethernet interface to connect the car's sensor suite and autonomous driving system to the virtual twin center, run the autonomous driving system, record data on quantitative indicators, and generate performance reports.

[0066] The technical effects and advantages of the virtual-real combined vehicle automatic driving test method of the present invention are as follows:

[0067] 1. Virtual and real combined testing method

[0068] By combining the real-world test site environment with a cloud-based twin simulation system, rapid generation and switching of virtual scenarios is achieved, significantly shortening the testing cycle and enabling accelerated, large-scale, and enhanced testing. This reduces the potential for dangerous situations during real-world vehicle testing, improves testing efficiency and diversifies traffic scenario simulation, while also reducing testing costs and risks.

[0069] 2. Modeling the physical properties of raindrops

[0070] By modeling detailed raindrop physics, including its generation, dynamic behavior, and visual effects, the system more realistically simulates the impact of rainy environments on autonomous driving sensors. Complex physical models and optical effect simulations enhance the realism of test scenarios. Dynamic adjustment of raindrop physical properties based on varying environmental conditions (such as rainfall intensity and wind speed) improves the model's adaptability. A complex wind field model is introduced to simulate raindrop motion under varying wind speeds and directions. The dynamic evolution of surface water is incorporated to simulate the accumulation of water after raindrops reach the ground. A spatial segmentation algorithm is used to perform collision detection on all raindrop pairs, improving computational efficiency. The discrete element method and smoothed particle fluid dynamics are combined to simulate the dynamic behavior of raindrops in the air. Visual rendering of the water splash effect enhances the visualization of the simulation results and provides a comprehensive description of the physical properties of raindrops.

[0071] 3. Modeling the physical impact of sensors

[0072] By accurately simulating the dynamic interference of sensors in rainy environments, the accuracy of test results is improved. Comprehensive sensor physical impact modeling is provided. The physical impact of sensors is updated in real time, improving the real-time performance and response speed of the test. Ray tracing technology is used to trace the path of each laser beam, calculate its intersection with raindrops, and simulate the scattering effect. Optical effects and wet effects are introduced to simulate the impact of raindrops on camera images. The ITU-R model is used to fit the rainfall intensity to simulate the attenuation and scattering of millimeter-wave radar signals. The interference of ultrasonic sensors in rainy environments is simulated to improve the accuracy of the model. The physical impact of sensors is updated in real time through the dynamic interference unit, improving the adaptability of the model. By simulating the dynamic interference of sensors in rainy environments, sensor performance can be more accurately evaluated. BRIEF DESCRIPTION OF THE DRAWINGS

[0073] Figure 1 A schematic diagram of the steps of a virtual-reality combined vehicle automatic driving testing method of the present invention;

[0074] Figure 2 This is a structural diagram of a virtual-reality combined automobile automatic driving testing method of the present invention. DETAILED DESCRIPTION

[0075] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0076] Example 1

[0077] See also Figure 1 As shown, the virtual-reality combined vehicle autonomous driving testing method described in this embodiment includes:

[0078] The discrepancy between simulation and reality is one of the core challenges in virtual-reality testing of autonomous driving. This discrepancy directly impacts the reliability of test results and the performance of autonomous driving systems in the real world. Real-world weather, lighting, road conditions, and other factors are highly diverse and dynamically changing, while simulation environments are typically based on limited assumptions and models. This means that virtual simulation environments cannot fully replicate the complexity of the real world. Simulated sensor models (such as lidar and cameras) may be overly idealized and unable to accurately simulate real-world errors or interference. Specifically, autonomous driving systems rely on a variety of sensors (such as lidar, cameras, millimeter-wave radar, and ultrasonic sensors) to perceive the environment. In virtual simulations, the behavior of these sensors is often based on theoretical models, ignoring the various errors and interferences that occur in the real world.

[0079] Rainy days are the most common weather conditions in life and are also the most common weather scenarios encountered during autonomous driving. If the test can be passed on rainy days, more than half of the entire test content has been completed. Therefore, this application mainly focuses on the test of autonomous driving in rainy days.

[0080] Rain not only alters the physical environment (such as road friction and lighting conditions) but also directly impacts sensor performance. Rain can cause camera blur and sparse LiDAR point clouds. Simulations may simply simulate the effects of rainfall (for example, by adding raindrops to renderings), but fail to accurately simulate the raindrops' interference with the LiDAR point cloud or their occlusion of the camera image. This "double simplification" (physical environment simplification + sensor model idealization) can result in systems that perform well in simulation but completely fail in real-world rainy conditions.

[0081] LiDAR can generate erroneous data under specular reflections (such as on slippery roads). In reality, multi-sensor fusion must comprehensively consider changes in the physical environment and sensor performance. For example, in rainy weather, LiDAR's perception range may be significantly reduced, cameras may fail, and millimeter-wave radar may become the primary reliance. However, simulation environments often assume that all sensor data is perfectly synchronized and fail to accurately simulate the specific impact of fog on individual sensor data. As a result, systems can easily achieve multi-sensor fusion in simulation, but may fail in real-world environments due to inconsistent or missing data.

[0082] Therefore, the interaction between the overly idealized sensor models in simulation and the simplified physical environment can lead to significant discrepancies between the performance of autonomous driving systems in simulation and in real-world conditions. This discrepancy not only increases the difficulty of development and testing but can also lead to safety hazards in the real world. For example, a system may pass all tests in simulation but cause an accident in real-world rainy conditions due to perception errors or poor decision-making.

[0083] A virtual-reality vehicle autonomous driving testing method includes deploying a sensor suite (e.g., lidar, camera, millimeter-wave radar, ultrasonic sensor, IMU, GPS, etc.) and an autonomous driving system on a test vehicle, connecting the vehicle's sensor suite and autonomous driving system to a virtual twin center using a CAN bus or Ethernet interface, running the autonomous driving system, and generating performance reports.

[0084] The vehicle's sensor suite and autonomous driving system are connected to the virtual twin center via the CAN bus or Ethernet interface, enabling integrated and synchronous operation of the vehicle, cloud, and field.

[0085] The virtual twin center is designed based on simulation platforms (such as CARLA, AirSim, and LGSVL Simulator, which already integrate some sensor and environment modeling capabilities and can be expanded upon). This includes modeling the physical properties of raindrops and the physical impact of sensors. Twin virtual test scenarios are preset based on real environments, and these models are imported into the twin virtual test scenarios for pre-training.

[0086] Through preset quantitative indicators and real-world environment data, detailed performance reports are generated to provide data support for the optimization of the autonomous driving system.

[0087] The physical characteristics of raindrops are modeled for raindrop generation, including initializing raindrop properties, setting raindrop transparency based on real-world environmental data, introducing complex wind field models and the dynamic evolution of surface water, using a spatial segmentation algorithm to perform collision detection on all raindrop pairs, simulating the dynamic behavior of raindrops in the air based on discrete element methods and smoothed particle fluid dynamics, and visually rendering the water splash effect.

[0088] The physical impact modeling of the sensor includes the sensor dynamic interference unit, which iteratively updates the outputs of the lidar point cloud scattering modeling, camera image blur modeling, millimeter wave radar signal attenuation and scattering modeling, and ultrasonic sensor interference modeling under the influence of raindrop physical characteristics modeling.

[0089] Modeling of raindrop physics includes:

[0090] Provide a parameterized interface (for example, develop a GUI interface in the simulation platform, such as Unreal Engine's UMG or Unity's UI system), where users can set evolution parameters through input boxes. The evolution parameters include rainfall intensity, wind speed, wind direction, and real environmental data used for testing;

[0091] The Monte Carlo method is used to sample the Marshall-Palmer distribution to generate the initial diameter of raindrops. The sampling range is 0.1 mm to 10 mm (covering the range from small raindrops to large raindrops).

[0092] In the present invention, it is preset;

[0093] Light rain (1 mm / h): Raindrop diameters are mainly concentrated below 0.5 mm.

[0094] Moderate rain (10 mm / h): Raindrop diameters are mainly concentrated around 1 mm.

[0095] Heavy rain (50 mm / h): Raindrop diameters range widely, and there may be more large raindrops larger than 3 mm.

[0096] Based on the rainfall intensity set by the user, use Python to calculate the volumetric microdensity of raindrops (number of raindrops per cubic meter); the total number of raindrops , is the rainfall intensity, is the volume of the test scene, is a constant representing the reference raindrop density.

[0097] If the initial diameter of the raindrop is less than 1 mm, Stokes' law is used to calculate the terminal velocity. If the initial diameter of the raindrop is ≥ 1 mm, the Beard model is used to calculate the terminal velocity. The terminal velocity is stored as a property of the raindrop for subsequent dynamic simulations.

[0098] Use the numerical calculation module of MATLAB or Python to implement speed calculation, and specifically use NumPy for vectorized calculation.

[0099] Set the transparency of raindrops based on real environment data;

[0100] Assign an initial position and initial velocity to each raindrop. For example, randomly generate a position (x_0, y_0, z_0) at the upper boundary of the simulation scene (height H), where (z_0 = H) and (x_0, y_0) are uniformly distributed in the horizontal plane. The initial velocity is usually set to zero or initialized to downward motion according to the terminal velocity.

[0101] A complex wind field model is designed based on the actual test environment and added to the raindrop physical property modeling. In each time step, the gravity, air resistance, and wind force of the raindrop are calculated. Subsequently, the velocity and position of the raindrop are updated using a numerical integration method. If the raindrop reaches the ground or other surface, the raindrop is removed and the splash effect is triggered. Otherwise, the update continues, and the operation within the iterative time step is repeated until the preset number of iterations or time is reached.

[0102] Use Python's NumPy and SciPy libraries for numerical integration, or accelerate motion simulations using physics engines such as PhysX or BulletPhysics.

[0103] At each time step, a spatial segmentation algorithm (such as octree or grid partitioning) is used to perform collision detection on all raindrop pairs. The dynamic behavior of raindrops in the air is simulated based on the discrete element method and smoothed particle fluid dynamics method. This includes: defining a collision detection range for each raindrop, namely a sphere with the raindrop diameter D as its diameter. When the distance between the centers of two raindrops is less than the sum of the two raindrop radii (that is, when the spheres of the two raindrops intersect), a collision is determined. Based on the magnitude and direction of the relative velocity, the two raindrops are determined to merge or split. If the relative velocity of the two raindrops is less than a preset threshold (small velocity difference), they merge into a larger raindrop, and the new raindrop diameter satisfies the conservation of volume and the velocity satisfies the conservation of momentum. If the relative velocity of the two raindrops is greater than or equal to the preset threshold (large velocity difference between the two raindrops), they randomly split into several smaller raindrops. The size and number of the raindrops after splitting conform to the Marshall-Palmer distribution, and the total volume of the raindrops is conserved. The raindrop properties (size, velocity, position) are updated, and the newly generated raindrops are added to the raindrop physical property modeling.

[0104] Methods for calculating the weight, air resistance, and wind force on raindrops include:

[0105] Raindrops will randomly absorb particulate impurities in the air during the falling process, causing the mass of the raindrops to increase, thereby changing gravity. The mass of the absorbed particles depends on the concentration of particulate matter in the air. , the surface area of ​​the raindrop (because adsorption mainly occurs on the surface of the raindrop), the time the raindrop is exposed to the air, and the adsorption efficiency (which indicates the ability of the raindrop to adsorb particulate matter, which is related to the size of the raindrop, the type of particulate matter, etc.).

[0106] For gravity, since the adsorption process is random, randomness can be introduced to simulate the mass of adsorbed particles. The specific method is: in each time step, a random number ξ~(0,1) is generated, where ξ represents the chance of adsorption. If ξ< , then adsorption occurs, and the adsorption mass is Δm, otherwise no adsorption occurs. is the adsorption probability, , is the probability adjustment coefficient, which requires experimental calibration, Indicates the concentration of particulate matter in the air. is the surface area of ​​the raindrop; the mass of the raindrop is iteratively updated as the sum of the mass of the previous time step and the adsorption mass, and the gravity of the raindrop is calculated based on the updated mass;

[0107] Air pollution (such as the concentration of particulate matter such as PM2.5 and PM10) may change the density, dynamic viscosity and drag coefficient of the air, thereby affecting air resistance.

[0108] For air resistance, the standard density of air is preset based on historical data or literature. , standard dynamic viscosity and the standard drag coefficient , adjusted air density , is the coefficient of influence of particulate matter on air density, in kg / μg, and the adjusted dynamic viscosity , is the coefficient of influence of particles on viscosity, and the adjusted resistance coefficient , is the influence coefficient of particulate matter on the drag coefficient. If the initial diameter of the raindrop is less than 1 mm, the Stokes law is used to calculate the air resistance; if the initial diameter of the raindrop is ≥ 1 mm, the Beard model is used to calculate the air resistance.

[0109] For wind forces, CFD software (such as ANSYS Fluent or OpenFOAM) is used to generate three-dimensional wind field data. This includes: defining the geometric model of the simulation scenario based on the actual test environment (e.g., city streets, open terrain), presetting wind speed boundary conditions (e.g., inlet wind speed, wind direction, and turbulence intensity), generating a wind speed vector field by solving the Navier-Stokes equations, and importing this into the raindrop physical property modeling. The wind speed at each raindrop position is interpolated and calculated, thereby adding wind forces to the raindrop motion. A complex wind field model is introduced using ANSYS Fluent and OpenFOAM to generate three-dimensional wind field data, and the impact of wind on raindrop trajectories is simulated in the simulation platform.

[0110] The transparency of raindrops is set based on real environment data. The transparency of raindrops is mainly determined by the optical interaction between the particles (such as PM2.5 and dust) adsorbed by them and the background environment. This includes the effect of particle adsorption and the effect of ambient light interaction. The particle adsorption effect is set to 、 , [PM] and Linear representation of, for example, transparency under the influence of particle adsorption , is the transparency adjustment coefficient, [PM] is the local PM concentration in the area where the raindrops are located, represents the number of particles in the raindrops at each time step, is the surface area of ​​the raindrop; particle adsorption influences transparency attenuation in an exponential manner (e.g., an extension of the Lambert-Beer law);

[0111] When raindrops overlap with background objects (such as buildings and vehicles), the transparency under the influence of ambient light intersection is modulated by the transmittance γ of the occluder. For example, the transparency under the influence of ambient light intersection is G=1-(1-γ)⋅Occlusion, where Occlusion is a binary occlusion factor (1 means occluded and 0 means no occlusion) and γ is the transmittance of the occluder (such as glass γ≈0.9 and concrete γ≈0.2). Moving light sources (such as car lights) will locally increase the transmittance of raindrops. Introducing light sources can correct the influence of ambient light intersection. For example, corrected ambient light intersection influence = ambient light intersection influence + β×I light / r 2 , I light is the light source intensity (unit: candela), r is the distance from the raindrop to the light source, and β is the light source-raindrop interaction coefficient (experimentally calibrated);

[0112] The transparency under the influence of corrected ambient light intersection is combined with the transparency under the influence of particle adsorption (e.g., weighted summation or linear accumulation) to obtain the transparency of raindrops in each time step.

[0113] The splash effect produced by raindrops impacting various surfaces (such as asphalt, dirt, and water) can blur the sensor to some extent, depending on the shape and height of the splash. Specifically, raindrops create different splash patterns, including crown-shaped splashes perpendicular to the surface and droplet-shaped splashes that quickly break away from the surface. These splash patterns can affect the sensor's field of view, resulting in irregular blurred areas in the image. Furthermore, the splash height depends on the impact velocity, liquid film thickness, and surface characteristics. A high splash height can cause the droplets to hit the sensor lens, leaving residual droplets and further affecting image quality.

[0114] Furthermore, different surface materials can have different effects. For example, when raindrops strike asphalt, they create a higher splash height and larger water splashes, potentially leading to greater optical blur and increased noise. Soil, with its high water absorption, may reduce the splash height but increase the spread of the splashes, causing more extensive blur. When raindrops strike water, they create complex splash patterns, including crown-shaped and droplet-shaped ones. These splashes can potentially splash onto the sensor lens, leaving residual droplets and affecting image quality. Therefore, splashes are a significant factor affecting sensor quality. However, these factors are generally not accounted for in virtual simulation environments, resulting in overly idealized sensor data and a discrepancy between the real sensor and the real world.

[0115] The visual rendering method of the water splash effect includes:

[0116] Set the location of the raindrop when it reaches the ground or other surface as the impact point, the size as the initial bonding size, and the mass as the original mass. Use raycast to determine whether the raindrop collides with the ground or other surface. Use the terminal velocity as the initial splash velocity, and preset the raindrop life cycle (e.g., 0.2-1 second).

[0117] The preset materials for reaching the ground or other object surfaces are smooth and hard materials (such as asphalt), non-smooth medium-hard materials (such as soil), and liquid materials (such as water surface); the material correlation coefficient is preset for each material;

[0118] Calculate the impact energy based on the original mass of the raindrop and the initial splash velocity , and record the normal direction of the collision point (for subsequent splash direction calculation); according to the impact energy Generate water splash particle clusters , is the number of particles per cluster, Represents the material correlation coefficient, which is used to describe the particle generation efficiency of different material surfaces. The material correlation coefficients of different material surfaces are determined based on experimental data or literature. For example, the material correlation coefficient of asphalt is 3; the material correlation coefficient of soil is 5, and the material correlation coefficient of water surface is 8;

[0119] Simulate evaporation effects based on temperature and humidity to decay the mass of water particles; for example, combine temperature and humidity to create a linear formula to calculate the decay update of water particles' mass.

[0120] The splash direction of water particles is set to be along the ground normal direction as the main velocity component, and the random horizontal deflection angle forms the tangential direction. The splash direction is obtained by combining the main velocity component and the tangential direction. A scene is added in which a car drives over the water surface, and a directional spray is triggered in this scene. The initial splash velocity is positively correlated with the vehicle speed.

[0121] Use the same method as raindrops to iteratively calculate the gravity, air resistance, and wind force of the water droplets, and update the speed and position;

[0122] At the time step, the speed, direction and position of the water splash particles are iteratively updated to generate the water splash effect.

[0123] Rainfall changes the reflective properties of the ground, and the ground's reflective parameters are updated synchronously in the material modeling module. For example, asphalt pavement will increase specular reflection when it becomes wet.

[0124] The dynamic evolution mechanism of surface water is introduced into the modeling of raindrop physical characteristics. The methods include:

[0125] For each material reaching the ground or other surface, a preset moisture saturation limit is set (i.e., the material stops absorbing water after reaching the moisture saturation limit and begins to accumulate water). At each time step, the number of raindrops per unit material is updated in combination with the evaporation effect. The material's specular reflection is set to be positively correlated with the number of raindrops. When the number of raindrops per unit material reaches the moisture saturation limit, the material's specular reflection is set to reach its limit and no longer change.

[0126] The drainage capacity of the rainwater wells was preset according to the actual environment used for testing. The depression water level capacity curve method was used to simulate the depression water accumulation process based on the number of raindrops reaching the ground at each time step. When the intensity of rainstorm runoff exceeded the drainage capacity of the rainwater wells, the runoff overflowed from the rainwater wells in the pipe network to the low-lying areas on the urban surface, forming water accumulation.

[0127] Starting from the depression water depth of 0, the water depth of the depression is iteratively updated at each time step, and the surface area of ​​the water in each depression is obtained. The mirror reflection of the corresponding material of the water surface is used for the water area; based on the relationship between the different water levels and the corresponding water volume of each depression, the water level capacity curve of each depression is obtained by interpolation or fitting and stored as a depression attribute.

[0128] Modeling the physical impact of raindrops on sensors aims to simulate the impact of raindrops on lidar point clouds and camera images. This involves simulating the impact of raindrops on multiple sensors in autonomous driving systems (lidar, cameras, millimeter-wave radar, and ultrasonic sensors) and generating realistic sensor data for testing and verifying the robustness of autonomous driving algorithms. Specific goals include:

[0129] 1. Simulate the scattering and noise interference of raindrops on lidar point clouds.

[0130] 2. Simulate the occlusion, blurring, and optical effects (such as refraction, reflection, and rainbow effects) of raindrops on camera images.

[0131] 3. Simulate the impact of raindrops on the ground reflection characteristics.

[0132] 4. Simulate the attenuation and scattering of millimeter-wave radar signals by raindrops.

[0133] 5. Simulate the blocking and scattering of ultrasonic sensor signals by raindrops.

[0134] Specific design contents include:

[0135] Preset the optical, electromagnetic, and acoustic properties of the raindrop material. For the optical properties, you can set the refractive index of the raindrop to 1.33 (the refractive index of water) and the absorption coefficient to approximately 0 (assuming pure water). For the electromagnetic properties, the dielectric constant of the raindrop can be set to the dielectric constant of water (about 80) for millimeter-wave radar modeling. For the acoustic properties, the acoustic impedance of the raindrop can be set close to the acoustic impedance of water for ultrasonic sensor modeling.

[0136] The sensor dynamic interference unit includes lidar point cloud scattering modeling, camera image blur modeling, millimeter wave radar signal attenuation and scattering modeling, and ultrasonic sensor interference modeling;

[0137] Specifically:

[0138] At each time step, the intersection point between the laser beam and the raindrop is iteratively calculated, and the noise points and intensity attenuation in the point cloud are updated;

[0139] Iteratively updates the position and shape of water droplets on the camera, recalculates occlusion, blur, and optical effects, and generates dynamic image sequences;

[0140] Iteratively updates the intersection point of the millimeter-wave beam and raindrops, recalculates attenuation and scattering effects, and generates dynamic radar signals;

[0141] The intersection point between the ultrasonic beam and the raindrops is iteratively updated, the occlusion and scattering effects are recalculated, and a dynamic echo signal is generated.

[0142] LiDAR point cloud scattering modeling;

[0143] Mie scattering theory is used to calculate the scattering effect between a laser beam and raindrops. Mie scattering is applicable when the raindrop size is close to the laser wavelength (approximately 905 nm or 1550 nm).

[0144] In the LiDAR point cloud generation module, a raindrop scattering model is added. For each laser beam, its intersection with a raindrop is calculated, and the scattering probability and intensity are calculated based on Mie scattering theory. If the laser beam is scattered, a noise point is added to the point cloud or the intensity of the reflection point is reduced.

[0145] Use ray tracing technology to trace the path of each laser beam and calculate its intersection with the raindrop. To improve efficiency, the voxel grid method can be used to divide the space into voxels. Scattering is calculated only in the voxels containing raindrops. The scattering probability at the intersection is determined by the Mie scattering cross section, and the scattering direction is determined by the Mie scattering phase function. The phase function can be calculated by numerical methods (such as the T-matrix method) or using a pre-calculated lookup table.

[0146] If the laser beam is scattered, a noise point is added to the point cloud. The position of the noise point is determined by the scattering direction and intensity. If the laser beam is not completely scattered, its intensity decays exponentially.

[0147] Camera image blur modeling;

[0148] Project raindrops onto the camera lens. The projected shape is the same as the raindrop shape, for example, a circle or an ellipse. The area is related to the raindrop size and the lens focal length. The transparency of the raindrop is obtained. A Gaussian blur filter is applied to the area of ​​the image obscured by the raindrop. The blur radius is proportional to the raindrop size to obtain the occlusion and blurring effect of the raindrop on the lens.

[0149] A linear formula is used to express the relationship between the gravity of raindrops, wind force, surface friction of the lens, and the dynamic movement of raindrops on the lens. At each time step, the position and shape of the raindrops are updated, and the occlusion and blur effects are recalculated.

[0150] Millimeter-wave radar signal attenuation and scattering modeling;

[0151] When millimeter waves (24 GHz or 77 GHz) propagate through raindrops, they are absorbed and scattered. The attenuation coefficient is related to the raindrop size, density, and wavelength and can be calculated using Rayleigh scattering or Mie scattering theory.

[0152] For each mmWave beam, ray tracing techniques are used to calculate its intersection with raindrops to determine attenuation and scattering effects.

[0153] The attenuation coefficient and scattering cross section were obtained by fitting the rainfall intensity using the ITU-R model;

[0154] The scattering direction of millimeter waves is determined by the Mie scattering phase function. Since the wavelength of millimeter waves (approximately 4 mm or 12.5 mm) is close to the size of raindrops, the scattering effect is significant. The scattering direction can be calculated by numerical methods (such as the T-matrix method) or using a pre-calculated lookup table.

[0155] The scattered millimeter-wave signal appears as noise or a false target in the radar receiver, which in turn affects the millimeter-wave radar signal. The position of the false target is determined by the scattering direction and intensity, and the intensity is determined by the scattering cross section.

[0156] Ultrasonic sensor interference modeling;

[0157] Ultrasonic waves (usually at a frequency of 40kHz) are blocked and scattered by raindrops during propagation. The blocking effect is determined by the geometric blocking area of ​​the raindrops, while the scattering effect is determined by the acoustic impedance and size of the raindrops.

[0158] For each ultrasonic beam, calculate its intersection with the raindrop. If the beam is completely blocked, the echo signal strength is 0. The blocking probability is determined by the projected area of ​​the raindrop and the beam cross-sectional area.

[0159] If the initial diameter of the raindrop is less than 1mm, Rayleigh scattering is used to calculate the scattering intensity of the raindrop on the ultrasonic wave. If the initial diameter of the raindrop is ≥1mm, Born scattering is used to calculate the scattering intensity of the raindrop on the ultrasonic wave. The scattered ultrasonic signal appears as noise or false echo in the receiver. The affected ultrasonic signal is then obtained. The distance of the false echo is determined by the scattering point position, and the intensity is determined by the scattering cross section.

[0160] In camera image blur modeling, optical effects and slippery effects are introduced;

[0161] Ray tracing technology generates images by simulating the propagation of light through a scene, accurately modeling light refraction and reflection. These optical effects directly impact the clarity and quality of camera images. The rainbow effect is caused by the difference in refractive index of raindrops for different wavelengths of light. Spectral decomposition is used to calculate the refraction and reflection of incident white light within raindrops. This rainbow effect can result in colored halos or blurred areas in images, especially under strong sunlight.

[0162] Ray tracing technology is used to simulate the refraction and reflection of light by raindrops. Refraction follows Snell's law, and reflection follows Fresnel's equation.

[0163] Snell's law describes the refraction of light when it passes from one medium into another. Refraction can cause light to bend at the interface between different media, resulting in blurred or distorted areas in the image.

[0164] Fresnel's equations describe the reflection and transmission coefficients of light at the interface between two media. Reflection can cause unwanted bright spots or reflected light in an image, affecting image contrast and clarity.

[0165] For raindrops on the camera, an incident ray is generated from the light source (or ambient light in the scene), the intersection point of the ray with the raindrop surface is calculated, and the intersection position and normal vector are determined using geometric optics methods. Based on Snell's law and Fresnel's equations, the direction and intensity of the refracted and reflected rays are calculated. The ray path is recursively traced at each time step until the ray leaves the raindrop or the intensity decays below a threshold.

[0166] The rainbow effect is caused by the difference in refractive index of raindrops for light of different wavelengths. Secondary refraction and primary reflection within raindrops decompose white light into different colors, creating a rainbow. Spectral decomposition is used to decompose the incident white light into multiple monochromatic lights (ranging from 400nm to 700nm). This is because the refractive index of water varies with wavelength. Based on the dispersion effect, the refraction angle of each wavelength of incident white light within the raindrop is calculated. Ray tracing technology is used to calculate the refraction and reflection paths of each wavelength within the raindrop. The rainbow effect is then synthesized based on the angle and intensity of the outgoing light.

[0167] For the wet effect, use a bidirectional reflectance distribution function model (such as the Phong model or the Cook-Torrance model) to model the specular reflection of the material.

[0168] Methods for connecting a vehicle's sensor suite and autonomous driving system to the virtual twin center using a CAN bus or Ethernet interface, running the autonomous driving system, and generating performance reports include:

[0169] Preset quantitative indicators for performance reporting (e.g., key performance indicators, perception accuracy, decision success rate, and safety) and a real-world environment for testing;

[0170] Use digital twin technology to virtually construct the real environment used for testing to produce a virtual twin center. Use real-world data to pre-train the raindrop physical property modeling until the similarity between the raindrop physical property modeling and the real-world data reaches a threshold. Use data from the sensor suite used in autonomous driving to pre-train the sensor physical impact modeling until the expected level is achieved, thus forming a fully trained virtual twin center.

[0171] Use the CAN bus or Ethernet interface to connect the car's sensor kit and autonomous driving system to the virtual twin center, run the autonomous driving system, record quantitative indicator data, generate performance reports, and complete the evaluation of autonomous driving vehicles under the combination of virtual and real. This not only allows for the setting of diverse environmental scenarios, but also saves test space, thereby saving car companies' costs and improving economic benefits.

[0172] Example 2

[0173] See also Figure 2 As shown, for the parts not described in detail in this embodiment, please refer to the description of Example 1. A virtual and real combined automobile automatic driving test system is provided, including:

[0174] Pre-preparation module: used to deploy the sensor suite and autonomous driving system on the test car, and connect the car's sensor suite and autonomous driving system to the virtual twin center using the CAN bus or Ethernet interface;

[0175] Virtual Twin Center Design Module: Expands the design of raindrop physical characteristics modeling and sensor physical impact modeling based on the simulation platform;

[0176] Pre-training module: Presets a twin virtual test scenario based on the real environment, imports the modeling of raindrop physical characteristics and sensor physical impact into the twin virtual test scenario, and performs pre-training;

[0177] Evaluation output module: Runs the autonomous driving system in the virtual twin center, records quantitative indicator data, and outputs performance reports.

[0178] The virtual-reality hybrid autonomous driving testing method achieves efficient, safe, and comprehensive autonomous driving testing by integrating a real-world test site environment with a cloud-based twin simulation system. Detailed modeling of raindrop physics and sensor physical impacts enhances the realism of test scenarios and the accuracy of sensor performance evaluation. Connecting the vehicle and virtual twin center via a CAN bus or Ethernet interface enables integrated, synchronized vehicle-cloud-field operation, improving the real-time and responsiveness of testing. Finally, detailed performance reports are generated, providing data support for optimizing the autonomous driving system. This not only improves testing efficiency and safety, but also provides strong support for the development of autonomous driving technology.

[0179] Example 3

[0180] This embodiment discloses an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the operation mode of the above-mentioned virtual-reality combined automobile autonomous driving testing method is implemented.

[0181] Since the electronic device described in this embodiment is an electronic device used to implement a virtual-reality combined automatic driving vehicle testing method in the embodiment of this application, based on the virtual-reality combined automatic driving vehicle testing method described in the embodiment of this application, those skilled in the art will be able to understand the specific implementation of the electronic device of this embodiment and its various variations, so how the electronic device implements the method in the embodiment of this application will not be described in detail here. As long as those skilled in the art can implement the electronic device used in the virtual-reality combined automatic driving vehicle testing method in the embodiment of this application, it falls within the scope of protection of this application.

[0182] The above formulas are all dimensionless and numerical calculations. The formulas are obtained by collecting a large amount of data and performing software simulation to obtain the most recent real situation. The preset parameters and thresholds in the formulas are set by technicians in this field according to actual conditions.

[0183] The above description is merely a preferred embodiment of the present invention. The scope of protection of the present invention is not limited to the above embodiment. All technical solutions based on the principles of the present invention are within the scope of protection of the present invention. It should be noted that for users of ordinary skill in the art, various improvements and modifications that do not depart from the principles of the present invention should also be considered within the scope of protection of the present invention.

Claims

1. A virtual-real combined vehicle automatic driving test method, characterized in that: include: Deploy the sensor suite and autonomous driving system on the test vehicle, connect the vehicle's sensor suite and autonomous driving system to the virtual twin center using the CAN bus or Ethernet interface, run the autonomous driving system, and generate performance reports; The virtual twin center is designed based on an extended simulation platform, including modeling of raindrop physical properties and sensor physical impacts. Twin virtual test scenarios are preset based on real environments, and these models are imported into the twin virtual test scenarios for pre-training. The physical characteristics of raindrops are modeled for raindrop generation, including: providing a parameterized interface where users can set evolution parameters through input boxes. The evolution parameters include rainfall intensity, wind speed, wind direction, and real environment data for testing; The Monte Carlo method is used to sample the Marshall-Palmer distribution to generate the initial diameter of raindrops, with a sampling range of 0.1 mm to 10 mm; Based on the rainfall intensity set by the user, the volumetric microscopic density of raindrops is calculated using Python; If the initial diameter of the raindrop is less than 1 mm, Stokes' law is used to calculate the terminal velocity. If the initial diameter of the raindrop is ≥ 1 mm, the Beard model is used to calculate the terminal velocity. The terminal velocity is stored as a property of the raindrop. Set the transparency of raindrops based on real environment data; Assign an initial position and initial velocity to each raindrop; A complex wind field model is designed based on the actual test environment and added to the raindrop physical property modeling. In each time step, the gravity, air resistance, and wind force of the raindrop are calculated. Subsequently, the velocity and position of the raindrop are updated using a numerical integration method. If the raindrop reaches the ground or other surface, the raindrop is removed and the splash effect is triggered. Otherwise, the update continues, and the operation within the iterative time step is repeated until the preset number of iterations or time is reached. At each time step, a spatial segmentation algorithm is used to perform collision detection on all raindrop pairs. The dynamic behavior of raindrops in the air is simulated based on the discrete element method and smoothed particle fluid dynamics method. This includes: defining a collision detection range for each raindrop. When the distance between the centers of two raindrops is less than the sum of the two raindrop radii, a collision is determined. Merger or splitting is determined based on the magnitude and direction of the relative velocity. If the relative velocity of the two raindrops is less than a preset threshold, they are merged into a larger raindrop, and the new raindrop diameter satisfies the conservation of volume and the velocity satisfies the conservation of momentum. If the relative velocity of the two raindrops is greater than or equal to the preset threshold, they are randomly split into several small raindrops. The size and number of the raindrops after splitting conform to the Marshall-Palmer distribution, and the total volume of the raindrops is conserved. The properties of the raindrops are updated, and the newly generated raindrops are added to the raindrop physical property modeling. The physical impact modeling of the sensor includes the sensor dynamic interference unit, which iteratively updates the outputs of the lidar point cloud scattering modeling, camera image blur modeling, millimeter wave radar signal attenuation and scattering modeling, and ultrasonic sensor interference modeling under the influence of raindrop physical characteristics modeling.

2. The method for testing an autonomous driving vehicle using a combination of virtual and real technology according to claim 1, wherein: The method for calculating the gravity, air resistance and wind force of raindrops includes: For gravity, within each time step, a random number ξ ∼ (0, 1) is generated. ξ represents the chance of adsorption occurring. If ξ < P, adsorption occurs with an adsorption mass of Δm; otherwise, no adsorption occurs. P is the adsorption probability, and P = k4 × C PM × A, where k4 is the probability adjustment coefficient, and C PM represents the concentration of particulate matter in the air, and A is the surface area of the raindrop. The mass of the raindrop is iteratively updated as the sum of the mass in the previous time step and the adsorption mass, and the gravity of the raindrop is calculated based on the updated mass. For air resistance, the standard density of air ρ is preset based on historical data or literature. 0 , standard dynamic viscosity μ 0 and the standard drag coefficient Adjusted air density ρ t =ρ 0 +k1×C PM , k1 is the coefficient of influence of particles on air density, and the adjusted dynamic viscosity μ t =μ 0 ×(1+k2×C PM ), k2 is the coefficient of influence of particles on viscosity, and the adjusted resistance coefficient φ t =φ 0 ×(1+k3×C PM ), k3 is the influence coefficient of particulate matter on the drag coefficient. If the initial diameter of the raindrop is less than 1 mm, the Stokes law is used to calculate the air resistance; if the initial diameter of the raindrop is ≥ 1 mm, the Beard model is used to calculate the air resistance; For wind forces, CFD software is used to generate three-dimensional wind field data, including: defining the geometric model of the simulation scene according to the actual test environment, presetting the wind speed boundary conditions, generating the wind speed vector field by solving the Navier-Stokes equation, and importing it into the raindrop physical characteristics modeling. The wind speed at each raindrop position is interpolated and calculated, and the wind force is added during the raindrop movement.

3. The method for testing an autonomous driving vehicle using a combination of virtual and real technology according to claim 2, wherein: The transparency of raindrops is set based on real environment data, including the effect of particle adsorption and the effect of ambient light intersection, wherein the particle adsorption effect is set as S PM , A, [PM] and k5 linear representation, k5 is the transparency adjustment coefficient, [PM] is the local PM concentration in the area where the raindrops are located, S PM represents the number of particles in the raindrop at each time step, and A is the surface area of ​​the raindrop; the effect of particle adsorption follows an exponential law that affects transparency attenuation; The transparency under the influence of ambient light is modulated by the transmittance γ of the obstruction, and the introduction of light source is used to correct the influence of ambient light. The transparency under the influence of corrected ambient light intersection is combined with the transparency under the influence of particle adsorption to obtain the transparency of raindrops in each time step.

4. The method for testing an autonomous driving vehicle using a combination of virtual and real technology according to claim 3, wherein: The visual rendering method of the water splash effect includes: Set the position of the raindrop when it reaches the ground or other surface as the impact point, the size as the initial bonding size, the mass as the original mass, the terminal velocity as the initial splash velocity, and preset the life cycle of the raindrop; The preset materials reaching the ground or other object surfaces are smooth hard material, non-smooth medium hard material and liquid material; the material correlation coefficient is preset for each material; Calculate the impact energy E based on the original mass of the raindrop and the initial splash velocity z , and record the normal direction of the collision point; according to the impact energy E z Generate water splash particle clusters LP is the number of particles per cluster, h represents the material correlation coefficient; The mass attenuation of water droplets is simulated by evaporation effect according to temperature and humidity; The splash direction of water particles is set to be along the ground normal direction as the main velocity component, and the random horizontal deflection angle forms the tangential direction. The splash direction is obtained by combining the main velocity component and the tangential direction. A scene is added in which a car drives over the water surface, and a directional spray is triggered in this scene. The initial splash velocity is positively correlated with the vehicle speed. Use the same method as raindrops to iteratively calculate the gravity, air resistance, and wind force of the water droplets, and update the speed and position; At the time step, the speed, direction and position of the water splash particles are iteratively updated to generate the water splash effect.

5. The method for testing an autonomous driving vehicle using a combination of virtual and real technology according to claim 4, wherein: The dynamic evolution mechanism of surface water is introduced into the modeling of raindrop physical characteristics, including: A humidity saturation limit is set for each material reaching the ground or other surface. At each time step, the number of raindrops per unit material is updated in combination with the evaporation effect. The specular reflection of the material is set to be positively correlated with the number of raindrops. When the number of raindrops per unit material reaches the humidity saturation limit, the specular reflection of the material is set to reach its limit and no longer change. The drainage capacity of the rainwater well is preset according to the actual environment used for testing. The depression water level capacity curve method is used to simulate the depression water accumulation process based on the number of raindrops reaching the ground at each time step. Starting from the depression water depth of 0, iteratively update the water depth of the depression at each time step, and obtain the surface area of ​​water in each depression. The mirror reflection of the corresponding material of the water surface is used on the water area.

6. The method for testing an autonomous driving vehicle using a combination of virtual and real technology according to claim 5, wherein: Modeling the physical impact of the sensor, the method comprising: Preset optical, electromagnetic, and acoustic properties of raindrop materials; At each time step, the intersection point between the laser beam and the raindrop is iteratively calculated, and the noise points and intensity attenuation in the point cloud are updated; Iteratively updates the position and shape of water droplets on the camera, recalculates occlusion, blur, and optical effects, and generates dynamic image sequences; Iteratively updates the intersection point of the millimeter-wave beam and raindrops, recalculates attenuation and scattering effects, and generates dynamic radar signals; The intersection point between the ultrasonic beam and the raindrops is iteratively updated, the occlusion and scattering effects are recalculated, and a dynamic echo signal is generated.

7. The method for testing an autonomous driving vehicle using a combination of virtual and real technology according to claim 6, wherein: The sensor dynamic interference unit includes: Modeling of LiDAR point cloud scattering; Using ray tracing technology, the path of each laser beam is traced and its intersection with the raindrop is calculated. The scattering probability at the intersection is determined by the Mie scattering cross section, and the scattering direction is determined by the Mie scattering phase function. If the laser beam is scattered, a noise point is added to the point cloud. The position of the noise point is determined by the scattering direction and intensity. If the laser beam is not completely scattered, its intensity decays exponentially. Modeling camera image blur; Project raindrops onto the camera lens and obtain the transparency of the raindrops. Apply a Gaussian blur filter to the areas of the image obscured by raindrops, with a blur radius proportional to the size of the raindrops, to obtain the occlusion and blurring effect of the raindrops on the lens. A linear formula is used to express the relationship between the gravity of raindrops, wind force, surface friction of the lens, and the dynamic movement of raindrops on the lens. At each time step, the position and shape of the raindrops are updated, and the occlusion and blur effects are recalculated. Modeling of millimeter-wave radar signal attenuation and scattering; For each mmWave beam, ray tracing techniques are used to calculate its intersection with raindrops to determine attenuation and scattering effects. The attenuation coefficient and scattering cross section were obtained by fitting the rainfall intensity using the ITU-R model; The scattering direction of millimeter waves is determined by the Mie scattering phase function; The scattered millimeter-wave signal will appear as noise or false targets in the radar receiver, thereby obtaining an affected millimeter-wave radar signal; Modeling interference with ultrasonic sensors; For each ultrasonic beam, calculate its intersection with the raindrop. If the beam is completely blocked, the echo signal strength is 0. The blocking probability is determined by the projected area of ​​the raindrop and the beam cross-sectional area. If the initial diameter of the raindrop is less than 1mm, Rayleigh scattering is used to calculate the scattering intensity of the raindrop on the ultrasonic wave; if the initial diameter of the raindrop is ≥1mm, Born scattering is used to calculate the scattering intensity of the raindrop on the ultrasonic wave; the scattered ultrasonic signal appears as noise or false echo in the receiver; and the affected ultrasonic signal is then obtained.

8. The method for testing an autonomous driving vehicle using a combination of virtual and real technology according to claim 7, wherein: In the camera image blur modeling, optical effect and slippery effect are introduced; For raindrops on the camera, an incident ray is generated from the light source. The intersection point of the ray with the raindrop surface is calculated. The intersection position and normal vector are determined using geometric optics methods. The direction and intensity of the refracted and reflected rays are calculated based on Snell's law and Fresnel equations. The ray path is recursively traced at each time step until the ray leaves the raindrop or the intensity decays below a threshold. The incident white light is decomposed into multiple monochromatic lights. The refraction angle of each wavelength of the incident white light inside the raindrop is calculated based on the dispersion effect. Ray tracing technology is used to calculate the refraction and reflection path of each wavelength of light inside the raindrop. The rainbow effect is synthesized based on the angle and intensity of the outgoing light. For the slippery-wet effect, the specular reflection of the material is modeled using the Bidirectional Reflectance Distribution Function model.

9. The method for testing an autonomous driving vehicle using a combination of virtual and real technology according to claim 8, wherein: The method of connecting the vehicle's sensor suite and autonomous driving system to the virtual twin center using a CAN bus or Ethernet interface, operating the autonomous driving system, and generating a performance report includes: Preset quantitative indicators for performance reports and real-world environments for testing; Use digital twin technology to virtually construct the real environment used for testing to produce a virtual twin center. Use real-world data to pre-train the raindrop physical property modeling until the similarity between the raindrop physical property modeling and the real-world data reaches a threshold. Use data from the sensor suite used in autonomous driving to pre-train the sensor physical impact modeling until the expected level is achieved, thus forming a fully trained virtual twin center. Use the CAN bus or Ethernet interface to connect the car's sensor suite and autonomous driving system to the virtual twin center, run the autonomous driving system, record data on quantitative indicators, and generate performance reports.

Images