A foggy scene generation method and system for autonomous driving simulation testing

Abstract

The present invention relates to the field of autonomous driving technology, and in particular to a method and system for generating foggy weather scenes for autonomous driving simulation testing. The method comprises the following steps: S1. According to a scene description file, an autonomous driving scene required for corresponding sensor configuration is generated in a CARLA simulator; S2. In the foggy weather scene in the scene description file, the CARLA simulator obtains the parameter configuration of the corresponding sensor of the autonomous driving main vehicle; S3. A sensor simulation model is constructed according to the parameter configuration of the corresponding sensor; S4. The autonomous driving scene is combined with the autonomous driving main vehicle to form an autonomous driving simulation test environment of the CARLA simulator, and the autonomous driving object under test is connected to the autonomous driving simulation test environment to perform in-the-loop testing. The present invention constructs an autonomous driving simulation scene based on the CARLA simulator, and innovatively designs a laser radar fog simulation algorithm based on physical modeling, which realizes the sensor simulation capability of foggy driving scenes.

Description

Technical Field

[0001] The present invention relates to the field of autonomous driving technology, and in particular to a method and system for generating foggy scenes for autonomous driving simulation testing. Background Art

[0002] With the rapid development and gradual commercialization of autonomous driving technology, its safety issues have received increasing attention. In order to ensure that the autonomous driving system can achieve stable and reliable operation in various complex and unpredictable driving scenarios, comprehensive and in-depth testing has become the key. However, traditional road testing methods are not only costly and time-consuming, but may also face safety hazards, especially in severe weather conditions, which further increases the difficulty and risk of testing. Therefore, using virtual simulation technology to verify the autonomous driving system has become an efficient and economical solution.

[0003] Fog, as a common adverse weather condition, poses a severe challenge to the perception, positioning and decision-making capabilities of autonomous driving systems. Thick fog can significantly reduce the performance of sensors. For example, the signals of lidar are easily scattered and absorbed in fog, resulting in a shortened detection range; cameras may have difficulty capturing clear images due to reduced visibility. These problems directly affect the vehicle's perception accuracy of the environment and the reliability of the system, increasing the risk of autonomous driving systems operating in foggy weather conditions.

[0004] Existing fog simulation methods are mainly based on clear weather data sets, and are implemented by superimposing fog effects in the later stage. These methods for processing lidar fog effects are generally divided into two categories: one is to simply superimpose Gaussian noise or random noise, and the other is to post-process the clear weather data after learning the data distribution of real foggy weather through a data-driven method. However, both methods have the following problems:

[0005] (1) Insufficient simulation authenticity: Since the method uses noise superposition or data-driven processing, it lacks interpretability. The generated foggy weather effect is difficult to fully reflect the sensor characteristics under actual foggy weather conditions and lacks physical accuracy.

[0006] (2) Limited scenario scalability: The generated scenario dataset is highly dependent on the original sunny day data and is difficult to flexibly expand to a variety of scenarios, which limits the scope of application of the simulation test.

[0007] (3) Poor sensor consistency: In foggy weather, the fog effect simulation of lidar and camera often cannot maintain consistency, and cannot accurately simulate the cross-perception effect between the two sensors, affecting the comprehensiveness and credibility of the test.

[0008] These problems significantly limit the application effect and applicability of existing methods in foggy weather simulation tests. Summary of the invention

[0009] The present invention provides a fog scene generation method and system for autonomous driving simulation testing, aiming to solve the problems of insufficient authenticity of existing fog simulation methods and inconsistency with camera fog concentration performance.

[0010] The present invention provides a method for generating a foggy scene for an autonomous driving simulation test, comprising the following steps:

[0011] S1. Generate the autonomous driving scenario required for the corresponding sensor configuration in the CARLA simulator according to the scenario description file;

[0012] S2. In the foggy scene in the scene description file, the CARLA simulator obtains the parameter configuration of the corresponding sensor of the autonomous driving main vehicle;

[0013] S3. Construct a sensor simulation model according to the parameter configuration of the corresponding sensor;

[0014] S4. The autonomous driving scenario is combined with the autonomous driving main vehicle to form the autonomous driving simulation test environment of the CARLA simulator. The autonomous driving test object is connected to the autonomous driving simulation test environment to perform in-the-loop testing.

[0015] In the autonomous driving simulation test environment, each frame of sensor perception data is generated according to the position of the autonomous driving test object in the autonomous driving simulation test environment and its surrounding environment; in the foggy weather scenario, for each frame of perception data during the simulation test, the construction of the sensor simulation model specifically includes:

[0016] S31. Obtain the simulated world position coordinates of the autonomous driving vehicle to which the current sensor is attached;

[0017] S32. Obtaining the surrounding environment information of the autonomous driving main vehicle;

[0018] S33. Generate perception data according to the sensor parameter configuration of the autonomous driving main vehicle and the sensor simulation model algorithm;

[0019] S34. transmitting the perception data of the current frame to the autonomous driving test object as the sensor input of the current frame;

[0020] S35. The autonomous driving test object executes the autonomous driving strategy according to the sensor input, and modifies the position of the autonomous driving main vehicle in the next frame according to the vehicle dynamics model of the CARLA simulator; this cycle is repeated.

[0021] As a further improvement of the present invention, the sensor includes a laser radar and a camera, the sensor simulation model includes a laser radar simulation model and a camera simulation model, and the perception data includes laser radar data and camera data.

[0022] As a further improvement of the present invention, for a laser radar, step S33 includes:

[0023] For each ray of light emitted by the LiDAR, the laser beam intensity P and the corresponding distance R are calculated according to the LiDAR simulation model of the foggy scene and based on the MOR of the current scene, and finally point cloud data is generated.

[0024] As a further improvement of the present invention, in the laser radar simulation model of the foggy scene, the intensity of the laser radar transmission signal is expressed as:

[0025] (1)

[0026] in, is the peak value of the signal strength, is the pulse half width;

[0027] After propagation, the laser signal is attenuated and deformed due to the propagation medium and the reflection characteristics of the target. The pulse intensity of the received signal is calculated by the following formula:

[0028] (2)

[0029] in, Indicates the distance between the target and the laser radar, represents the speed of light constant, Represents the optical system constant of the laser radar, which determines the hardware performance of the system. Represents the spatial impulse response function, which is used to describe the ability of the reflected signal strength at various locations.

[0030] As a further improvement of the present invention, in the laser radar simulation model of foggy weather scenes, in foggy weather conditions, the total signal power received by the laser radar is divided into hard target reflection power according to the medium in contact during the propagation process. and the reflected power of the medium in fog transmission Two, the formula is:

[0031] (3)

[0032] in, R Indicates the distance between the target and the laser radar;

[0033] The distance measured by the laser radar is usually based on the first reception of the reflected pulse or the reception of the strongest pulse signal, expressed as:

[0034] (4)

[0035] Hard target reflected power According to the attenuation characteristics of the signal in the fog environment, the modeling expression is directly calculated as follows:

[0036] (5)

[0037] in, represents the optical system constant of the lidar, represents the peak power of the laser pulse, represents the reflectivity of the hard target, represents the attenuation coefficient of fog, Indicates the target distance, represents the speed of light constant, Indicates the pulse half width;

[0038] Reflected power of the medium in fog transmission Modeled as:

[0039] (6)

[0040] in is a step function, represents the cross-section function, The back reflection coefficient represents the change in the reflection intensity of the incident light from the surface of an object.

[0041] Cross-Section Function It is used to define the ratio between the laser radar transmitter illumination area and the laser radar receiver receiving area, and is modeled as:

[0042] (7)

[0043] in, R 1 represents the starting distance of the overlapping area between the transmitter irradiation area and the receiver receiving area. R 2 represent the maximum distance of the overlapping area between the transmitter irradiation area and the receiver receiving area.

[0044] As a further improvement of the present invention, the laser beam intensity is calculated P And the corresponding distance R , and finally generate point cloud data, including:

[0045] For each ray, determine whether it is blocked by an object. If so, the corresponding position of the object R , object reflection coefficient , directly substitute into formulas (3)~(5) for calculation. If there is no obstacle, use formula (6) for calculation. Then, the reflection coefficient can be obtained according to the fog concentration. , substituting into the above formulas (5) and (6), we can obtain the laser radar intensity in foggy weather: and , and finally the laser intensity of the light can be obtained as:

[0046] (8)

[0047] Finally, we get the laser intensity of a ray in the foggy scene. P And the corresponding distance R , that is, the information of a point in the point cloud. This cycle is repeated and the point cloud data of the lidar is obtained through cumulative calculation.

[0048] As a further improvement of the present invention, step S33 includes:

[0049] For the camera, a convolutional neural network is used to establish a mapping relationship between fog concentration and MOR in the autonomous driving scene of the CARLA simulator to construct camera data;

[0050] For LiDAR, LiDAR data is generated according to the LiDAR simulation model based on the same MOR of the current scene.

[0051] As a further improvement of the present invention, in step S33, a mapping relationship between fog density and MOR is established in the autonomous driving scene of the CARLA simulator by using a convolutional neural network, specifically including: adjusting the Fog Density parameter in CARLA with a step size of 1, recording the camera image, and using the convolutional neural network to calculate the MOR value; conducting experiments at different map locations, repeating the experiment multiple times at each location, taking the average of the results, and then averaging the multiple pairs of<Fog Density, MOR> The data points are linearly interpolated to establish a direct relationship between fog concentration and MOR.

[0052] As a further improvement of the present invention, in step S2, the parameter configuration of the sensor corresponding to the autonomous driving main vehicle includes installation position, resolution, lidar laser beam, and number of point clouds per second.

[0053] The present invention also provides a foggy scene generation system for autonomous driving simulation testing, comprising:

[0054] Scenario generation module: used to generate the corresponding configured autonomous driving scenario in CARLA according to the scenario description file;

[0055] Sensor configuration module: used to obtain the parameter configuration of the corresponding sensor of the autonomous driving vehicle in the foggy scene of the scene description file;

[0056] Sensor simulation module: used to build a sensor simulation model according to the parameter configuration of the corresponding sensor;

[0057] Simulation test module: It is used to combine the autonomous driving scenario with the sensor simulation model to form an autonomous driving simulation test environment, connect the autonomous driving test object to the autonomous driving simulation test environment, and perform in-the-loop testing;

[0058] In foggy weather scenarios, for each frame of perception data during the simulation test, the sensor simulation module is specifically as follows:

[0059] Used to obtain the simulated world position coordinates of the autonomous driving main vehicle to which the current sensor is attached; obtain the surrounding environment information of the autonomous driving main vehicle; configure the algorithm of the sensor simulation model according to the sensor parameters of the autonomous driving main vehicle, and generate perception data; transmit the perception data of the current frame to the autonomous driving test object as the sensor input of the current frame; the autonomous driving test object executes the autonomous driving strategy according to the sensor input, and modifies the position of the autonomous driving main vehicle in the next frame according to the vehicle dynamics model; and repeats this cycle.

[0060] The beneficial effects of the present invention are as follows: based on the CARLA simulator, an autonomous driving simulation scenario is constructed, and a laser radar fog simulation algorithm based on physical modeling is innovatively designed to achieve sensor simulation capabilities for foggy driving scenarios. By implementing the algorithm in the CARLA simulator, the system is provided with high-precision simulation capabilities for autonomous driving sensors under foggy weather conditions, ultimately providing an efficient and reliable solution for autonomous driving virtual simulation testing, which can comprehensively evaluate the perception and decision-making capabilities of the autonomous driving system in foggy environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] Figure 1 It is an overall simulation flow chart of the foggy scene generation method for autonomous driving simulation testing of the present invention;

[0062] Figure 2 is a schematic diagram of the autonomous driving joint simulation in the present invention;

[0063] Figure 3 is a schematic diagram of the ratio of the transmitting and receiving areas of the laser radar sensor in the present invention;

[0064] Figure 4 It is a schematic diagram of the simulation effect of the laser radar and the camera in the present invention under the same fog concentration. DETAILED DESCRIPTION

[0065] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments.

[0066] The present invention proposes a foggy scene generation method and system for autonomous driving simulation testing, aiming to solve the key problems and deficiencies of the existing technology in foggy simulation. Through innovative design, this method can truly restore the impact of foggy weather on sensors (such as lidar and cameras) and ensure the consistency of simulation effects between different sensors. At the same time, it gets rid of the excessive dependence on sunny data sets and realizes the flexible generation of diversified scenes. This system provides a more realistic, efficient and scalable solution for foggy environment testing of autonomous driving systems, effectively improving the accuracy and comprehensiveness of simulation tests.

[0067] like Figure 1 As shown, a foggy scene generation method for autonomous driving simulation testing of the present invention comprises the following steps:

[0068] S1. Generate the autonomous driving scenario required for the corresponding sensor configuration in the CARLA simulator according to the scenario description file.

[0069] Scenario Description Language (SDL) is a standardized file used to describe scenarios in an autonomous driving simulation environment. The purpose is to reproduce certain specific traffic conditions, weather conditions, road conditions, and behaviors of traffic participants in the simulation environment. Scenario description files can be designed based on language specifications such as OpenSCENARIO or SCENIC.

[0070] S2. In the foggy scene of the scene description file, the CARLA simulator obtains the parameter configuration of the corresponding sensors of the autonomous driving main vehicle. The sensors include lidar and camera. The parameter configuration includes installation position, resolution, lidar laser beam, number of point clouds per second and other parameter settings.

[0071] S3. Construct a sensor simulation model according to the parameter configuration of the corresponding sensor, the sensor simulation model includes a lidar simulation model and a camera simulation model. When there is a foggy day parameter in the weather configuration item, the sensor simulation model of the foggy day scene of the present invention is used.

[0072] S4. The autonomous driving scenario is combined with the autonomous driving main vehicle to form the autonomous driving simulation test environment of the CARLA simulator. The autonomous driving test object is connected to the autonomous driving simulation test environment to perform in-the-loop testing.

[0073] like Figure 2As shown in the figure, the autonomous driving test object is equipped with sensors, perception modules, decision modules, and control modules. The CARLA simulator is equipped with two main simulation modules, including the autonomous driving main vehicle and the autonomous driving scene. The autonomous driving main vehicle contains a sensor simulation model and a vehicle dynamics model, and the autonomous driving scene includes map construction, simulation environment, traffic participants, etc. The autonomous driving test object and the CARLA simulator are connected through a joint simulation communication bridge layer. The test signal flow flows from the sensor simulation model of the autonomous driving main vehicle into the sensor of the autonomous driving test object, and then flows out to the vehicle dynamics model of the autonomous driving main vehicle after passing through the perception module, decision module, and control module in turn, forming an in-loop test. The map construction, simulation environment, and traffic participants constitute the virtual autonomous driving scene during the test.

[0074] Sensor data is provided by the CARLA simulator. The input of the autonomous driving test object requires sensor data, and the output is control instructions. The control instructions update the vehicle motion state based on the vehicle dynamics model in the CARLA simulator. At the next moment, the sensor's perspective will change to form new sensor data.

[0075] The vehicle dynamics model is a function provided by the CARLA simulator itself; in the sensor simulation model, the CARLA simulator itself provides the camera fog simulation capability, but does not provide the lidar fog simulation capability. Therefore, the present invention constructs a lidar fog simulation model and adjusts the consistency of the lidar and camera simulation effects through the MOR parameters.

[0076] In the autonomous driving simulation test environment, each frame of sensor (such as lidar and camera) perception data is generated according to the position of the autonomous driving test object in the autonomous driving simulation test environment and its surrounding environment; in foggy weather scenarios, for each frame of perception data during the simulation test, the construction of the lidar and camera sensor simulation model specifically includes:

[0077] S31. Obtain the simulated world position coordinates of the autonomous driving main vehicle to which the current sensor is attached.

[0078] S32. Obtain information about the environment surrounding the autonomous driving vehicle, including the location, shape, material, and other information of various assets.

[0079] S33. Generate perception data, including lidar and camera data, based on the sensor parameter configuration of the autonomous driving main vehicle and the sensor simulation model algorithm.

[0080] S34. Transmit the perception data of the current frame to the autonomous driving test object as the sensor input of the current frame.

[0081] S35. The autonomous driving object under test executes the autonomous driving strategy according to the sensor input, and modifies the position of the autonomous driving main vehicle in the next frame according to the vehicle dynamics model of the CARLA simulator; this cycle is repeated. The autonomous driving strategy refers to the autonomous driving strategy decision-making including perception, decision-making planning, control and other modules for autonomous driving by receiving sensor data such as lidar and cameras, and finally forms an autonomous driving trajectory planning for autonomous driving. The vehicle dynamics model refers to the model based on the vehicle dynamics equation used by the CARLA simulator to simulate the vehicle's motion behavior. The vehicle's motion is determined by multiple factors such as vehicle mass, tire mechanics, engine and braking system.

[0082] For the laser radar, step S33 includes: for each light ray emitted by the laser radar, according to the laser radar simulation model of the foggy scene and based on the MOR of the current scene, calculating the laser beam intensity P and the corresponding distance R, and finally generating point cloud data.

[0083] Specifically, the lidar fog simulation model and algorithm are introduced in detail below.

[0084] LiDAR simulation algorithm in foggy scenes:

[0085] LiDAR (Light Detection and Ranging) is a sensor that uses laser technology to measure and model long-distance targets. The working steps of LiDAR can be briefly divided into the following steps:

[0086] (1) Laser emission: LiDAR emits a beam of narrow wavelength laser pulses through a light source (usually a laser diode).

[0087] (2) Laser propagation: During laser propagation, reflection, scattering, and absorption may occur. When the laser encounters the surface of an object, reflection occurs and part of the beam returns to the radar. In bad weather (such as fog), the laser may be scattered or absorbed by particles or water droplets in the air, resulting in signal attenuation.

[0088] (3) Signal reception: The laser radar receiver (usually a photodetector) receives the returned laser pulse and measures the intensity and arrival time of the returned optical signal.

[0089] (4) Point cloud formation: LiDAR usually uses a rapidly rotating laser emitting device (such as a mechanical rotating or MEMS scanning mirror) to scan the environment in all directions. Information at different angles and distances is recorded and composed into point cloud data to form a three-dimensional image of the environment.

[0090] In the foggy scene lidar modeling process proposed in the present invention, the influence of fog on the lidar laser propagation is taken into account and represented by physical formulas, and finally the working process is implemented in the form of a plug-in in the CARLA simulator.

[0091] The laser pulses emitted by vehicle-mounted laser radar are generally sinusoidal pulses, and the intensity of the emitted signal can be expressed as:

[0092] (1)

[0093] in, is the peak value of the signal strength, is the pulse half-width. After propagation, the laser signal will be attenuated and deformed due to the propagation medium and the target reflection characteristics. The pulse intensity of the received signal can be calculated by the following formula:

[0094] (2)

[0095] in, Indicates the distance between the target and the laser radar, represents the speed of light constant, Represents the optical system constant of the laser radar, which determines the hardware performance of the system. represents the spatial impulse response function, which is used to describe the ability of the reflected signal strength at each location. Formula (2) indicates that the laser radar distributes the energy of the pulse signal on the propagation path With reflection attenuation function Accumulate to calculate the distance R The total power of the received signal.

[0096] In particular, in foggy weather conditions, the total signal power received by the lidar can be divided into hard target reflection power and and the reflected power in the fog medium Two, the formula can be expressed as:

[0097] (3)

[0098] According to the working principle of LiDAR, the distance measured is usually based on the first reflected pulse received or the strongest pulse signal received (generally the strongest pulse signal is used as the default configuration), which can be expressed as:

[0099] (4)

[0100] Hard target reflected power It can be directly calculated based on the attenuation characteristics of the signal in the fog environment, and its modeling expression is as follows:

[0101] (5)

[0102] in, Represents the optical system constant of the LiDAR, which determines the hardware performance of the system; represents the peak power of the laser pulse; Indicates the reflectivity of the hard target. Hard targets of different materials have different reflectivities. Indicates the attenuation coefficient of fog, which indicates the energy attenuation degree of laser signal in fog; Indicates the target distance; represents the speed of light constant; Indicates the pulse half width.

[0103] Reflected power of the medium in fog transmission Modeled as:

[0104] (6)

[0105] in is a step function; represents the cross-section function; The back reflection coefficient represents the change in the reflection intensity of the incident light on the surface of an object. Objects of different materials have different back reflection coefficients.

[0106] like Figure 2 As shown, the cross-section function , which defines the ratio between the area illuminated by the transmitter and the portion of the area received by the receiver. Figure 2 In the figure, TX stands for transmission, RX stands for reception, and the solid line and dotted line represent the boundaries of the transmission and reception light areas, respectively.

[0107] It can be modeled as:

[0108] (7)

[0109] in, R 1 and R 2 respectively represent the starting distance and the maximum distance of the overlapping area of ​​the transmitter irradiation area and the receiver receiving area. This model can better simulate and model different types of lidar geometric structures and hardware configurations.

[0110] The corresponding ray casting function in CARLA corresponds to the physical implementation of the laser propagation process. The ray casting function refers to the code implementation process of simulating the emission of light in the CARLA simulator. For each ray, it can be obtained whether the ray is blocked by an object. If an object is blocked, the corresponding position of the object R , object reflection coefficient , directly substitute into formulas (3)~(5) for calculation. If there is no obstacle, use formula (6) for calculation. Then, the reflection coefficient can be obtained according to the fog concentration. , substituting into the above formulas (5) and (6), we can obtain the laser radar intensity in foggy weather: and , and finally the laser intensity of the light can be obtained as:

[0111] (8)

[0112] The laser intensity of a ray in foggy weather is obtained by the above method. P And the corresponding distance R , which is the information corresponding to a point in the point cloud. This cycle repeats and accumulates the calculation to obtain the point cloud data of the laser radar. P Then, the laser intensity is determined inversely by formula (4): P Corresponding obstacle distance R .

[0113] The position of the object R , object reflection coefficient , fog reflectivity In the CARLA simulator, the corresponding positions of all objects in the virtual simulation world R can be determined directly. In the CARLA simulator, the object reflection coefficient , fog reflectivity For a determined constant, the corresponding value can be obtained in building a simulated autonomous driving scenario.

[0114] The present invention accurately simulates the signal propagation and reflection characteristics of LiDAR in foggy weather environment through physical modeling based on the working principle of LiDAR, taking into account the multi-dimensional influence of fog scattering, absorption, and target surface reflection characteristics. The simulation model can not only accurately describe the attenuation behavior of laser signals in foggy environment, but also generate high-precision point cloud data, providing a real and reliable simulation environment for virtual testing of autonomous driving systems.

[0115] The simulation model of the LiDAR includes modeling based on the physical working principle of the LiDAR, covering the physical modeling of the laser during emission, propagation, reception, and point cloud generation. In the process of LiDAR signal propagation, whether it encounters obstacles or not, there are corresponding mathematical models and formulas to support it. Specifically, the present invention accurately describes the propagation characteristics and signal change process of the laser in different scenarios, and combines physical principles to model reflection, scattering, and attenuation, providing a scientific basis for the high-precision generation of point cloud data. This method based on physical modeling significantly improves the authenticity and applicability of the simulation results.

[0116] Step S33 includes: for the camera, a mapping relationship between fog concentration and MOR is established in the autonomous driving scene of the CARLA simulator through a convolutional neural network to construct camera data; for the lidar, based on the same MOR of the current scene, lidar data is generated according to the lidar simulation model.

[0117] Specifically, the design of consistency between lidar and camera data in foggy scenes is introduced in detail below.

[0118] In order to ensure the consistency of lidar and camera in foggy simulation, the present invention proposes a unified standard method based on meteorological optical range (MOR). In the CARLA simulator, the fog density is controlled by the parameter Fog Density, ranging from 0 to 100. However, the relationship between Fog Density and MOR is not clearly stated. To solve this problem, the present invention adopts a convolutional neural network (CNN) to realize the mapping between fog density and minimum visible distance (MOR). In the training stage, the training is carried out by using a public real data set, which includes information such as fog density, minimum visible distance (MOR) and corresponding visual images. The CNN model automatically extracts spatial features in the image through the convolution layer, and combines the fog density data to learn the complex nonlinear relationship between fog density and MOR. In this process, the network optimizes the weights through back propagation to minimize the error between the predicted MOR value and the true value, thereby improving the accuracy of the model. In the data inference stage, given the new fog density data and visual image, the trained CNN model is calculated through forward propagation, extracts features in the input image and combines the fog density, and finally outputs the predicted minimum visible distance (MOR). This reasoning process does not involve parameter updates and can provide the autonomous driving system with real-time visibility distance estimates under different foggy weather conditions.

[0119] In the previous part, the neural network model has been trained with a real calibrated dataset to learn the correspondence between the camera visual effect and the minimum visible distance (MOR). Next, the model will be used to establish a mapping relationship between fog density and MOR in the CARLA simulation environment. Specifically, by adjusting the fog density (FogDensity) parameter in CARLA with a step size of 1, the camera image is recorded and the corresponding MOR value is calculated using the neural network model. Experiments are conducted at different map locations, and the experiment is repeated ten times at each location, and the average of the results is taken. Then, using these 100 pairs of<Fog Density,MOR> Data points, a direct mapping relationship between fog concentration and MOR is established through linear interpolation. In addition, for lidar point cloud data, MOR, as a simulation parameter of lidar, can be directly set, thus providing a reliable prediction basis for the perception ability of lidar under different foggy weather conditions.

[0120] The CARLA simulator itself provides a camera-based fog simulation model, that is, by providing the configuration of the fog density parameters (for example, FogDensity=50), a visual-based fog simulation effect can be constructed in the CARLA simulator. The CARLA simulator itself provides the ability to simulate camera fog, and the present invention supplements the fog simulation capability of the laser radar in the CARLA simulator and maintains the consistency with the camera fog simulation. That is, when the CARLA simulator obtains the fog density during the camera fog simulation, it combines the calculation process of the laser radar simulation model to obtain the laser beam intensity corresponding to the fog density. P And the corresponding distance R , and because the point cloud data is the accumulation of multiple laser beams at the same time, it can eventually form lidar point cloud data.

[0121] like Figure 3 As shown in the figure, through the above method, using MOR as a unified indicator, the consistency of the simulation effects of the lidar and the camera under the same fog concentration was successfully established.

[0122] In order to solve the problem of data consistency of multiple sensors in foggy environment, this paper designs a unified fog density evaluation standard based on meteorological optical range (MOR). Through neural network learning the mapping relationship between images and fog density in real data sets, the conversion method between fog concentration parameters and MOR in CARLA is clarified to ensure that the lidar point cloud data and camera image data have consistent perception effects under the same fog density conditions. This consistency design not only improves the reliability of multi-sensor fusion testing, but also provides important technical guarantees for the verification of autonomous driving perception algorithms under complex weather conditions.

[0123] In the implementation process, the present invention fully considers the practical problems encountered when building a fog environment in the CARLA simulator based on the simulation model, especially how to ensure the consistency of the lidar with the existing visual fog simulation effect. To this end, a systematic calibration method is adopted to adjust the correspondence between the fog concentration parameters in CARLA and the meteorological optical range (MOR), so that the lidar and the camera can achieve synchronous perception effects under the same fog conditions. Such a design effectively solves the consistency problem of multiple sensors under complex weather conditions, and provides a more realistic and reliable solution for autonomous driving simulation testing.

[0124] The present invention also provides a foggy scene generation system for autonomous driving simulation testing, comprising:

[0125] Scenario generation module: used to generate the corresponding configured autonomous driving scenario in CARLA according to the scenario description file;

[0126] Sensor configuration module: used to obtain the parameter configuration of the corresponding sensor of the autonomous driving vehicle in the foggy scene of the scene description file;

[0127] Sensor simulation module: used to build a sensor simulation model according to the parameter configuration of the corresponding sensor;

[0128] Simulation test module: It is used to combine the autonomous driving scenario with the sensor simulation model to form an autonomous driving simulation test environment, connect the autonomous driving test object to the autonomous driving simulation test environment, and perform in-the-loop testing;

[0129] In foggy weather scenarios, for each frame of perception data during the simulation test, the sensor simulation module is specifically as follows:

[0130] Used to obtain the simulated world position coordinates of the autonomous driving main vehicle to which the current sensor is attached; obtain the surrounding environment information of the autonomous driving main vehicle; configure the algorithm of the sensor simulation model according to the sensor parameters of the autonomous driving main vehicle, and generate perception data; transmit the perception data of the current frame to the autonomous driving test object as the sensor input of the current frame; the autonomous driving test object executes the autonomous driving strategy according to the sensor input, and modifies the position of the autonomous driving main vehicle in the next frame according to the vehicle dynamics model; and repeats this cycle.

[0131] The foggy scene generation method and system for autonomous driving simulation testing of the present invention have the following advantages:

[0132] (1) Based on physical modeling, the simulation is highly realistic.

[0133] The present invention uses a physical modeling method to accurately simulate the signal propagation, attenuation and reflection characteristics of LiDAR in a foggy environment. Compared with the traditional data-driven method, the model not only greatly improves the simulation realism, but also has good interpretability.

[0134] (2) Strong scenario scalability.

[0135] Existing methods usually rely on specific data sets for post-processing, and the generated scenarios are limited by the coverage of the data sets, and the scalability is poor. By combining the CARLA simulation platform with the independently designed simulation model algorithm, the present invention can be tested in a variety of virtual scenarios, including different maps, vehicle placements, traffic participant configurations, and various complex road conditions. This high flexibility significantly expands the scope of application of simulation testing.

[0136] (3) Multi-sensor consistency design.

[0137] The present invention fully considers the consistency of the lidar and camera under foggy weather conditions during the simulation process. By establishing a unified evaluation standard based on the meteorological optical range (MOR) and clarifying the conversion method between the fog concentration parameter and the MOR, it ensures that the perception effect of the lidar point cloud data and the camera image data in the same foggy environment is consistent. This design not only improves the reliability of multi-sensor fusion, but also provides support for the verification of multi-sensor fusion perception algorithms under complex weather conditions.

[0138] The above contents are further detailed descriptions of the present invention in combination with specific preferred embodiments, and it cannot be determined that the specific implementation of the present invention is limited to these descriptions. For ordinary technicians in the technical field to which the present invention belongs, several simple deductions or substitutions can be made without departing from the concept of the present invention, which should be regarded as falling within the protection scope of the present invention.

Claims

1. A method for generating foggy scenes for autonomous driving simulation testing, characterized in that: The following steps are involved: S1. Generate the autonomous driving scenario required for the corresponding sensor configuration in the CARLA simulator according to the scenario description file; S2. In the foggy scene in the scene description file, the CARLA simulator obtains the parameter configuration of the corresponding sensor of the autonomous driving main vehicle; S3. Construct a sensor simulation model according to the parameter configuration of the corresponding sensor; S4. The autonomous driving scenario is combined with the autonomous driving main vehicle to form the autonomous driving simulation test environment of the CARLA simulator. The autonomous driving test object is connected to the autonomous driving simulation test environment to perform in-the-loop testing. In the autonomous driving simulation test environment, each frame of sensor perception data is generated according to the position of the autonomous driving test object in the autonomous driving simulation test environment and its surrounding environment; in the foggy weather scenario, for each frame of perception data during the simulation test, the construction of the sensor simulation model specifically includes: S31. Obtain the simulated world position coordinates of the autonomous driving vehicle to which the current sensor is attached; S32. Obtaining the surrounding environment information of the autonomous driving main vehicle; S33. Generate perception data according to the sensor parameter configuration of the autonomous driving main vehicle and the sensor simulation model algorithm; S34. transmitting the perception data of the current frame to the autonomous driving test object as the sensor input of the current frame; S35. The autonomous driving test object executes the autonomous driving strategy according to the sensor input, and modifies the position of the autonomous driving main vehicle in the next frame according to the vehicle dynamics model of the CARLA simulator; and the cycle is repeated; The sensor includes a laser radar and a camera, the sensor simulation model includes a laser radar simulation model and a camera simulation model, and the perception data includes laser radar data and camera data; The S33 includes: For the camera, a convolutional neural network is used to establish a mapping relationship between fog concentration and MOR in the autonomous driving scene of the CARLA simulator to construct camera data; For LiDAR, based on the same MOR of the current scene, LiDAR data is generated according to the LiDAR simulation model; In S33, a mapping relationship between fog concentration and MOR is established in the autonomous driving scene of the CARLA simulator through a convolutional neural network, specifically including: By adjusting the Fog Density parameter in CARLA with a step size of 1, recording the camera image, and using a convolutional neural network to calculate the MOR value; experiments were conducted at different map locations, and the experiment was repeated multiple times at each location, and the average of the results was taken.<Fog Density, MOR> The data points are linearly interpolated to establish a direct relationship between fog concentration and MOR.

2. The method for generating foggy scenes for autonomous driving simulation testing according to claim 1, characterized in that: For laser radar, the S33 includes: For each laser radar light, the laser intensity of a light ray and the distance between the corresponding target and the laser radar are calculated according to the laser radar simulation model of the foggy scene and based on the MOR of the current scene. R , and finally generate point cloud data.

3. The method for generating foggy scenes for autonomous driving simulation testing according to claim 2, characterized in that: In the lidar simulation model of the foggy scene, the intensity of the lidar emission signal is expressed as: (1) in, is the peak value of the signal strength, is the pulse half width; After propagation, the laser signal is attenuated and deformed due to the propagation medium and the reflection characteristics of the target. The pulse intensity of the received signal is calculated by the following formula: (2) in, Indicates the distance between the target and the laser radar, represents the speed of light constant, Represents the optical system constant of the laser radar, which determines the hardware performance of the system. Represents the spatial impulse response function, which is used to describe the ability of the reflected signal strength at various locations.

4. The method for generating foggy scenes for autonomous driving simulation testing according to claim 2, characterized in that: In the lidar simulation model of foggy scenes, in foggy weather conditions, the total signal power received by the lidar is divided into hard target reflection power and and the reflected power of the medium in fog transmission Two, the formula is: (3) in, R Indicates the distance between the target and the laser radar; The distance measured by the laser radar is usually based on the first reception of the reflected pulse or the reception of the strongest pulse signal, expressed as: (4) in R Indicates the distance between the target and the laser radar; Hard target reflected power According to the attenuation characteristics of the signal in the fog environment, the modeling expression is directly calculated as follows: (5) in, represents the optical system constant of the lidar, Indicates the peak value of the signal strength. represents the reflectivity of the hard target, represents the attenuation coefficient of fog, Indicates the target distance, represents the speed of light constant, Indicates the pulse half width; Reflected power of the medium in fog transmission Modeled as: (6) in is a step function, represents the cross-section function, The back reflection coefficient represents the change in the reflection intensity of the incident light from the surface of the object; Cross Section Function It is used to define the ratio between the laser radar transmitter illumination area and the laser radar receiver receiving area, and is modeled as: (7) in, R 1 represents the starting distance of the overlapping area between the transmitter irradiation area and the receiver receiving area. R 2 represents the maximum distance of the overlapping area between the transmitter's irradiation area and the receiver's receiving area.

5. The method for generating foggy scenes for autonomous driving simulation testing according to claim 4, characterized in that: Calculate the laser intensity of a ray and the distance between the corresponding target and the lidar R , and finally generate point cloud data, including: For each ray, determine whether it is blocked by an object. If so, the distance between the corresponding target and the LiDAR is R , the reflectivity of the hard target , directly substitute into formulas (3)~(5) for calculation. If no obstacle is encountered, use formula (6) for calculation, and we get and , and finally the laser intensity of the light is: (8) Finally, we get the laser intensity of a ray and the distance between the corresponding target and the lidar in the foggy scene. R , is the information of a point in the point cloud. This cycle is repeated and the point cloud data of the lidar is obtained through cumulative calculation.

6. The method for generating foggy scenes for autonomous driving simulation testing according to claim 1, characterized in that: In S2, the parameter configuration of the corresponding sensor of the autonomous driving main vehicle includes the installation position, resolution, lidar laser beam, and the number of point clouds per second.

7. A foggy scene generation system for autonomous driving simulation testing, executing the foggy scene generation method for autonomous driving simulation testing as claimed in any one of claims 1 to 6, characterized in that: include Scenario generation module: used to generate the corresponding configured autonomous driving scenario in CARLA according to the scenario description file; Sensor configuration module: used to obtain the parameter configuration of the corresponding sensor of the autonomous driving vehicle in the foggy scene of the scene description file; Sensor simulation module: used to build a sensor simulation model according to the parameter configuration of the corresponding sensor; Simulation test module: It is used to combine the autonomous driving scenario with the sensor simulation model to form an autonomous driving simulation test environment, connect the autonomous driving test object to the autonomous driving simulation test environment, and perform in-the-loop testing; In foggy weather scenarios, for each frame of perception data during the simulation test, the sensor simulation module is specifically as follows: Used to obtain the simulated world position coordinates of the autonomous driving vehicle to which the current sensor is attached; obtain the surrounding environment information of the autonomous driving vehicle; configure the algorithm of the sensor simulation model according to the sensor parameters of the autonomous driving vehicle, and generate perception data; transmit the perception data of the current frame to the autonomous driving object under test as the sensor input of the current frame; The autonomous driving test object executes the autonomous driving strategy based on the sensor input and modifies the position of the autonomous driving main vehicle in the next frame based on the vehicle dynamics model; This cycle is repeated repeatedly.

Images