An automatic driving accident prevention simulation scene generation technology based on swarm intelligence

Abstract

The application provides an automatic driving accident prevention simulation scene generation technology based on swarm intelligence, which can realize random sampling in a road accident area, design an algorithm for splitting a lane to establish a KD tree, effectively reduce the time for searching a nearest neighbor lane, quickly determine the direction of a generated accident vehicle, and greatly reduce the search time compared with a method without optimization. Then, the constraint relationship of participants in the automatic driving accident prevention simulation scene is analyzed, an evaluation function is designed, a lane trajectory and a normal direction are searched along a lane in the road accident area based on a firework algorithm, and a suitable vehicle generation area is gradually approached. A sensor object is created through an interface provided by a Carla simulator, and an accident image is generated according to simulation data received by the sensor object. Compared with an existing scheme, the application can effectively improve the speed of automatic driving accident prevention simulation scene generation.

Description

Technical fields:

[0001] This invention discloses a simulation scenario generation technology for autonomous driving accident prevention based on swarm intelligence, which relates to the research of simulation testing technology in autonomous driving technology and belongs to the field of autonomous driving simulation testing. Background technology:

[0002] Before autonomous vehicles are officially put into use, they need to undergo systematic testing and verification. Considering the time, economic costs, and potential dangers of testing in real-world environments, a large amount of testing is conducted using simulation environments to execute test cases. Scenario-based virtual testing technology offers flexible test scenario configuration, high testing efficiency, process safety, and low cost, which can accelerate autonomous driving testing. Therefore, scenario-based virtual testing has become an indispensable part of the testing and evaluation of autonomous vehicles. Scenario data mainly comes from three sources: real data, simulated data, and expert experience. Scenario generation based on real data generally comes from real driving data or traffic datasets, such as pre-crash scenario data from the China In-Depth Accident Study (CIDAS) and the National Highway Traffic Safety Administration (NHTSA) in the United States. However, these data cannot cover all driving scenarios, therefore, many methods for constructing simulated scenario data have been proposed.

[0003] Currently, the main methods for constructing simulation scene data are as follows:

[0004] Paracosm parameterizes objects in a scene and provides a set of programming interfaces for scene generation. It generates scenes based on random search and tests the visual perception module in autonomous driving. However, considering the enormous size of the searchable parameter space, it is difficult to efficiently identify safety issues in the perception module through random search.

[0005] GeoScenario: A tool-independent, domain-specific language for scene description. Static objects are represented by nodes, and the nodes are located at fixed latitude and longitude. However, it cannot generate all possible suitable scenes using sampling methods.

[0006] Scenic is a probabilistic programming language used to describe scenarios. It can quickly determine the location of objects that meet the requirements by cutting and placing the area based on the vehicle's orientation and the area range. However, it is difficult to quickly find a suitable generation location in scenarios where the participants are far apart, such as at intersections or curves.

[0007] While significant progress has been made in generating autonomous driving scenarios based on scene description languages, there are still many areas for further research and improvement. Issues include the inflexibility of requiring users to specify participant positions using coordinates when describing their locations, slow search speeds, and difficulty in finding suitable locations in certain scenarios. Summary of the Invention:

[0008] This invention proposes a swarm intelligence-based technology for generating simulation scenarios for autonomous driving accident prevention. Addressing the limitations of existing autonomous driving scenario description languages, which require users to specify the positions of participants in the accident scene using coordinates (lacking flexibility), slow search speed, and difficulty in finding suitable locations for certain accident scenarios, this invention utilizes the Fireworks algorithm to design a swarm intelligence-based technology for generating simulation scenarios for autonomous driving accident prevention. The main objectives are: first, to propose a high-precision map processing method capable of random sampling within the road accident area, and second, to design an algorithm for segmenting lanes and building a KD-tree, effectively reducing the time spent searching for nearest neighbor lanes and quickly determining the orientation of the vehicles involved in the accident. This significantly reduces search time compared to unoptimized methods. Then, the constraints of participants in the autonomous driving accident prevention simulation scenario are analyzed, an evaluation function is designed, and the Fireworks algorithm is used to search along lane trajectories and normals within the road accident area, gradually approaching a suitable vehicle generation region. Sensor objects are created through the interface provided by the Carla simulator, and accident images are generated based on the simulation data received by these sensor objects. Compared to existing solutions, this invention effectively improves the speed of generating simulation scenarios for autonomous driving accident prevention.

[0009] This invention mainly includes the following steps:

[0010] Step (1): Calculate and sample the driving area to achieve uniform sampling of location points in the road accident area on the map;

[0011] Step (2): Use the KD-tree-based method for calculating vehicle orientation in accident scenarios to specify the orientation of vehicles in the accident scenario;

[0012] Step (3): Using the proposed autonomous driving accident prevention simulation scenario participant generation point search algorithm, the sampled location points are evaluated based on the distance and orientation relationship of the accident scenario participants. New points are generated by calculating the evaluation value, and new points are filtered to gradually approach the suitable ones, so as to quickly search for accident scenario participant generation locations that meet the requirements.

[0013] Step (4): After determining the generation position and orientation of the participants in the autonomous driving accident prevention simulation scenario, connect the Carla simulator and generate a static autonomous driving accident prevention simulation scenario based on the simulation data received by the sensor objects.

[0014] Each step includes the following specific implementation process:

[0015] Step (1.1) Driving Area Calculation. First, generate polygons based on the lane boundary points. Then, expand the boundaries of all polygons outward by 0.5 meters to eliminate the influence of gaps between lane boundaries. Next, take the union of all the polygons formed by the lanes and merge them into a large polygon. Then, erode the boundary of the large polygon inward by 0.5 meters to restore its original size, connecting all road accident areas. Finally, decompose the resulting large polygon into smaller triangles to fill all road accident areas.

[0016] Step (1.2) Sampling of the driving area. First, the area of ​​the triangle is used as the weight for sampling, and triangles are obtained by uniform sampling. The road is decomposed into multiple triangular regions, and individual triangular regions can be sampled. Then, based on the vertex coordinates of the region, the smallest rectangle whose length and width are parallel to the two coordinate axes and enclose the triangular region is obtained. A value is randomly selected from the length and width intervals of the rectangle. The selected value is the coordinate value of the sampling point. If the sampling point is within the triangular region, the location point is successfully sampled; otherwise, a value is reselected from the two intervals until the selected coordinate point is within the triangle. This method of decomposing into triangles and then sampling saves a lot of time compared to sampling directly in the large area of ​​the map, because the road area accounts for a small proportion of the accident area on the map, and it is easy to sample outside the road accident area, resulting in sampling failure.

[0017] Step (2.1) Segmenting lanes and constructing a KD tree. First, connect adjacent points on the lane centerlines to form basic lane segments. All basic lane segments form a large set. Determine the segmentation dimension and segmentation value based on the coordinate boundary range of the set. Divide the set into two parts according to the segmentation dimension and segmentation value, and obtain the segmentation nodes for this segmentation. Then, recursively segment the two parts until the maximum length of the lane segment set boundary in both coordinate directions is less than the set value. All segmentation nodes and the final set of lane segments that cannot be further segmented are then formed.

[0018] Step (2.2) involves performing a nearest neighbor search on the KD-tree. This process is divided into two stages: generating the search path and backtracking. Generating the search path is essentially a binary search. Starting from the root node of the KD-tree, the search proceeds along the left side of the tree based on the current layer's partitioning dimension. If the coordinates of the current point in that dimension are less than the current layer's partitioning value, the search continues recursively on the left side of the KD-tree; otherwise, it continues recursively on the right side. This continues until the current node has no children. Then, backtracking begins, calculating the minimum distance between the current layer's lane segment set and the current point, and updating the minimum distance and the nearest lane segment. If the current node has sibling nodes, and the nearest distance between the lane segment set in the sibling node and the current point is smaller than the current nearest distance, the search continues in the sibling node. Then, the search continues backtracking to the partitioning node of the previous layer, repeating this process until all nodes on the search path have been found. Finally, the nearest distance and the nearest lane segment are obtained, and the coordinates of the lane centerline point closest to the current point are determined based on the lane segment.

[0019] Step (3.1) Analysis of the positional relationships of participants in the autonomous driving accident prevention simulation scenario. This invention extracts the participants with relative relationships and the relationships between them. For accident scenario participants without reference to other accident scenario participants, random sampling points are used as their generation points. Then, the positions and orientations of the remaining participants are calculated based on their relative relationships to calculate the evaluation value of the sampling points. The process of calculating the evaluation value of each sampling point x based on its positional relationships is shown below.

[0020]

[0021] Where distance(c i ) is a participant in the accident scene, c i The length from the road boundary, where angle_bias(c i ) is a participant in the accident scene, c i The deviation between the actual angle and the target angle formed by other accident participants. k is the number of participants other than the randomly sampled participants. A small evaluation value indicates that the sampled location is close to the suitable area, and an evaluation value of 0 indicates that the sampled location is in the suitable area. The total sum of deviations takes into account the distance and orientation relationships between participants, and can represent the suitability of the location point for the positional relationships of the participants in the accident scene.

[0022] Step (3.2) Search for suitable generation points. Calculate the evaluation value of the location points, generate new location points, filter the next generation of location points, and then continue generating new points until a suitable location is found.

[0023] Step (4.1) connects to the Carla simulation environment via network, then selects the corresponding participant blueprint and generates vehicles or pedestrians facing the corresponding directions at the generated location points. Camera and LiDAR sensor objects are then set on these participant objects; the cameras include RGB, depth, and semantic segmentation cameras.

[0024] Step (4.2) sets up listening functions on these sensors. Whenever a sensor receives data, it parses the data. The Carla sensor object generates a data stream in carla.SensorData format. It uses carla.convert to convert the data into carla.Image data of the corresponding sensor type, and then converts it into a NumPy array object in the form of a stream. The size of the array is modified to correspond to the resolution of the image to be generated, so as to realize the custom simulation scene resolution.

[0025] Step (4.3) renders the sensor data on the pygame window and uses the Carla interface to save scene images, generating an autonomous driving accident prevention simulation scene.

[0026] This invention is used for generating simulation scenarios to prevent autonomous driving accidents, which can accelerate the generation of initial accident scenarios and has the following advantages:

[0027] This invention proposes a simulation scenario generation technology for autonomous driving accident prevention based on swarm intelligence. It effectively solves problems such as the inflexibility of existing autonomous driving scenario description languages ​​requiring users to specify participant positions using coordinates, insufficient speed in searching for participant positions in accident scenarios, and difficulty in finding suitable positions in certain accident scenarios.

[0028] A high-precision map processing method is proposed, which can achieve random sampling in road accident areas and design an algorithm to segment lanes and build a KD tree. This can effectively reduce the time of searching for the nearest neighbor lane and quickly determine the orientation of the generated vehicle. Compared with the method without optimization, it can greatly reduce the search time.

[0029] By analyzing the constraints of participants in an accident scenario, an evaluation function is designed. Based on the fireworks algorithm, the system searches along lane trajectories and normals in the road region, gradually approaching a suitable vehicle generation area. This can effectively improve the speed of generating simulation scenarios for autonomous driving accident prevention.

[0030] Sensor objects are created using the interface provided by the Carla simulator, and images are generated based on the simulation data received by the sensor objects. This allows for the generation of data from various sensor types and different perspectives. Attached image description:

[0031] To make the technical solution of the invention clearer, the invention will be further described in detail below with reference to the accompanying drawings, wherein...

[0032] Figure 1 A framework for generating simulation scenarios for autonomous driving accident prevention based on swarm intelligence. Detailed implementation method:

[0033] The present invention will be further described in detail below with reference to the accompanying drawings.

[0034] like Figure 1 As shown, the input to the algorithm is a static accident scenario of autonomous driving described using scene language, including the area where the accident scenario occurs, all participants in the accident scenario and their positions and orientation relationships.

[0035] The map processing module identifies road elements that meet the requirements based on the description of the test area. Then, it generates polygon objects based on the boundary information of the road elements. All drivable areas are connected through buffer-based dilation and erosion operations. Finally, the polygon objects are filled with triangles. When sampling location points, the area of ​​the triangles is used as the weight to achieve uniform sampling of location points in the road accident area. Simultaneously, lanes are segmented, and a KD-tree composed of lane segments is constructed, which is used when calculating the orientation of generated location points.

[0036] After identifying the drivable accident area, the accident scene participant generation and location search module begins analyzing the relationships between participants. It first randomly samples location points, and then gradually approximates a suitable generation area based on a swarm intelligence algorithm.

[0037] A suitable location was searched to serve as the generation point for participants in the accident scenario. Finally, the autonomous driving accident prevention simulation scenario generation module connected to the simulation environment, selected vehicle and pedestrian models, and generated participants in the simulation environment based on the coordinates and orientation of the searched generation points. Simultaneously, various sensors were set up, sending data from cameras, LiDAR, and other sources to the front-end rendering module. The rendering module processed the received sensor data, converting it into images of a specified resolution, and finally used Pygame to display the simulation environment information.

[0038] Finally, it should be noted that the present invention can also have many other applications. Without departing from the spirit and essence of the present invention, those skilled in the art can make various corresponding changes and modifications according to the present invention, but these corresponding changes and modifications should all fall within the protection scope of the present invention.

Claims

1. A method for generating simulation scenarios for autonomous driving accident prevention based on swarm intelligence, characterized in that, This includes implementing random sampling in road accident areas using high-precision map processing methods, designing an algorithm to segment lanes and build KD trees to effectively reduce the time for searching nearest neighbor lanes, quickly determining the orientation of the generated accident vehicle, analyzing the constraints of participants in the autonomous driving accident prevention simulation scenario, designing an evaluation function, searching along lane trajectories and normals in the road accident area based on the fireworks algorithm to gradually approach a suitable vehicle generation area, creating sensor objects through the interface provided by the Carla simulator, and generating accident images based on the simulation data received by the sensor objects; Includes the following steps: Step (1): Calculate and sample the driving area to achieve uniform sampling of location points in the road accident area on the map; Step (2): Use the KD-tree-based method for calculating the orientation of accident vehicles to specify the orientation of the accident vehicles; Step (3): Using the proposed autonomous driving accident prevention simulation scenario participant generation point search algorithm, the sampled location points are evaluated based on the distance and orientation relationship of the accident scenario participants. New points are generated by calculating the evaluation value, and new points are filtered to gradually approach suitable location points, thereby achieving a fast search for accident scenario participant generation locations that meet the requirements. Step (4): After determining the generation position and orientation of the participants in the autonomous driving accident prevention simulation scenario, connect the Carla simulator and generate a static autonomous driving accident prevention simulation scenario based on the simulation data received by the sensor objects. Step (2) includes the following steps: Step (2.1) Segmenting lanes and constructing a KD tree: First, connect adjacent points of the lane centerline to form basic lane segments. All basic lane segments form a large set. Determine the segmentation dimension and segmentation value according to the coordinate boundary range of the set. Divide the set into two parts according to the segmentation dimension and segmentation value, and obtain the segmentation node for this segmentation. Then, recursively segment the two parts after segmentation until the maximum length of the lane segment set boundary in the two coordinate directions is less than the set value. All segmentation nodes and the last set of lane segments that cannot be further segmented form a KD tree. Step (2.2) performs nearest neighbor search on the KD tree, which is divided into two stages: generating the search path and backtracking. Generating the search path is the process of performing a binary search. Starting from the root node of the KD tree, it determines which side of the KD tree to continue searching based on the dimension of the current layer's segmentation. If the coordinate value of the position point in this dimension is less than the segmentation value of the current layer, it recursively searches on the left side of the KD tree; otherwise, it recursively searches on the right side. This continues until the current node has no child nodes. Then, it starts backtracking, calculates the minimum distance between the current layer's lane segment set and the position point, and updates the minimum distance and the nearest lane segment. If the current node has sibling nodes, and the nearest distance between the lane segment set in the sibling node and the position point is smaller than the current nearest distance, it needs to enter the sibling node for searching. Then, it backtracks to the segmentation node of the previous layer to continue searching. This operation is repeated until all nodes on the search path have been searched. Finally, the nearest distance and the nearest lane segment are obtained. Based on the lane segment, the coordinates of the lane centerline point closest to the position point are determined.

2. The method for generating simulation scenarios for autonomous driving accident prevention based on swarm intelligence according to claim 1, characterized in that, Step (1) includes the following steps: Step (1.1) Calculation of driving area: First, generate polygons based on the boundary points of the lanes. Then, expand the boundaries of all polygons outward by 0.5 meters to eliminate the influence of gaps between lane boundaries. After that, take the union of all the polygons formed by the lanes and merge them into a large polygon. Then, erode the boundary of the large polygon inward by 0.5 meters to restore the size before expansion. All road accident areas are then connected together. Finally, decompose the large polygon into small triangles to fill all road accident areas. Step (1.2) Sampling of the driving area: First, the area of ​​the triangle is used as the weight for sampling. Triangles are obtained by uniform sampling. The road is decomposed into multiple triangular regions. A single triangular region can be sampled. Then, based on the vertex coordinates of the region, the smallest rectangle with length and width parallel to the two coordinate axes and enclosing the triangular region is obtained. A value is randomly selected from the length and width intervals of the rectangle. The selected value is the coordinate value of the sampling point. If the sampling point is within the triangular region, the location point is successfully sampled. Otherwise, the value is reselected from the two intervals until the selected coordinate point is within the triangle. This method of decomposing into triangles and then sampling saves a lot of time compared to sampling directly in the large area of ​​the map. This is because the road area accounts for a small proportion in the accident area of ​​the map, and it is easy to sample outside the road accident area, resulting in sampling failure.

3. The method for generating simulation scenarios for autonomous driving accident prevention based on swarm intelligence according to claim 1, characterized in that, Step (3) includes the following steps: Step (3.1) Analyze the positional relationships of participants in the autonomous driving accident prevention simulation scenario. Extract the participants with relative relationships and the relationships between them. For accident scenario participants without reference to other accident scenario participants, randomly sample location points as their generation points. Then, calculate the position and orientation of the remaining participants based on the relative relationships to calculate the evaluation value of the sample points. The process of calculating the evaluation value of each sample point x based on the positional relationships is as follows: , in Participants in the accident scene The length from the road boundary, of which Participants in the accident scene The deviation between the actual angle formed by other accident participants and the target angle, k is the number of participants other than the randomly sampled participants. A small evaluation value means that the sampled location is closer to the suitable area, and an evaluation value of 0 means that the sampled location is in the suitable area. The total deviation is calculated to take into account the distance and orientation relationship between participants, which can indicate the suitability of the location point for the positional relationship of the accident scene participants. Step (3.2) Search for suitable generation points, calculate the evaluation value of the location points, generate new location points, filter the next generation of location points, and then continue to generate new points until a suitable location is found.

4. The method for generating simulation scenarios for autonomous driving accident prevention based on swarm intelligence according to claim 1, characterized in that, Step (4) includes the following steps: Step (4.1) Connect to the Carla simulation environment via network, then select the corresponding participant blueprint, generate vehicles or pedestrians facing the corresponding direction at the generated location point, and set up camera and LiDAR sensor objects on these participant objects. The cameras include RGB cameras, depth cameras and semantic segmentation cameras. Step (4.2) Set up listening functions on these sensors. Whenever a sensor receives data, it parses the data. The Carla sensor object generates a data stream in carla.SensorData format. Use carla.convert to convert it into carla.Image data of the corresponding sensor type. Then convert it into a NumPy array object in the form of a stream. Modify the size of the array to correspond to the resolution of the image to be generated to achieve a custom simulation scene resolution. Step (4.3) renders the sensor data on the pygame window and uses the Carla interface to save scene images, generating an autonomous driving accident prevention simulation scene.

Images