Simulation data acquisition method and device, equipment and storage medium

Abstract

The embodiment of the invention provides a simulation data acquisition method and device, equipment and a storage medium. The method comprises the following steps: creating a target directed map; the target directed map comprises road points, and the road points at least comprise a starting point and an ending point; traversing the target directed map to obtain a first target route; traversing the target directed map to obtain a second target route according to the starting point and the ending point on the target directed map; determining a local path planning scheme according to the current position of the target virtual vehicle; controlling the target virtual vehicle to perform simulated driving according to the first target route or the second target route according to the local route planning scheme; and when the target virtual vehicle simulates driving, collecting simulation data collected by the virtual fisheye camera on the target virtual vehicle. According to the embodiment of the invention, the method can meet the changes of different environments and the demands of data collection, and a tester can collect diversified data of various automatic driving regions, thereby remarkably improving the generalization capability of a trained automatic driving model.

Description

Technical Field

[0001] The present application relates to the field of autonomous driving technology, and in particular to a simulation data acquisition method and device, an electronic device, and a storage medium. Background Art

[0002] With the continuous advancement of autonomous driving technology, the perception system of vehicles is undergoing a profound transformation. Modern autonomous driving systems are gradually shifting from relying on a single surround pinhole camera to a comprehensive solution that combines surround fisheye cameras and pinhole cameras with multiple perspectives. This shift not only reflects the evolution of technology, but also aims to improve the safety and efficiency of vehicles during driving and parking. Fisheye cameras have become key sensors in the driving and parking solution because of their ability to capture a wider field of view. At the same time, the high-quality simulation data obtained in the autonomous driving simulation system provides important support for the training, verification and testing of autonomous driving algorithms.

[0003] However, in the development process of autonomous driving systems, there are still two major technical difficulties. First, the collection and annotation cycle of real scene data is long, which is difficult to meet data needs, and traditional manual annotation is inefficient. Secondly, although autonomous driving simulation systems such as CARLA (Car Learning to Act, an autonomous driving simulation platform) can provide a large amount of simulation data, the algorithms of autonomous driving simulation systems are insufficient when traversing the entire map and fixed routes, resulting in uneven data distribution, which in turn affects the generalization ability of the trained autonomous driving model. Summary of the invention

[0004] The embodiment of the present application provides a simulation data acquisition method to solve or at least partially solve the above-mentioned problem.

[0005] Correspondingly, an embodiment of the present application also provides a simulation data acquisition device, an electronic device and a storage medium to ensure the implementation and application of the above method.

[0006] In order to solve the above problems, the embodiment of the present application discloses a simulation data acquisition method, which is applied to an autonomous driving simulation system. The autonomous driving simulation system includes a target virtual vehicle, and a virtual fisheye camera is provided on the target virtual vehicle. The method includes:

[0007] Creating a target directed map; the target directed map includes waypoints, and the waypoints include at least a starting point and an end point;

[0008] Traversing the target directed map to obtain a first target route;

[0009] According to the starting point and the end point on the target directed map, traverse the target directed map to obtain a second target route;

[0010] Determining a local path planning scheme according to the current position of the target virtual vehicle;

[0011] Controlling the target virtual vehicle to simulate driving along the first target route or the second target route according to the local path planning scheme;

[0012] When the target virtual vehicle simulates driving, simulation data acquired by the virtual fisheye camera on the target virtual vehicle is collected.

[0013] Optionally, traversing the target directed map to obtain a first target route includes:

[0014] Create a first route list;

[0015] Traversing the waypoints of the target directed map to obtain the waypoint coordinates;

[0016] Extracting waypoint attribute information corresponding to the waypoint in the target directed map according to the waypoint coordinates;

[0017] Creating a waypoint object according to the waypoint coordinates and the waypoint attribute information corresponding to the waypoint;

[0018] The waypoint object is stored in the first route list to obtain a first target route.

[0019] Optionally, the target directed map includes a path, the waypoint exists on the path, the path is composed of directed edges, and traversing the target directed map according to the starting point and the end point on the target directed map to obtain the second target route includes:

[0020] Create a second route list;

[0021] Acquire a target route consisting of the paths between the starting point and the end point on the target directed map;

[0022] storing the starting point and the end point in the second route list;

[0023] Obtaining the current waypoint on the target route;

[0024] Determine the edge type of the target directed edge where the current waypoint is located;

[0025] If the target directed edge is of a non-lane following type, obtaining the next directed edge of the target directed edge;

[0026] Determine whether there is a corresponding index path for the next directed edge;

[0027] If the next directed edge has a corresponding index path, obtaining the index waypoint closest to the current waypoint on the index path;

[0028] If the next directed edge does not have a corresponding index path, obtaining the index waypoint of the exit of the next directed edge;

[0029] If the target directed edge is of lane following type, obtaining an index path corresponding to the target directed edge;

[0030] Acquire the index waypoint on the index path;

[0031] The current waypoint and the index waypoint are stored in the second route list to obtain a second target route including all waypoints on the target route.

[0032] Optionally, the method further comprises:

[0033] If the current waypoint is located before the end point and the distance between the current waypoint and the end point is less than a first preset distance, or if the current waypoint is located after the end point, then the acquisition of the current waypoint on the target route is stopped.

[0034] Optionally, determining a local path planning scheme according to the current position of the target virtual vehicle includes:

[0035] Obtaining the current positions of the target virtual vehicle and other virtual vehicles;

[0036] Determining whether a bounding box detection is required according to a current position of the target virtual vehicle and the first target route or the second target route;

[0037] If bounding box detection is required, then generating bounding boxes corresponding to the target virtual vehicle, the other virtual vehicles and the remaining routes respectively; the remaining routes are the remaining target routes that the target virtual vehicle needs to travel;

[0038] Determine the intersection between the bounding box of the target virtual vehicle and the bounding boxes of the other virtual vehicles, and between the bounding box of the target virtual vehicle and the bounding boxes of the remaining target routes to be traveled;

[0039] A local path planning scheme is formulated according to the intersection situation.

[0040] Optionally, the target directed map has a corresponding traffic light list, the target directed map includes a path, the waypoint exists on the path, and determining the local path planning scheme according to the current position of the target virtual vehicle includes:

[0041] Obtaining the current position of the target virtual vehicle;

[0042] According to the current position of the target virtual vehicle and the first target route or the second target route, traverse the traffic light list;

[0043] Obtaining a target traffic light closest to the current position of the target virtual vehicle;

[0044] If the distance between the target traffic light and the target virtual vehicle is less than a second preset distance, and the path of the target traffic light is the same as the path of the target virtual vehicle, determining the orientation relationship between the target traffic light and the target virtual vehicle;

[0045] If the positional relationship between the target traffic light and the target virtual vehicle is that the target traffic light is in front of the target virtual vehicle, obtaining a current lighting state of the target traffic light;

[0046] A local path planning scheme is formulated according to the current lighting status.

[0047] Optionally, the first target route and the second target route are composed of the waypoints, and the step of controlling the target virtual vehicle to simulate driving along the first target route or the second target route according to the local path planning scheme includes:

[0048] On the first target route or the second target route, obtaining a next waypoint corresponding to the current waypoint where the target virtual vehicle is located;

[0049] Acquire a target speed and a target steering angle of the target virtual vehicle according to the local path planning scheme;

[0050] Calculating a target throttle value and a target brake value of the target virtual vehicle according to the target speed;

[0051] Calculating a target steering value of the target virtual vehicle according to the target steering angle;

[0052] The target throttle value, the target brake value, and the target steering value are used to control the target virtual vehicle to simulate driving toward the next waypoint.

[0053] The present application also discloses a simulation data acquisition device, which is applied to an autonomous driving simulation system. The autonomous driving simulation system includes a target virtual vehicle, and a virtual fisheye camera is provided on the target virtual vehicle. The device includes:

[0054] A map creation module, used to create a target directed map; the target directed map includes waypoints, and the waypoints include at least a starting point and an end point;

[0055] A first target route planning module, used for traversing the target directed map to obtain a first target route;

[0056] A second target route planning module, configured to traverse the target directed map according to the starting point and the end point on the target directed map to obtain a second target route;

[0057] A local path planning module, used to determine a local path planning solution according to the current position of the target virtual vehicle;

[0058] A simulated driving module, used for controlling the target virtual vehicle to simulate driving along the first target route or the second target route according to the local path planning scheme;

[0059] The simulation data acquisition module is used to collect the simulation data collected by the virtual fisheye camera on the target virtual vehicle when the target virtual vehicle simulates driving.

[0060] An embodiment of the present application further discloses an electronic device, including: a processor; and a memory, on which executable code is stored. When the executable code is executed, the processor executes the simulation data acquisition method as described in one or more of the embodiments of the present application.

[0061] The embodiments of the present application also disclose one or more machine-readable media on which executable codes are stored. When the executable codes are executed, the processor executes the simulation data acquisition method as described in one or more of the embodiments of the present application.

[0062] Compared with the prior art, the embodiments of the present application have the following advantages:

[0063] In an embodiment of the present application, a target directed map is created; the target directed map includes waypoints, and the waypoints include at least a starting point and an end point; the target directed map is traversed to obtain a first target route; according to the starting point and the end point on the target directed map, the target directed map is traversed to obtain a second target route; a local path planning scheme is determined according to the current position of the target virtual vehicle; the target virtual vehicle is controlled to simulate driving along the first target route or the second target route according to the local path planning scheme; when the target virtual vehicle simulates driving, the simulation data collected by the virtual fisheye camera on the target virtual vehicle is collected. The embodiment of the present application can not only plan the first target route for the entire map of the target directed map and the second target route for the preset starting point and end point, but also dynamically adjust the local path of the target virtual vehicle according to the local path planning scheme during the simulated driving of the target virtual vehicle. The embodiment of the present application can adapt to the changes in different environments and the needs of data collection, and provides a complete set of simulation data collection schemes for autonomous driving simulation systems that traverse the entire map and fixed routes. Testers can collect diversified data from various autonomous driving areas, thereby significantly improving the generalization ability of the trained autonomous driving model. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] Figure 1 is a flowchart of the steps of an embodiment of a simulation data acquisition method of the present application;

[0065] Figure 2 is a path planning flow chart of an embodiment of a simulation data collection method of the present application;

[0066] Figure 3 It is a flow chart for determining a local path planning solution of an embodiment of a simulation data collection method of the present application;

[0067] Figure 4 is a simulation data collection flow chart of a simulation data collection method embodiment of the present application;

[0068] Figure 5 It is a structural block diagram of an embodiment of a simulation data acquisition device of the present application;

[0069] Figure 6 It is a schematic diagram of the structure of a device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0070] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0071] With the continuous development of autonomous driving technology, the perception system of vehicles is undergoing significant changes. Modern autonomous driving systems are gradually shifting from relying on a single surround pinhole camera to a comprehensive solution that combines surround fisheye cameras and pinhole cameras with multiple perspectives. This change is not only a technological evolution, but also aims to improve the safety and efficiency of vehicles during driving and parking. Fisheye cameras have become a key sensor for driving and parking solutions due to their ability to capture a wider field of view. They have the advantages of a wide field of view, reduced blind spots, high dynamic adaptability, and reduced costs.

[0072] The process of training the vehicle's autonomous driving model requires a large amount of high-quality simulation data to support it. The generation of simulation data mainly includes physical camera modeling, map and scene construction, and graphics engine rendering. The high-quality simulation data obtained in the autonomous driving simulation system provides important support for the training, verification, and testing of autonomous driving algorithms.

[0073] However, there are still two major technical difficulties in the development of autonomous driving systems. First, the collection and annotation cycle of real scene data is long, which is difficult to meet data needs, and traditional manual annotation is inefficient. Second, although autonomous driving simulation systems such as CARLA can provide a large amount of simulation data, the algorithms of autonomous driving simulation systems are insufficient when traversing the entire map and fixed routes, resulting in uneven data distribution, which in turn affects the generalization ability of the trained autonomous driving model.

[0074] Reference Figure 1 , is a flowchart of a simulation data acquisition method embodiment of the present application, comprising the following steps:

[0075] Step 101: Create a target directed map; the target directed map includes waypoints, and the waypoints include at least a start point and an end point.

[0076] A simulation data collection method of an embodiment of the present application is applied to an autonomous driving simulation system. In one embodiment, the autonomous driving simulation system may be CARLA. The autonomous driving simulation system includes a target virtual vehicle, and a virtual fisheye camera is provided on the target virtual vehicle. In the autonomous driving simulation system, the target virtual vehicle can simulate driving along a target route on a specified map, and collect simulation data using the virtual fisheye camera during the simulated driving process.

[0077] In step 101, a target directed map is first created in the autonomous driving simulation system, the target directed map including waypoints, and the waypoints include at least a start point and an end point for setting a second target route.

[0078] In an optional embodiment, a target directed map may be created through the following steps.

[0079] Step 1011: Generate a road topology map. In this embodiment, the topology information of the road network can be obtained from the vectorized map data and converted into a topology dictionary. The specific process is as follows.

[0080] First, create an empty list to store the topological dictionary. Then, obtain a list of road segments through vectorized map data, each road segment consists of two waypoints. For each road segment, extract the two waypoints and their location information, and round the location coordinates of the two waypoints to avoid floating-point precision issues. In one example, the np.round method (a method in the NumPy library that rounds the elements in an array to a specified number of decimal places) can be used to round the location coordinates of the waypoints. After that, create a topological dictionary, which contains multiple path lists. The topological dictionary can store the entry and exit waypoints of the path and the location coordinates of the entry and exit waypoints. The path list can store the entry and exit waypoints of the corresponding path, the waypoints between the entrance and the exit, and the location coordinates of these waypoints.

[0081] Traverse each path and check whether the distance between the entry and exit waypoints of the path is greater than the sampling resolution. If the distance is greater than the sampling resolution, add a new waypoint between the entry and exit waypoints of the path, and add the new waypoint and the original waypoints of the path to the path list until the exit waypoint is reached or there are no more waypoints; if the distance is less than or equal to the sampling resolution, directly obtain the next waypoint of the path entry waypoint and add it to the path list. After the traversal is completed, store the topology dictionary containing the path list in an empty list to obtain a road topology map.

[0082] Step 1012: Generate a NetworkX directed graph (Networkx is a Python library for creating, operating and studying complex network structures). In this embodiment, the road topology map obtained in step 1011 can be used to further generate a NetworkX directed graph.

[0083] Specifically, a network graph (networkx DiGraph, DiGraph is a class in networkx for representing directed graphs) can be constructed to represent the road topology, and multiple class attributes can be created. The NetworkX directed graph can include a path consisting of directed edges, and the waypoints represented by the nodes exist on the path. In an example, a NetworkX graph representing a world map can be generated by the graph(networkx.DiGraph) instruction, and the class attributes of the NetworkX directed graph can include:

[0084] Node attributes: the location of each waypoint (x, y, z) on the world map;

[0085] Directed edge attributes;

[0086] Entry vector: the unit vector of the tangent line of the entry waypoint;

[0087] Exit vector: unit vector of the tangent line of the exit waypoint;

[0088] Net vector: the unit vector of the chord from the entry waypoint to the exit waypoint;

[0089] intersection: Boolean value indicating whether the directed edge belongs to an intersection;

[0090] id_map: mapping from waypoints (x, y, z) to node IDs;

[0091] road_id_to_edge: A mapping from road segment IDs to directed edges in the graph.

[0092] Step 1013: Add unconnected paths to the directed graph. In this embodiment, unconnected paths that are not traversed in step 1011, such as dead-end paths, etc., can be found and added to the internal graph representation.

[0093] First, the road counter can be set to 0 initially to record the number of unconnected paths in the future. After that, traverse each path in the road topology map, obtain the relevant information of its exit, and check whether the exit of the current path exists in the directed graph. If it exists, it means that the exit is connected and the path is skipped; if it does not exist, increase the count of the road counter, create a new path for the path in the road topology map, use the count of the road counter to number the new path, add the new path to the path list, obtain the updated road topology map, and update the directed graph according to the updated road topology map.

[0094] In one example, if a path without an exit is found, the coordinates of the last waypoint on the path are obtained, and the corresponding node is added to the directed graph, directed edges are added to connect the entry and exit waypoints of the path, and the properties of the directed edges are set (such as length, path, entry and exit waypoints, etc.), thereby completing the update of the directed graph.

[0095] Step 1014: Adding a changeable lane. In this embodiment, a changeable lane can be found in the road topology map and added to the directed graph for subsequent zero-cost lane change / obstacle avoidance in virtual vehicle simulation driving.

[0096] Traverse the road topology map and initialize the left and right changeable states for each road segment to track whether the road segment has found a lane change on the left or right side. Traverse the waypoints in the path and for each waypoint, check whether the waypoint is an intersection in the path of the current road segment. If the waypoint is not an intersection, check the lane change state of the waypoint.

[0097] Check the right lane change of the current waypoint. If the right lane mark of the current waypoint exists and the right lane change is allowed, and the right lane change has not been found, then get the next waypoint of the right lane of the current waypoint; if the next waypoint is valid, and its lane type is a driving lane, and it is on the same path as the current waypoint, then get the segment information of the next waypoint; if the road segment where the next waypoint is located is valid, add a directed edge from the current waypoint to the next waypoint in the directed graph, set relevant attributes (such as entry and exit waypoints, path, length, etc.), and update the right transformable state of the current road segment.

[0098] Check the left lane change of the current waypoint. If the left lane mark of the current waypoint exists and the left lane change is allowed, and the left lane change has not been found, then get the next waypoint of the left lane of the current waypoint; if the next waypoint is valid, and its lane type is a driving lane, and it is on the same path as the current waypoint, then get the segment information of the next waypoint; if the road segment where the next waypoint is located is valid, add a directed edge from the current waypoint to the next waypoint in the directed graph, set relevant attributes (such as entry and exit waypoints, path, length, etc.), and update the left transformable state of the current road segment.

[0099] If the current waypoint finds lane changes on both the left and right sides at the same time, exit the traversal of the current waypoint early.

[0100] The directed graph obtained according to steps 1011 to 1014 is the target directed map.

[0101] Step 102: traverse the target directed map to obtain a first target route.

[0102] In step 102, the target directed map is fully traversed to obtain a first target route, which includes all waypoints of the target directed map.

[0103] Step 103: According to the starting point and the end point on the target directed map, traverse the target directed map to obtain a second target route.

[0104] In step 103, according to the starting point and the end point on the target directed map, the target directed map is traversed to obtain a second target route, where the second target route is a route between the starting point and the end point. In one embodiment, the starting point and the end point may be waypoints set in advance by the user.

[0105] Step 104: Determine a local path planning solution according to the current position of the target virtual vehicle.

[0106] In step 104, during the simulated driving of the target virtual vehicle, it is necessary to obtain the current position of the target virtual vehicle and determine a local path planning scheme according to the current position of the target virtual vehicle. In one embodiment, the local path planning scheme may include dynamic obstacle avoidance, traffic light detection, vehicle control and other schemes.

[0107] Step 105: Control the target virtual vehicle to simulate driving along the first target route or the second target route according to the local path planning solution.

[0108] In step 105, the target virtual vehicle is controlled to simulate driving on the target directed map along the first target route or the second target route according to the local path planning solution.

[0109] Step 106: When the target virtual vehicle simulates driving, collect simulation data acquired by the virtual fisheye camera on the target virtual vehicle.

[0110] In step 106, when the target virtual vehicle simulates driving, simulation data acquired by the virtual fisheye camera on the target virtual vehicle is collected.

[0111] In an embodiment of the present application, a target directed map is created; the target directed map includes waypoints, and the waypoints include at least a starting point and an end point; the target directed map is traversed to obtain a first target route; according to the starting point and the end point on the target directed map, the target directed map is traversed to obtain a second target route; a local path planning scheme is determined according to the current position of the target virtual vehicle; the target virtual vehicle is controlled to simulate driving along the first target route or the second target route according to the local path planning scheme; when the target virtual vehicle simulates driving, the simulation data collected by the virtual fisheye camera on the target virtual vehicle is collected. The embodiment of the present application can not only plan the first target route for the entire map of the target directed map and the second target route for the preset starting point and end point, but also dynamically adjust the local path of the target virtual vehicle according to the local path planning scheme during the simulated driving of the target virtual vehicle. The embodiment of the present application can adapt to the changes in different environments and the needs of data collection, and provides a complete set of simulation data collection schemes for autonomous driving simulation systems that traverse the entire map and fixed routes. Testers can collect diversified data from various autonomous driving areas, thereby significantly improving the generalization ability of the trained autonomous driving model.

[0112] In an optional embodiment of the present application, in step 102, traversing the target directed map to obtain a first target route includes:

[0113] Create a first route list;

[0114] Traversing the waypoints of the target directed map to obtain the waypoint coordinates;

[0115] Extracting waypoint attribute information corresponding to the waypoint in the target directed map according to the waypoint coordinates;

[0116] Creating a waypoint object according to the waypoint coordinates and the waypoint attribute information corresponding to the waypoint;

[0117] The waypoint object is stored in the first route list to obtain a first target route.

[0118] In this embodiment, when the target directed map is fully traversed to obtain a first target route including all waypoints of the target directed map, a DFS (Depth-First Search) or BFS (Breadth-First Search) algorithm can be used to traverse the target directed map to generate the first target route.

[0119] In one embodiment, the user can select the traversal algorithm. The system checks the value entered by the user. If it is 'DFS' or 'dfs', NetworkX's dfs_preorder_nodes method (a method used for depth-first search) is used to perform a depth-first search, and the starting node (i.e., waypoint) can be 0; if it is 'BFS' or 'bfs', NetworkX's bfs_edges method (used for breadth-first search) is used to perform a breadth-first search.

[0120] Afterwards, an empty list is initialized as the first route list for storing the first target route obtained through traversal. The first target route consists of all waypoints of the target directed map.

[0121] According to the pre-selected traversal algorithm, the waypoints of the target directed map are traversed one by one. When the current waypoint is traversed, the waypoint coordinates of the current waypoint are obtained, and the waypoint attribute information corresponding to the current waypoint is extracted from the target directed map according to the waypoint coordinates, and the waypoint coordinates and waypoint attribute information corresponding to the current waypoint are used to create a waypoint object for the current waypoint. The waypoint objects corresponding to all the obtained waypoints are stored in the first route list, and the first target route can be obtained.

[0122] In one embodiment, when the waypoint object is stored in the first route list, the lane change status of the waypoint obtained in step 1014 may also be stored in the first route list together with the waypoint object.

[0123] The embodiment of the present application obtains the first target route by traversing all waypoints of the target directed map, and can obtain global route planning for the entire target directed map, thereby avoiding excessive attention to certain areas. Testers can collect diverse data from various autonomous driving areas, thereby significantly improving the generalization ability of the trained autonomous driving model.

[0124] In an optional embodiment of the present application, the target directed map includes a path, the waypoint exists on the path, and the path is composed of directed edges. In step 103, traversing the target directed map to obtain a second target route according to the starting point and the end point on the target directed map includes:

[0125] Create a second route list;

[0126] Acquire a target route consisting of the paths between the starting point and the end point on the target directed map;

[0127] storing the starting point and the end point in the second route list;

[0128] Obtaining the current waypoint on the target route;

[0129] Determine the edge type of the target directed edge where the current waypoint is located;

[0130] If the target directed edge is of a non-lane following type, obtaining the next directed edge of the target directed edge;

[0131] Determine whether there is a corresponding index path for the next directed edge;

[0132] If the next directed edge has a corresponding index path, obtaining the index waypoint closest to the current waypoint on the index path;

[0133] If the next directed edge does not have a corresponding index path, obtaining the index waypoint of the exit of the next directed edge;

[0134] If the target directed edge is of lane following type, obtaining an index path corresponding to the target directed edge;

[0135] Acquire the index waypoint on the index path;

[0136] The current waypoint and the index waypoint are stored in the second route list to obtain a second target route including all waypoints on the target route.

[0137] In this embodiment, a second target route connecting the starting point and the end point pre-selected by the user may be generated, which may specifically include the following steps.

[0138] First, an empty list is initialized as the second route list, which is used to store all waypoints on the target route from the starting point to the end point, as well as the waypoints corresponding to the starting point and the end point.

[0139] Get a target route consisting of paths between a starting point and an end point on a target directed map. In one embodiment, the nx.astar method (used to execute an A* search algorithm in a target directed map, which is a heuristic search algorithm commonly used for path planning and graph traversal) can be used to get the target route. Get the starting point and the end point, store the starting point and the end point in a second route list, and then traverse the waypoints on the target route.

[0140] When traversing to the current waypoint, the current waypoint and the next waypoint on the target route are obtained, the current waypoint and the next waypoint are connected by a target directed edge, and the edge type of the target directed edge where the current waypoint is located is determined, and the edge type includes a non-lane following type and a lane following type. In an optional embodiment, the target directed edge has a corresponding road option, and the road option indicates the travel options that the vehicle can choose at the directed edge, which may include going straight, turning left, turning right, turning around, etc.

[0141] If the target directed edge is of a non-lane following type, that is, the target directed edge is an edge with special conditions, such as roundabouts, sharp turns and other abnormal driving conditions, the current waypoint and the road options of the target directed edge are stored in the second route list, and the exit waypoint of the target directed edge is obtained, and the node corresponding to the exit waypoint of the target directed edge is found to obtain the next directed edge of the target directed edge.

[0142] Determine whether there is a corresponding index path for the next directed edge, that is, determine whether there is a path for the next directed edge. If there is a corresponding index path for the next directed edge, obtain the index waypoint closest to the current waypoint on the index path, and update the current waypoint to the index waypoint; if there is no corresponding index path for the next directed edge, that is, there is a special situation for the next directed edge, such as a dead end, obtain the index waypoint of the exit of the next directed edge, and update the current waypoint to the index waypoint of the exit of the next directed edge. Store the index waypoint and the road options of the next directed edge in the second route list.

[0143] If the target directed edge is of lane following type, the current path corresponding to the target directed edge can be directly set to the entry waypoint of the target directed edge, the target directed edge and the combination of the target directed edge. Specifically, the index path corresponding to the target directed edge and the index waypoint closest to the current waypoint on the index path are obtained, and the index waypoints on the index path are traversed from the index waypoint. For each traversed index waypoint, the index waypoint and the road options of the target directed edge are stored in the second route list.

[0144] Finally, after traversing the waypoints on the target route, a second target route including all the waypoints and road options on the target route can be obtained.

[0145] The embodiment of the present application obtains a second target route by traversing a preset starting point and an end point, and can obtain route planning for the second target route to meet the data collection needs for a specific route. The tester can collect data for the second target route, thereby significantly improving the generalization ability of the trained autonomous driving model.

[0146] In an optional embodiment of the present application, when generating the second target route and traversing the waypoints on the target route, the method further includes:

[0147] If the current waypoint is located before the end point and the distance between the current waypoint and the end point is less than a first preset distance, or if the current waypoint is located after the end point, then the acquisition of the current waypoint on the target route is stopped.

[0148] When traversing waypoints on the target route, this embodiment needs to determine whether the current waypoint meets the end condition, that is, whether the traversal of all waypoints on the target route has been completed. The end condition may include that the current waypoint is located before the end point and the distance between the current waypoint and the end point is less than a first preset distance, or that the current waypoint is located after the end point, then it is determined that the end condition is met, and the acquisition of the current waypoint on the target route is stopped, that is, the traversal of the waypoints on the target route is stopped.

[0149] In an optional embodiment, the first preset distance may be a preset multiple of the sampling resolution, such as 2 times the sampling resolution.

[0150] In an optional embodiment, if the road ID, segment ID and lane ID of the directed edge where the current waypoint is located are the same as those of the directed edge where the endpoint is located, then when the current waypoint is located after the endpoint, it is determined that the endpoint condition is met.

[0151] The embodiment of the present application sets an end point condition. When it is determined that the end point condition is met, the traversal of the waypoints on the target route is stopped, thereby obtaining a second target route for subsequent simulated driving of the target virtual vehicle and obtaining simulation data.

[0152] Reference Figure 2 , is a path planning flow chart of an embodiment of a simulation data acquisition method of the present application.

[0153] Figure 2 Detailed description of the process of generating a target directed map, a first target route and a second target route in a simulation data acquisition method in an embodiment of the present application is shown in FIG. , which includes the following steps.

[0154] Step 201: Input a vectorized map.

[0155] Step 202: Generate a road topology map. Step 202 includes steps 2021 to 2026.

[0156] Step 2021, initialize the topology list.

[0157] Step 2022: Obtain road segments.

[0158] Step 2023: Process the road segment.

[0159] Step 2024: Create a dictionary.

[0160] Step 2025: Path search.

[0161] Step 2026: Generate a topological map.

[0162] Step 203: Generate a NetworkX directed graph.

[0163] Step 204: adding unconnected road segments. Step 204 includes steps 2041 to 2045.

[0164] Step 2041, initialize the counter.

[0165] Step 2042: traverse the road segments.

[0166] Step 2043, check connectivity.

[0167] Step 2044, path search.

[0168] Step 2045: Update the directed graph.

[0169] Step 205, adding a changeable lane. Step 205 includes steps 2051 to 2055.

[0170] Step 2051, traverse the topology map.

[0171] Step 2052: traverse the path waypoints.

[0172] Step 2053, check the right lane change of the current waypoint, if the right lane change is found, jump to step 2054, otherwise jump to step 2052.

[0173] Step 2054, check the left lane change of the current waypoint, if the left lane change is found, jump to step 2055, otherwise jump to step 2052.

[0174] Step 2055: If a right lane change and a left lane change have been found at the current waypoint, then exit the loop of the current waypoint in advance.

[0175] Step 206 : Generate a global traversal path, ie, a first target route. Step 206 includes steps 2061 to 2065 .

[0176] Step 2061, input traversal method.

[0177] Step 2062: If the algorithm input by the user is DFS, the traversal method is determined to be depth-first search; if the algorithm input by the user is BFS, the traversal method is determined to be breadth-first search.

[0178] Step 2063: Initialize the route.

[0179] Step 2064: traverse the nodes, that is, traverse the waypoints.

[0180] Step 2065: Generate a path.

[0181] Step 207 : Generate a path plan, that is, a second target route. Step 207 includes steps 2071 to 2075 .

[0182] Step 2071, initialize variables.

[0183] Step 2072: traverse the directed edges in the path.

[0184] Step 2073: Process directed edges of non-lane following type.

[0185] Step 2074: Process directed edges of the lane following type.

[0186] Step 2075, return the result.

[0187] Step 208: Output the first target route and the second target route of the global path planning.

[0188] In an optional embodiment of the present application, in step 104, determining a local path planning scheme according to the current position of the target virtual vehicle includes:

[0189] Obtaining the current positions of the target virtual vehicle and other virtual vehicles;

[0190] Determining whether a bounding box detection is required according to a current position of the target virtual vehicle and the first target route or the second target route;

[0191] If bounding box detection is required, then generating bounding boxes corresponding to the target virtual vehicle, the other virtual vehicles and the remaining routes respectively; the remaining routes are the remaining target routes that the target virtual vehicle needs to travel;

[0192] Determine the intersection between the bounding box of the target virtual vehicle and the bounding boxes of the other virtual vehicles, and between the bounding box of the target virtual vehicle and the bounding boxes of the remaining target routes to be traveled;

[0193] A local path planning scheme is formulated according to the intersection situation.

[0194] In this embodiment, in step 104, during the simulated driving of the target virtual vehicle, it is necessary to obtain the current position of the target virtual vehicle and determine a local path planning scheme for dynamic obstacle avoidance according to the current position of the target virtual vehicle. Specifically, determining the local path planning scheme for dynamic obstacle avoidance may include the following process.

[0195] First, the current positions of the target virtual vehicle and other virtual vehicles are obtained, and the target virtual vehicle and other virtual vehicles are simulated to travel on a preset route. In an optional embodiment, the lane change information of the target virtual vehicle and the waypoint object of the current waypoint where the target virtual vehicle is located can also be obtained at the same time, and the lateral offset of the target virtual vehicle relative to the center line of the lane is adjusted according to the lane change information of the target virtual vehicle and the waypoint object to ensure that the target virtual vehicle is on the right or left side of the current lane.

[0196] According to the current position of the target virtual vehicle, and the first target route or the second target route, it is determined whether a bounding box detection is required. In one embodiment, a detection area can be set using a preset distance in front of the target virtual vehicle, and for other virtual vehicles, the detection area is used to determine whether a bounding box detection is required for the current position of the other virtual vehicle. If the current position of the other virtual vehicle is within the detection area, or the target virtual vehicle is within the boundary of a lane of the first target route or the second target route, a bounding box detection is required.

[0197] If bounding box detection is required, that is, the target virtual vehicle is located at an intersection or other situations where bounding box detection is required, bounding boxes corresponding to the target virtual vehicle, other virtual vehicles and remaining routes are generated respectively; the remaining route is the remaining target route that the target virtual vehicle needs to travel.

[0198] In one embodiment, an empty list is initialized to store the boundary points corresponding to the target virtual vehicle, other virtual vehicles, and the remaining routes. The width and offset of each vehicle are calculated, and the left and right boundary points are generated, thereby obtaining the boundary boxes of the target virtual vehicle and other virtual vehicles. Each waypoint in the remaining target route that the target virtual vehicle needs to travel is traversed, and a polygonal boundary box is generated based on these waypoints and added to the list; if these waypoints are less than 3 waypoints, a result without a valid polygon is returned, and there is no need to generate the boundary box corresponding to the remaining route.

[0199] Determine the intersection between the bounding box of the target virtual vehicle and the bounding boxes of other virtual vehicles, and between the bounding box of the target virtual vehicle and the bounding boxes of the remaining target routes to be driven, and formulate a local path planning scheme based on the intersection. The existence of intersections between bounding boxes indicates that abnormal situations such as collisions may occur, and a local path planning scheme needs to be determined for dynamic obstacle avoidance.

[0200] In an optional embodiment, if bounding box detection is not required, that is, the target virtual vehicle is located at a non-intersection or other situation where bounding box detection is not required, it is determined whether the target virtual vehicle and other virtual vehicles are on the same path, and whether the lane deviation between the target virtual vehicle and the other virtual vehicles is less than a preset threshold. If the target virtual vehicle and other virtual vehicles are on the same path, and the lane deviation between the target virtual vehicle and the other virtual vehicles is less than a preset threshold, it is determined whether the preset collision area behind the other virtual vehicles and the detection area in front of the target virtual vehicle intersect, and a local path planning scheme is formulated according to the intersection.

[0201] During the simulated driving process of the target virtual vehicle in the embodiment of the present application, the local path of the target virtual vehicle can also be dynamically adjusted according to the local path planning scheme to achieve dynamic obstacle avoidance, so that testers can collect diverse data on autonomous driving under different circumstances, thereby significantly improving the generalization ability of the trained autonomous driving model.

[0202] In an optional embodiment of the present application, the target directed map has a corresponding traffic light list, the target directed map includes a path, the waypoint exists on the path, and in step 104, determining a local path planning scheme according to the current position of the target virtual vehicle includes:

[0203] Obtaining the current position of the target virtual vehicle;

[0204] According to the current position of the target virtual vehicle and the first target route or the second target route, traverse the traffic light list;

[0205] Obtaining a target traffic light closest to the current position of the target virtual vehicle;

[0206] If the distance between the target traffic light and the target virtual vehicle is less than a second preset distance, and the path of the target traffic light is the same as the path of the target virtual vehicle, determining the orientation relationship between the target traffic light and the target virtual vehicle;

[0207] If the positional relationship between the target traffic light and the target virtual vehicle is that the target traffic light is in front of the target virtual vehicle, obtaining a current lighting state of the target traffic light;

[0208] A local path planning scheme is formulated according to the current lighting status.

[0209] In this embodiment, in step 104, during the simulated driving of the target virtual vehicle, it is necessary to obtain the current position of the target virtual vehicle, and determine the local path planning scheme for traffic light detection according to the current position of the target virtual vehicle. Specifically, determining the local path planning scheme for traffic light detection may include the following process.

[0210] The current position of the target virtual vehicle is obtained, and according to the current position of the target virtual vehicle and the first target route or the second target route, a traffic light list is traversed to filter out a target traffic light which is closest to the current position of the target virtual vehicle and is on the target route currently traveled by the vehicle.

[0211] Detect the distance between the target traffic light and the target virtual vehicle. If the distance between the target traffic light and the target virtual vehicle is less than a second preset distance, and the path of the target traffic light is the same as the path of the target virtual vehicle, determine the orientation relationship between the target traffic light and the target virtual vehicle; if the above conditions are not met, skip the target traffic light and do not need to perform subsequent steps.

[0212] In one embodiment, the azimuth relationship between the target traffic light and the target virtual vehicle can be determined by obtaining the forward direction vector of the target virtual vehicle and the forward direction vector of the traffic light and calculating the dot product of the two. If the dot product is less than 0, it means that the target traffic light is behind the target virtual vehicle.

[0213] If the azimuth relationship between the target traffic light and the target virtual vehicle is that the target traffic light is in front of the target virtual vehicle, the current lighting state of the target traffic light is obtained, and a local path planning scheme is formulated according to the current lighting state. In one embodiment, if the current lighting state is red, the target virtual vehicle needs to be controlled to stop driving and wait for the current lighting state of the target traffic light to switch to green.

[0214] During the simulated driving process of the target virtual vehicle in the embodiment of the present application, the local path of the target virtual vehicle can also be dynamically adjusted according to the local path planning scheme to achieve traffic light detection, so that testers can collect diversified data on autonomous driving under different circumstances, thereby significantly improving the generalization ability of the trained autonomous driving model.

[0215] In an optional embodiment of the present application, the first target route and the second target route are composed of the waypoints. In step 105, controlling the target virtual vehicle to simulate driving along the first target route or the second target route according to the local path planning scheme includes:

[0216] On the first target route or the second target route, obtaining a next waypoint corresponding to the current waypoint where the target virtual vehicle is located;

[0217] Acquire a target speed and a target steering angle of the target virtual vehicle according to the local path planning scheme;

[0218] Calculating a target throttle value and a target brake value of the target virtual vehicle according to the target speed;

[0219] Calculating a target steering value of the target virtual vehicle according to the target steering angle;

[0220] The target throttle value, the target brake value, and the target steering value are used to control the target virtual vehicle to simulate driving toward the next waypoint.

[0221] In this embodiment, a specific scheme for controlling the target virtual vehicle to simulate driving along the first target route or the second target route according to the local path planning scheme is shown. The control of the target virtual vehicle can be divided into longitudinal control and lateral control. The longitudinal control adjusts the speed of the target virtual vehicle by determining the required throttle value or brake value, and the lateral control controls the forward direction of the target virtual vehicle to head towards the waypoint.

[0222] In one embodiment, the simulated driving of the target virtual vehicle may be controlled by a PID controller (Proportional-Integral-Derivative Controller).

[0223] On the first target route or the second target route, a next waypoint corresponding to the current waypoint where the target virtual vehicle is located is obtained.

[0224] According to the local path planning scheme, the target speed and target steering angle of the target virtual vehicle are obtained. According to the target speed, the target throttle value and target brake value of the target virtual vehicle are calculated, and according to the target steering angle, the target steering value of the target virtual vehicle is calculated.

[0225] In one embodiment, if the calculated acceleration corresponding to the target speed is positive (indicating that the vehicle needs to accelerate), the target throttle value can be set to the smaller value of the required acceleration and the target throttle value; if the acceleration corresponding to the target speed is negative (indicating that the vehicle needs to decelerate), the target throttle value is set to zero, and the target braking value is set according to the maximum braking limit to implement braking.

[0226] In one embodiment, a target turning value of the target virtual vehicle can be restricted by using a preset turning threshold to restrict the target turning value of the target virtual vehicle each time it turns. If the angle that the target virtual vehicle needs to turn is greater than the preset turning threshold, the turning is completed in multiple times. In one example, the preset turning threshold can be set to plus or minus 0.1 radians.

[0227] The target throttle value, target brake value and target steering value are used to control the target virtual vehicle to simulate driving to the next waypoint.

[0228] During the simulated driving process of the target virtual vehicle in the embodiment of the present application, the local path of the target virtual vehicle can be dynamically adjusted according to the local path planning scheme, and the target virtual vehicle can be controlled to simulate driving according to the target throttle value, target braking value and target steering value calculated in real time, so that the test personnel can collect more robust simulation data, thereby significantly improving the generalization ability of the trained autonomous driving model.

[0229] Reference Figure 3 , is a flow chart for determining a local path planning scheme in an embodiment of a simulation data acquisition method of the present application.

[0230] Figure 3 The specific process of generating a local path planning scheme in a simulation data acquisition method of an embodiment of the present application and controlling the target virtual vehicle to simulate driving is shown, including the following steps.

[0231] Step 301: Input the agent position (ie, the current position of the target virtual vehicle).

[0232] Step 302: Formulate a local path planning scheme for dynamic obstacle avoidance. Step 302 includes steps 3021 to 3023.

[0233] Step 3021: Obtain agent vehicle and road information, that is, obtain relevant information of the target virtual vehicle, other virtual vehicles, and remaining routes.

[0234] Step 3022: Obtain the front information of the target virtual vehicle.

[0235] Step 3023: Detect obstacles.

[0236] Step 303: Formulate a local path planning scheme for traffic light detection. Step 303 includes steps 3031 to 3038.

[0237] Step 3031: Detect the historical status of the traffic light.

[0238] Step 3032: Obtain the agent vehicle location and waypoint information.

[0239] Step 3033, traverse the traffic light list.

[0240] Step 3034: Determine whether the traffic light is in front of the agent. If yes, jump to step 3035; if no, jump to step 3038.

[0241] Step 3035: Determine whether the light is red. If yes, jump to step 3036; if no, jump to step 3038.

[0242] Step 3036: Determine whether the distance is within the range. If yes, jump to step 3037; if no, jump to step 3038.

[0243] Step 3037: Output the impact status.

[0244] Step 3038: Skip the current traffic light.

[0245] Step 304: PID control. Step 304 includes steps 3041 to 3046.

[0246] Step 3041: Longitudinal PID control.

[0247] Step 3042: lateral PID control.

[0248] Step 3043: throttle and brake control.

[0249] Step 3044: Steering adjustment.

[0250] Step 3045, status update.

[0251] Step 3046, output the result.

[0252] Step 305: Generate local path planning.

[0253] In an embodiment of the present application, a target directed map is created; the target directed map includes waypoints, and the waypoints include at least a starting point and an end point; the target directed map is traversed to obtain a first target route; according to the starting point and the end point on the target directed map, the target directed map is traversed to obtain a second target route; a local path planning scheme is determined according to the current position of the target virtual vehicle; the target virtual vehicle is controlled to simulate driving along the first target route or the second target route according to the local path planning scheme; when the target virtual vehicle simulates driving, the simulation data collected by the virtual fisheye camera on the target virtual vehicle is collected. The embodiment of the present application can not only plan the first target route for the entire map of the target directed map and the second target route for the preset starting point and end point, but also dynamically adjust the local path of the target virtual vehicle according to the local path planning scheme during the simulated driving of the target virtual vehicle. The embodiment of the present application can adapt to the changes in different environments and the needs of data collection, and provides a complete set of simulation data collection schemes for autonomous driving simulation systems that traverse the entire map and fixed routes. Testers can collect diversified data from various autonomous driving areas, thereby significantly improving the generalization ability of the trained autonomous driving model.

[0254] Reference Figure 4 , is a simulation data collection flow chart of a simulation data collection method embodiment of the present application.

[0255] like Figure 4 As shown, Figure 4 The complete simulation data collection process of a simulation data collection method of an embodiment of the present application is shown in FIG.

[0256] Step 4001, input simulation map.

[0257] Step 4002: Generate a road topology map.

[0258] Step 4003: Generate a NetworkX directed graph.

[0259] Step 4004: Add the unconnected road segments.

[0260] Step 4005: Add changeable lanes.

[0261] Step 4006: Generate a global traversal path (ie, the first target route).

[0262] Step 4007: Generate path planning (ie, the second target route).

[0263] Step 4008: The user can select and input the starting point, end point and traversal method to generate a global path plan (i.e., the driving route that the target virtual vehicle ultimately decides to collect simulation data) and the target virtual vehicle.

[0264] Step 4009: Develop a local path planning scheme for dynamic obstacle avoidance.

[0265] Step 4010: Develop a local path planning scheme for traffic light control.

[0266] Step 4011, generate PID control.

[0267] Step 4012: Generate local path planning.

[0268] Step 4013: Determine whether the number of frames of the collected simulation data meets the threshold. If so, end the collection; if not, continue to collect data.

[0269] In an embodiment of the present application, a target directed map is created; the target directed map includes waypoints, and the waypoints include at least a starting point and an end point; the target directed map is traversed to obtain a first target route; according to the starting point and the end point on the target directed map, the target directed map is traversed to obtain a second target route; a local path planning scheme is determined according to the current position of the target virtual vehicle; the target virtual vehicle is controlled to simulate driving along the first target route or the second target route according to the local path planning scheme; when the target virtual vehicle simulates driving, the simulation data collected by the virtual fisheye camera on the target virtual vehicle is collected. The embodiment of the present application can not only plan the first target route for the entire map of the target directed map and the second target route for the preset starting point and end point, but also dynamically adjust the local path of the target virtual vehicle according to the local path planning scheme during the simulated driving of the target virtual vehicle. The embodiment of the present application can adapt to the changes in different environments and the needs of data collection, and provides a complete set of simulation data collection schemes for autonomous driving simulation systems that traverse the entire map and fixed routes. Testers can collect diversified data from various autonomous driving areas, thereby significantly improving the generalization ability of the trained autonomous driving model.

[0270] It should be noted that, for the method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the embodiments of the present application are not limited by the described order of actions, because according to the embodiments of the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of the present application.

[0271] On the basis of the above embodiments, this embodiment further provides a simulation data acquisition device, which is applied to electronic devices such as terminal devices and servers.

[0272] Reference Figure 5 , shows a structural block diagram of an embodiment of a simulation data acquisition device of the present application, which may specifically include the following modules:

[0273] A map creation module 501 is used to create a target directed map; the target directed map includes waypoints, and the waypoints include at least a starting point and an end point;

[0274] A first target route planning module 502, configured to traverse the target directed map to obtain a first target route;

[0275] A second target route planning module 503 is used to traverse the target directed map to obtain a second target route according to the starting point and the end point on the target directed map;

[0276] A local path planning module 504, used to determine a local path planning solution according to the current position of the target virtual vehicle;

[0277] A simulated driving module 505, configured to control the target virtual vehicle to simulate driving along the first target route or the second target route according to the local path planning scheme;

[0278] The simulation data acquisition module 506 is used to collect the simulation data acquired by the virtual fisheye camera on the target virtual vehicle when the target virtual vehicle simulates driving.

[0279] Optionally, the first target route planning module 502 includes:

[0280] A first route list creation submodule, used to create a first route list;

[0281] A full map traversal submodule, used to traverse the waypoints of the target directed map to obtain the waypoint coordinates;

[0282] A waypoint attribute extraction submodule, used to extract waypoint attribute information corresponding to the waypoint in the target directed map according to the waypoint coordinates;

[0283] A waypoint object creation submodule, used to create a waypoint object according to the waypoint coordinates and the waypoint attribute information corresponding to the waypoint;

[0284] The first waypoint storage submodule is used to store the waypoint object into the first route list to obtain a first target route.

[0285] Optionally, the target directed map includes a path, the waypoint exists on the path, the path is composed of directed edges, and the second target route planning module 503 includes:

[0286] A second route list creation submodule, used to create a second route list;

[0287] A target route acquisition submodule, used to acquire a target route consisting of the paths between the starting point and the end point on the target directed map;

[0288] A second waypoint storage submodule, used for storing the starting point and the end point in the second route list;

[0289] A current waypoint acquisition submodule is used to acquire the current waypoint on the target route;

[0290] An edge type determination submodule, used to determine the edge type of the target directed edge at the current waypoint;

[0291] a first non-lane following type processing submodule, configured to obtain a next directed edge of the target directed edge if the target directed edge is of a non-lane following type;

[0292] A second non-lane following type processing submodule is used to determine whether there is a corresponding index path for the next directed edge;

[0293] a third non-lane following type processing submodule, configured to obtain an index waypoint closest to the current waypoint on the index path if the next directed edge has a corresponding index path;

[0294] a fourth non-lane following type processing submodule, configured to obtain the index waypoint of the exit of the next directed edge if the next directed edge does not have a corresponding index path;

[0295] A first lane following type processing submodule, configured to obtain an index path corresponding to the target directed edge if the target directed edge is of a lane following type;

[0296] A second lane following type processing submodule, used for acquiring the index waypoint on the index path;

[0297] The third waypoint storage submodule is used to store the current waypoint and the index waypoint in the second route list to obtain a second target route including all waypoints on the target route.

[0298] Optionally, the second target route planning module 503 further includes:

[0299] The traversal stop submodule is used to stop acquiring the current waypoint on the target route if the current waypoint is located before the end point and the distance between the current waypoint and the end point is less than a first preset distance, or if the current waypoint is located after the end point.

[0300] Optionally, the local path planning module 504 includes:

[0301] A first current position acquisition submodule, used to acquire the current positions of the target virtual vehicle and other virtual vehicles;

[0302] A bounding box use determination submodule, used to determine whether a bounding box detection is required according to the current position of the target virtual vehicle and the first target route or the second target route;

[0303] A bounding box detection submodule, for generating bounding boxes corresponding to the target virtual vehicle, the other virtual vehicles and the remaining routes respectively if bounding box detection is required; the remaining routes are the remaining target routes that the target virtual vehicle needs to travel;

[0304] An intersection determination submodule, used to determine the intersection between the bounding box of the target virtual vehicle and the bounding boxes of the other virtual vehicles, and between the bounding box of the target virtual vehicle and the bounding boxes of the remaining target routes to be traveled;

[0305] The first plan formulation submodule is used to formulate a local path planning plan according to the intersection situation.

[0306] Optionally, the target directed map has a corresponding traffic light list, the target directed map includes a path, the waypoint exists on the path, and the local path planning module 504 includes:

[0307] A second current position acquisition submodule is used to acquire the current position of the target virtual vehicle;

[0308] A traffic light traversal submodule, configured to traverse the traffic light list according to the current position of the target virtual vehicle and the first target route or the second target route;

[0309] A target traffic light acquisition submodule is used to acquire a target traffic light closest to the current position of the target virtual vehicle;

[0310] an orientation relationship determination submodule, configured to determine the orientation relationship between the target traffic light and the target virtual vehicle if the distance between the target traffic light and the target virtual vehicle is less than a second preset distance, and the path where the target traffic light is located is the same as the path where the target virtual vehicle is located;

[0311] A lighting state acquisition submodule, configured to acquire a current lighting state of the target traffic light if the positional relationship between the target traffic light and the target virtual vehicle is that the target traffic light is in front of the target virtual vehicle;

[0312] The second plan formulation submodule is used to formulate a local path planning plan according to the current lighting state.

[0313] Optionally, the first target route and the second target route are composed of the waypoints, and the driving simulation module 505 includes:

[0314] A next waypoint acquisition submodule, used to acquire a next waypoint corresponding to the current waypoint where the target virtual vehicle is located on the first target route or the second target route;

[0315] A driving state acquisition submodule, used for acquiring a target speed and a target steering angle of the target virtual vehicle according to the local path planning scheme;

[0316] A first driving parameter calculation submodule, used for calculating a target throttle value and a target brake value of the target virtual vehicle according to the target speed;

[0317] A second driving parameter calculation submodule, used for calculating a target steering value of the target virtual vehicle according to the target steering angle;

[0318] The driving control submodule is used to control the target virtual vehicle to simulate driving toward the next waypoint by using the target throttle value, the target brake value and the target steering value.

[0319] In an embodiment of the present application, a target directed map is created; the target directed map includes waypoints, and the waypoints include at least a starting point and an end point; the target directed map is traversed to obtain a first target route; according to the starting point and the end point on the target directed map, the target directed map is traversed to obtain a second target route; a local path planning scheme is determined according to the current position of the target virtual vehicle; the target virtual vehicle is controlled to simulate driving along the first target route or the second target route according to the local path planning scheme; when the target virtual vehicle simulates driving, the simulation data collected by the virtual fisheye camera on the target virtual vehicle is collected. The embodiment of the present application can not only plan the first target route for the entire map of the target directed map and the second target route for the preset starting point and end point, but also dynamically adjust the local path of the target virtual vehicle according to the local path planning scheme during the simulated driving of the target virtual vehicle. The embodiment of the present application can adapt to the changes in different environments and the needs of data collection, and provides a complete set of simulation data collection schemes for autonomous driving simulation systems that traverse the entire map and fixed routes. Testers can collect diversified data from various autonomous driving areas, thereby significantly improving the generalization ability of the trained autonomous driving model.

[0320] The embodiment of the present application also provides a non-volatile readable storage medium, which stores one or more modules (programs). When the one or more modules are applied to a device, the device can execute instructions (instructions) of each method step in the embodiment of the present application.

[0321] The present application embodiment provides one or more machine-readable media on which instructions are stored, and when executed by one or more processors, an electronic device executes one or more of the methods described in the above embodiments. In the present application embodiment, the electronic device includes various types of devices such as terminal devices and servers (clusters).

[0322] The embodiments of the present disclosure may be implemented as a device configured as desired using any appropriate hardware, firmware, software, or any combination thereof, and the device may include electronic devices such as terminal devices and servers (clusters). Figure 6 An exemplary apparatus 600 that may be used to implement various embodiments described in this application is schematically illustrated.

[0323] For one embodiment, Figure 6 An exemplary apparatus 600 is shown having one or more processors 602, a control module (chip set) 604 coupled to at least one of the processor(s) 602, a memory 606 coupled to the control module 604, a non-volatile memory (NVM) / storage device 608 coupled to the control module 604, one or more input / output devices 610 coupled to the control module 604, and a network interface 612 coupled to the control module 604.

[0324] The processor 602 may include one or more single-core or multi-core processors, and the processor 602 may include any combination of general-purpose processors or special-purpose processors (such as graphics processors, application processors, baseband processors, etc.). In some embodiments, the device 600 can be used as a terminal device, server (cluster), etc. described in the embodiments of the present application.

[0325] In some embodiments, the apparatus 600 may include one or more computer-readable media (e.g., memory 606 or NVM / storage device 608) having instructions 614 and one or more processors 602 combined with the one or more computer-readable media and configured to execute the instructions 614 to implement a module to perform the actions described in the present disclosure.

[0326] For one embodiment, control module 604 may include any suitable interface controller to provide any suitable interface to at least one of processor(s) 602 and / or any suitable device or component in communication with control module 604 .

[0327] The control module 604 may include a memory controller module to provide an interface to the memory 606. The memory controller module may be a hardware module, a software module, and / or a firmware module.

[0328] The memory 606 may be used, for example, to load and store data and / or instructions 614 for the device 600. For one embodiment, the memory 606 may include any suitable volatile memory, such as a suitable DRAM. In some embodiments, the memory 606 may include double data rate type four synchronous dynamic random access memory (DDR4 SDRAM).

[0329] For one embodiment, control module 604 may include one or more input / output controllers to provide an interface to NVM / storage device 608 and input / output device(s) 610 .

[0330] For example, NVM / storage 608 may be used to store data and / or instructions 614. NVM / storage 608 may include any suitable non-volatile memory (e.g., flash memory) and / or may include any suitable non-volatile storage device(s) (e.g., one or more hard disk drives (HDDs), one or more compact disk (CD) drives, and / or one or more digital versatile disk (DVD) drives).

[0331] NVM / storage device 608 may include storage resources that are physically part of the device on which apparatus 600 is installed, or it may be accessible to the device without being part of the device. For example, NVM / storage device 608 may be accessed via input / output device(s) 610 over a network.

[0332] (One or more) input / output devices 610 may provide an interface for the apparatus 600 to communicate with any other appropriate device, and the input / output device 610 may include a communication component, an audio component, a sensor component, etc. The network interface 612 may provide an interface for the apparatus 600 to communicate through one or more networks, and the apparatus 600 may wirelessly communicate with one or more components of a wireless network according to any of one or more wireless network standards and / or protocols, for example, access a wireless network based on a communication standard, such as WiFi, 2G, 3G, 4G, 5G, etc., or a combination thereof for wireless communication.

[0333] For one embodiment, at least one of the processor(s) 602 may be packaged together with the logic of one or more controllers (e.g., a memory controller module) of the control module 604. For one embodiment, at least one of the processor(s) 602 may be packaged together with the logic of one or more controllers of the control module 604 to form a system-in-package (SiP). For one embodiment, at least one of the processor(s) 602 may be integrated on the same die with the logic of one or more controllers of the control module 604. For one embodiment, at least one of the processor(s) 602 may be integrated on the same die with the logic of one or more controllers of the control module 604 to form a system-on-chip (SoC).

[0334] In various embodiments, the device 600 may be, but is not limited to, a terminal device such as a server, a desktop computing device, or a mobile computing device (e.g., a laptop computing device, a handheld computing device, a tablet computer, a netbook, etc.). In various embodiments, the device 600 may have more or fewer components and / or different architectures. For example, in some embodiments, the device 600 includes one or more cameras, a keyboard, a liquid crystal display (LCD) screen (including a touch screen display), a non-volatile memory port, multiple antennas, a graphics chip, an application-specific integrated circuit (ASIC), and a speaker.

[0335] Among them, the main control chip can be used as a processor or control module in the detection device, sensor data, location information, etc. are stored in a memory or NVM / storage device, the sensor group can be used as an input / output device, and the communication interface may include a network interface.

[0336] As for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0337] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0338] The embodiments of the present application are described with reference to the flowcharts and / or block diagrams of the methods, terminal devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable simulation data acquisition terminal device to generate a machine, so that the instructions executed by the processor of the computer or other programmable simulation data acquisition terminal device generate instructions for implementing the process in the flowchart. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0339] These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable simulation data acquisition terminal device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0340] These computer program instructions can also be loaded onto a computer or other programmable simulation data acquisition terminal device, so that a series of operating steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable terminal device provide for implementing the process in the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0341] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present application.

[0342] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the elements.

[0343] The above is a detailed introduction to a simulation data acquisition method and device, an electronic device and a storage medium provided by the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea; at the same time, for general technical personnel in this field, according to the idea of ​​the present application, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A simulation data acquisition method, characterized in that: Applied to an autonomous driving simulation system, the autonomous driving simulation system includes a target virtual vehicle, and the target virtual vehicle is provided with a virtual fisheye camera. The method includes: Creating a target directed map; the target directed map includes waypoints, and the waypoints include at least a starting point and an end point; Traversing the target directed map to obtain a first target route; According to the starting point and the end point on the target directed map, traverse the target directed map to obtain a second target route; Determining a local path planning scheme according to the current position of the target virtual vehicle; Controlling the target virtual vehicle to simulate driving along the first target route or the second target route according to the local path planning scheme; When the target virtual vehicle simulates driving, simulation data acquired by the virtual fisheye camera on the target virtual vehicle is collected.

2. The method according to claim 1, characterized in that: The traversing the target directed map to obtain a first target route includes: Create a first route list; Traversing the waypoints of the target directed map to obtain the waypoint coordinates; Extracting waypoint attribute information corresponding to the waypoint in the target directed map according to the waypoint coordinates; Creating a waypoint object according to the waypoint coordinates and the waypoint attribute information corresponding to the waypoint; The waypoint object is stored in the first route list to obtain a first target route.

3. The method according to claim 1, characterized in that The target directed map includes a path, the waypoint exists on the path, the path is composed of directed edges, and traversing the target directed map according to the starting point and the end point on the target directed map to obtain a second target route includes: Create a second route list; Acquire a target route consisting of the paths between the starting point and the end point on the target directed map; storing the starting point and the end point in the second route list; Obtaining the current waypoint on the target route; Determine the edge type of the target directed edge where the current waypoint is located; If the target directed edge is of a non-lane following type, obtaining the next directed edge of the target directed edge; Determine whether there is a corresponding index path for the next directed edge; If the next directed edge has a corresponding index path, obtaining the index waypoint closest to the current waypoint on the index path; If the next directed edge does not have a corresponding index path, obtaining the index waypoint of the exit of the next directed edge; If the target directed edge is of lane following type, obtaining an index path corresponding to the target directed edge; Acquire the index waypoint on the index path; The current waypoint and the index waypoint are stored in the second route list to obtain a second target route including all waypoints on the target route.

4. The method according to claim 3, characterized in that The method further comprises: If the current waypoint is located before the end point and the distance between the current waypoint and the end point is less than a first preset distance, or if the current waypoint is located after the end point, then the acquisition of the current waypoint on the target route is stopped.

5. The method according to claim 1, characterized in that Determining a local path planning scheme according to the current position of the target virtual vehicle includes: Obtaining the current positions of the target virtual vehicle and other virtual vehicles; Determining whether a bounding box detection is required according to a current position of the target virtual vehicle and the first target route or the second target route; If bounding box detection is required, then generating bounding boxes corresponding to the target virtual vehicle, the other virtual vehicles and the remaining routes respectively; the remaining routes are the remaining target routes that the target virtual vehicle needs to travel; Determine the intersection between the bounding box of the target virtual vehicle and the bounding boxes of the other virtual vehicles, and between the bounding box of the target virtual vehicle and the bounding boxes of the remaining target routes to be traveled; A local path planning scheme is formulated according to the intersection situation.

6. The method according to claim 1, characterized in that The target directed map has a corresponding traffic light list, the target directed map includes a path, the waypoint exists on the path, and the local path planning scheme is determined according to the current position of the target virtual vehicle, including: Obtaining the current position of the target virtual vehicle; According to the current position of the target virtual vehicle and the first target route or the second target route, traverse the traffic light list; Obtaining a target traffic light closest to the current position of the target virtual vehicle; If the distance between the target traffic light and the target virtual vehicle is less than a second preset distance, and the path of the target traffic light is the same as the path of the target virtual vehicle, determining the orientation relationship between the target traffic light and the target virtual vehicle; If the positional relationship between the target traffic light and the target virtual vehicle is that the target traffic light is in front of the target virtual vehicle, obtaining a current lighting state of the target traffic light; A local path planning scheme is formulated according to the current lighting status.

7. The method according to claim 1, characterized in that The first target route and the second target route are composed of the waypoints, and the step of controlling the target virtual vehicle to simulate driving along the first target route or the second target route according to the local path planning scheme includes: On the first target route or the second target route, obtaining a next waypoint corresponding to the current waypoint where the target virtual vehicle is located; Acquire a target speed and a target steering angle of the target virtual vehicle according to the local path planning scheme; Calculating a target throttle value and a target brake value of the target virtual vehicle according to the target speed; Calculating a target steering value of the target virtual vehicle according to the target steering angle; The target throttle value, the target brake value, and the target steering value are used to control the target virtual vehicle to simulate driving toward the next waypoint.

8. A simulation data acquisition device, characterized in that: Applied to an autonomous driving simulation system, the autonomous driving simulation system includes a target virtual vehicle, the target virtual vehicle is provided with a virtual fisheye camera, and the device includes: A map creation module, used to create a target directed map; the target directed map includes waypoints, and the waypoints include at least a starting point and an end point; A first target route planning module, used for traversing the target directed map to obtain a first target route; A second target route planning module, configured to traverse the target directed map according to the starting point and the end point on the target directed map to obtain a second target route; A local path planning module, used to determine a local path planning solution according to the current position of the target virtual vehicle; A simulated driving module, used for controlling the target virtual vehicle to simulate driving along the first target route or the second target route according to the local path planning scheme; The simulation data acquisition module is used to collect the simulation data collected by the virtual fisheye camera on the target virtual vehicle when the target virtual vehicle simulates driving.

9. An electronic device, characterized in that: include: processor; and A memory having executable codes stored thereon, which, when executed, enables the processor to execute the simulation data acquisition method as described in one or more of claims 1-7.

10. One or more machine-readable media having executable codes stored thereon, which, when executed, cause a processor to execute the simulation data acquisition method according to one or more of claims 1-7.

Images