Perception algorithm verification method and device for unmanned vehicle
By simulating the laser beam emission behavior of lidar in an unmanned vehicle simulation environment and determining the intersection information, the problem of insufficient accuracy and efficiency in the verification of perception algorithms in the existing technology is solved, and efficient verification of perception algorithms is achieved.
Patent Information
- Application Number
- CN202511072902.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-07-31
- Publication Date
- 2025-11-18
AI Technical Summary
In existing technologies, the accuracy and efficiency of ground truth data in the simulation environment of autonomous vehicle perception algorithms are insufficient, making it impossible to effectively verify the effectiveness of the perception algorithms.
A target simulation scenario is constructed, simulating the projection of laser rays by a lidar in the target simulation environment. The intersection points between the laser rays and the entities are determined by ray tracing or depth mapping technology, generating perception data such as the location information, material information, and semantic features of the intersection points. This data serves as the baseline truth value for the perception algorithm and is used to verify the perception algorithm.
This improves the accuracy and efficiency of perception algorithm verification. By simulating the laser beam emission behavior of lidar in a simulation environment, the intersection information consistent with the perception range of the simulated vehicle is determined, and more realistic perception data is obtained, thus improving the verification effect of the perception algorithm.
Smart Images
Figure CN120971039A_ABST
Abstract
Description
Technical Field
[0001] This disclosure relates to the fields of autonomous driving and simulation technology, and in particular to a method and apparatus for verifying perception algorithms for autonomous vehicles. Background Technology
[0002] In the field of autonomous driving, autonomous vehicles need to use perception algorithms to identify the environmental conditions around the vehicle and provide a basis for downstream planning and control algorithms to plan and output driving paths. To ensure the effectiveness of perception algorithms, they can be verified in a simulation environment. Therefore, data that can serve as the true value of perception data is needed as a benchmark for verifying perception algorithms. In related technologies, the true value of perception algorithms in simulation environments is often obtained through annotation, which is insufficient in both accuracy and efficiency, and cannot provide an appropriate benchmark for verifying perception algorithms. Summary of the Invention
[0003] To overcome the problems existing in related technologies, this disclosure provides a method and apparatus for verifying perception algorithms for unmanned vehicles.
[0004] According to a first aspect of this disclosure, a method for verifying the perception algorithm of an autonomous vehicle is provided, comprising: Construct a target simulation scenario, which includes: a target simulation environment and a target simulation unmanned vehicle; In the target simulation environment, the target simulation unmanned vehicle is run, and the target features within the perception range of the target simulation unmanned vehicle in the target simulation environment are identified using the perception algorithm to be verified, so as to obtain the perception data to be verified. The laser beam is simulated in the target simulation environment to obtain target perception data in the target simulation environment; wherein, the target perception data is associated with the attribute of the first intersection point between the laser beam and the entity in the target simulation environment; The target perception data is used to verify the perception algorithm to be verified.
[0005] In some embodiments, simulating laser beams in the target simulation environment to obtain target perception data in the target simulation environment includes: Using the target location in the target simulation environment as the origin of the lidar, the lidar projects multiple laser beams into the target simulation environment. Based on the emission direction of each laser beam, the first intersection point between each laser beam and the entity in the target simulation environment is determined. The target perception data is obtained based on the first intersection point between each laser beam and an entity in the target simulation environment. The target perception data includes at least one of the following: location information of the intersection point, material information, echo intensity, and semantic features.
[0006] In some embodiments, determining the first intersection point between each laser beam and an entity in the target simulation environment based on the emission direction of each laser beam includes: Put pre-built orientation texture data into the ray tracing pipeline; The directional texture data is sampled to obtain the emission direction of each of the multiple light beams; wherein, the directional texture data is used to show the emission direction of all laser beams emitted by the lidar within one lidar scan cycle; For each of the multiple laser beams, the propagation path of the laser beam is simulated based on its emission direction using ray tracing technology. Then, the intersection of the laser beam with an entity in the target simulation environment is calculated to obtain the position of the first intersection point between the laser beam and the entity in the target simulation environment.
[0007] In some embodiments, the method further includes: Based on the scanning resolution and scanning angle of the specified lidar, the emission direction of multiple laser beams emitted by the lidar within one lidar scanning cycle is calculated. The directional texture data is constructed using the unit direction vectors of the emission directions of the multiple laser beams; wherein the directional texture data is stored in the form of a two-dimensional array, each row of the two-dimensional array is used to record the emission direction of each row of laser beams in the horizontal direction of the lidar, and each column of the two-dimensional array is used to record the emission direction of each column of laser beams in the vertical direction of the lidar.
[0008] In some embodiments, simulating laser beams in the target simulation environment to obtain target perception data in the target simulation environment includes: Using the target setting position in the target simulation environment as the origin of the depth camera's view frustum, an environmental depth map of the target simulation environment is generated; For each laser beam emitted by the lidar, the following processing is performed: Based on the emission direction of the laser beam, the texture coordinates corresponding to the laser beam are determined in the environmental depth map; The depth value of the texture coordinates is obtained by sampling the position corresponding to the texture coordinates in the environment depth map. Using the origin of the view cone as the setting origin of the lidar, extend the distance of the depth value in the direction of the laser beam's emission to obtain the first intersection point between the laser beam and the entity in the target simulation environment; In the target simulation environment, acquire the target perception data of the intersection point; The target perception data includes at least one of the following: location information of the intersection point, material information, echo intensity, and semantic features.
[0009] In some embodiments, generating an environmental depth map of the target simulation environment using the target location in the target simulation environment as the origin of the depth camera's view frustum includes: Using the target setting position in the target simulation environment as the origin of the depth camera's view frustum, depth maps of the simulation environment are rendered at different view frustum angles to obtain multiple depth maps; wherein, the environment depth map includes the multiple depth maps, and the sum of the view frustum angles of the multiple depth maps covers the acquisition angle of the lidar.
[0010] In some embodiments, determining the texture coordinates corresponding to the laser beam in the environmental depth map based on the emission direction of the laser beam includes: Based on the emission direction of the laser beam, the target depth map that intersects with the laser beam is determined from the multiple depth maps; In the target depth map, determine the location of the intersection point between the laser beam and the target depth map; The texture coordinates corresponding to the intersection point in the target depth map are used as the texture coordinates corresponding to the laser beam.
[0011] In some embodiments, sampling the position corresponding to the texture coordinates in the environment depth map to obtain the depth value of the texture coordinates includes: The depth value of the texture coordinate is obtained by sampling the pixel point that is closest to the texture coordinate in the environment depth map.
[0012] In some embodiments, before identifying target features within the perception range of the target simulation unmanned vehicle in the target simulation environment using the perception algorithm to be verified, and obtaining the perception data to be verified, the method further includes: The point cloud data collected by the target simulation unmanned vehicle in the target simulation environment is acquired, and the point cloud data is stored in the GPU's video memory; Based on the acquisition angle of the point cloud data, each frame of point cloud data in the video memory is decomposed into multiple point cloud sub-data, and each point cloud sub-data is sent to the memory one by one; The process of using a perception algorithm to be verified to identify target features within the perception range of the target simulation unmanned vehicle in the target simulation environment, and obtaining perception data to be verified, includes: After each point cloud sub-data is sent to the memory, the point cloud sub-data stored in the memory is input into the sensing algorithm to be verified. The sensing algorithm to be verified processes the point cloud sub-data and outputs the sensing data to be verified.
[0013] According to a second aspect of this disclosure, a perception algorithm verification device for an autonomous vehicle is provided, comprising: A construction module is used to construct a target simulation scenario, which includes a target simulation environment and a target simulation unmanned vehicle. The generation module is used to run the target simulation unmanned vehicle in the target simulation environment and use the perception algorithm to be verified to identify target features within the perception range of the target simulation unmanned vehicle in the target simulation environment to obtain the perception data to be verified. A simulation module is used to simulate laser rays in the target simulation environment to obtain target perception data in the target simulation environment; wherein, the target perception data is associated with the attributes of the first intersection point between the laser ray and the entity in the target simulation environment; The verification module is used to verify the perception algorithm to be verified using the target perception data.
[0014] The solution provided in this disclosure can construct a target simulation scenario, then run a target simulation unmanned vehicle in the target simulation environment, and use the perception algorithm to be verified to identify target features within the perception range of the target simulation unmanned vehicle in the target simulation environment, obtaining the perception data to be verified. Laser rays are simulated in the target simulation environment to obtain target perception data in the target simulation environment. The target perception data is associated with the attributes of the first intersection point between the laser ray and an entity in the target simulation environment. The target perception data can be used to verify the perception algorithm to be verified. This disclosure, by simulating the laser ray emission behavior of a lidar in a simulation environment, can determine intersection point information consistent with the perception range of the simulated vehicle, and further, through pre-configured entity attributes in the simulation scenario, can obtain more realistic perception data, improving the accuracy and efficiency of perception algorithm verification. Attached Figure Description
[0015] Figure 1 This diagram illustrates a flowchart of a perception algorithm verification method for an unmanned vehicle according to an embodiment of the present disclosure.
[0016] Figure 2 A schematic flowchart of a laser beam simulation method according to an embodiment of this disclosure is shown.
[0017] Figure 3 The diagram shows a flowchart of an intersection calculation method according to an embodiment of this disclosure.
[0018] Figure 4 A flowchart illustrating another laser beam simulation method in an embodiment of this disclosure is shown.
[0019] Figure 5 This diagram illustrates the structure of a perception algorithm verification device for an unmanned vehicle according to an embodiment of the present disclosure. Detailed Implementation
[0020] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.
[0021] Furthermore, the terms “first,” “second,” etc., used in this disclosure are used only for distinguishing descriptions and should not be construed as indicating or implying relative importance.
[0022] First, this disclosure provides a method for verifying the perception algorithm of an unmanned vehicle, which can be executed by an electronic device equipped with an unmanned vehicle simulation system.
[0023] Figure 1 This diagram illustrates a flowchart of a perception algorithm verification method for an autonomous vehicle according to an embodiment of the present disclosure. Figure 1 As shown, the method includes the following steps S101 to S104.
[0024] S101, Construct the target simulation scenario, which includes: the target simulation environment and the target simulation unmanned vehicle.
[0025] For example, the target simulation environment can be constructed based on point cloud data collected by the autonomous vehicle in a real scene, or obtained by scanning the real scene with a stereo camera. This method can create a realistic simulation scene. Of course, the target simulation environment can also be arbitrarily set by the user according to actual needs. For example, the user can adjust the position of obstacles, road structure, weather conditions, traffic signs, etc. in the target simulation environment according to actual needs.
[0026] For example, the target simulation unmanned vehicle can be an unmanned vehicle deployed in a target simulation environment for collecting perception data. The vehicle model of the target simulation unmanned vehicle can be a real vehicle model or a simplified model that only retains the general outline or bounding box of the vehicle; this disclosure does not limit this.
[0027] S102, In the target simulation environment, run the target simulation unmanned vehicle and use the perception algorithm to be verified to identify the target features within the perception range of the target simulation unmanned vehicle in the target simulation environment, and obtain the perception data to be verified.
[0028] For example, since the execution of the perception algorithm to be verified depends on the sensor data of the autonomous vehicle, in order to verify the accuracy of the perception algorithm, the types, parameters, and locations of the sensors carried by the target simulated autonomous vehicle can be pre-configured when deploying the model of the target simulated autonomous vehicle in the simulation environment. This allows the target simulated autonomous vehicle to identify target features within its perception range in the target simulation environment based on the pre-configured virtual sensors and the perception algorithm to be verified, thereby obtaining the perception data to be verified. The sensors carried by the target simulated autonomous vehicle may include one or more of the following: lidar sensors, vision sensors, inertial measurement units, and ultrasonic sensors. The perception algorithm to be verified can generate the perception data based on multimodal sensor data.
[0029] S103, simulate laser beams in the target simulation environment to obtain target perception data in the target simulation environment.
[0030] Among them, the target perception data is associated with the properties of the first intersection point between the laser beam and the entity in the target simulation environment.
[0031] For example, the laser beam emission behavior of a lidar sensor can be simulated in a simulated scenario to determine the first intersection point between the laser beam and the target entity in the simulated environment, thereby obtaining realistic point cloud data. Based on this, the location information, material information, echo intensity, semantic features, and other attributes of the intersection point can be further extracted from the simulated environment. These intersection point attributes extracted from the simulated environment can serve as target perception data, i.e., the true values of the perception data, for verifying the perception algorithm under test.
[0032] S104, using target perception data, to verify the perception algorithm to be verified.
[0033] For example, the target perception data can be organized into a benchmark ground truth dataset containing the same perception data type as the perception data to be verified. For instance, assuming the perception data to be verified contains the location information and semantic features of the target object, the target perception data can also be constructed into a benchmark ground truth dataset containing location information and semantic features. By comparing the target perception data and the perception data to be verified frame by frame, the error between the two can be quantitatively evaluated, thereby verifying the effectiveness of the perception algorithm to be verified. Furthermore, the parameters of the perception algorithm to be verified can be adjusted based on the obtained error results to obtain a more accurate perception algorithm.
[0034] This embodiment of the disclosure simulates the laser beam emission behavior of a lidar in a simulation environment, thereby determining intersection information consistent with the perception range of the simulated vehicle. Furthermore, by using pre-configured entity attributes in the simulation scene, more realistic perception data can be obtained, improving the accuracy and efficiency of perception algorithm verification.
[0035] In some embodiments, laser beams can be simulated in a target simulation environment using ray tracing technology to obtain target perception data in the target simulation environment.
[0036] Please refer to Figure 2 , Figure 2 A schematic flowchart of a laser beam simulation method according to an embodiment of this disclosure is shown, as follows: Figure 2 As shown, the method includes the following steps S201 to S203.
[0037] S201 uses the target location in the target simulation environment as the origin of the lidar, and simulates the lidar to project multiple laser beams into the target simulation environment.
[0038] For example, the target setting location can be the LiDAR setting location of the target simulation unmanned vehicle, so that the target perception data obtained by simulating laser rays can be spatially aligned with the perception data to be verified, so as to verify the perception algorithm to be verified.
[0039] S202, based on the emission direction of each laser beam, determine the first intersection point between each laser beam and the entity in the target simulation environment.
[0040] For example, ray tracing technology can be used to calculate the first intersection point between a laser ray and an entity in the target simulation environment. Ray tracing technology is a computer graphics rendering technique based on physical optics principles, capable of simulating the propagation path of light in a 3D scene (including interactive behaviors such as reflection, refraction, and shadows). Using ray tracing technology, the path propagation of a laser ray can be simulated relatively well in the simulation environment to determine the first intersection point between the laser ray and an entity in the target simulation environment.
[0041] S203: Target perception data is obtained based on the first intersection point between each laser beam and an entity in the target simulation environment.
[0042] The target perception data includes at least one of the following: location information of the intersection point, material information, echo intensity, and semantic features. After the first intersection point is determined, the spatial coordinates of the intersection point are the location information of the intersection point. As for the material information and semantic features, they can be obtained by using the attributes of the entity being intersected in the simulation environment. The echo intensity can be further calculated based on the material information.
[0043] In some embodiments, please refer to Figure 3 The above S202 can be implemented in the following ways: S2021 places pre-built orientation texture data into the ray tracing pipeline.
[0044] S2022, sample the directional texture data to obtain the emission direction of each ray in the multiple rays.
[0045] S2023, for each of the multiple laser beams, according to the emission direction of the laser beam, and based on ray tracing technology, the propagation path of the laser beam is simulated, and the intersection of the laser beam with the entity in the target simulation environment is calculated to obtain the position of the first intersection point between the laser beam and the entity in the target simulation environment.
[0046] In this embodiment, the orientation texture data is used to show the emission direction of all laser beams emitted by the lidar within one lidar scan cycle.
[0047] For example, directional texture data can be constructed as follows: based on the scanning resolution and scanning angle of a specified lidar, the emission directions of multiple laser beams emitted by the lidar within one lidar scan cycle are calculated. The directional texture data is then constructed using the unit direction vectors of the emission directions of the multiple laser beams.
[0048] In other words, the orientation texture data records the laser beam emission direction of the actual LiDAR. To eliminate differences caused by different sensor performance during the verification process of the perception algorithm to be verified, the LiDAR sensor deployed on the target simulated unmanned vehicle can be used as the designated LiDAR, so that the laser beam emission direction recorded in the orientation texture data is consistent with the laser beam emission direction of the LiDAR sensor on the target simulated unmanned vehicle.
[0049] For example, the orientation texture data can be stored in the form of a two-dimensional array for retrieval during ray tracing. Each row of the two-dimensional array records the emission direction of each row of laser rays in the horizontal direction of the lidar, and each column records the emission direction of each column of laser rays in the vertical direction of the lidar.
[0050] During simulation runtime, by incorporating directional texture data into the hardware ray tracing pipeline, ray tracing technology can be used to render the propagation path of laser rays in their emission direction. Combined with the position and pose of the lidar, the unit direction vector of any laser ray in the world coordinate system can be obtained. After calculating the unit direction vector of the laser ray in the simulation environment, the hardware ray tracing core can be used to solve in parallel for the first intersection point of each laser ray with an entity in the 3D simulation scene. The set of the first intersection points of all laser rays emitted by the lidar within one scan cycle with entities can be used as the single-frame point cloud data acquired by the lidar.
[0051] Since directional texture data only needs to be generated once based on the parameters of a specified LiDAR, and hardware ray tracing technology has a high rendering rate, the method provided in this embodiment can quickly calculate the intersection point of the laser ray and the entity. Furthermore, because ray tracing technology, in its implementation principle, directly calculates the intersection point through line-surface intersection, the results obtained also have high accuracy.
[0052] In some embodiments, a depth map can be used to simulate laser beams in a target simulation environment to obtain target perception data in the target simulation environment.
[0053] In a simulated scene, a depth camera can capture and render the scene within its view frustum using perspective projection. During this process, a depth map is generated to determine the depth occlusion relationships of pixels within the perspective projection. The depth map records the distances between several rays arranged in a matrix emanating from the depth camera's view frustum and their nearest intersection points in the simulated scene.
[0054] For example, the target location in the target simulation environment can be used as the origin of the depth camera's view frustum to generate an environmental depth map of the target simulation environment. The target location can be the location of the lidar sensor of the target simulation autonomous vehicle, thereby aligning the target perception data obtained through simulated laser rays spatially with the perception data to be verified, thus enabling the verification of the perception algorithm.
[0055] Please refer to Figure 4 After generating the environmental depth map, each laser beam emitted by the lidar can be analyzed separately. Figure 4 The following steps S401 to S404 are performed to obtain target perception data in the target simulation environment.
[0056] S401 determines the texture coordinates corresponding to the laser ray in the environment depth map based on the emission direction of the laser ray.
[0057] S402 samples the position corresponding to the texture coordinates in the environment depth map to obtain the depth value of the texture coordinates.
[0058] S403 uses the origin of the view cone as the setting origin of the lidar, and extends the depth value in the direction of the laser beam to obtain the first intersection point between the laser beam and the entity in the target simulation environment.
[0059] S404: In the target simulation environment, acquire target perception data at the intersection point.
[0060] Understandably, similar to the process of simulating laser beams using ray tracing technology to obtain target perception data, this embodiment also obtains target perception data by calculating the first intersection point between the laser beam and an entity in the simulation environment. Therefore, the types of target perception data that can be obtained are also similar, including at least one of the following: intersection point location information, material information, echo intensity, and semantic features. Once the first intersection point is determined, its spatial coordinates constitute its location information. Material information and semantic features can be obtained from the attributes of the entity in the simulation environment, and the echo intensity can be further calculated based on the material information.
[0061] For example, environmental depth maps can be acquired as follows: using the target location in the simulated environment as the origin of the depth camera's view frustum, depth maps of the simulated environment are rendered at different view frustum angles to obtain multiple depth maps. The environmental depth map comprises multiple depth maps, and the sum of the view frustum angles of these multiple depth maps covers the acquisition angle of the LiDAR.
[0062] Understandably, depth cameras have limited viewing angles, while LiDAR typically covers a 360-degree horizontal acquisition angle. Therefore, the horizontal acquisition angle of the LiDAR can be decomposed into multiple horizontal angle ranges, and multiple depth cameras with overlapping viewing angle origins can be set up to acquire depth maps for different horizontal angle ranges. The depth maps acquired by each depth camera, when superimposed, can cover the horizontal acquisition angle of the LiDAR. For example, at least four depth cameras with a horizontal viewing angle of 90 degrees can be set, with their viewing angles meeting at their boundaries, thus covering a 360-degree horizontal acquisition angle. The vertical viewing angles of each depth camera can be set with reference to the vertical viewing angle of the LiDAR, and can be greater than or equal to the vertical viewing angle of the LiDAR. This setup ensures that the viewing angle range of the environmental depth map completely covers the acquisition angle range of the LiDAR, allowing the sampling of the environmental depth map to obtain the first intersection point between any laser ray emitted by the LiDAR and an entity in the simulated environment.
[0063] For example, S401 described above can be implemented as follows: Based on the emission direction of the laser ray, determine the target depth map that intersects with the laser ray from multiple depth maps. In the target depth map, determine the intersection point between the laser ray and the target depth map. Use the texture coordinates corresponding to the intersection point in the target depth map as the texture coordinates corresponding to the laser ray.
[0064] As mentioned earlier, since the environmental depth map is composed of multiple depth maps, each corresponding to a different horizontal acquisition angle range of the lidar, when determining the texture coordinates corresponding to the laser ray, we can first select the corresponding target depth map (i.e., the depth map that intersects with the laser ray) based on the horizontal and vertical angles of the laser ray, and then determine the corresponding texture coordinates (i.e., the coordinates of the intersection point between the target depth map and the laser ray in the target depth map) within the target depth map.
[0065] For example, the above S402 can be implemented as follows: sample the pixel point in the environment depth map that is closest to the texture coordinates to obtain the depth value of the texture coordinates.
[0066] Understandably, the resolution of the environmental depth map captured by a depth camera is limited, and the texture coordinates corresponding to a laser ray may not exactly correspond to a pixel in the environmental depth map; that is, the laser ray does not overlap with any ray generated by the depth camera when generating the depth map. In this case, an approximate depth value can be obtained by sampling the pixel in the depth map that is closest to the texture coordinates.
[0067] After obtaining the depth value, the origin of the view frustum can be used as the setting origin of the lidar. The depth value is extended to the direction of the laser beam's emission to obtain the first intersection point between the laser beam and the entity in the target simulation environment, and then the target perception data at the intersection point is obtained.
[0068] Although the above methods may have some errors in obtaining depth values, these errors can be reduced by increasing the resolution of the depth map. Furthermore, the method of simulating laser rays based on the depth map does not require the electronic device to support ray tracing technology in hardware, so it can be implemented by any electronic device and has a wider range of applications.
[0069] The above examples, with specific embodiments, illustrate methods for generating target perception data based on ray tracing technology and depth map technology to simulate the emission behavior of laser rays.
[0070] The following section will describe the process of generating the sensing data to be verified, using specific examples.
[0071] In some embodiments, when performing S102 above, point cloud data collected by the target simulation unmanned vehicle in the target simulation environment can be acquired and stored in the GPU's video memory. Subsequently, based on the acquisition angle of the point cloud data, each frame of point cloud data in the video memory can be decomposed into multiple point cloud sub-data, and each point cloud sub-data can be sent to memory one by one. After each point cloud sub-data is sent to memory, the point cloud sub-data stored in memory can be input into the perception algorithm to be verified, and the perception algorithm to be verified can process the point cloud sub-data and output the perception data to be verified.
[0072] For example, when decomposing point cloud data into point cloud sub-data, the decomposition can be based on the horizontal acquisition angle of the point cloud data. For instance, the point cloud data can be decomposed into three point cloud sub-data with a horizontal acquisition angle of 120 degrees, or four point cloud sub-data with a horizontal acquisition angle of 90 degrees.
[0073] Furthermore, by decomposing the point cloud data collected in the simulation environment and transferring it one by one to memory for processing, the efficiency of transferring point cloud data can be improved, thereby improving the efficiency of converting point cloud data into sensing data to be verified. That is, there is no need to wait for each frame of point cloud data to be completely transferred to memory before using the sensing algorithm to be verified to process it.
[0074] Based on the same inventive concept, this disclosure also provides a perception algorithm verification device for autonomous vehicles, as shown in the following embodiment. Since the principle of solving the problem in this embodiment of the perception algorithm verification device for autonomous vehicles is the same as that described above... Figure 1 The method embodiments shown are similar, therefore the implementation of the perception algorithm verification device embodiment for this unmanned vehicle can refer to the above. Figure 1 The implementation of the method embodiments shown will not be repeated here.
[0075] Figure 5 This diagram illustrates the structure of a perception algorithm verification device for an unmanned vehicle according to an embodiment of this disclosure. Figure 5 As shown, the perception algorithm verification device 500 for the unmanned vehicle includes: a construction module 501, a generation module 502, a simulation module 503, and a verification module 504.
[0076] Among them, the construction module 501 is used to construct the target simulation scene, which includes the target simulation environment and the target simulation unmanned vehicle.
[0077] The generation module 502 is used to run the target simulation unmanned vehicle in the target simulation environment and use the perception algorithm to be verified to identify the target features within the perception range of the target simulation unmanned vehicle in the target simulation environment, and obtain the perception data to be verified.
[0078] The simulation module 503 is used to simulate the laser beam in the target simulation environment to obtain target perception data in the target simulation environment; wherein, the target perception data is associated with the attributes of the first intersection point between the laser beam and the entity in the target simulation environment.
[0079] The verification module 504 is used to verify the perception algorithm to be verified using the target perception data.
[0080] In some embodiments, the simulation module 503 is specifically used to: use the target setting position in the target simulation environment as the setting origin of the lidar, and simulate the lidar to project multiple laser beams into the target simulation environment; determine the first intersection point between each laser beam and an entity in the target simulation environment based on the emission direction of each laser beam; and obtain target perception data based on the first intersection point between each laser beam and an entity in the target simulation environment; wherein the target perception data includes at least one of the following: location information of the intersection point, material information, echo intensity, and semantic features.
[0081] In some embodiments, the simulation module 503 is specifically used to: place pre-built directional texture data into the ray tracing pipeline; sample the directional texture data to obtain the emission direction of each of the multiple laser beams; wherein the directional texture data is used to show the emission direction of all laser beams emitted by the lidar in one radar scan cycle; for each of the multiple laser beams, simulate the propagation path of the laser beam according to its emission direction and based on ray tracing technology, and perform line-surface intersection calculation between the laser beam and an entity in the target simulation environment to obtain the intersection point position of the first intersection point between the laser beam and the entity in the target simulation environment.
[0082] In some embodiments, the simulation module 503 is specifically used to calculate the emission direction of multiple laser beams emitted by the lidar within one lidar scanning cycle based on the specified lidar scanning resolution and scanning angle; and to construct directional texture data using the unit direction vector of the emission direction of the multiple laser beams; wherein the directional texture data is stored in the form of a two-dimensional array, each row of the two-dimensional array is used to record the emission direction of each row of laser beams in the horizontal direction of the lidar, and each column of the two-dimensional array is used to record the emission direction of each column of laser beams in the vertical direction of the lidar.
[0083] In some embodiments, the simulation module 503 is specifically used to generate an environmental depth map of the target simulation environment, using the target setting position in the target simulation environment as the origin of the depth camera's view frustum; for each laser beam emitted by the lidar, the following processing is performed: based on the emission direction of the laser beam, the texture coordinates corresponding to the laser beam are determined in the environmental depth map; the position corresponding to the texture coordinates in the environmental depth map is sampled to obtain the depth value of the texture coordinates; using the view frustum as the setting origin of the lidar, the depth value is extended to the emission direction of the laser beam to obtain the first intersection point between the laser beam and the entity in the target simulation environment; in the target simulation environment, the target perception data of the intersection point is acquired; wherein, the target perception data includes at least one of the following: the position information, material information, echo intensity, and semantic features of the intersection point.
[0084] In some embodiments, the simulation module 503 is specifically used to render depth maps of the simulation environment with different view frustums, using the target setting position of the target simulation environment as the origin of the depth camera's view frustum, to obtain multiple depth maps; wherein, the environment depth map includes multiple depth maps, and the sum of the view frustums of the multiple depth maps covers the acquisition angle of the lidar.
[0085] In some embodiments, the simulation module 503 is specifically used to: determine a target depth map that intersects with the laser beam in multiple depth maps based on the emission direction of the laser beam; determine the intersection point of the laser beam and the target depth map in the target depth map; and use the texture coordinates of the intersection point in the target depth map as the texture coordinates of the laser beam.
[0086] In some embodiments, the simulation module 503 is specifically used to sample the pixel point in the environment depth map that is closest to the texture coordinates to obtain the depth value of the texture coordinates.
[0087] In some embodiments, the autonomous vehicle perception algorithm verification device 500 further includes a transmission module (not shown in the figure). The transmission module is used to acquire point cloud data collected by the target simulated autonomous vehicle in the target simulation environment and store the point cloud data in the GPU's video memory; based on the acquisition angle of the point cloud data, it decomposes each frame of point cloud data in the video memory into multiple point cloud sub-data, and sends each point cloud sub-data to memory one by one. The verification module 504 is specifically used to, after each point cloud sub-data is sent to memory, input the point cloud sub-data stored in memory into the perception algorithm to be verified, process the point cloud sub-data through the perception algorithm to be verified, and output the perception data to be verified.
[0088] Through the description of the various embodiments above, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of this disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) or on a network, including several instructions to cause a computing device (such as a personal computer, server, mobile terminal, or network device, etc.) to execute the methods according to the embodiments of this disclosure.
[0089] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the appended claims.
Claims
1. A method for verifying the perception algorithm of an autonomous vehicle, characterized in that, include: Construct a target simulation scenario, which includes: a target simulation environment and a target simulation unmanned vehicle; In the target simulation environment, the target simulation unmanned vehicle is run, and the target features within the perception range of the target simulation unmanned vehicle in the target simulation environment are identified using the perception algorithm to be verified, so as to obtain the perception data to be verified. The laser beam is simulated in the target simulation environment to obtain target perception data in the target simulation environment; wherein, the target perception data is associated with the attribute of the first intersection point between the laser beam and the entity in the target simulation environment; The target perception data is used to verify the perception algorithm to be verified.
2. The method according to claim 1, characterized in that, The process of simulating laser beams in the target simulation environment to obtain target perception data in the target simulation environment includes: Using the target location in the target simulation environment as the origin of the lidar, the lidar projects multiple laser beams into the target simulation environment. Based on the emission direction of each laser beam, the first intersection point between each laser beam and the entity in the target simulation environment is determined. The target perception data is obtained based on the first intersection point between each laser beam and an entity in the target simulation environment. The target perception data includes at least one of the following: location information of the intersection point, material information, echo intensity, and semantic features.
3. The method according to claim 2, characterized in that, The determination of the first intersection point between each laser beam and an entity in the target simulation environment, based on the emission direction of each laser beam, includes: Put pre-built orientation texture data into the ray tracing pipeline; The directional texture data is sampled to obtain the emission direction of each of the multiple light beams; wherein, the directional texture data is used to show the emission direction of all laser beams emitted by the lidar within one lidar scan cycle; For each of the multiple laser beams, the propagation path of the laser beam is simulated based on its emission direction using ray tracing technology. Then, the intersection of the laser beam with an entity in the target simulation environment is calculated to obtain the position of the first intersection point between the laser beam and the entity in the target simulation environment.
4. The method according to claim 3, characterized in that, The method further includes: Based on the scanning resolution and scanning angle of the specified lidar, the emission direction of multiple laser beams emitted by the lidar within one lidar scanning cycle is calculated. The directional texture data is constructed using the unit direction vectors of the emission directions of the multiple laser beams; wherein the directional texture data is stored in the form of a two-dimensional array, each row of the two-dimensional array is used to record the emission direction of each row of laser beams in the horizontal direction of the lidar, and each column of the two-dimensional array is used to record the emission direction of each column of laser beams in the vertical direction of the lidar.
5. The method according to claim 1, characterized in that, The process of simulating laser beams in the target simulation environment to obtain target perception data in the target simulation environment includes: Using the target setting position in the target simulation environment as the origin of the depth camera's view frustum, an environmental depth map of the target simulation environment is generated; For each laser beam emitted by the lidar, the following processing is performed: Based on the emission direction of the laser beam, the texture coordinates corresponding to the laser beam are determined in the environmental depth map; The depth value of the texture coordinates is obtained by sampling the position corresponding to the texture coordinates in the environment depth map. Using the origin of the view cone as the setting origin of the lidar, extend the distance of the depth value in the direction of the laser beam's emission to obtain the first intersection point between the laser beam and the entity in the target simulation environment; In the target simulation environment, acquire the target perception data of the intersection point; The target perception data includes at least one of the following: location information of the intersection point, material information, echo intensity, and semantic features.
6. The method according to claim 5, characterized in that, The step of generating an environmental depth map of the target simulation environment, using the target location in the target simulation environment as the origin of the depth camera's view frustum, includes: Using the target setting position in the target simulation environment as the origin of the depth camera's view frustum, depth maps of the simulation environment are rendered at different view frustum angles to obtain multiple depth maps; wherein, the environment depth map includes the multiple depth maps, and the sum of the view frustum angles of the multiple depth maps covers the acquisition angle of the lidar.
7. The method according to claim 6, characterized in that, Determining the texture coordinates corresponding to the laser beam in the environmental depth map based on the emission direction of the laser beam includes: Based on the emission direction of the laser beam, the target depth map that intersects with the laser beam is determined from the multiple depth maps; In the target depth map, determine the location of the intersection point between the laser beam and the target depth map; The texture coordinates corresponding to the intersection point in the target depth map are used as the texture coordinates corresponding to the laser beam.
8. The method according to claim 5, characterized in that, The step of sampling the position corresponding to the texture coordinates in the environment depth map to obtain the depth value of the texture coordinates includes: The depth value of the texture coordinate is obtained by sampling the pixel point that is closest to the texture coordinate in the environment depth map.
9. The method according to claim 1, characterized in that, Before using the perception algorithm to be verified to identify target features within the perception range of the target simulation unmanned vehicle in the target simulation environment and obtain the perception data to be verified, the method further includes: The point cloud data collected by the target simulation unmanned vehicle in the target simulation environment is acquired, and the point cloud data is stored in the GPU's video memory; Based on the acquisition angle of the point cloud data, each frame of point cloud data in the video memory is decomposed into multiple point cloud sub-data, and each point cloud sub-data is sent to the memory one by one; The process of using a perception algorithm to be verified to identify target features within the perception range of the target simulation unmanned vehicle in the target simulation environment, and obtaining perception data to be verified, includes: After each point cloud sub-data is sent to the memory, the point cloud sub-data stored in the memory is input into the sensing algorithm to be verified. The sensing algorithm to be verified processes the point cloud sub-data and outputs the sensing data to be verified.
10. A perception algorithm verification device for an unmanned vehicle, characterized in that, include: A construction module is used to construct a target simulation scenario, which includes a target simulation environment and a target simulation unmanned vehicle. The generation module is used to run the target simulation unmanned vehicle in the target simulation environment and use the perception algorithm to be verified to identify target features within the perception range of the target simulation unmanned vehicle in the target simulation environment to obtain the perception data to be verified. A simulation module is used to simulate laser rays in the target simulation environment to obtain target perception data in the target simulation environment; wherein, the target perception data is associated with the attributes of the first intersection point between the laser ray and the entity in the target simulation environment; The verification module is used to verify the perception algorithm to be verified using the target perception data.