Test method and device for autonomous driving perception results

Abstract

The application discloses a kind of automatic driving perception result test method and device. Among them, the method comprises: obtaining test sample carrying sample label, the test sample is used to simulate the journey image collected by intelligent car in the course of driving, and the sample label is used to identify the object in the test sample;Using image algorithm, the object in the test sample is identified, and the identification result is obtained, wherein the image algorithm is the algorithm used on the intelligent car;According to the identification result and the sample label, the test result of the image algorithm is determined.The application solves the technical problem that the efficiency of algorithm test in the related art is relatively low.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of automatic driving, in particular to a method and device for testing automatic driving perception results. BACKGROUND

[0002] With the rapid development of the field of automatic driving, various perception schemes are also rapidly updated and iterated, mainly including vision-based, laser radar-based, and vision and laser radar fusion-based perception schemes. From the current large-scale mass-produced vehicles with assisted driving or automatic driving functions, the vision-based perception scheme is still the mainstream scheme at present.

[0003] In the image algorithm development process and when performing automatic testing of image algorithms on a vehicle-mounted embedded terminal, how to quantitatively evaluate the perception results given by the algorithm is a prerequisite for subsequent algorithm iteration and optimization. At present, the perception results given by the algorithm are mainly evaluated by manual means, for example, by one or more experts to score, which not only has high manpower and material costs, but also cannot guarantee accuracy.

[0004] At present, no effective solution has been proposed for the above problems. SUMMARY

[0005] Embodiments of the present application provide a method and device for testing automatic driving perception results to at least solve the technical problem of low efficiency of algorithm testing in related technologies.

[0006] According to an aspect of an embodiment of the present application, a method for testing automatic driving perception results is provided, including: obtaining a test sample carrying a sample label, wherein the test sample is used to simulate a travel image collected by an intelligent vehicle during driving, and the sample label is used to identify an object in the test sample; identifying the object in the test sample using an image algorithm to obtain an identification result, wherein the image algorithm is an algorithm used on the intelligent vehicle; and determining a test result of the image algorithm according to the identification result and the sample label.

[0007] Optionally, obtaining the test sample carrying the sample label includes: generating the test sample using point cloud collected by a laser radar and an original image collected by a camera; and identifying the object in the test sample to generate the sample label of the test sample.

[0008] Optionally, generating the test sample using the point cloud collected by the laser radar and the original image collected by the camera includes: obtaining the point cloud collected by the laser radar and an original image collected by the camera at the same time; and projecting points in the point cloud onto pixel points at the same position in the original image to obtain the test sample.

[0009] Optionally, the point cloud collected by the laser radar and the original image collected by the camera at the same time are acquired by triggering the camera to collect an original image in a pulse triggered manner when the laser radar collects a frame of point cloud, so as to ensure that the point cloud collected by the laser radar and the image collected by the camera are synchronized in the time domain.

[0010] Optionally, the points in the point cloud are projected onto the pixel points at the same position in the original image to obtain the test sample, including: using an external parameter calibration algorithm to align the coordinate system of the laser radar to the coordinate system of the camera, and then establishing a mapping relationship between the points in the point cloud and the pixel points in the original image according to the camera internal parameter, so as to ensure that the point cloud collected by the laser radar and the image collected by the camera are synchronized in space; using the mapping relationship to project each point in the point cloud onto the pixel points at the same position in the original image to obtain the test sample.

[0011] Optionally, the objects in the test sample are identified to generate a sample label of the test sample, including: inputting the test sample into a 3D target detection network, inputting a perception result output by the 3D target detection network into a multi-target tracking algorithm to obtain a fused perception result, wherein the perception result is used to represent the objects identified by the 3D target detection network from the test sample, and the fused perception result is used to represent the objects identified by the multi-target tracking algorithm from the test sample; and generating the sample label of the test sample according to the fused perception result.

[0012] Optionally, generating the sample label of the test sample according to the fused perception result includes: displaying the test sample and the fused perception result on a GUI tool; in a case where a confirmation operation is detected, taking the fused perception result as the sample label of the test sample; and in a case where a correction operation on the fused perception result is detected, taking the corrected content as the sample label of the test sample.

[0013] According to another aspect of the embodiments of the present application, an automatic driving perception result testing device is also provided, including: an acquisition unit configured to acquire a test sample carrying a sample label, wherein the test sample is used to simulate a travel image collected by an intelligent vehicle during driving, and the sample label is used to identify an object in the test sample; an identification unit configured to identify the object in the test sample using an image algorithm to obtain an identification result, wherein the image algorithm is an algorithm used on the intelligent vehicle; and a testing unit configured to determine a test result of the image algorithm according to the identification result and the sample label.

[0014] Optionally, the acquisition unit is further configured to: generate the test sample by using the point cloud collected by the lidar and the original image collected by the camera; and identify the object in the test sample to generate a sample label of the test sample.

[0015] Optionally, the acquisition unit is further configured to: acquire the point cloud collected by the lidar and the original image collected by the camera at the same time; and project the points in the point cloud onto the pixel points at the same positions in the original image to obtain the test sample.

[0016] Optionally, the acquisition unit is further configured to: when the lidar collects a frame of point cloud, trigger the camera to collect a frame of original image in a pulse trigger manner to ensure that the point cloud collected by the lidar and the image collected by the camera are synchronized in time domain.

[0017] Optionally, the acquisition unit is further configured to: use an extrinsic parameter calibration algorithm to map the coordinate system of the lidar into the coordinate system of the camera, and then establish a mapping relationship between the points in the point cloud and the pixel points in the original image according to the intrinsic parameter of the camera to ensure that the point cloud collected by the lidar and the image collected by the camera are synchronized in space; and project each point in the point cloud onto the pixel points at the same positions in the original image by using the mapping relationship to obtain the test sample.

[0018] Optionally, the acquisition unit is further configured to: input the test sample into a 3D target detection network, input a perception result output by the 3D target detection network into a multi-target tracking algorithm to obtain a fused perception result, wherein the perception result is used to represent the object identified by the 3D target detection network from the test sample, and the fused perception result is used to represent the object identified by the multi-target tracking algorithm from the test sample; and generate a sample label of the test sample according to the fused perception result.

[0019] Optionally, the acquisition unit is further configured to: display the test sample and the fused perception result on a GUI tool; in a case where a confirmation operation is detected, use the fused perception result as the sample label of the test sample; and in a case where a correction operation on the fused perception result is detected, use the corrected content as the sample label of the test sample.

[0020] According to another aspect of the embodiments of the present application, an electronic device is provided, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor executes the method described above by using the computer program.

[0021] According to an aspect of the present application, a computer program product or computer program is provided, which comprises computer instructions stored in a computer readable storage medium. A processor of a computer device reads the computer instructions from the computer readable storage medium, and the processor executes the computer instructions, so that the computer device performs the steps of any of the above-mentioned methods.

[0022] By applying the technical solution of the present application, a test sample carrying a sample label is obtained, an object in the test sample is identified using an image algorithm to obtain an identification result, and a test result of the image algorithm is determined according to the identification result and the sample label. By providing a standard answer in the form of a test sample, the testing of the image algorithm can be automatically completed throughout the process without human intervention, and the technical problem of low efficiency of algorithm testing in the related art can be solved.

[0023] In addition to the purposes, features and advantages described above, the present application has other purposes, features and advantages. The present application will be further described below with reference to the drawings. BRIEF DESCRIPTION OF DRAWINGS

[0024] The drawings accompanying the specification of the present application serve to provide a further understanding of the present application, the illustrative embodiments thereof, and the description thereof, and do not constitute an improper limitation of the present application. In the drawings:

[0025] Figure 1 is a flowchart of an optional test method for automatic driving perception results according to an embodiment of the present application;

[0026] Figure 2 is a schematic diagram of an optional test scheme for automatic driving perception results according to an embodiment of the present application;

[0027] Figure 3 is a schematic diagram of an optional test device for automatic driving perception results according to an embodiment of the present application. DETAILED DESCRIPTION

[0028] In order to enable persons skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by persons skilled in the art without creative labor should fall within the scope of protection of the present application.

[0029] It is to be understood that the terms "first", "second", and the like in the description and in the claims of the present application are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the use of data "or" herein includes that a selection maybe made from the group of items that are conjunctively described. Furthermore, the terms "comprise", "comprising", "include", "including", and the like are to be construed open-ended, i.e. in the sense of "including, but not limited to", to indicate the inclusion of a process, method, product, or apparatus with not only those steps or elements specified, but also other possible steps or elements not specified.

[0030] In the image algorithm development process and in the automatic testing of image algorithms on the vehicle-mounted embedded end, how to quantitatively evaluate the perception results given by the algorithm is the premise of subsequent algorithm iteration optimization. At present, the perception results given by the algorithm are mainly evaluated by manual, for example, by one or more experts to score, which not only has high cost of manpower and material resources, but also cannot guarantee the accuracy. Obviously, a "standard answer" is needed to be used as a reference to evaluate these perception results and to measure the performance of the image algorithm.

[0031] Therefore, a system needs to be constructed to automatically generate these "standard answers" at a low cost, and finally determine the final "standard answer" after review, which is called "ground truth" (i.e. the test sample carrying the sample label of the present application), and then the detection results of each frame can be automatically evaluated according to these "ground truth", so that the automatic testing of the image algorithm can be performed, and this system is called "ground truth system".

[0032] According to an aspect of an embodiment of the present application, a method for testing the perception results of autonomous driving is provided. The method for testing the perception results of autonomous driving can be executed by a server or a terminal. The terminal can also execute the method for testing the perception results of autonomous driving by a client installed thereon. Figure 1 is a flowchart of an optional method for testing the perception results of autonomous driving according to an embodiment of the present application, as shown in Figure 1 The method can include the following steps:

[0033] In step S102, a test sample carrying a sample label is obtained, the test sample is used to simulate the travel images collected by an intelligent vehicle during driving, and the sample label is used to identify the objects in the test sample, such as pedestrians, other vehicles, bicycles, electric vehicles, obstacles, etc.

[0034] 1) can first obtain the point cloud collected by the laser radar and the original image collected by the camera at the same time, for example, when the laser radar collects a frame of point cloud, the camera collects a frame of original image in a pulse triggered manner, to ensure that the point cloud collected by the laser radar and the image collected by the camera are synchronized in time domain;

[0035] 2) project the points in the point cloud onto the pixel points at the same position in the original image to obtain the test samples. For example, using an external parameter calibration algorithm, the coordinate system of the laser radar is mapped into the coordinate system of the camera, and then the mapping relationship between the points in the point cloud and the pixel points in the original image is established according to the camera internal parameter, to ensure that the point cloud collected by the laser radar and the image collected by the camera are synchronized in space; using the mapping relationship, each point in the point cloud is projected onto the pixel point at the same position in the original image to obtain the test sample.

[0036] For example, first, the vehicle carrying the true value system is stationary at a certain position, then the chessboard calibration board is placed in front of the vehicle at any position, as long as the laser radar and the camera can see the chessboard calibration board in the field of view, then the point cloud data and camera image data at this time are recorded by using the above-mentioned synchronous acquisition method, then the position of the chessboard calibration board is changed, and the above steps are repeated to obtain several frames of the above-mentioned data; the several frames of data obtained above are processed, specifically, first, the three-dimensional coordinates (x, y, z) of the four corner points of the chessboard calibration board are extracted from each frame of laser radar point cloud data, then the pixel coordinates (u, v) of the four corner points of the chessboard calibration board are extracted from each frame of camera image data, because the internal parameter of the camera is provided by the manufacturer when it is manufactured, it is known data, so several groups of three-dimensional coordinates and pixel coordinates are added to the camera internal parameter, and the PnP method is used to solve, to obtain the external parameter transformation matrix of the laser radar coordinate system and the camera coordinate system. Thus, the mapping relationship between the point cloud points and the pixel points can be established.

[0037] 3) identify the objects in the test samples to generate sample labels of the test samples.

[0038] For example, the test sample can be first input into the 3D target detection network, and then the perception result output by the 3D target detection network is input into the multi-target tracking algorithm to obtain a fused perception result, the perception result being used to represent an object identified by the 3D target detection network from the test sample, and the fused perception result being used to represent an object identified by the multi-target tracking algorithm from the test sample; then the sample label of the test sample is generated according to the fused perception result, and the test sample and the fused perception result can be displayed on the GUI tool, in the case that a confirmation operation is detected, the fused perception result is taken as the sample label of the test sample, and in the case that a correction operation on the fused perception result is detected, the corrected content is taken as the sample label of the test sample.

[0039] The perception result is an inference result of the 3D target detection network, including the center point position, length, width, height and heading angle of each detected target in the laser radar coordinate system, that is, 7-dimensional data (x, y, z, l, w, h, theta).

[0040] Due to the mechanism of the Kalman filtering algorithm and the Hungarian data association algorithm in the 3D multi-target tracking algorithm, for any frame of point cloud data, as long as the target in the frame does not completely disappear in the field of view of the laser radar, even if most of it is blocked or only a small part is exposed, it can be tracked. Without the multi-target tracking algorithm, only relying on 3D target detection, due to the sparsity of point cloud and the incompleteness of the overall target point cloud caused by the blocking of the target, almost all of them cannot be detected, so the multi-target tracking algorithm can reduce the false detection and missed detection probability caused by target detection, and improve the accuracy of the fused perception result.

[0041] The ground truth system of the present application not only guarantees high accuracy, but also solves the problem of the lack of spatial distance dimension in image data. Therefore, the selection of hardware cannot rely solely on cameras, and other sensors need to be introduced as a supplement for fusion. Laser radar has good three-dimensional perception capability, and the point cloud formed is relatively dense. It not only can form a correspondence in space with image pixels to achieve synchronous fusion perception, but also can calculate the distance to centimeter level. Therefore, laser radar and camera are the preferred sensors of the ground truth system. The present application comprehensively utilizes the advantages of laser radar three-dimensional perception and camera image category information richness to construct a ground truth system for automatic testing of image algorithms.

[0042] In step S104, an object in the test sample is identified using an image algorithm to obtain an identification result, the image algorithm being an algorithm used on the intelligent vehicle.

[0043] Step S106, determining the test result of the image algorithm according to the recognition result and the sample label.

[0044] For example, one round of testing can test m test samples, if the recognition result of the image algorithm is the same as the sample label n, then the test accuracy is n / m, if the test accuracy needs to reach 99%, the image algorithm can be used in the intelligent automobile, then only need to compare n / m with 99% can be compared.

[0045] Through the above steps, the test sample carrying the sample label is obtained, the object in the test sample is identified by using the image algorithm, the recognition result is obtained, and the test result of the image algorithm is determined according to the recognition result and the sample label. By providing the test sample as a standard answer, the test of the image algorithm can be automatically completed throughout the process without manual intervention, and the technical problem of low efficiency of algorithm test in the related art can be solved.

[0046] In addition, the present application combines the respective advantages of laser radar and camera, and obtains a high-precision perception result by offline processing of the data synchronously collected by the two, so as to generate the required true value for image algorithm automatic test. The laser radar point cloud data and image data are aligned in time and space. In the time domain, the problem of synchronous collection of laser radar and camera is solved. In the space domain, the problem of camera external parameter calibration of laser radar and camera distributed in different positions of the vehicle, different installation angles and different FOV angles is solved. The point cloud and image fusion perception result precision is improved, the manual intervention cost is minimized, and the reliability of the true value is ensured.

[0047] As an optional embodiment, the technical solutions of the present application are further described in detail in the specific implementation mode in combination with the contents shown in the following Figure 2

[0048] In the time domain, the synchronization of laser radar point cloud data and camera image data is ensured in a pulse triggered manner. In the space domain, the laser radar coordinate system is transformed into the camera coordinate system, the point cloud is mapped to the image pixel according to the camera internal parameter, the time and space synchronized point cloud and image are input into the 3D target detection network, and a multi-target tracking algorithm is introduced. The fusion perception result output by the algorithm is saved. Through the GUI tool of manual review, the fusion perception result of the above step is read in, the cases of missed detection and false detection are modified, and the final version of the true value is generated. The specific implementation steps are as follows:

[0049] Step 1, laser radar and camera pre-fusion.

[0050] ​First, the image of the camera is semantically segmented. After completing the image semantic segmentation, according to the extrinsic matrix of the laser radar and the camera and the intrinsic matrix of the camera itself, the projection calculation is performed on each point cloud. After filtering out the point cloud points projected outside the image range, the remaining point cloud points are projected onto the pixel points of the semantic segmentation image to obtain the target class represented by the pixel points, thereby giving the original point cloud points a class dimension.

[0051] Step 2, 3D target detection.

[0052] The new point cloud in the previous step is input into the 3D target detection network to obtain the 3D target detection result.

[0053] Step 3, multi-target tracking.

[0054] The 3D target detection result in the previous step is input into the multi-target tracking algorithm to reduce the probability of false detection and missed detection.

[0055] Step 4, manual or machine review.

[0056] After the above steps, the preliminary true value has been automatically generated by the algorithm. In order to make the true value result more reliable, manual review is needed. The true value generated in the above steps is read through the GUI tool, and the original image is referred to for review. If there is false detection or missed detection, the corresponding button in the GUI tool can be clicked to modify the true value. After the review is completed, the true value of all frames is determined, and all operations of the true value system are completed.

[0057] In the technical solution of the present application:

[0058] (1) In a pulse-triggered manner, when the laser radar collects a frame of point cloud, the camera is triggered to expose and collect a frame of image, ensuring the synchronous collection of laser radar point cloud data and camera image data in time domain; using an extrinsic calibration algorithm, the laser radar coordinate system is mapped to the camera coordinate system, and then according to the camera intrinsic parameter, the mapping relationship between the point cloud points and the pixel points is established, and finally the laser radar and the camera are time and space aligned.

[0059] (2) After inputting the synchronous data generated in the above steps into the 3D target detection network, the perception result obtained is output to the multi-target tracking algorithm to further reduce the false detection and missed detection probability caused by target detection and improve the accuracy of the fused perception result.

[0060] (3) The fused perception result generated in the above steps is read into the GUI tool for manual review, and the frame result of the few existing missed detection and false detection is modified to further improve the reliability of the true value given by the previous algorithm.

[0061] This solution addresses the issue of missing ground truth distance values ​​in current automated testing of image algorithms, making the testing of image algorithm perception functions more comprehensive. It establishes a complete workflow for a ground truth system, first using a LiDAR and camera fusion algorithm combined with a multi-target tracking algorithm to provide high-precision ground truth values, and then further improving the reliability of the ground truth values ​​through minimal human intervention.

[0062] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0063] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0064] According to another aspect of the embodiments of this application, an apparatus for testing autonomous driving perception results for implementing the above-described method for testing autonomous driving perception results is also provided. Figure 3 This is a schematic diagram of an optional testing device for autonomous driving perception results according to an embodiment of this application, such as... Figure 3 As shown, the device may include:

[0065] The acquisition unit 31 is used to acquire a test sample carrying a sample label, wherein the test sample is used to simulate the travel images collected by the intelligent vehicle during its journey, and the sample label is used to identify objects in the test sample; the recognition unit 33 is used to use an image algorithm to recognize the objects in the test sample and obtain a recognition result, wherein the image algorithm is an algorithm used on the intelligent vehicle; the testing unit 35 is used to determine the test result of the image algorithm based on the recognition result and the sample label.

[0066] Optionally, the acquisition unit is further configured to: generate the test sample by using the point cloud collected by the lidar and the original image collected by the camera; and identify the object in the test sample to generate a sample label of the test sample.

[0067] Optionally, the acquisition unit is further configured to: acquire the point cloud collected by the lidar and the original image collected by the camera at the same time; and project the points in the point cloud onto the pixel points at the same positions in the original image to obtain the test sample.

[0068] Optionally, the acquisition unit is further configured to: when collecting a frame of point cloud by the lidar, trigger the camera to collect a frame of original image in a pulse trigger manner to ensure that the point cloud collected by the lidar and the image collected by the camera are synchronized in time domain.

[0069] Optionally, the acquisition unit is further configured to: use an extrinsic parameter calibration algorithm to map the coordinate system of the lidar into the coordinate system of the camera, and then establish a mapping relationship between the points in the point cloud and the pixel points in the original image according to the intrinsic parameter of the camera to ensure that the point cloud collected by the lidar and the image collected by the camera are synchronized in space; and project each point in the point cloud onto the pixel points at the same positions in the original image by using the mapping relationship to obtain the test sample.

[0070] Optionally, the acquisition unit is further configured to: input the test sample into a 3D target detection network, input a perception result output by the 3D target detection network into a multi-target tracking algorithm to obtain a fused perception result, wherein the perception result is used to represent the object identified by the 3D target detection network from the test sample, and the fused perception result is used to represent the object identified by the multi-target tracking algorithm from the test sample; and generate a sample label of the test sample according to the fused perception result.

[0071] Optionally, the acquisition unit is further configured to: display the test sample and the fused perception result on a GUI tool; in a case where a confirmation operation is detected, use the fused perception result as the sample label of the test sample; and in a case where a correction operation on the fused perception result is detected, use the corrected content as the sample label of the test sample.

[0072] Optionally, specific examples in the embodiment can refer to the examples described in the above embodiments, and the embodiment will not be described here again.

[0073] Optionally, in the embodiment, the storage medium can include, but is not limited to, a U disk, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk, and various storage media that can store program codes.

[0074] The serial numbers of the embodiments of the present application are only for description, and do not represent the advantages or disadvantages of the embodiments.

[0075] The integrated units in the above embodiments, if realized in the form of software function units and sold or used as independent products, can be stored in the above computer-readable storage medium. Based on such understanding, the technical solutions of the present application, essentially or in part or all or part of the technical solutions, can be embodied in the form of a software product. The computer software product is stored in a storage medium, and includes a plurality of instructions for causing one or more computer devices (which can be personal computers, servers or network devices, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present application.

[0076] In the above embodiments of the present application, the description of each embodiment has its own focus, and the parts not described in detail in a certain embodiment can be referred to the relevant description of other embodiments.

[0077] In the several embodiments provided by the present application, it should be understood that the disclosed client can be implemented in other ways. Of course, the above device embodiment is only illustrative, for example, the division of the units is only a logical function division, and there can be another division manner in actual implementation, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units shown or discussed can be indirect coupling or communication connection through some interface, unit or module, and can be electrical or other forms.

[0078] The units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, i.e., they can be located in one place or distributed on multiple network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiment.

[0079] In addition, each functional unit in each embodiment of the present application can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software function unit.

[0080] The above merely preferred embodiments of the present application, it should be noted that for those of ordinary skill in the art, without departing from the principles of the present application, can also make a number of improvements and refinements, these improvements and refinements should also be considered as the scope of protection of the present application.

Claims

1. A method for testing the perception results of autonomous driving, characterized in that, include: Obtain test samples carrying sample tags, wherein the test samples are used to simulate the travel images collected by the intelligent vehicle during driving, and the sample tags are used to identify objects in the test samples; The image algorithm is used to identify the objects in the test sample to obtain the identification result, wherein the image algorithm is the algorithm used in the intelligent vehicle; The test results of the image algorithm are determined based on the recognition results and the sample labels; Obtain test samples carrying sample labels, including: The test sample is generated using point clouds acquired by lidar and raw images acquired by a camera; The objects in the test sample are identified to generate sample labels for the test sample; The process of identifying objects in the test samples to generate sample labels for the test samples includes: After the test sample is input into the 3D target detection network, the perception result output by the 3D target detection network is input into the multi-target tracking algorithm to obtain the fused perception result. The perception result is used to represent the object identified by the 3D target detection network from the test sample, and the fused perception result is used to represent the object identified by the multi-target tracking algorithm from the test sample. Generate sample labels for the test samples based on the fusion perception results; Based on the fusion perception results, sample labels for the test samples are generated, including: Display the test samples and the fusion perception results on a GUI tool; If a confirmation operation is detected, the fusion perception result is used as the sample label of the test sample; If a correction operation is detected on the fusion perception result, the corrected content is used as the sample label of the test sample.

2. The method according to claim 1, characterized in that, The test sample is generated using point clouds acquired by lidar and raw images acquired by a camera, including: Acquire the point cloud collected by the lidar and the original image collected by the camera at the same time; The test sample is obtained by projecting the points in the point cloud onto the pixels at the same positions in the original image.

3. The method according to claim 2, characterized in that, Acquiring the point cloud acquired by the lidar and the original image acquired by the camera at the same time includes: When the lidar acquires a frame of point cloud, the camera is triggered to acquire a frame of original image in a pulse-triggered manner to ensure that the point cloud acquired by the lidar and the image acquired by the camera are synchronized in the time domain.

4. The method according to claim 2, characterized in that, The test sample is obtained by projecting the points in the point cloud onto the pixels at the same positions in the original image, including: Using an extrinsic calibration algorithm, the coordinate system of the lidar is mapped to the coordinate system of the camera. Then, based on the camera intrinsic parameters, a mapping relationship is established between the points in the point cloud and the pixels in the original image to ensure that the point cloud acquired by the lidar and the image acquired by the camera are spatially synchronized. The test sample is obtained by projecting each point in the point cloud onto the pixel at the same position in the original image using the mapping relationship.

5. A testing device for autonomous driving perception results, characterized in that, include: An acquisition unit is used to acquire test samples carrying sample tags, wherein the test samples are used to simulate the travel images collected by an intelligent vehicle during its journey, and the sample tags are used to identify objects in the test samples; The recognition unit is used to recognize objects in the test sample using an image algorithm to obtain a recognition result, wherein the image algorithm is an algorithm used in the intelligent vehicle; A testing unit is used to determine the test results of the image algorithm based on the recognition results and the sample labels; The acquisition unit is further configured to generate the test sample using point clouds acquired by the lidar and raw images acquired by the camera; and to identify objects in the test sample to generate sample labels for the test sample. The acquisition unit is further configured to input the test sample into a 3D target detection network, input the perception result output by the 3D target detection network into a multi-target tracking algorithm, and obtain a fused perception result, wherein the perception result is used to represent the object identified by the 3D target detection network from the test sample, and the fused perception result is used to represent the object identified by the multi-target tracking algorithm from the test sample; and generate sample labels for the test sample based on the fused perception result. The acquisition unit is also used to display the test sample and the fusion perception result on a GUI tool; when a confirmation operation is detected, the fusion perception result is used as the sample label of the test sample; when a correction operation is detected on the fusion perception result, the corrected content is used as the sample label of the test sample.

6. A storage medium, characterized in that, The storage medium includes a stored program, wherein the program executes the method described in any one of claims 1 to 4 when it is run.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, The processor executes the method described in any one of claims 1 to 4 through the computer program.

Images