A sensor model performance evaluation method, device, equipment and storage medium

Abstract

The present application discloses a sensor model performance evaluation method, device, equipment and storage medium, the method comprising: obtaining the raw data output by the physical model of the sensor; inputting the raw data into at least two algorithms respectively, so that each algorithm analyzes and processes the raw data respectively; obtaining the environmental perception result data output by at least two algorithms respectively; and determining the performance evaluation data of the physical model according to the obtained environmental perception result data and environmental true value data. In the present application, the performance of the physical model is evaluated using the environmental perception result data output by at least two algorithms. Compared with evaluating the performance of the physical model using the environmental perception result data output by one algorithm, the influence of the algorithm itself on the performance evaluation of the physical model can be avoided as much as possible, so that the performance evaluation data can more truly reflect the performance of the physical model, thereby realizing effective evaluation of the performance of the physical model.

Description

Technical Field

[0001] The embodiments of the present invention relate to simulation technology, and in particular to a sensor model performance evaluation method, device, equipment and storage medium. Background Art

[0002] Through daily reconnaissance, technical research and life practice, it is found that using simulation technology to test fully automated driving (FAD) is a time-saving and economical method.

[0003] The simulation system is generally composed of a simulation environment, a sensor model, etc. The sensor model is usually composed of a physical model and a corresponding algorithm for data analysis and processing. The physical model processes the environmental data input by the simulator and outputs the raw data. The algorithm analyzes and processes the raw data and outputs the environmental perception result data. For example, the sensor model can be a camera model, in which the physical model outputs image data, and the algorithm performs entity object recognition on the image data and outputs a list of recognized entity objects; for another example, the sensor model can also be a lidar model, in which the physical model of the lidar outputs point cloud data, and the algorithm performs entity object recognition on the point cloud data and outputs a list of recognized entity objects. The above entity object list contains the entity objects existing in the simulation environment.

[0004] In order to confirm the accuracy of the simulation system, it is necessary to effectively evaluate the performance of the physical model in the sensor model. How to evaluate it is a problem that needs to be solved at present. Summary of the invention

[0005] The present invention provides a sensor model performance evaluation method, device, equipment and storage medium to achieve effective evaluation of the performance of a physical model in a sensor model.

[0006] In a first aspect, an embodiment of the present invention provides a sensor model performance evaluation method, comprising:

[0007] Get the raw data output by the physical model of the sensor;

[0008] Inputting the raw data into at least two algorithms respectively, so that each algorithm analyzes and processes the raw data respectively;

[0009] Obtaining environmental perception result data outputted by the at least two algorithms respectively;

[0010] The performance evaluation data of the physical model is determined based on the acquired environmental perception result data and environmental true value data.

[0011] One embodiment of the above application has the following advantages or beneficial effects: by using at least two algorithms to analyze and process the raw data output by the physical model of the sensor respectively, and based on the environmental perception result data output by each algorithm, the performance of the physical model can be effectively evaluated. Because the performance of the physical model is evaluated using the environmental perception result data output by at least two algorithms, compared with evaluating the performance of the physical model using the environmental perception result data output by one algorithm, the impact of the performance evaluation of the physical model due to the algorithm itself can be avoided as much as possible, so that the performance evaluation data can more truly reflect the performance of the physical model, thereby achieving an effective evaluation of the performance of the physical model. The solution of this embodiment is particularly suitable for effectively evaluating the performance of the physical model of the sensor when the data analysis and processing algorithm adopted by the manufacturer of the sensor hardware cannot be obtained, thereby evaluating the actual performance of the physical model.

[0012] Optionally, determining the performance evaluation data of the physical model according to the acquired environment perception result data and environment true value data includes:

[0013] By comparing the environment perception result data output by each algorithm with the environment true value data, the correct environment perception result data among the environment perception result data output by each algorithm is determined;

[0014] The performance evaluation data of the physical model is determined based on the correct environmental perception result data and the environmental true value data.

[0015] One embodiment of the above application has the following advantages or beneficial effects: by comparing the environmental perception result data output by each algorithm with the environmental true value data respectively, the correct environmental perception result data output by each algorithm can be determined, so that the performance of the physical model can be evaluated based on the correct environmental perception result data output by each algorithm, and a more accurate performance evaluation result can be obtained.

[0016] Optionally, determining the performance evaluation data of the physical model according to the correct environment perception result data and the environment true value data includes:

[0017] Determine the intersection of the correct environment perception result data output by each of the algorithms;

[0018] The proportion of the environmental perception result data in the intersection to the environmental true value data is determined as the performance evaluation data of the physical model.

[0019] One embodiment of the above application has the following advantages or beneficial effects: by determining the intersection of the correct environmental perception result data output by each algorithm, the correct environmental perception result that the physical model can obtain under any algorithm can be obtained, and the proportion of the environmental perception result data in the intersection to the environmental true value data is determined as the performance evaluation data of the physical model. The performance evaluation data characterizes that the physical model is close to the minimum performance that can be guaranteed to be provided under any algorithm. Then, the same evaluation can be performed on different physical models to screen out the physical model with the best performance without considering algorithm factors.

[0020] Optionally, determining the performance evaluation data of the physical model according to the correct environment perception result data and the environment true value data includes:

[0021] Determine the correct environmental perception result data output by each algorithm, and the proportion of the correct environmental true value data;

[0022] According to each of the proportion data, performance evaluation data of the physical model is determined.

[0023] One embodiment of the above application has the following advantages or beneficial effects: by respectively determining the correct environmental perception result data output by each algorithm and the proportion data of the environmental true value data, and using the proportion data to evaluate the performance of the physical model, a more accurate performance evaluation result can be obtained.

[0024] Optionally, determining the performance evaluation data of the physical model according to each of the proportion data includes:

[0025] Determine the average value of each of the proportion data as the performance evaluation data of the physical model; or,

[0026] The minimum proportion data among the proportion data is determined as the performance evaluation data of the physical model.

[0027] One embodiment of the above application has the following advantages or beneficial effects: by determining the average value of each proportion data as the performance evaluation data of the physical model, the influence of different algorithms on the performance of the physical model is taken into account, and then the average performance of the physical model under any algorithm can be measured based on the performance evaluation data. Alternatively, by determining the minimum proportion data among the proportion data as the performance evaluation data of the physical model, the minimum performance of the physical model under any algorithm can be measured based on the performance evaluation data, thereby avoiding the performance evaluation of the physical model from being affected by the algorithm itself, making the performance evaluation data more valuable for reference.

[0028] Optionally, the method further comprises:

[0029] By comparing the environment perception result data respectively output by the algorithms, the similarity between the environment perception result data respectively output by the algorithms is determined;

[0030] Prompt information including the similarity is issued, so that the user can determine the confidence of the performance evaluation data according to the similarity.

[0031] One embodiment of the above application has the following advantages or beneficial effects: by determining the similarity between the environmental perception result data output by various algorithms, and sending a prompt message containing the similarity to the user, the user can be prompted to determine the confidence of the performance evaluation data based on the similarity, so that the user can better understand the reference value of the performance evaluation data.

[0032] Optionally, after determining the performance evaluation data of the physical model, the method further includes:

[0033] Acquiring performance evaluation data of the vehicle in at least one driving performance angle when a driving test is performed on the vehicle equipped with the physical model;

[0034] Driving performance evaluation data of the vehicle is determined based on the performance evaluation data of the physical model and the performance evaluation data of the at least one driving performance angle.

[0035] One embodiment of the above application has the following advantages or beneficial effects: by conducting a driving test on a vehicle equipped with a physical model, performance evaluation data of the vehicle in at least one driving performance angle is obtained, and based on the performance evaluation data of the physical model and the performance evaluation data of each driving performance angle, a comprehensive evaluation of the vehicle's driving performance based on the performance evaluation data of the physical model is achieved.

[0036] Optionally, determining the driving performance evaluation data of the vehicle according to the performance evaluation data of the physical model and the performance evaluation data of the at least one driving performance angle comprises:

[0037] The driving performance evaluation data of the vehicle is determined based on the performance evaluation data of the physical model, the weight corresponding to the performance evaluation data of the physical model, the performance evaluation data of each driving performance angle, and the weight corresponding to the performance evaluation data of each driving performance angle.

[0038] One embodiment of the above application has the following advantages or beneficial effects: By adopting this scheme, the weights of the performance evaluation data of the physical model and the weights of the performance evaluation data of each driving performance angle can be used to adjust the degree of influence of the performance evaluation data of the physical model and the performance evaluation data of each driving performance angle on the driving performance of the vehicle, thereby more accurately determining the driving performance of the vehicle.

[0039] Optionally, the physical model of the sensor is a physical model of a camera, the raw data is image data, the environmental perception result data is a list of entity objects perceived by the model, and the environmental truth data is a list of entity objects actually existing in the environment; or,

[0040] The physical model of the sensor is a physical model of a radar, the raw data is point cloud data, the environmental perception result data is a list of entity objects perceived by the model, and the environmental truth data is a list of entity objects that actually exist in the environment.

[0041] One embodiment in the above application has the following advantages or beneficial effects: This embodiment can effectively evaluate the performance of the physical model of the camera or the physical model of the radar.

[0042] In a second aspect, an embodiment of the present invention further provides a sensor model performance evaluation device, comprising:

[0043] A raw data acquisition module, used to acquire raw data output by the physical model of the sensor;

[0044] A raw data analysis and processing module, used for inputting the raw data into at least two algorithms respectively, so that each algorithm analyzes and processes the raw data respectively;

[0045] An environment perception result data acquisition module, used to acquire the environment perception result data respectively output by the at least two algorithms;

[0046] The performance evaluation data determination module is used to determine the performance evaluation data of the physical model according to the acquired environment perception result data and environment true value data.

[0047] In a third aspect, an embodiment of the present invention further provides a device, the device comprising:

[0048] one or more processors;

[0049] A memory for storing one or more programs;

[0050] When the one or more programs are executed by the one or more processors, the one or more processors implement the sensor model performance evaluation method provided by any embodiment of the present invention.

[0051] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements a sensor model performance evaluation method as provided in any embodiment of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 is a flow chart of a sensor model performance evaluation method provided in Embodiment 1 of the present invention;

[0053] Figure 2 This is an example of a sensor model performance evaluation process involved in the first embodiment of the present invention;

[0054] Figure 3 is a flow chart of a sensor model performance evaluation method provided in Embodiment 2 of the present invention;

[0055] Figure 4 is a flow chart of a sensor model performance evaluation method provided by Embodiment 3 of the present invention;

[0056] Figure 5 is a flow chart of a sensor model performance evaluation method provided by Embodiment 4 of the present invention;

[0057] Figure 6 is a flow chart of a sensor model performance evaluation method provided by Embodiment 5 of the present invention;

[0058] Figure 7 is a structural schematic diagram of a sensor model performance evaluation device provided by Embodiment 6 of the present invention;

[0059] Figure 8 It is a structural schematic diagram of a device provided in Embodiment 7 of the present invention. DETAILED DESCRIPTION

[0060] The present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the present invention, rather than to limit the present invention. It should also be noted that, for ease of description, only parts related to the present invention, rather than all structures, are shown in the accompanying drawings.

[0061] Embodiment 1

[0062] Figure 1 A flowchart of a sensor model performance evaluation method provided in Example 1 of the present invention. This embodiment is applicable to the situation where performance evaluation of a physical model in a sensor model is performed. The method can be performed by a sensor model performance evaluation device, which can be implemented by software and / or hardware and integrated into a device with simulation function.

[0063] Before introducing the sensor model performance evaluation method provided by the embodiment of the present invention, the application scenario of the embodiment of the present invention is briefly described. The simulation system can be composed of a simulation environment and a sensor model. The sensor model can be composed of a physical model and an algorithm for data analysis and processing. Among them, the environment simulation is implemented by a simulator. The physical model in the sensor model can simulate the sensor hardware function through simulation to restore the physical parameters of the sensor hardware. Usually, the algorithm for data analysis and processing corresponding to the sensor hardware is designed by the manufacturer of the sensor hardware, and the algorithm can be converted into a logic circuit and integrated in the sensor hardware, so that the source code of the algorithm designed by the manufacturer is often not available, so that the algorithm designed by the sensor manufacturer cannot be restored by simulation in the simulation system. In view of this, this embodiment can effectively evaluate the performance of the physical model of the sensor when the algorithm inside the sensor hardware cannot be restored in the simulation system, thereby evaluating the actual performance of the physical model of the sensor.

[0064] like Figure 1 As shown, the sensor model performance evaluation method provided in this embodiment specifically includes the following steps:

[0065] S110, obtaining raw data output by the physical model of the sensor.

[0066] Among them, the sensor may refer to a hardware device for sensing the surrounding environment information. For example, the sensor may be, but is not limited to, a camera and a radar. Among them, the radar may include, but is not limited to, a laser radar and a millimeter wave radar. The laser radar may include a rotating laser radar and a solid-state laser radar. The camera in the camera may include a telephoto camera and a wide-angle camera. The telephoto camera has a long focal length and a small viewing angle, and is suitable for capturing images of distant objects. The wide-angle camera has a very short focal length, a wide viewing angle, and a deep depth of field, and is suitable for capturing images of larger scenes. The physical model of the sensor may be a hardware virtual model simulated based on the physical parameters of the sensor hardware, which is used to simulate the hardware functions of the actual sensor. The raw data may refer to the raw data obtained after the physical model processes the environmental data input by the simulator. For example, when the physical model of the sensor is a physical model of a camera, the output raw data may be image data. When the physical model of the sensor is a physical model of a radar, the output raw data is point cloud data. For example, the point cloud data may include a set of position vectors in a three-dimensional coordinate system, which may also include RGB (Red, Green, Blue) color information, gray value information, depth information, etc. of each position point.

[0067] S120, inputting the original data into at least two algorithms respectively, so that each algorithm analyzes and processes the original data respectively.

[0068] Among them, the algorithm may refer to a general algorithm for analyzing and processing environmental perception data. Exemplarily, the algorithm may refer to a common target detection algorithm to detect and identify various object information in the surrounding environment. For example, the target detection algorithm may be an object recognition algorithm based on a neural network model, such as a region convolutional neural network model (Region-Convolutional Neural Networks, R-CNN) algorithm, a fast region convolutional neural network model (Faster R-CNN) algorithm, a single shot multi-box model (Single Shot MultiBox Detector, SSD) algorithm, a YOLO (You Only Look Once) algorithm, a RetinaNet algorithm, an end-to-end 3D object detection network model based on point cloud (VoxelNet) algorithm, a 3D point cloud classification and segmentation model based on deep learning (FPointNet) algorithm, and a constant false alarm rate (CFAR) algorithm based on FPGA.

[0069] Specifically, based on business needs and sensor categories, at least two matching algorithms for analyzing and processing environmental perception data can be selected from existing general algorithms, and each algorithm can be used to analyze and process the raw data output by the physical model, so that at least two existing known algorithms can be used to replace the actual algorithm inside the sensor.

[0070] It should be noted that since the algorithm used is not the real algorithm inside the sensor, there may be a situation where the performance of the physical model is poor due to the algorithm itself rather than the physical model. Therefore, it is impossible to accurately evaluate the performance of the physical model. Therefore, it is necessary to use at least two algorithms to analyze and process the raw data output by the physical model. Compared with using one algorithm to analyze and process the raw data output by the physical model, the impact of the algorithm itself on the performance evaluation of the physical model can be avoided as much as possible, thereby ensuring the accuracy of the physical model performance evaluation.

[0071] S130. Obtain environmental perception result data outputted by at least two algorithms respectively.

[0072] The environmental perception result data may refer to the environmental perception result obtained by processing the environmental data input by the simulator by the physical model of the sensor. For example, when the physical model of the sensor is a physical model of a camera or a physical model of a radar, the output environmental perception result data may be a list of entity objects perceived by the model. Entity objects may refer to any object that actually exists in the simulation environment, such as a human body, a vehicle, a tree, etc. The list of entity objects may include information of all entity objects perceived by the model stored in a table.

[0073] Specifically, each algorithm can output environmental perception result data after analyzing and processing the raw data output by the physical model. It should be noted that the environmental perception result data output by different algorithms may be the same or different. For example, the environmental perception result data output by one algorithm includes: pedestrians, vehicles and street lights; the environmental perception result data output by another algorithm includes: pedestrians and vehicles. When the similarity between the environmental perception result data output by different algorithms is higher, it indicates that the difference in the perception ability of the physical model under different algorithms is smaller, so that the output environmental perception result data can more truly reflect the performance of the physical model, making the performance evaluation more accurate.

[0074] S140. Determine performance evaluation data of the physical model based on the acquired environment perception result data and environment true value data.

[0075] The environmental truth data may include information about all objects that actually exist in the simulation environment. For example, when the physical model of the sensor is a physical model of a camera or a physical model of a radar, the environmental truth data may be a list of physical objects that actually exist in the environment. The environmental truth data may include information about all physical objects that actually exist in the environment stored in a table. The performance evaluation data may refer to an indicator used to evaluate the performance of a physical model. For example, the performance evaluation data may include the perception accuracy of the physical model, etc.

[0076] Specifically, this embodiment can compare and analyze the environmental perception result data output by various algorithms based on the environmental true value data, determine the performance of the physical model under each algorithm, and comprehensively determine the performance evaluation data of the physical model under any algorithm based on the performance of various algorithms, thereby avoiding the impact of the selected algorithm itself on the performance of the physical model, making the performance of the physical model represented by the performance evaluation data closer to the actual performance of the physical model. Furthermore, when the algorithm inside the sensor cannot be restored in the simulation system, at least two general algorithms can be used to evaluate the performance of the physical model of the sensor, thereby ensuring the accuracy of the performance evaluation of the physical model.

[0077] Figure 2 An example of a sensor model performance evaluation process is given. Figure 2As shown, the sensor model performance evaluation process can be as follows: after constructing the simulation environment and the physical model in the sensor model, the environment data can be input into the physical model through the simulator to obtain the raw data output by the physical model, and the raw data can be input into at least two pre-selected general algorithms respectively, so that each algorithm analyzes and processes the raw data and outputs the corresponding environment perception result data. The evaluation index is determined according to the environment true value data corresponding to the simulation environment and each environment perception result data, so that the performance evaluation data of the physical model can be obtained, and the effective evaluation of the performance of the physical model can be realized, so that the physical model with better performance can be accurately selected.

[0078] The technical solution of this embodiment uses at least two algorithms to analyze and process the raw data output by the physical model of the sensor respectively, and the performance of the physical model can be effectively evaluated based on the environmental perception result data output by each algorithm. Because the performance of the physical model is evaluated using the environmental perception result data output by at least two algorithms, compared with evaluating the performance of the physical model using the environmental perception result data output by one algorithm, the impact of the performance evaluation of the physical model due to the algorithm itself can be avoided as much as possible, so that the performance evaluation data can more truly reflect the performance of the physical model, thereby achieving an effective evaluation of the performance of the physical model. The solution of this embodiment is particularly suitable for effectively evaluating the performance of the physical model of the sensor when the data analysis and processing algorithm adopted by the manufacturer of the sensor hardware cannot be obtained, thereby evaluating the actual performance of the physical model.

[0079] On the basis of the above technical solution, when determining the performance evaluation data of the physical model in S140 or after determining the performance evaluation data of the physical model, the method may further include: determining the similarity between the environmental perception result data output by each algorithm by comparing the environmental perception result data output by each algorithm; issuing a prompt message containing the similarity so that the user can determine the confidence of the performance evaluation data based on the similarity.

[0080] Among them, similarity can refer to the degree of similarity between the environmental perception result data output by various algorithms, which can be used to reflect the reference value of the obtained performance evaluation data. For example, the higher the similarity between the environmental perception result data corresponding to each algorithm, the less each environmental perception result data output is affected by the algorithm itself. At this time, the performance evaluation data can better reflect the actual performance of the physical model, making the reference value of the performance evaluation data higher. Confidence can refer to the true degree of physical model performance that can be reflected by the performance evaluation data of the determined physical model. When the similarity between the environmental perception result data output by each algorithm is higher, the confidence of the performance evaluation data is also higher.

[0081] Specifically, the similarity between the environmental perception result data output by various algorithms can be determined based on the similarity calculation algorithm. The similarity calculation algorithm may include but is not limited to the Euclidean distance algorithm, the Manhattan distance algorithm and the cosine similarity algorithm. This embodiment can also send a prompt message containing similarity to the user, prompting the user to determine the confidence of the performance evaluation data of the physical model based on the similarity, so that the user can better understand the reference value of the performance evaluation data and improve the user evaluation experience.

[0082] Embodiment 2

[0083] Figure 3 This is a flow chart of a sensor model performance evaluation method provided in Embodiment 2 of the present invention. Based on the above embodiment, this embodiment optimizes "determining the performance evaluation data of the physical model according to the acquired environmental perception result data and environmental true value data". The explanations of the terms that are the same or corresponding to the above embodiment are not repeated here.

[0084] See also Figure 3 The sensor model performance evaluation method provided in this embodiment specifically includes the following steps:

[0085] S210: Acquire raw data output by the physical model of the sensor.

[0086] S220, inputting the original data into at least two algorithms respectively, so that each algorithm analyzes and processes the original data respectively.

[0087] S230: Obtain environmental perception result data outputted by at least two algorithms respectively.

[0088] S240. Determine the correct environmental perception result data among the environmental perception result data output by each algorithm by comparing the environmental perception result data output by each algorithm with the environmental true value data.

[0089] Specifically, for the environmental perception result data output by each algorithm, the environmental perception result data is compared with the environmental truth data. If a certain sub-data in the environmental perception result data is included in the environmental truth data, it can be determined that the sub-data is the correct environmental perception result data, otherwise it is the wrong environmental perception result data, so that all the correct environmental perception result data in the environmental perception result data output by each algorithm can be determined. For example, the environmental truth data includes: pedestrians, vehicles and street lights, and the environmental perception result data output by a certain algorithm includes: pedestrians, vehicles and trees, then it can be determined that the correct environmental perception result data output by the algorithm includes pedestrians and vehicles. Since each algorithm is aimed at the same simulation environment, the perception accuracy of the physical model under each algorithm can be measured based on the number of correct environmental perception result data output by the algorithm. For example, the more correct environmental perception result data the algorithm outputs, the higher the perception accuracy of the physical model under the algorithm.

[0090] S250. Determine the performance evaluation data of the physical model according to the correct environmental perception result data and the environmental true value data.

[0091] Specifically, by comprehensively evaluating the correct environmental perception result data and environmental true value data output by each algorithm, the performance evaluation data of the physical model can be made so that the performance of the physical model represented by the performance evaluation data is closer to the actual performance of the physical model, thereby improving the accuracy of the performance evaluation results.

[0092] The technical solution of this embodiment can determine the correct environmental perception result data output by each algorithm by comparing the environmental perception result data output by each algorithm with the environmental true value data, so as to evaluate the performance of the physical model based on the correct environmental perception result data output by each algorithm, and obtain more accurate performance evaluation results.

[0093] Embodiment 3

[0094] Figure 4 This is a flow chart of a sensor model performance evaluation method provided in Embodiment 3 of the present invention. Based on Embodiment 2 above, this embodiment optimizes "determining the performance evaluation data of the physical model according to the correct environmental perception result data and environmental true value data". The explanations of the terms that are the same or corresponding to the above embodiments are not repeated here.

[0095] See also Figure 4 The sensor model performance evaluation method provided in this embodiment specifically includes the following steps:

[0096] S310: Obtaining raw data output by the physical model of the sensor.

[0097] S320, inputting the original data into at least two algorithms respectively, so that each algorithm analyzes and processes the original data respectively.

[0098] S330: Obtain environmental perception result data outputted by at least two algorithms respectively.

[0099] S340. Determine the correct environmental perception result data among the environmental perception result data output by each algorithm by comparing the environmental perception result data output by each algorithm with the environmental true value data.

[0100] S350: Determine the intersection of the correct environment perception result data output by each algorithm.

[0101] Specifically, the correct environmental perception result data output by various algorithms can be compared to determine the correct environmental perception result data that can be identified by each algorithm, thereby obtaining the intersection of the correct environmental perception result data output by various algorithms. For example, the correct environmental perception result data output by one algorithm includes: pedestrians, vehicles and street lights; the correct environmental perception result data output by another algorithm includes: pedestrians, vehicles and trees, then it can be determined that the intersection of the correct environmental perception result data output by the two algorithms includes pedestrians and vehicles.

[0102] S360: Determine the ratio of the environmental perception result data in the intersection to the environmental true value data as the performance evaluation data of the physical model.

[0103] Specifically, the environmental perception result data in the intersection may refer to the environmental perception result that the physical model can correctly identify under any algorithm. The ratio of the environmental perception result data in the intersection to the environmental true value data can be used to reflect the minimum perception accuracy that the physical model can guarantee under any algorithm. By using the ratio data as the performance evaluation indicator of the physical model, the performance of the physical model can be avoided from being affected by the algorithm itself. The larger the ratio data, the higher the minimum perception accuracy that the physical model can achieve, that is, the better the performance of the physical model. Exemplarily, by evaluating different physical models of the sensor, the minimum perception accuracy corresponding to each physical model is obtained, so that based on each minimum perception accuracy, the physical model with the best physical performance can be accurately screened out without considering algorithm factors, such as taking the physical model corresponding to the maximum value of each minimum perception accuracy as the physical model with the best physical performance.

[0104] The technical solution of this embodiment can obtain the correct environmental perception results that the physical model can obtain under any algorithm by determining the intersection of the correct environmental perception result data output by each algorithm, and determine the proportion of the environmental perception result data in the intersection to the environmental true value data as the performance evaluation data of the physical model. The performance evaluation data characterizes that the physical model is close to the minimum performance that can be ensured under any algorithm. The same evaluation can then be performed on different physical models to screen out the physical model with the best performance without considering algorithm factors.

[0105] Embodiment 4

[0106] Figure 5 This is a flow chart of a sensor model performance evaluation method provided in Embodiment 4 of the present invention. Based on Embodiment 2 above, this embodiment optimizes "determining the performance evaluation data of the physical model according to the correct environmental perception result data and environmental true value data". The explanations of the terms that are the same or corresponding to the above embodiments are not repeated here.

[0107] See also Figure 5 The sensor model performance evaluation method provided in this embodiment specifically includes the following steps:

[0108] S410: Acquire raw data output by the physical model of the sensor.

[0109] S420, inputting the original data into at least two algorithms respectively, so that each algorithm analyzes and processes the original data respectively.

[0110] S430: Obtain environmental perception result data outputted by at least two algorithms respectively.

[0111] S440. Determine the correct environmental perception result data among the environmental perception result data output by each algorithm by comparing the environmental perception result data output by each algorithm with the environmental true value data.

[0112] S450, determining the proportion of the correct environmental perception result data output by each algorithm to the environmental true value data.

[0113] Specifically, for each algorithm, the proportion of the correct environmental perception result data outputted to the environmental true value data can be determined. For example, the first quantity of environmental true value data and the second quantity of correct environmental perception result data outputted by each algorithm can be obtained, and the ratio of the second quantity to the first quantity can be used as the proportion data corresponding to the corresponding algorithm. The proportion data corresponding to each algorithm can reflect the perception accuracy that the physical model can achieve under each algorithm. For example, the environmental true value data includes: pedestrians, vehicles, trees and street lights, and the correct environmental perception result data outputted by an algorithm includes: pedestrians and vehicles, then it can be determined that the correct environmental perception result data outputted by the algorithm accounts for 50% of the environmental true value data.

[0114] S460: Determine performance evaluation data of the physical model according to each proportion data.

[0115] Specifically, by comprehensively determining the performance evaluation data of the physical model based on the proportional data corresponding to each algorithm, the performance of the physical model represented by the performance evaluation data is closer to the actual performance of the physical model, thereby improving the accuracy of the performance evaluation results.

[0116] Exemplarily, S460 may include: determining performance evaluation data of the physical model according to each proportion data, including: determining the average value of each proportion data as the performance evaluation data of the physical model; or determining the minimum proportion data among each proportion data as the performance evaluation data of the physical model.

[0117] Specifically, by determining the average value of each proportion data as the performance evaluation data of the physical model, the impact of different algorithms on the performance of the physical model is taken into account, and then the average performance of the physical model under any algorithm can be measured based on the performance evaluation data. For example, if the proportion data corresponding to the three algorithms are 50%, 60% and 70% respectively, then it can be determined that the average value of these three proportion data is 60%, that is, the performance evaluation data of the physical model is a perception accuracy of 60%.

[0118] Alternatively, by determining the minimum proportion data among the proportion data as the performance evaluation data of the physical model, the minimum performance of the physical model under any algorithm can be measured based on the performance evaluation data, thereby avoiding the performance evaluation of the physical model from being affected by the algorithm itself, making the performance evaluation data more valuable for reference. For example, if the proportion data corresponding to the three algorithms are 50%, 60% and 70% respectively, then it can be determined that the minimum proportion data among the three proportion data is 50%, that is, the performance evaluation data of the physical model is a perception accuracy of 50%.

[0119] The technical solution of this embodiment can obtain more accurate performance evaluation results by respectively determining the proportion of the correct environmental perception result data output by each algorithm to the environmental true value data, and using the proportion data to evaluate the performance of the physical model.

[0120] Embodiment 5

[0121] Figure 6 This is a flow chart of a sensor model performance evaluation method provided in Embodiment 5 of the present invention. Based on the above embodiments, this embodiment describes in detail the evaluation process of the driving performance of a vehicle equipped with a physical model after determining the performance evaluation data of the physical model. The explanations of the terms that are the same or corresponding to the above embodiments are not repeated here.

[0122] See also Figure 6 The sensor model performance evaluation method provided in this embodiment specifically includes the following steps:

[0123] S510: Acquire raw data output by the physical model of the sensor.

[0124] S520: Input the original data into at least two algorithms respectively, so that each algorithm analyzes and processes the original data respectively.

[0125] S530: Obtain environment perception result data outputted by at least two algorithms respectively.

[0126] S540: Determine performance evaluation data of the physical model according to the acquired environment perception result data and environment true value data.

[0127] S550: Acquire performance evaluation data of the vehicle in at least one driving performance angle when performing a driving test on the vehicle equipped with the physical model.

[0128] The driving performance angle may refer to an angle used to measure the driving performance. For example, the driving performance angle may include but is not limited to whether a collision occurs and whether the driving is completed. Accordingly, the performance evaluation data of the driving performance angle may include a collision score and a driving score.

[0129] Specifically, the physical model of the sensor is installed in the vehicle, and each algorithm is used to analyze and process the raw data output by the installed physical model, so that the vehicle can perform driving tests based on the environmental perception result data output by the algorithm, and obtain the vehicle driving test results under each algorithm, such as whether the vehicle has collided and whether the vehicle has completed the entire driving journey under the algorithm. Based on the vehicle driving test results under various algorithms, it can be determined to obtain performance evaluation data from at least one driving performance angle.

[0130] Exemplarily, when the performance evaluation data from the driving performance perspective is a collision score, it can be determined based on whether the physical model carried by the vehicle collides under various algorithms. For example, the number of first algorithms that collide is counted. When the number of first algorithms is equal to 0, it indicates that the physical model carried by the vehicle will not collide when each algorithm is used. At this time, the collision score can be determined as the maximum score value, such as 100; when the number of first algorithms is greater than or equal to 1, it indicates that there is at least one algorithm that causes the vehicle to collide. At this time, the collision score can be directly determined as the minimum value, such as 0. Of course, the collision score can also be reduced as the number of first algorithms increases, such as the collision score can be reduced linearly or exponentially until the number of first algorithms is equal to the total number of algorithms. The collision score is reduced to the minimum value, such as 0.

[0131] Exemplarily, when the performance evaluation data from the driving performance perspective is the driving score for the entire journey, it can be determined based on whether the physical model carried by the vehicle completes the entire journey under various algorithms. For example, the number of second algorithms that cannot complete the entire journey can be counted. When the number of second algorithms is equal to 0, it indicates that the physical model carried by the vehicle can complete the entire journey when each algorithm is used. At this time, the driving score for the entire journey can be determined as the maximum score value, such as 100; when the number of second algorithms is greater than or equal to 1, it indicates that there is at least one algorithm that cannot complete the entire journey. At this time, the driving score for the entire journey can be directly determined as the minimum value, such as 0. Of course, the driving score for the entire journey can also be reduced as the number of second algorithms increases. For example, the driving score for the entire journey can be reduced linearly or exponentially until the number of second algorithms is equal to the total number of algorithms. The driving score for the entire journey is reduced to the minimum value, such as 0.

[0132] S560: Determine driving performance evaluation data of the vehicle according to the performance evaluation data of the physical model and the performance evaluation data of at least one driving performance angle.

[0133] Specifically, the performance evaluation data of the physical model and the performance evaluation data of each driving performance angle can be added, and the obtained addition result is determined as the driving performance evaluation data of the vehicle, thereby realizing the evaluation of the vehicle's driving performance and further reflecting the performance of the physical model in the vehicle driving scenario.

[0134] Exemplarily, S560 may include: determining the driving performance evaluation data of the vehicle based on the performance evaluation data of the physical model, the weights corresponding to the performance evaluation data of the physical model, the performance evaluation data of each driving performance angle, and the weights corresponding to the performance evaluation data of each driving performance angle.

[0135] Among them, the weight of the performance evaluation data of the physical model and the weight of the performance evaluation data of each driving performance angle can be used to adjust the degree of influence of the performance evaluation data of the physical model and the performance evaluation data of each driving performance angle on the overall driving performance of the vehicle. For example, the higher the weight of the performance evaluation data, the greater the influence of the performance evaluation data on the overall driving performance of the vehicle. By setting a corresponding weight for each performance evaluation data, different business needs can be met.

[0136] Specifically, this embodiment can perform weighted superposition on the performance evaluation data of the physical model and the performance evaluation data of each driving performance angle, and determine the obtained superposition result as the driving performance evaluation data of the vehicle, so as to more accurately determine the driving performance of the vehicle.

[0137] The technical solution of this embodiment obtains performance evaluation data of the vehicle in at least one driving performance angle by conducting a driving test on a vehicle equipped with a physical model, and based on the performance evaluation data of the physical model and the performance evaluation data of each driving performance angle, a comprehensive evaluation of the vehicle's driving performance based on the performance evaluation data of the physical model is achieved.

[0138] It should be noted that how to determine the performance evaluation data of the physical model based on the acquired environmental perception result data and environmental true value data is not limited to the scheme provided in the embodiments of the present application. Any other scheme that can obtain the performance evaluation data of the physical model based on the environmental perception result data and environmental true value data output by multiple algorithms is within the protection scope of the present application. For example, the environmental perception result data output by each algorithm can be compared with the environmental true value data respectively to determine the erroneous environmental perception result data in the environmental perception result data output by each algorithm, and then the performance evaluation data of the physical model can be determined based on the erroneous environmental perception result data and environmental true value data output by each algorithm. Specifically, the maximum proportion data of the proportion data of the erroneous environmental perception result data output by each algorithm to the environmental true value data can be used as the performance evaluation data of the physical model, or the average data of the proportion data of the erroneous environmental perception result data output by each algorithm to the environmental true value data can be used as the performance evaluation data of the physical model, and so on. We will not go into details here.

[0139] The following is an embodiment of a sensor model performance evaluation device provided in an embodiment of the present invention. The device and the sensor model performance evaluation methods of the above-mentioned embodiments belong to the same inventive concept. For details not described in detail in the embodiment of the sensor model performance evaluation device, reference can be made to the embodiment of the above-mentioned sensor model performance evaluation method.

[0140] Embodiment 6

[0141] Figure 7 This is a schematic diagram of the structure of a sensor model performance evaluation device provided by Embodiment 6 of the present invention. This embodiment is applicable to the case of evaluating the performance of a physical model in a sensor model. Figure 7 As shown, the device specifically includes: a raw data acquisition module 610, a raw data analysis and processing module 620, an environment perception result data acquisition module 630 and a performance evaluation data determination module 640.

[0142] Among them, the raw data acquisition module 610 is used to obtain the raw data output by the physical model of the sensor; the raw data analysis and processing module 620 is used to input the raw data into at least two algorithms respectively, so that each algorithm analyzes and processes the raw data respectively; the environmental perception result data acquisition module 630 is used to obtain the environmental perception result data output by at least two algorithms respectively; the performance evaluation data determination module 640 is used to determine the performance evaluation data of the physical model based on the acquired environmental perception result data and environmental true value data.

[0143] The embodiment of the present invention uses at least two algorithms to analyze and process the raw data output by the physical model of the sensor respectively, and can effectively evaluate the performance of the physical model based on the environmental perception result data output by each algorithm. Because the environmental perception result data output by at least two algorithms is used to evaluate the performance of the physical model, compared with the environmental perception result data output by one algorithm, the impact of the performance evaluation of the physical model caused by the algorithm itself can be avoided as much as possible, so that the performance evaluation data can more truly reflect the performance of the physical model, thereby achieving an effective evaluation of the performance of the physical model. The solution of this embodiment is particularly suitable for effectively evaluating the performance of the physical model of the sensor when the data analysis and processing algorithm adopted by the manufacturer of the sensor hardware cannot be obtained, thereby evaluating the actual performance of the physical model.

[0144] Optionally, the performance evaluation data determination module 640 includes:

[0145] A correct environment perception result data determination unit, used to determine the correct environment perception result data among the environment perception result data output by each algorithm by comparing the environment perception result data output by each algorithm with the environment true value data;

[0146] The performance evaluation data determination unit is used to determine the performance evaluation data of the physical model according to the correct environmental perception result data and environmental true value data.

[0147] Optionally, the performance evaluation data determination unit is specifically used to: determine the intersection of correct environmental perception result data output by each algorithm; and determine the proportion of environmental perception result data in the intersection to environmental true value data as the performance evaluation data of the physical model.

[0148] Optionally, the performance evaluation data determining unit includes:

[0149] The proportion data determination subunit is used to determine the proportion of the correct environmental perception result data output by each algorithm to the environmental true value data;

[0150] The performance evaluation data determination subunit is used to determine the performance evaluation data of the physical model according to each proportion data.

[0151] Optionally, the performance evaluation data determination subunit is specifically used to: determine the average value of each proportion data as the performance evaluation data of the physical model; or determine the minimum proportion data among each proportion data as the performance evaluation data of the physical model.

[0152] Optionally, the device further comprises:

[0153] A similarity determination module is used to determine the similarity between the environment perception result data output by each algorithm by comparing the environment perception result data output by each algorithm;

[0154] The prompt information issuing module is used to issue prompt information containing similarity, so that the user can determine the confidence of the performance evaluation data according to the similarity.

[0155] Optionally, the device further comprises:

[0156] A data acquisition module, for acquiring, after determining the performance evaluation data of the physical model, performance evaluation data of the vehicle at at least one driving performance angle when performing a driving test on the vehicle equipped with the physical model;

[0157] The driving performance evaluation data determination module is used to determine the driving performance evaluation data of the vehicle according to the performance evaluation data of the physical model and the performance evaluation data of at least one driving performance angle.

[0158] Optionally, the driving performance evaluation data determination module is specifically used to determine the driving performance evaluation data of the vehicle based on the performance evaluation data of the physical model, the weights corresponding to the performance evaluation data of the physical model, the performance evaluation data of each driving performance angle, and the weights corresponding to the performance evaluation data of each driving performance angle.

[0159] Optionally, the physical model of the sensor is a physical model of a camera, the raw data is image data, the environmental perception result data is a list of entity objects perceived by the model, and the environmental truth data is a list of entity objects actually existing in the environment; or,

[0160] The physical model of the sensor is the physical model of the radar, the raw data is point cloud data, the environmental perception result data is a list of entity objects perceived by the model, and the environmental truth data is a list of entity objects that actually exist in the environment.

[0161] The sensor model performance evaluation device provided in the embodiment of the present invention can execute the sensor model performance evaluation method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects for executing the sensor model performance evaluation method.

[0162] It is worth noting that in the embodiment of the above-mentioned sensor model performance evaluation device, the various modules included are only divided according to functional logic, but are not limited to the above-mentioned division, as long as the corresponding functions can be achieved; in addition, the specific names of the functional units are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of the present invention.

[0163] Embodiment 7

[0164] Figure 8 A schematic diagram of the structure of a device provided in Embodiment 4 of the present invention. Figure 8 A block diagram of an exemplary device 12 suitable for use in implementing embodiments of the present invention is shown. Figure 8 The device 12 shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0165] like Figure 8 As shown, device 12 is in the form of a general purpose computing device. Components of device 12 may include, but are not limited to, one or more processors or processing units 16, system memory 28, and bus 18 that connects various system components (including system memory 28 and processing unit 16).

[0166] Bus 18 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processor or a local bus using any of a variety of bus architectures. By way of example, these architectures include, but are not limited to, an Industry Standard Architecture (ISA) bus, a Micro Channel Architecture (MAC) bus, an Enhanced ISA bus, a Video Electronics Standards Association (VESA) local bus, and a Peripheral Component Interconnect (PCI) bus.

[0167] Device 12 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by device 12, including volatile and non-volatile media, removable and non-removable media.

[0168] System memory 28 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 30 and / or cache memory 32. Device 12 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, storage system 34 may be used to read and write non-removable, non-volatile magnetic media ( Figure 8 not shown, usually called a "hard drive"). Although Figure 8 Not shown in the figure, a disk drive for reading and writing to a removable non-volatile disk (e.g., a "floppy disk"), and an optical disk drive for reading and writing to a removable non-volatile optical disk (e.g., a CD-ROM, a DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to the bus 18 via one or more data medium interfaces. The system memory 28 may include at least one program product having a set (e.g., at least one) of program modules that are configured to perform the functions of the various embodiments of the present invention.

[0169] A program / utility 40 having a set (at least one) of program modules 42 may be stored, for example, in system memory 28, such program modules 42 including, but not limited to, an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment. Program modules 42 generally perform the functions and / or methods of the embodiments described herein.

[0170] The device 12 may also communicate with one or more external devices 14 (e.g., keyboards, pointing devices, displays 24, etc.), may communicate with one or more devices that enable a user to interact with the device 12, and / or may communicate with any device that enables the device 12 to communicate with one or more other computing devices (e.g., network cards, modems, etc.). Such communication may be performed via an input / output (I / O) interface 22. Furthermore, the device 12 may also communicate with one or more networks (e.g., local area networks (LANs), wide area networks (WANs), and / or public networks, such as the Internet) via a network adapter 20. As shown, the network adapter 20 communicates with other modules of the device 12 via a bus 18. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the device 12, including, but not limited to, microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0171] The processing unit 16 executes various functional applications and data processing by running the program stored in the system memory 28, for example, implementing a sensor model performance evaluation method provided by the embodiment of the present invention, the method comprising:

[0172] Get the raw data output by the physical model of the sensor;

[0173] Inputting the original data into at least two algorithms respectively, so that each algorithm analyzes and processes the original data respectively;

[0174] Obtaining environmental perception result data outputted by at least two algorithms respectively;

[0175] The performance evaluation data of the physical model is determined based on the acquired environmental perception result data and environmental true value data.

[0176] Of course, those skilled in the art can understand that the processor can also implement the technical solution of the sensor model performance evaluation method provided by any embodiment of the present invention.

[0177] Embodiment 8

[0178] This embodiment provides a computer-readable storage medium having a computer program stored thereon. When the program is executed by a processor, the steps of the sensor model performance evaluation method provided in any embodiment of the present invention are implemented. The method includes:

[0179] Get the raw data output by the physical model of the sensor;

[0180] Inputting the original data into at least two algorithms respectively, so that each algorithm analyzes and processes the original data respectively;

[0181] Obtaining environmental perception result data outputted by at least two algorithms respectively;

[0182] The performance evaluation data of the physical model is determined based on the acquired environmental perception result data and environmental true value data.

[0183] The computer storage medium of the embodiment of the present invention can adopt any combination of one or more computer-readable media. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium can be, for example, but not limited to: an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples (non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this document, a computer-readable storage medium can be any tangible medium containing or storing a program, which can be used by an instruction execution system, device or device or used in combination with it.

[0184] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, which carry computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0185] The program code embodied on the computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0186] Computer program code for performing the operations of the present invention may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0187] It should be understood by those skilled in the art that the modules or steps of the present invention described above can be implemented by a general-purpose computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, optionally, they can be implemented by a program code executable by a computer device, so that they can be stored in a storage device and executed by the computing device, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present invention is not limited to any specific combination of hardware and software.

[0188] Note that the above are only preferred embodiments of the present invention and the technical principles used. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and that various obvious changes, readjustments and substitutions can be made by those skilled in the art without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in more detail through the above embodiments, the present invention is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of the present invention, and the scope of the present invention is determined by the scope of the appended claims.

Claims

1. A sensor model performance evaluation method, characterized in that: include: Get the raw data output by the physical model of the sensor; Inputting the raw data into at least two algorithms respectively, so that each algorithm analyzes and processes the raw data respectively; Acquire environment perception result data respectively output by the at least two algorithms, wherein the environment perception result data is a list of entity objects perceived by the model; Determine the performance evaluation data of the physical model according to the acquired environment perception result data and environment truth data, wherein the environment truth data is a list of entity objects that actually exist in the environment, Wherein, determining the performance evaluation data of the physical model according to the acquired environment perception result data and environment true value data includes: By comparing the environment perception result data output by each algorithm with the environment true value data, if a certain sub-data in the environment perception result data is included in the environment true value data, it is determined that the sub-data is correct environment perception result data, otherwise it is wrong environment perception result data, thereby determining the number of correct environment perception result data in the environment perception result data output by each algorithm; Determine the performance evaluation data of the physical model according to the amount of correct environmental perception result data, Wherein, determining the performance evaluation data of the physical model according to the number of correct environmental perception result data includes: determining the intersection of the correct environmental perception result data outputted by each algorithm respectively; Determine the ratio of the environmental perception result data to the environmental true value data in the intersection as the performance evaluation data of the physical model; or Among them, the performance evaluation data of the physical model is determined according to the number of correct environmental perception result data, including: determining the proportion data of the correct environmental perception result data output by each algorithm to the environmental true value data; and determining the performance evaluation data of the physical model according to the proportion data.

2. The method according to claim 1, characterized in that Determining the performance evaluation data of the physical model according to each proportion data includes: Determine the average value of each proportion data as the performance evaluation data of the physical model; or The minimum scale data among the scale data is determined as the performance evaluation data of the physical model.

3. The method according to claim 1, characterized in that The method further comprises: By comparing the environment perception result data respectively output by the algorithms, the similarity between the environment perception result data respectively output by the algorithms is determined; Prompt information including the similarity is issued, so that the user can determine the confidence of the performance evaluation data according to the similarity.

4. The method according to any one of claims 1 to 3, characterized in that After determining the performance evaluation data of the physical model, the method further includes: Acquiring performance evaluation data of the vehicle in at least one driving performance angle when a driving test is performed on the vehicle equipped with the physical model; Driving performance evaluation data of the vehicle is determined based on the performance evaluation data of the physical model and the performance evaluation data of the at least one driving performance angle.

5. The method according to claim 4, characterized in that Determining driving performance evaluation data of the vehicle according to the performance evaluation data of the physical model and the performance evaluation data of the at least one driving performance angle includes: The driving performance evaluation data of the vehicle is determined based on the performance evaluation data of the physical model, the weight corresponding to the performance evaluation data of the physical model, the performance evaluation data of each driving performance angle, and the weight corresponding to the performance evaluation data of each driving performance angle.

6. The method according to any one of claims 1 to 3, characterized in that The physical model of the sensor is a physical model of a camera, and the raw data is image data; or The physical model of the sensor is a physical model of a radar, and the raw data is point cloud data.

7. A sensor model performance evaluation device, characterized in that: include: A raw data acquisition module, used to acquire raw data output by the physical model of the sensor; A raw data analysis and processing module, used for inputting the raw data into at least two algorithms respectively, so that each algorithm analyzes and processes the raw data respectively; An environment perception result data acquisition module, used to acquire the environment perception result data respectively output by the at least two algorithms; a performance evaluation data determination module, configured to determine the performance evaluation data of the physical model according to the acquired environment perception result data and environment true value data, Wherein, the sensor model performance evaluation device is configured to execute the sensor model performance evaluation method as described in any one of claims 1-6.

8. A device, characterized in that The device comprises: one or more processors; A memory for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the sensor model performance evaluation method as described in any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the sensor model performance evaluation method as described in any one of claims 1 to 6 is implemented.

Images