Method for evaluating sensing performance of sensor, vehicle, and storage medium

Abstract

The application discloses a kind of sensor's sensing performance evaluation method, vehicle and storage medium.Therein, the method includes: obtaining the environment data of the environment where vehicle is located collected by vehicle-mounted sensor, obtaining the first target identified by vehicle-mounted sensor at the time corresponding to the collection of the environment data and its corresponding confidence; according to the target tracking result of the environment data, determine the verification result of the first target, whether the first target is real existence, the target tracking result is obtained by target tracking system according to the environment data analysis;According to the verification result, update the model parameter in the preset model, obtain prediction model, the preset model represents the corresponding relationship between the confidence of the sensor and the identification accuracy;According to the prediction model, determine the target identification accuracy corresponding to the confidence.The application aims to accurately evaluate the sensing performance of vehicle-mounted sensor.

Description

Technical Field

[0001] This invention relates to the field of vehicle technology, and more particularly to a method for evaluating the sensing performance of sensors, a vehicle, and a storage medium. Background Technology

[0002] Perception is one of the key technologies for autonomous driving. Accurately judging the authenticity of detected targets is a prerequisite for autonomous driving systems to make safe and reasonable control strategies, which depends on a correct understanding of the perception performance of sensors.

[0003] Currently, sensor perception performance is generally calibrated offline, that is, manually calibrated when the sensor is not installed on the vehicle. However, the environment in which vehicles are used is diverse and constantly changing, making it difficult to obtain accurate sensor perception performance through offline calibration. Summary of the Invention

[0004] The main objective of this invention is to provide a method for evaluating the sensing performance of a sensor, a vehicle, and a storage medium, with the aim of accurately evaluating the sensing performance of an on-board sensor.

[0005] To achieve the above objectives, the present invention provides a method for evaluating the sensing performance of a sensor, the method comprising the following steps:

[0006] The system acquires environmental data of the vehicle's environment collected by the vehicle-mounted sensors, and acquires the first target identified by the vehicle-mounted sensors at the time corresponding to the acquisition of the environmental data and its corresponding confidence level.

[0007] The verification result of the first target is determined based on the target tracking result of the environmental data. The verification result is whether the first target actually exists. The target tracking result is obtained by the target tracking system based on the analysis of the environmental data.

[0008] The model parameters in the preset model are updated based on the verification results to obtain the prediction model. The preset model represents the correspondence between the confidence level and the recognition accuracy of the sensor.

[0009] The target recognition accuracy corresponding to the confidence level is determined based on the prediction model.

[0010] Optionally, the step of updating the model parameters in the preset model based on the verification result to obtain the prediction model includes:

[0011] The predicted value of the recognition accuracy corresponding to the confidence level is calculated according to the preset model, and a reference value of the recognition accuracy corresponding to the verification result is determined. Different verification results correspond to different reference values.

[0012] Determine the deviation between the predicted value and the reference value;

[0013] The model parameters in the preset model are updated based on the deviation value to obtain the prediction model.

[0014] Optionally, the step of updating the model parameters in the preset model based on the deviation value to obtain the prediction model includes:

[0015] Construct the loss function of the preset model;

[0016] The change in the model parameters is determined based on the deviation value and the loss function;

[0017] The model parameters are updated according to the change in the direction of the decrease in the error of the loss function to obtain the prediction model.

[0018] Optionally, the step of determining the change in the model parameters based on the deviation value and the loss function includes:

[0019] Determine the partial derivatives of the loss function with respect to the model parameters;

[0020] The gradient is obtained by calculating the partial derivative based on the deviation value;

[0021] The amount of change is determined based on the gradient and the target learning rate.

[0022] Optionally, after the step of determining the target recognition accuracy corresponding to the confidence level based on the prediction model, the method further includes:

[0023] Record the prediction model and set the prediction model as a new preset model;

[0024] Return to the steps of acquiring environmental data of the vehicle's environment collected by the vehicle-mounted sensors, and acquiring the first target identified by the vehicle-mounted sensors at the time corresponding to the acquisition of the environmental data and its corresponding confidence level.

[0025] Optionally, the step of determining the target recognition accuracy corresponding to the confidence level based on the prediction model includes:

[0026] Obtain a target model set, which includes more than one of the prediction models recorded before the current time.

[0027] Determine the reference accuracy corresponding to the confidence level for each model in the target model set;

[0028] The target recognition accuracy is determined based on more than one of the reference accuracy rates.

[0029] Optionally, the preset model is a polynomial model.

[0030] Optionally, the target tracking system is constructed based on one of the following algorithms: Kalman filter tracking algorithm, particle filter tracking algorithm, multiple hypothesis tracking algorithm, and joint probability data association algorithm;

[0031] And / or, the sensor's sensing performance evaluation method further includes:

[0032] Obtain the status information of the vehicle;

[0033] When the status information indicates that the vehicle is in a driving state, the steps of acquiring environmental data of the vehicle's environment collected by the vehicle-mounted sensors and acquiring the first target identified by the vehicle-mounted sensors at the time corresponding to the acquisition of the environmental data and its corresponding confidence level are executed.

[0034] Furthermore, in order to achieve the above objectives, this application also proposes a vehicle comprising:

[0035] Vehicle-mounted sensors;

[0036] A sensor perception performance evaluation device, wherein the vehicle-mounted sensor is connected to the perception performance evaluation device, and the perception performance evaluation device includes:

[0037] A memory, a processor, and a sensor perception performance evaluation program stored in the memory and executable on the processor, wherein the sensor perception performance evaluation program, when executed by the processor, implements the steps of the sensor perception performance evaluation method as described in any of the preceding claims.

[0038] In addition, to achieve the above objectives, this application also proposes a storage medium storing a sensor perception performance evaluation program, which, when executed by a processor, implements the steps of the sensor perception performance evaluation method as described in any of the preceding claims.

[0039] This invention proposes a sensor perception performance evaluation method. The method verifies the authenticity of the first target detected by the onboard sensor by analyzing environmental data from the vehicle's environment using a target tracking system. Based on the verification results, it updates the model parameters in a preset model representing the correspondence between confidence and recognition accuracy to obtain a prediction model. The prediction model then determines the target recognition accuracy corresponding to the current confidence level of the onboard sensor's target recognition. Based on this, the output of the target tracking system is used as an instantaneous truth system, which can compensate for the lack of truth data due to the sensor's operation on the vehicle, preventing accurate evaluation of perception performance. By verifying the authenticity of the onboard sensor's recognition results based on the target tracking results, a prediction model is further constructed, ensuring that the predicted target recognition accuracy accurately reflects the sensor's perception performance in the current vehicle usage scenario, thus accurately evaluating the onboard sensor's perception performance. Attached Figure Description

[0040] Figure 1 This is a schematic diagram of the hardware structure involved in the operation of a vehicle according to an embodiment of the present invention;

[0041] Figure 2 This is a flowchart illustrating an embodiment of the sensor sensing performance evaluation method of the present invention.

[0042] Figure 3 This is a schematic diagram of a prediction model involved in an embodiment of the sensor sensing performance evaluation method of the present invention;

[0043] Figure 4 This is a flowchart illustrating another embodiment of the sensor sensing performance evaluation method of the present invention;

[0044] Figure 5 for Figure 4 A detailed flowchart of step S33;

[0045] Figure 6 This is a flowchart illustrating another embodiment of the sensor sensing performance evaluation method of the present invention;

[0046] Figure 7 for Figure 6 A detailed flowchart of step S40.

[0047] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0048] It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0049] This invention provides a vehicle, which can be any type of motor vehicle such as a car or a truck.

[0050] In this embodiment, the vehicle includes an onboard sensor 2 and a sensor perception performance evaluation device 1 connected to the onboard sensor 2. The sensor perception performance evaluation device 1 can be used to identify the performance of the onboard sensor 2 in the vehicle.

[0051] Vehicle sensor 2 is a sensor installed on the vehicle body. Vehicle sensor 2 is a detection module used to collect information about the scene in which the vehicle is located while driving. Vehicle sensor 2 includes, but is not limited to, one or more of the following sensors: camera, millimeter-wave radar, ultrasonic radar, lidar, etc. Vehicle sensor 2 can detect image data, video data, point cloud images, electromagnetic waves, and other information.

[0052] In this embodiment of the invention, reference is made to Figure 1 The sensor performance evaluation device 1 includes a processor 1001 (e.g., CPU), a memory 1002, etc. The components in the sensor performance evaluation device 1 are connected via a communication bus. The memory 1002 can be a high-speed RAM or a stable, non-volatile memory, such as a disk drive. Optionally, the memory 1002 can also be a storage device independent of the aforementioned processor 1001.

[0053] Those skilled in the art will understand that Figure 1 The device structure shown does not constitute a limitation on the device and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0054] like Figure 1 As shown, the memory 1002, which serves as a storage medium, may include a sensor's sensing performance evaluation program. Figure 1 In the device shown, the processor 1001 can be used to call the sensor perception performance evaluation program stored in the memory 1002 and execute the relevant steps of the sensor perception performance evaluation method in the following embodiments.

[0055] This invention also provides a method for evaluating the sensing performance of sensors, which is applied to evaluate the sensing performance of onboard sensors in the aforementioned vehicles.

[0056] Reference Figure 2 This application proposes an embodiment of a sensor sensing performance evaluation method. In this embodiment, the sensor sensing performance evaluation method includes:

[0057] Step S10: Obtain environmental data of the vehicle's environment collected by the vehicle-mounted sensor, and obtain the first target identified by the vehicle-mounted sensor at the time corresponding to the collection of the environmental data and its corresponding confidence level.

[0058] The number of vehicle-mounted sensors may be one or more. Vehicle-mounted sensors include, but are not limited to, one or more of the following: cameras, millimeter-wave radar, ultrasonic radar, lidar, etc. Vehicle-mounted sensors can detect environmental data such as image data, video data, point cloud images, and electromagnetic waves.

[0059] Vehicle-mounted sensors can detect targets within their detection range in the environment based on environmental data. The target identified by the vehicle-mounted sensors is called the "first target," and there may be one or more first targets. Due to detection errors inherent in vehicle-mounted sensors, the first target may be a real target in the environment or a target that does not actually exist in the environment.

[0060] Each time the vehicle-mounted sensors identify a first target, they also output the corresponding confidence level for that first target. Different first targets can correspond to different confidence levels.

[0061] Here, the environmental data, its corresponding first target, and confidence level refer to the data collected in real time. In other embodiments, the environmental data, its corresponding first target, and confidence level may also be historical data collected before the current moment. Alternatively, the environmental data, its corresponding first target, and confidence level may be historical data collected within a preset time period.

[0062] Step S20: Determine the verification result of the first target based on the target tracking result of the environmental data. The verification result is whether the first target actually exists. The target tracking result is obtained by the target tracking system based on the analysis of the environmental data.

[0063] The target tracking system can predict and track targets identified by sensors. In particular, it can track multiple targets from multiple sensors, constructing a perception output for the tracking system, which is more robust. The target tracking result is regarded as an instantaneous truth system, providing a reference for the quality of sensor recognition. For example, if the sensor currently identifies a target and loses it, the target tracking system can provide a prediction value to indicate that the target has appeared before, so as to further determine whether the current sensor perception system's recognition effect is poor, resulting in the loss of the target.

[0064] The target tracking system is constructed based on one of the following algorithms: Kalman filter tracking algorithm, particle filter tracking algorithm, multiple hypothesis tracking algorithm, and joint probability data association algorithm.

[0065] The target tracking system can track and match all first targets detected by vehicle-mounted sensors in environmental data and output the target tracking results. Based on the target tracking results, an evaluation score can be assigned to determine whether the first target detected by the vehicle-mounted sensors actually exists. If the evaluation score is greater than a preset threshold, the corresponding first target is considered to exist; if the evaluation score is less than or equal to the preset threshold, the corresponding first target is considered not to exist.

[0066] Step S30: Update the model parameters in the preset model according to the verification result to obtain the prediction model. The preset model represents the correspondence between the confidence level of the sensor and the recognition accuracy.

[0067] The preset model can be a model whose parameters are pre-set as initial values. The predictive model can also be a predictive model whose parameters are obtained by updating the initial values ​​or further updating the parameters after updating the initial values ​​during the vehicle's operation before the current time. Different vehicle sensors can correspond to different preset models.

[0068] In this embodiment, the preset model is a polynomial model, such as a quadratic polynomial model, which is beneficial for improving the model's update and prediction efficiency while ensuring a high degree of agreement between the sensor's predicted recognition accuracy and the actual recognition accuracy. In other embodiments, the preset model can also be a machine learning model.

[0069] Specifically, a correspondence between validation results and model parameter adjustment parameters can be established in advance. This correspondence can include calculation formulas, mapping relationships, etc. Based on this correspondence, the adjustment parameters corresponding to the current validation results can be determined. After adjusting the current model parameters of the preset model according to the adjustment parameters, the prediction model is obtained.

[0070] In this embodiment, the model parameters in the preset model are updated based on the verification results and confidence levels to obtain the prediction model. In other embodiments, adjustment values ​​for model parameters that have a mapping relationship with the verification results can also be determined, and the prediction model is obtained by updating the model parameters based on the adjustment values.

[0071] Specifically, such as Figure 3 As shown, Figure 3The horizontal axis represents confidence level, and the vertical axis represents recognition accuracy. Different bars represent the actual recognition accuracy of the sensor determined by the training set corresponding to different confidence level intervals. The solid line represents the curve model of the relationship between confidence level calculated from the true values ​​of the dataset and recognition accuracy, which can be used as a reference model for the relationship between confidence level and sensor recognition accuracy, i.e., the reference model for the prediction model. Based on the positional relationship between the curve and the bar distribution plot, it can be seen that the recognition accuracy predicted by the prediction model after the model parameter update is closer to the actual recognition accuracy of the sensor than the recognition accuracy predicted by the preset model before the model parameter update. The SGD line represents the curve corresponding to the prediction model obtained by online updating with small batches of data using the method mentioned in this embodiment. BGD is the curve corresponding to the model obtained by offline updating the model parameters using a large batch of data using the preset model mentioned in this paper. Among them, the data in the 0.9-1 confidence level interval accounts for 60%. This proves that the prediction models constructed by both SGD and BGD methods can fit well with the reference model.

[0072] Step S40: Determine the target recognition accuracy corresponding to the confidence level based on the prediction model.

[0073] Target recognition accuracy is the proportion of real targets detected by the vehicle sensor out of all targets detected (including real targets and non-real targets): precision = TP / (TP+FP), where TP is the number of real targets and FP is the number of non-real targets.

[0074] The result obtained by substituting the confidence level into the prediction model can be used as the target recognition accuracy.

[0075] This invention proposes a sensor perception performance evaluation method. The method verifies the authenticity of the first target detected by the vehicle-mounted sensor by analyzing environmental data from a target tracking system based on the vehicle's environment. Based on the verification results, it updates the model parameters in a preset model representing the correspondence between confidence and recognition accuracy to obtain a prediction model. The prediction model then determines the target recognition accuracy corresponding to the current confidence level of the vehicle-mounted sensor. Based on this, the output of the target tracking system is used as an instantaneous truth system, which can compensate for the lack of truth values ​​due to the sensor's operation on the vehicle, preventing accurate evaluation of perception performance. By verifying the authenticity of the vehicle-mounted sensor's recognition results based on the target tracking results, a prediction model is further constructed, ensuring that the predicted target recognition accuracy accurately reflects the sensor's perception performance in the current vehicle usage scenario and accurately evaluates the vehicle-mounted sensor's perception performance.

[0076] Furthermore, in this embodiment, the current state information of the vehicle is obtained, and step S10 is executed when the state information indicates that the vehicle is in a driving state. Based on this, it is beneficial to accurately reflect the sensor's recognition accuracy rate in the vehicle's driving state, and to improve the matching degree between the target recognition accuracy rate and the vehicle's driving environment, thereby further improving the accuracy of the target recognition rate.

[0077] Furthermore, based on the above embodiments, another embodiment of the sensor sensing performance evaluation method of this application is proposed. In this embodiment, reference is made to... Figure 4 Step S30 includes:

[0078] Step S31: Calculate the predicted value of the recognition accuracy corresponding to the confidence level according to the preset model, and determine the reference value of the recognition accuracy corresponding to the verification result. Different verification results correspond to different reference values.

[0079] Specifically, the result obtained by substituting the confidence level into the preset model is used as the predicted value here.

[0080] If the verification result indicates that the first target actually exists, the first value is determined as a reference value; if the verification result indicates that the first target does not actually exist, the second value is determined as a reference value. In this embodiment, the first value is 1, and the second value is 0.

[0081] Step S32: Determine the deviation between the predicted value and the reference value;

[0082] The deviation value is the result of subtracting the reference value from the predicted value.

[0083] Step S33: Update the model parameters in the preset model according to the deviation value to obtain the prediction model.

[0084] The backpropagation algorithm updates the model parameters in the preset model based on the deviation value, and the updated model is used as the current prediction model.

[0085] Specifically, in this embodiment, the preset model is a multinomial model with confidence level as the independent variable and recognition accuracy as the dependent variable. The prediction model is obtained by updating the coefficients of each term in the multinomial model based on the deviation value using the backpropagation algorithm. For example, the preset model is precision = f(conf) = a0 + a1conf + a2conf 2 Where conf is the confidence level and precision is the recognition accuracy, then after updating a0, a1, and a2 based on the deviation values, we obtain a0′, a1′, and a2′ respectively. Therefore, the prediction model is precision = f(conf) = a0′ + a1′conf + a2′conf 2 .

[0086] In this embodiment, a reference value for the recognition accuracy is determined based on the target tracking results output by the target tracking system. The confidence level of the current first target is input into the predicted value of the recognition accuracy obtained by the current preset model. The deviation value can reflect the accuracy of the preset model in predicting the recognition accuracy at the current confidence level. After updating the preset model based on the deviation value, a prediction model is obtained to predict the target recognition accuracy corresponding to the current confidence level. This helps to reduce the deviation between the performance of the vehicle sensor reflected by the target recognition accuracy and the actual performance of the vehicle sensor, so as to further improve the accuracy of the target recognition accuracy and further improve the accuracy of the vehicle sensor's perception performance.

[0087] Furthermore, in this embodiment, referring to Figure 5 Step S33 includes:

[0088] Step S331: Construct the loss function of the preset model;

[0089] A loss function is specifically used to evaluate the degree of deviation between the predicted values ​​and the actual values ​​of a prediction model. Loss functions can include absolute value loss functions, squared loss functions, logarithmic loss functions, exponential loss functions, mean squared error loss functions, or mean absolute error loss functions, etc.

[0090] In this embodiment, the preset model is the aforementioned precision = f(conf) = a0 + a1conf + a2conf. 2 The loss function Loss is:

[0091]

[0092] Among them, batch size The number of parameters currently used to update the preset model is GT. In this embodiment, the deviation values ​​corresponding to all currently identified first targets can be used to update the preset model, so the number of parameters is the total number of first targets; precision f(conf) represents the actual value (i.e., the reference value) of the preset model. i Characterizes the predicted value of the preset model.

[0093] Step S332: Determine the change in the model parameters based on the deviation value and the loss function;

[0094] The change can be understood as the adjustment value of the model parameters. An objective function corresponding to the gradient of the model parameters is constructed based on the loss function, and the gradient is obtained by solving the objective function based on the bias value.

[0095] In this embodiment, the partial derivatives of the loss function with respect to the model parameters are determined; the gradient is obtained by calculating the partial derivatives based on the deviation value; and the amount of change is determined based on the gradient and the target learning rate.

[0096] Here, the partial derivatives are obtained by differentiating the loss function. Different model parameters correspond to different partial derivatives. The gradient is obtained by solving for the partial derivatives corresponding to the model parameters using the bias value as a known value of the loss function.

[0097] The target learning rate can be a pre-fixed value or a value determined based on the vehicle's state parameters. A larger target learning rate results in faster training but can easily lead to parameter divergence. Conversely, a smaller target learning rate slows down training but makes parameter convergence easier. For example, the total driving time can be obtained, and the target learning rate can be determined based on this total time; the target learning rate can be negatively correlated with the total time. Alternatively, the scene type of the vehicle's environment can be obtained, and the target learning rate can be determined based on this scene type, with different scene types corresponding to different target learning rates.

[0098] Specifically, the change Δa is: Where ε is the target learning rate. The partial derivatives corresponding to the model parameters are used to solve for the gradient. The product of the gradient and the target learning rate is the change.

[0099] Step S333: Update the model parameters according to the amount of change in the direction of the decrease in the error of the loss function to obtain the prediction model.

[0100] Specifically, the model parameters are optimized using the following iterative formula;

[0101]

[0102] Among them, a i To preset the current model parameters, a i ′ represents the model parameters in the updated prediction model. The model parameters are iteratively updated in the direction of gradient descent according to the above formula, so that the error of the loss function changes in the direction of reduction.

[0103] In this embodiment, updating the model parameters based on gradient descent using the above method helps to further reduce the error between the predicted and actual values ​​of the prediction function, thereby further improving the accuracy of target recognition based on the current confidence level determined by the prediction function.

[0104] Furthermore, in this embodiment, the model parameters in the preset parameters are updated using the stochastic gradient descent method. Specifically, the model parameters of the preset model can be updated based on the multiple deviation values ​​corresponding to the multiple first targets identified at the current time; or the model parameters of the preset model can be updated based on the multiple deviation values ​​corresponding to all the first targets identified in the preset time period or preset frame before the current time.

[0105] Furthermore, based on any of the above embodiments, another embodiment of the sensor sensing performance evaluation method of this application is proposed. In this embodiment, reference is made to... Figure 6 After step S40, the method further includes:

[0106] Step S50: Record the prediction model and set the prediction model as a new preset model;

[0107] After step S50, return to step S10.

[0108] Based on this, the model parameters of the prediction model can be continuously updated iteratively in the manner mentioned in the above embodiments during the use of vehicle-mounted sensors, thereby making the prediction results of the prediction model increasingly accurate during vehicle use, so as to improve the accuracy of the perception performance evaluation of vehicle-mounted sensors in their usage scenarios.

[0109] Furthermore, in this embodiment, during the iterative update of the prediction model parameters, reference is made to... Figure 7 The step of determining the target recognition accuracy corresponding to the confidence level based on the prediction model includes:

[0110] Step S41: Obtain the target model set, which includes more than one prediction model recorded before the current time.

[0111] Specifically, the multiple prediction models here refer to the prediction models of the preset records that are closest to the current time.

[0112] Step S42: Determine the reference accuracy corresponding to the confidence level for each model in the target model set;

[0113] The confidence level is input into more than one prediction model to obtain the recognition accuracy of each prediction model as a reference accuracy, thus obtaining more than one reference accuracy.

[0114] Step S43: Determine the target recognition accuracy based on more than one reference accuracy. The average of the more than one reference accuracy is determined as the target recognition accuracy.

[0115] In this embodiment, the target recognition accuracy corresponding to the current confidence level is determined based on the recognition accuracy corresponding to the current confidence level predicted by more than one previous prediction model in the above manner, which helps to improve the stability of the prediction results of the prediction model.

[0116] In other embodiments, each reference accuracy may also have a corresponding weight value determined based on the scene type in which the vehicle was located when the corresponding prediction model was constructed, and the weighted average result calculated based on more than one reference accuracy and its corresponding weight value is used as the target recognition accuracy.

[0117] Furthermore, based on any of the above embodiments, another embodiment of the sensor perception performance evaluation method of this application is proposed. In this embodiment, taking a tracking system built based on a multi-hypothesis tracking algorithm as an example, the target tracking matching process and the process of determining the verification result of the target tracking system involved in this application are explained:

[0118] (1) Target tracking

[0119] At the initial moment of environmental data acquisition, the targets observed by the vehicle-mounted sensors in the environmental data are represented by a four-dimensional vector X = [xy v]. x v y ] T , where v x and v y This represents the velocity of the corresponding target in a two-dimensional plane. Assuming the inter-frame time interval is ΔT, the target's position and state at the next moment can be predicted using the following formula:

[0120]

[0121] superscript This is represented as a priori state. This is represented as the posterior state.

[0122] Assume the observations are represented as Z = [xy] T :

[0123] Z k =HX k-1 +R#(3)

[0124] H is defined as: H = [I 2×2 0 2×5 R is the estimated noise of the zero-mean sensor.

[0125] Once the observed target is assigned to a track, the state can be updated by a Kalman filter:

[0126]

[0127] Each observation target is assigned a tracker.

[0128] (2) Target association

[0129] The tracking system associates the predicted targets with the targets observed by the onboard sensors and updates the current tracking system score using the likelihood ratio. Specifically, the Euclidean distance between the positions of all predicted targets and all positions observed by the onboard sensors is calculated. A matching matrix is ​​determined based on the Euclidean distance, and the matching matrix is ​​solved using a matching algorithm. Based on the solution of the matching matrix, a one-to-one association between the predicted targets and the observed targets of the onboard sensors is established. The tracking system score is calculated for each associated predicted target and observed target in the manner described below.

[0130] New observations from onboard sensors are correlated with the existing tracking system, extending or generating new tracking hypotheses based on existing ones. The likelihood ratio (LR) is defined as the probability expression used to evaluate the data correlation.

[0131]

[0132] H1 and H0 represent the assumptions that the target is real or non-real. i (H i ) is defined as H i The prior probability, p(D|H) i H is the result after receiving the target. i The conditional probability density function is true. LR can be directly converted into the probability of a real target:

[0133]

[0134] LR consists of two parts: the first part is the motion factor, and the second part is the observation factor. Motion factor LR k This is related to the update equation of the Kalman filter mentioned above:

[0135]

[0136] p(D k |H1) is defined as the positional distribution based on the assumption that the target actually exists, and that it follows a Gaussian distribution. Observing Z k Substituting the values ​​into the Kalman filter equation yields the mean. Sum of variances ∑ k p(D) k |H0) is based on the assumption that the target does not actually exist, and that it conforms to the predicted volume V. c A uniform distribution, where p(D) k|H0)=1 / V c .

[0137] Observation factor LR s Defined as:

[0138]

[0139] p(z s |Det,H i An observed event is defined as an event that is observed. Assuming that the onboard sensors only return one signal indicating the presence of a target, then regardless of whether a target is detected... Both are 0. P(Det|H1) is considered to be the detection probability P D P(Det|H0) is considered the false positive rate (that is, the probability that the observed target does not actually exist). FPR If no target appears, then

[0140] In the continuous tracking hypothesis, LR will be iteratively updated in log-fractional form:

[0141] LLR k =LLR k-1 +ΔL(k)#(9)

[0142] If the target appears, otherwise

[0143] Here LLR k This can be understood as the evaluation score mentioned in the above embodiments used to determine the verification result corresponding to the first target. By checking whether the evaluation score corresponding to the first target reaches a preset value, it can be determined whether the first target actually exists.

[0144] Furthermore, this embodiment of the invention also proposes a storage medium storing a sensor perception performance evaluation program. When the sensor perception performance evaluation program is executed by a processor, it implements the relevant steps of any embodiment of the sensor perception performance evaluation method described above.

[0145] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or system. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or system that includes that element.

[0146] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0147] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, vehicle, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0148] The above are merely preferred embodiments of the present invention and do not limit the scope of the patent. Any equivalent structural or procedural transformations made based on the description and drawings of the present invention, or direct or indirect applications in other related technical fields, are similarly included within the scope of patent protection of the present invention.

Claims

1. A method of evaluating a sensing performance of a sensor, characterized by, The method for evaluating the sensing performance of the sensor comprises the following steps: obtaining environment data of an environment in which a vehicle is located, which is collected by a vehicle-mounted sensor, and obtaining a first target recognized by the vehicle-mounted sensor at a time corresponding to the collection of the environment data and a confidence degree corresponding to the first target; determining a verification result of the first target according to a target tracking result of the environment data, the verification result being whether the first target actually exists, and the target tracking result being obtained by a target tracking system according to analysis of the environment data; updating a model parameter in a preset model according to the verification result to obtain a prediction model, the preset model representing a corresponding relationship between the confidence degree and an identification accuracy rate of the sensor; determining the target identification accuracy rate corresponding to the confidence degree according to the prediction model.

2. The method of claim 1, wherein The step of updating the model parameter in the preset model according to the verification result to obtain the prediction model comprises: calculating a predicted value of the identification accuracy rate corresponding to the confidence degree according to the preset model, determining a reference value of the identification accuracy rate corresponding to the verification result, and different verification results corresponding to different reference values; determining a deviation value between the predicted value and the reference value; updating the model parameter in the preset model according to the deviation value to obtain the prediction model.

3. The method of claim 2, wherein The step of updating the model parameter in the preset model according to the deviation value to obtain the prediction model comprises: constructing a loss function of the preset model; determining a change amount of the model parameter according to the deviation value and the loss function; updating the model parameter according to the change amount in a direction in which an error of the loss function is reduced to obtain the prediction model.

4. The method of claim 3, wherein The step of determining the change amount of the model parameter according to the deviation value and the loss function comprises: determining a partial derivative of the loss function with respect to the model parameter; calculating a gradient of the partial derivative according to the deviation value; determining the change amount according to the gradient and a target learning rate.

5. The method of claim 2, wherein the sensor's sensing performance is evaluated by: After the step of determining the target identification accuracy rate corresponding to the confidence degree according to the prediction model, the method further comprises: recording the prediction model and setting the prediction model as a new preset model; returning to the step of obtaining the environment data of the environment in which the vehicle is located, which is collected by the vehicle-mounted sensor, and obtaining the first target recognized by the vehicle-mounted sensor at a time corresponding to the collection of the environment data and the confidence degree corresponding to the first target.

6. The method of claim 5, wherein The step of determining the target identification accuracy rate corresponding to the confidence degree according to the prediction model comprises: obtaining a target model set, the target model set comprising more than one prediction model recorded before a current time; determining a reference accuracy rate corresponding to the confidence degree according to each model in the target model set; determining the target identification accuracy rate according to more than one reference accuracy rate.

7. The method of claim 2, wherein the sensor's sensing performance is evaluated by, The preset model is a polynomial model.

8. The method of evaluating the sensing performance of a sensor according to any one of claims 1 to 7, wherein, The target tracking system is constructed based on one of the following algorithms: a Kalman filter tracking algorithm, a particle filter tracking algorithm, a multiple hypothesis tracking algorithm and a joint probabilistic data association algorithm. Moreover, the method for evaluating the sensing performance of the sensor further comprises: obtaining state information of the vehicle; When the state information is that the vehicle is in the driving state, the step of acquiring environment data of an environment where the vehicle is located collected by the vehicle-mounted sensor, and acquiring a first target recognized by the vehicle-mounted sensor at a time corresponding to the acquisition of the environment data and a corresponding confidence level is performed.

9. A vehicle characterized by comprising: The vehicle comprises: a vehicle-mounted sensor; a sensor perception performance evaluation device, the vehicle-mounted sensor being connected to the sensor perception performance evaluation device, the sensor perception performance evaluation device comprising a memory, a processor, and a sensor perception performance evaluation program stored in the memory and executable on the processor, the sensor perception performance evaluation program, when executed by the processor, implementing the steps of the sensor perception performance evaluation method according to any one of claims 1 to 8.

10. A storage medium, characterized by The storage medium stores a sensor perception performance evaluation program, the sensor perception performance evaluation program, when executed by the processor, implementing the steps of the sensor perception performance evaluation method according to any one of claims 1 to 8.

Images