Method and device for determining confidence of sensing target and method and device for responding to sensing target

Abstract

The invention relates to the field of vehicle automatic driving, and more specifically relates to a method for determining the confidence of a perceived target, a smart device, a computer readable storage medium, and a method for responding to the perceived target, a smart device, and a computer readable storage medium. The method for determining the confidence coefficient of the sensing target in one aspect of the invention comprises the following steps: acquiring feature data of one or more frames of sensing targets; and inputting the feature data of the one or more frames of perceptual targets into a perceptual target confidence model to determine the confidence of the perceptual targets, the training data of the perceptual target confidence model including response data of automatic driving to the perceptual targets and label data corresponding to the response data.

Description

Technical Field

[0001] The present application relates to the field of vehicle autonomous driving, and more specifically to a method, an intelligent device, a computer-readable storage medium for determining the confidence of a perceived target, and a method, an intelligent device, and a computer-readable storage medium for responding to a perceived target. Background Art

[0002] In the autonomous driving system architecture of perception, decision-making, planning and control, the perception model trained based on massive training data has good basic performance, but it is still difficult to avoid the occurrence of false perception in practical applications. Usually, the erroneous output of the perception model will affect the decision-making and planning modules' planning and control of the vehicle. For example, if the perception model gives an incorrect target or target attribute, it is easy to cause an erroneous response from the decision-making and planning modules, such as incorrect braking, incorrect lane change, and jitter in the planned trajectory.

[0003] In order to improve the perception accuracy of the perception large model, more training data is usually mined to train the perception large model. However, the above training process requires mining, cleaning and labeling a large amount of data, which has high data costs and long model training time, resulting in low model version iteration efficiency. Summary of the invention

[0004] In order to solve or at least alleviate one or more of the above problems, the following technical solutions are provided.

[0005] According to a first aspect of the present application, a method for determining the confidence of a perceived target is provided, the method comprising the following steps: acquiring feature data of the perceived target of one or more frames; inputting the feature data of the perceived target of the one or more frames into a perception target confidence model to determine the confidence of the perceived target, wherein the training data of the perception target confidence model comprises response data of the autonomous driving to the perceived target and label data corresponding to the response data.

[0006] According to a method for determining the confidence of a perceived target described in an embodiment of the present application, obtaining feature data of the perceived target of one or more frames includes: obtaining feature data of the perceived target of one or more frames based on output data of a vehicle perception model.

[0007] According to the method for determining the confidence of a perceived target according to one embodiment or any of the above embodiments of the present application, the feature data of the perceived target includes one or more of the following: the type of perceived target, the occlusion ratio of the perceived target, the tracking period of the perceived target, the loss period of the perceived target, the attribute label of the perceived target, the position of the perceived target, the speed of the perceived target, the heading of the perceived target, the observation continuity of the perceived target, the observation noise of the perceived target, the motion consistency of the perceived target, the key frame association quality of the perceived target, and the observation association quality of the perceived target.

[0008] According to the method for determining the confidence of a perceived target according to one embodiment of the present application or any of the above embodiments, the perceived target confidence model is used to determine the confidence of the perceived target based on at least one of the following: feature correlation between feature data of the perceived target in each frame; temporal correlation between feature data of the perceived target in multiple frames.

[0009] According to the method for determining the confidence of a perceived target according to one embodiment of the present application or any one of the above embodiments, the response data of the autonomous driving to the perceived target and the label data corresponding to the response data include: the response data obtained by the decision planning module of the autonomous driving in response to the perceived target; and the label data used to mark the response data as a correct response or an incorrect response.

[0010] According to the method for determining the confidence of a perceived target according to one embodiment of the present application or any of the above embodiments, the decision-making planning module of the autonomous driving is applied to one or more of the following scenarios: a simulation scenario, a road test scenario, and a shadow mode scenario.

[0011] According to the method for determining the confidence of a perceived target according to one embodiment of the present application or any of the above embodiments, the perceived target confidence model is a perceived target confidence model based on a self-attention mechanism.

[0012] According to the method for determining the confidence of a perceived target according to one embodiment of the present application or any of the above embodiments, the perceived target confidence model based on the self-attention mechanism includes: an embedding layer, which is used to determine the quantile feature vector of the feature data of the perceived target of the one or more frames relative to the feature data set of the perceived target; an encoder, which is used to obtain a query vector, a key vector and a value vector based on a linear mapping of the quantile feature vector, and determine an attention vector based on the query vector, the key vector and the value vector; and a classification layer, which is used to determine the confidence of the perceived target based at least on the quantile feature vector and the attention vector.

[0013] According to the method for determining the confidence of a perceived target according to one embodiment of the present application or any of the above embodiments, the classification layer is configured to: perform residual connection on the quantile feature vector and the attention vector to obtain a residual connection vector; normalize the residual connection vector to obtain a normalized vector; and perform binary classification on the normalized vector to determine the confidence of the perceived target.

[0014] According to the method for determining the confidence of a perceptual target according to one embodiment of the present application or any of the above embodiments, the encoder includes multiple Transformer Encoders.

[0015] According to the second aspect of the present application, there is provided an intelligent device for determining the confidence of a perceived target, the intelligent device comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein a computer program is stored in the memory, and when the computer program is executed by the at least one processor, the method for determining the confidence of a perceived target according to the first aspect of the present application is implemented.

[0016] According to a third aspect of the present application, a computer-readable storage medium is provided, in which a plurality of program codes are stored, wherein the program codes are suitable for being loaded and run by a processor to execute the method for determining the confidence of a perceived target according to the first aspect of the present application.

[0017] According to a fourth aspect of the present application, a method for responding to a perceived target is provided, the method comprising the following steps: determining the confidence of the perceived target according to the method for determining the confidence of a perceived target according to the first aspect of the present application; and selectively responding to the perceived target according to the confidence of the perceived target.

[0018] According to a method for responding to a perceived target described in an embodiment of the present application, selectively responding to the perceived target based on the confidence of the perceived target includes: when the confidence is greater than or equal to a first confidence threshold, performing an autonomous driving function response to the perceived target; when the confidence is less than the first confidence threshold and greater than or equal to a second confidence threshold, performing an autonomous driving function response to the perceived target after a delay of a preset time period; and when the confidence is less than the second confidence threshold, not performing an autonomous driving function response to the perceived target.

[0019] According to the fifth aspect of the present application, there is provided an intelligent device for responding to a perceived target, the intelligent device comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein a computer program is stored in the memory, and when the computer program is executed by the at least one processor, the method for responding to a perceived target according to the fourth aspect of the present application is implemented.

[0020] According to a sixth aspect of the present application, a computer-readable storage medium is provided, in which a plurality of program codes are stored, wherein the program codes are suitable for being loaded and run by a processor to execute the method for responding to a perceived target according to the fourth aspect of the present application.

[0021] According to one or more embodiments of the present application, the perception target confidence model in the scheme for determining the confidence of the perception target can use autonomous driving to perform parallel training on the response data of the perception target and its corresponding label data, without mining a large amount of training data, reducing the cost of training data and improving the efficiency of model iteration. According to one or more embodiments of the present application, the scheme for determining the confidence of the perception target can obtain accurate confidence of the perception target through the perception target confidence model, thereby guiding the vehicle's decision-making planning control to selectively respond to the perception target, further improving autonomous driving performance and user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The above and / or other aspects and advantages of the present application will become clearer and easier to understand through the following description of various aspects in conjunction with the accompanying drawings, in which the same or similar units are represented by the same reference numerals. The accompanying drawings include:

[0023] Figure 1 A flowchart of a method for determining the confidence of a perceived target according to one or more embodiments of the present application is shown.

[0024] Figure 2 A schematic diagram of a perceptual target confidence model based on a self-attention mechanism according to an embodiment of the present application is shown.

[0025] Figure 3 A flowchart of a method for responding to a perceived target according to one or more embodiments of the present application is shown.

[0026] Figure 4 A schematic diagram of a typical scenario for responding to a perceived target is shown.

[0027] Figure 5 A schematic diagram of a typical scenario for responding to a perceived target is shown.

[0028] Figure 6A schematic block diagram of an intelligent device for determining the confidence of a perceived target according to one or more embodiments of the present application is shown.

[0029] Figure 7 A schematic block diagram of a smart device for responding to a perception target according to one or more embodiments of the present application is shown.

[0030] Figures 8A-8B The verification results of the perceptual target confidence model according to one or more embodiments of the present application are shown. DETAILED DESCRIPTION

[0031] The present application is described more fully below with reference to the accompanying drawings in which illustrative embodiments of the present application are illustrated. However, the present application may be implemented in different forms and should not be interpreted as being limited to the embodiments given herein. The above embodiments are given to make the disclosure herein comprehensive and complete, so as to more fully convey the scope of protection of the present application to those skilled in the art.

[0032] In this specification, terms such as "comprise" and "include" indicate that in addition to the units and steps directly and explicitly stated in the specification and claims, the technical solution of the present application does not exclude the situation where there are other units and steps that are not directly or explicitly stated.

[0033] Unless otherwise specified, terms such as "first" and "second" do not indicate the order of the units in terms of time, space, size, etc., but are merely used to distinguish the units.

[0034] Hereinafter, various exemplary embodiments according to the present application will be described in detail with reference to the accompanying drawings.

[0035] Figure 1 A flowchart of a method for determining the confidence of a perceived target according to one or more embodiments of the present application is shown.

[0036] like Figure 1 As shown in FIG. 1 , in step S101 , feature data of a perceived target of one or more frames is obtained.

[0037] In step S103, feature data of one or more frames of the perceived target is input into a perceived target confidence model to determine the confidence of the perceived target, wherein the training data of the perceived target confidence model includes response data of the autonomous driving to the perceived target and label data corresponding to the response data.

[0038] Optionally, in step S101, feature data of one or more frames of perception targets may be acquired based on output data of the vehicle perception model.

[0039] In one embodiment, the vehicle perception model can perceive the vehicle's surrounding environment based on data from multiple sensors (e.g., laser radar, camera, millimeter wave radar, etc.), and output the location, shape, category and speed information of obstacles, as well as the speed, direction, trajectory prediction of moving pedestrians and vehicles, etc. Exemplarily, the vehicle perception model can be implemented as a convolutional neural network, a recurrent neural network, a deep learning model based on Tranformer, etc.

[0040] In one embodiment, the characteristic data of the perceived target may include any combination of one or more of the following: the type of the perceived target, the occlusion ratio of the perceived target, the tracking period of the perceived target, the loss period of the perceived target, the attribute label of the perceived target, the position of the perceived target, the speed of the perceived target, the heading of the perceived target, the observation continuity of the perceived target, the observation noise of the perceived target, the motion consistency of the perceived target, the key frame association quality of the perceived target, and the observation association quality of the perceived target. Through any combination of the above items, multi-dimensional and multi-level customized characteristic data of the perceived target can be designed, which is conducive to the lightweight of the perceived target confidence model, reduces the data volume requirement of the model, and improves the accuracy of the confidence of the perceived target determined by the model.

[0041] Exemplarily, the types of perceived targets may include other vehicles, pedestrians, cyclists, and other general obstacles (e.g., animals, roadblocks, buildings, bridges, trees, etc.). Exemplarily, the occlusion ratio of a perceived target refers to the degree to which the perceived target is partially or completely obscured by other objects in an image or data acquired by a sensor (e.g., a camera, a lidar, etc.), which can be determined, for example, by calculating the area ratio of the visible area of ​​the perceived target to the total area. Exemplarily, the attribute labels of the perceived target may include a group vulnerable road user (Vulnerable Road Users, VRU) label (e.g., a label for classifying and marking a group of target objects with similar spatial positions) and a label for indicating whether a person exists. Exemplarily, the observation continuity of the perceived target can be used to indicate whether the observation is continuous within the life cycle of the perceived target, which can be determined by the ratio of the actual number of observations to the theoretical number of observations within the observation time window. Exemplarily, the observation noise of the perceived target can be used to indicate the error or uncertainty introduced in the process of sensor data acquisition. Exemplarily, the motion consistency of the perceived target can be used to indicate whether the motion state of the perceived target is consistent between the previous and next frames. Exemplarily, the key frame association quality of the perceived target can be expressed as the mean association score and the standard deviation of the association score of the observations participating in the state optimization, which is used to indicate whether the observation associations participating in the state optimization are accurate and stable. Exemplarily, a portion of the observation frames can be selected as key frames based on the observation quality, for example, observation frames with no abnormal values ​​in the fitting error can be selected as key frames. Exemplarily, the observation association quality of the perceived target can be expressed as the mean association score and the standard deviation of the association score of the targets observed by each sensor in the historical observation of the target, which is used to indicate whether the association of the perceived target observed by each sensor is reliable.

[0042] In one embodiment, in step S103, feature data of the perceived target of one or more frames may be input into a perceived target confidence model to determine at least one of feature correlations between feature data of the perceived target of each frame and temporal correlations between feature data of the perceived targets of multiple frames, so as to determine the confidence of the perceived target based on at least one of feature correlations between feature data of the perceived target of each frame and temporal correlations between feature data of the perceived targets of multiple frames.

[0043] In one embodiment, the perception target confidence model can be implemented as a perception target confidence model based on a self-attention mechanism. In the perception target confidence model based on a self-attention mechanism, for each element in the input sequence, the correlation between it and other elements can be calculated, and a weighted vector can be generated accordingly. This weighted vector can be regarded as a representation of the element in the current context. In this way, the model can automatically learn the dependencies between different element positions, thereby capturing richer contextual information and improving the accuracy of confidence calculation.

[0044] In one embodiment, the perception target confidence model based on the self-attention mechanism can map the input sequence based on the feature data of the perception target to the representation space of the query vector, the key vector and the value vector. For each element in the input sequence, the similarity between the query vector and the key vectors of all other elements can be determined by dot product calculation or scaled dot product calculation, and the attention weight (which can represent the attention of each element to all other elements) can be assigned to each element according to the similarity, and the attention weight can be normalized by the softmax function, for example. Then, for each element in the input sequence, the value vector can be weighted and summed using the attention weight to obtain a weighted context vector. By implementing the perception target confidence model as a perception target confidence model based on the self-attention mechanism, global dependencies can be captured based on the feature data of the perception target, which helps to improve the accuracy of confidence calculation, especially when there is a mutual relationship between the perception targets. In addition, the self-attention mechanism can effectively capture the long-distance dependencies in the feature data of the perception target, which is particularly important for understanding the behavior of the perception target changing over time or space, thereby improving the accuracy of confidence calculation.

[0045] In one embodiment, the perceptual target confidence model based on the self-attention mechanism can be implemented based on a multi-head attention mechanism. Each head in the multi-head attention mechanism can map the input sequence based on the feature data of the perceptual target to multiple different representation subspaces of the query vector, key vector and value vector, so as to allow the perceptual target confidence model to obtain weighted context vectors in parallel in different representation subspaces. The context vectors of each head can be concatenated together and then transformed and output through a linear layer. By capturing information from different representation subspaces in parallel, the model can more comprehensively understand the feature data of the perceptual target, accelerate the training process of the model, and enable it to better process the feature data of complex perceptual targets.

[0046] In one embodiment, the response data of the autonomous driving to the perceived target and the label data corresponding to the response data may include the response data obtained by the decision planning module of the autonomous driving in response to the perceived target and the label data used to mark the response data as a correct response or an incorrect response. In one embodiment, the decision planning module of the autonomous driving can be applied to simulation scenarios, road test scenarios and shadow mode scenarios, wherein the shadow mode scenario refers to a mode scenario in which the vehicle transmits data back during actual road operation and tests the autonomous driving performance of the vehicle. Exemplarily, in the shadow mode scenario, the vehicle can operate under actual road conditions and simulate corresponding driving decisions with the help of the autonomous driving algorithm. These driving decisions will not directly control the vehicle but will be compared with the actual operation of the driver to optimize and improve the autonomous driving algorithm.

[0047] In one embodiment, the response data of the autonomous driving to the perceived target may include specific vehicle behaviors determined based on the real-time perception data, such as whether to brake, change lanes, overtake, avoid pedestrians or other vehicles, etc. By using the response data of the autonomous driving to the perceived target and the label data corresponding to the response data to train the perceived target confidence model, the cost of training data can be significantly reduced by means of secondary learning of the correct response and the wrong response of the decision planning module. For example, the perceived target confidence model can be trained with only hundreds of data fragments returned in the shadow mode scene of millions of kilometers. In addition, the perceived target confidence model can learn the relationship between the features and the time series characteristics of the training data, and the actual application effect is better. The structure of the model can support customization, easy plug-in and fine-tuning. In addition, the perceived target confidence model can be trained in parallel using a graphics processing unit (GPU), and gradient optimization is performed based on the loss function, so that the consistency of the model indicators is better and the balance is stronger. The perception target confidence model in the method for determining the confidence of a perception target according to one or more embodiments of the present application can utilize autonomous driving to perform parallel training on the response data of the perception target and its corresponding label data, without mining a large amount of training data, thereby reducing the cost of training data and improving the efficiency of model iteration. The method for determining the confidence of a perception target according to one or more embodiments of the present application can obtain accurate confidence of the perception target through the perception target confidence model, thereby guiding the decision-making planning control of the vehicle to selectively respond to the perception target, further improving autonomous driving performance and user experience.

[0048] The following combination Figure 2 A schematic diagram describing a perceptual target confidence model based on a self-attention mechanism according to an embodiment of the present application.

[0049] like Figure 2As shown in , the perceptual target confidence model 200 based on the self-attention mechanism includes an embedding layer 210, an encoder 220 and a classification layer 230, wherein the encoder 220 may include multiple Transformer Encoders, each of which includes a multi-head self-attention module 2201, a normalization layer 2202, a multi-layer feedforward neural network module 2203 and a normalization layer 2204.

[0050] The embedding layer 210 is used to determine the quantile feature vector of the feature data of the perceived target of one or more frames relative to the feature data set of the perceived target. Exemplarily, the embedding layer 210 can determine the quantile feature vector based on the chi-square binning method. Assuming that all samples in the feature data set of the perceived target have m features and they are quantile n, a quantile value of dimension (m, n) can be obtained. Assuming that the number of samples of the feature data of the perceived target is k and has features of dimension (k, m), subtract the quantile value of dimension (m, n) to obtain a feature vector of dimension (k, m, n), calculate the eigenvalue of the feature vector of dimension (k, m, n) and then use the SoftMax activation function to map the eigenvalue to 0-1 space, and then multiply it by the embedding layer weight of dimension (n, j), that is, obtain a quantile feature vector of dimension (k, m, j), where j represents the dimension of the embedding layer 210. Through the above operation of the embedding layer 210, the perceptual target confidence model 200 based on the self-attention mechanism can learn key information in relatively sparse training data.

[0051] The encoder 220 is used to obtain a query vector, a key vector and a value vector based on a linear mapping of the quantile feature vector output by the embedding layer 210, and determine an attention vector based on the query vector, the key vector and the value vector. In one embodiment, a similarity vector between the query vector and the key vector can be determined (for example, by calculating the dot product between the query vector and the key vector), the similarity vector is scaled and softmaxed to obtain an attention weight of each query vector and the key vector, and the value vector is weighted according to the attention weight to determine the attention vector.

[0052] Exemplarily, assuming that the dimension of the quantile feature vector is (k, m, j), the encoder 220 may linearly map the quantile feature vector of dimension (k, m, j) into a query vector, a key vector, and a value vector of dimension (k, m, head_size*head_num), and determine the attention vector based on the query vector, the key vector, and the value vector as:

[0053]

[0054] Among them, query represents the query vector, key represents the key vector, dk represents the dimension of the key vector, and value represents the value vector. The dimension of the attention vector is (k, head_num, m, head_size), where head num represents the number of self-attention heads, and head size represents the dimension of the hidden layer of each self-attention head.

[0055] For example, refer to Figure 2 As shown, the encoder 220 may include a plurality of Transformer Encoders, each Transformer Encoder including a multi-head self-attention module 2201, a normalization layer 2202, a multi-layer feedforward neural network module 2203 and a normalization layer 2204. The multi-head self-attention module 2201 is used to determine the attention vector based on the query vector, key vector and value vector obtained by the linear mapping of the quantile feature vector output by the embedding layer 210. The normalization layer 2202 is used to normalize the residual connection between the query vector, key vector and value vector and the attention vector. The multi-layer feedforward neural network module 2203 can be implemented as two linear transformations and a nonlinear activation function set between the two linear transformations, such as ReLU or GELU, to perform nonlinear transformation on the vector output by the normalization layer 2202. The normalization layer 2204 is used to normalize the residual connection between the input and output of the multi-layer feedforward neural network module 2203.

[0056] The classification layer 230 is used to determine the confidence of the perceived target based on at least the quantile feature vector and the attention vector.

[0057] In one embodiment, the classification layer 230 can be configured to perform residual connection on the quantile feature vector and the attention vector to obtain a residual connection vector, normalize the residual connection vector to obtain a normalized vector, and perform binary classification on the normalized vector to determine the confidence of the perceived target. Through residual connection and normalization, the gradient vanishing problem can be prevented, the training process can be accelerated, and the stability of the model can be improved. Exemplarily, assuming that the dimension of the quantile feature vector is (k, m, j) and the dimension of the attention vector is (k, head_num, m, head_size), the quantile feature vector and the attention vector can be residually connected (Add) and layer normalized (Normalize) to output a normalized vector with a dimension of (k, m, head_size*head_num). Next, the classification layer 230 can establish a mapping from the normalized vector to the binary classification result. For example, the normalized vector (k, m, head_size*head_num) is processed by a feedforward neural network and relu to obtain a vector of dimension (k, m*head_size*head_num), and then processed by linear mapping and sigmoid to obtain the confidence (0~1) of the perceived target of dimension (m).

[0058] It should be noted that Figure 2 A typical perception target confidence model based on the self-attention mechanism is described only as an example. Without departing from the spirit and scope of the present application, the method for determining the confidence of the perception target according to one or more embodiments of the present application may also adopt other forms of deep learning models based on the self-attention mechanism.

[0059] Figure 3 A flowchart of a method for responding to a perceived target according to one or more embodiments of the present application is shown.

[0060] like Figure 3 As shown in , in step S301, the confidence of the perceived target is determined according to the method for determining the confidence of the perceived target according to one or more embodiments of the present application.

[0061] In step S303, the perceived target is selectively responded to according to the confidence level of the perceived target.

[0062] Optionally, in step S303, when the confidence level is greater than or equal to a first confidence threshold (e.g., 0.48), the autonomous driving function response may be performed to the perceived target; when the confidence level is less than the first confidence threshold and greater than or equal to a second confidence threshold (e.g., 0.42), the autonomous driving function response may be performed to the perceived target after a preset time period (e.g., a delay of several milliseconds); and when the confidence level is less than the second confidence threshold, the autonomous driving function response is not performed to the perceived target.

[0063] In one embodiment, taking the autonomous driving function as controlling brakes as an example, when the confidence of the perceived target is determined to be greater than or equal to 0.48, the braking operation can be controlled to be performed directly; when the confidence of the perceived target is determined to be less than 0.48 and greater than or equal to 0.42, the braking operation can be controlled after a preset time period (for example, a delay of several milliseconds); when the confidence of the perceived target is determined to be less than 0.42, the braking operation can be controlled not to be performed. Exemplarily, the autonomous driving function response can include vehicle control such as acceleration, braking, steering, etc., in combination with environmental information (for example, other vehicles, pedestrians, obstacles, and traffic rules information) obtained by the perception module.

[0064] The method for responding to a perceived target according to one or more embodiments of the present application can guide the vehicle's decision-making planning control to selectively respond to the perceived target based on the confidence of the perceived target obtained by the perceived target confidence model, thereby further improving autonomous driving performance and user experience.

[0065] Figure 4 A schematic diagram of a typical scenario for responding to a perceived target is shown.

[0066] like Figure 4 As shown in the figure, in the scene 400 for responding to a perceived target, the vehicle 410 is moving straight, and the vehicle perception model recognizes that there is a stationary obstacle 420 in front of the vehicle 410. According to the method for determining the confidence of a perceived target in one or more embodiments of the present application, it is determined that the confidence of the stationary obstacle 420 is lower than 0.42, and the autonomous driving function response to the stationary obstacle 420 may not be performed, that is, the vehicle 410 may be controlled not to perform a braking operation. It should be noted that although Figure 4 Multiple perception targets different from the vehicle 410 identified by the vehicle perception model are shown. For the sake of clarity, a stationary obstacle 420 is taken as an example to describe the scenario 400 for responding to the perception target.

[0067] Figure 5 A schematic diagram of a typical scenario for responding to a perceived target is shown.

[0068] like Figure 5 As shown in the figure, in the scene 500 for responding to a perceived target, the vehicle 510 is moving straight, and the vehicle perception model recognizes that there is an obstacle 520 with a lateral speed in front of the vehicle 510. According to the method for determining the confidence of a perceived target in one or more embodiments of the present application, the confidence of the obstacle 520 is determined to be lower than 0.42, and the autonomous driving function response to the obstacle 520 may be omitted, that is, the vehicle 510 may be controlled not to perform a braking operation. It should be noted that although Figure 5A plurality of perception targets different from the vehicle 510 recognized by the vehicle perception model are shown. For the sake of clarity, an obstacle 520 with a lateral velocity is taken as an example to describe the scenario 500 for responding to the perception target.

[0069] Figure 6 A schematic block diagram of an intelligent device for determining the confidence of a perceived target according to one or more embodiments of the present application is shown.

[0070] like Figure 6 As shown in FIG. 6 , the smart device 60 for determining the confidence of a perceived target includes at least one processor 610, a memory 620 communicatively connected to the at least one processor 610, and a computer program 630 stored in the memory 620 and executable on the processor 610. The processor 610 executes the computer program 630 to implement a method for determining the confidence of a perceived target according to one aspect of the present application.

[0071] Figure 7 A schematic block diagram of a smart device for responding to a perception target according to one or more embodiments of the present application is shown.

[0072] like Figure 7 As shown in FIG. 7 , the smart device 70 for determining the confidence of a perceived target includes at least one processor 710, a memory 720 communicatively connected to the at least one processor 710, and a computer program 730 stored in the memory 720 and executable on the processor 710. The processor 710 executes the computer program 730 to implement a method for responding to a perceived target according to one aspect of the present application.

[0073] Figures 8A-8B The verification results of the perceptual target confidence model according to one or more embodiments of the present application are shown.

[0074] like Fig. 8AAs shown, in the verification result 800A for the vehicle, the horizontal axis represents the false positive rate (FalsePositive Rate) / recall rate (Recall), the vertical axis represents the true positive rate (TruePositive Rate) / precision rate (Precision), the ROC (Receiver Operating Characteristic) curve (Positive ROCcurve, that is, the orange curve shown in the figure) area of ​​the correct response is 0.97, the ROC curve of the wrong response (Negative ROC curve, that is, the red curve shown in the figure) area is 0.97, the PR (Precision Recall) curve of the correct response (Positive PRcurve, that is, the blue curve shown in the figure) area is 0.98, and the PR curve of the wrong response (Negative PR curve, that is, the green curve shown in the figure) area is 0.95. It should be noted that the curve area here refers to the area of ​​the area formed by the curve and the horizontal and vertical axes. In addition, experiments on vehicles show that the single training time of the perception target confidence model is 2 minutes.

[0075] like Figure 8B As shown, in the verification result 800B for VRU, the horizontal axis represents the false positive rate / recall rate, the vertical axis represents the true positive rate / precision rate, the ROC curve of the correct response (Positive ROC curve, i.e. the orange curve shown in the figure) has an area of ​​0.88, the ROC curve of the wrong response (Negative ROC curve, i.e. the red curve shown in the figure) has an area of ​​0.88, the PR curve of the correct response (Positive PR curve, i.e. the blue curve shown in the figure) has an area of ​​0.94, and the PR curve of the wrong response (Negative PR curve, i.e. the green curve shown in the figure) has an area of ​​0.77. It should be noted that the curve area here refers to the area of ​​the region formed by the curve and the horizontal and vertical axes. In addition, experiments on VRU show that the single training time of the perception target confidence model is 1.5 minutes.

[0076] In addition, the data of real vehicle reflow can be used to verify the generalization of the perception target confidence model according to one or more embodiments of the present application, as shown in the following Table 1:

[0077] Table 1

[0078]

[0079] Table 1 exemplarily shows the generalization of the perception target confidence model verified using 2.3621 million kilometers of data returned from real vehicles, where the second row represents the indicators of the decision planning control verified for VRU and vehicle that incorrectly respond to the perception target without using the perception target confidence model, and the third row represents the indicators of the decision planning control verified for VRU and vehicle that incorrectly respond to the perception target with the perception target confidence model.

[0080] In Table 1, as shown in the second row, without using the perception target confidence model, for VRU, the number of kilometers that the decision-making planning control incorrectly responds to the perception target once is 62,200 kilometers, and for vehicles, the number of kilometers that the decision-making planning control incorrectly responds to the perception target once is 40,000 kilometers. As shown in the third row, when using the perception target confidence model, for VRU, the number of kilometers that the decision-making planning control incorrectly responds to the perception target once is 181,700 kilometers, and for vehicles, the number of kilometers that the decision-making planning control incorrectly responds to the perception target once is 131,200 kilometers.

[0081] By comparing the second and third rows in Table 1, it can be seen that compared with the baseline version that does not use the perception target confidence model, for VRU, the use of the perception target confidence model can reduce the mileage of the decision-making planning control that incorrectly responds to the perception target once from 62,200 kilometers to 181,700 kilometers. For vehicles, the use of the perception target confidence model can reduce the mileage of the decision-making planning control that incorrectly responds to the perception target once from 40,000 kilometers to 131,200 kilometers.

[0082] In addition, the present application may also be implemented as a computer storage medium, in which a program is stored for causing a computer to execute a method for determining the confidence of a perceived target according to one aspect of the present application. In addition, the present application may also be implemented as a computer storage medium, in which a program is stored for causing a computer to execute a method for responding to a perceived target according to one aspect of the present application.

[0083] Computer storage media can be implemented as disks (e.g., magnetic disks, optical disks, etc.), cards (e.g., memory cards, optical cards, etc.), semiconductor memories (e.g., ROM, non-volatile memory, etc.), tapes (e.g., magnetic tapes, cassettes, etc.), etc.

[0084] In the applicable situation, hardware, software or a combination of hardware and software can be used to realize the various embodiments provided by the application. Moreover, in the applicable situation, without departing from the scope of the application, the various hardware components and / or software components set forth herein can be combined into a composite component comprising software, hardware and / or both. In the applicable situation, without departing from the scope of the application, the various hardware components and / or software components set forth herein can be divided into subcomponents comprising software, hardware or both. In addition, in the applicable situation, it is contemplated that the software component can be implemented as a hardware component, and vice versa.

[0085] Software (such as program code and / or data) according to the present application can be stored on one or more computer storage media. It is also contemplated that the software identified herein can be implemented using one or more general or special-purpose computers and / or computer systems, networked and / or otherwise. Where applicable, the order of the various steps described herein can be changed, combined into composite steps, and / or divided into sub-steps to provide the features described herein.

[0086] The embodiments and examples set forth herein are provided to best illustrate embodiments according to the present application and its specific applications, and thereby enable those skilled in the art to implement and use the present application. However, those skilled in the art will appreciate that the above description and examples are provided only for ease of illustration and example. The description set forth is not intended to cover all aspects of the present application or to limit the present application to the precise form disclosed.

Claims

1. A method for determining the confidence of a perceived target, characterized in that The method comprises the following steps: Acquire feature data of a perceived target for one or more frames; Inputting the feature data of the one or more frames of the perceived target into a perceived target confidence model to determine the confidence of the perceived target, The training data of the perception target confidence model includes response data of the autonomous driving to the perception target and label data corresponding to the response data.

2. The method according to claim 1, wherein acquiring feature data of the perceived target of one or more frames comprises: Acquire feature data of the one or more frames of perception targets based on output data of the vehicle perception model, The characteristic data of the perceived target includes one or more of the following: the type of perceived target, the occlusion ratio of the perceived target, the tracking period of the perceived target, the loss period of the perceived target, the attribute label of the perceived target, the position of the perceived target, the speed of the perceived target, the heading of the perceived target, the observation continuity of the perceived target, the observation noise of the perceived target, the motion consistency of the perceived target, the key frame association quality of the perceived target, and the observation association quality of the perceived target.

3. The method according to claim 1, wherein the perceived target confidence model is used to determine the confidence of the perceived target based on at least one of the following: The feature correlation between the feature data of the perceived target in each frame; The temporal correlation between the feature data of multiple frames of perceived targets, The response data of the autonomous driving to the perceived target and the label data corresponding to the response data include: The response data obtained by the autonomous driving decision-making planning module in response to the perceived target; as well as The label data used to mark the response data as a correct response or an error response, The autonomous driving decision-making planning module is applied to one or more of the following scenarios: simulation scenario, road test scenario and shadow mode scenario.

4. The method according to claim 1, wherein the perception target confidence model is a perception target confidence model based on a self-attention mechanism, The perception target confidence model based on the self-attention mechanism includes: an embedding layer for determining a quantile feature vector of the feature data of the perceptual target of the one or more frames relative to a feature data set of the perceptual target; An encoder configured to obtain a query vector, a key vector, and a value vector based on a linear mapping of the quantile feature vector, and determine an attention vector based on the query vector, the key vector, and the value vector; as well as a classification layer for determining the confidence of the perceived target based at least on the quantile feature vector and the attention vector, The classification layer is configured as follows: Performing a residual connection on the quantile feature vector and the attention vector to obtain a residual connection vector; Normalizing the residual connection vector to obtain a normalized vector; and Performing binary classification on the normalized vector to determine the confidence of the perceived target, The encoder includes multiple Transformer Encoders.

5. An intelligent device for determining the confidence of a perceived target, characterized in that: The smart device includes: at least one processor; and a memory communicatively coupled to the at least one processor; Wherein, a computer program is stored in the memory, and when the computer program is executed by the at least one processor, the method for determining the confidence of a perceived target as described in any one of claims 1-4 is implemented.

6. A computer-readable storage medium storing a plurality of program codes, characterized in that: The program code is suitable for being loaded and run by a processor to execute the method for determining the confidence of a perceived target according to any one of claims 1-4.

7. A method for responding to a perceived target, characterized in that The method comprises the following steps: According to the method for determining the confidence of a perceived target according to any one of claims 1 to 4, determining the confidence of the perceived target; The perceived target is selectively responded to according to the confidence level of the perceived target.

8. The method of claim 7, wherein selectively responding to the perceived target according to the confidence level of the perceived target comprises: When the confidence level is greater than or equal to a first confidence level threshold, performing an automatic driving function response to the perceived target; When the confidence level is less than the first confidence level threshold and greater than or equal to the second confidence level threshold, performing an autonomous driving function response to the perceived target after a preset time period; and When the confidence level is less than the second confidence level threshold, no autonomous driving function response is performed on the perceived target.

9. An intelligent device for responding to a sensing target, characterized in that: The smart device includes: at least one processor; and a memory communicatively coupled to the at least one processor; Wherein, a computer program is stored in the memory, and when the computer program is executed by the at least one processor, the method for responding to a perceived target as described in any one of claims 7-8 is implemented.

10. A computer-readable storage medium storing a plurality of program codes, characterized in that: The program code is suitable for being loaded and executed by a processor to perform the method for responding to a perceived target according to any one of claims 7-8.

Images