Method, device, medium and electronic device for determining false perception of obstacle

Abstract

Embodiments of the present application disclose a method, device, medium and electronic equipment for judging obstacle misperception. The method comprises: regarding an obstacle perceived by a perception module in an autonomous vehicle as a target obstacle to be judged, determining feature description data of the target obstacle and vehicle state data associated with the target obstacle; determining a misperception probability of the target obstacle based on the feature description data and the vehicle state data; and determining whether the target obstacle is an obstacle misperceived by the perception module according to the misperception probability, the feature description data and the vehicle state data. The technical solution of the present application is beneficial to improving the safety and reliability of autonomous driving.

Description

Technical Field

[0001] This application relates to the field of computer technology, specifically to the fields of autonomous driving and deep learning, and in particular to a method, apparatus, medium, and electronic device for judging the false perception of obstacles. Background Technology

[0002] In autonomous driving technology, the perception module uses perception hardware such as cameras and various radars to perceive environmental information around the vehicle, providing necessary information for the downstream prediction module to make the next decision.

[0003] Obstacle perception is a crucial task that the perception module must perform. Due to limitations in the perception hardware itself, as well as the influence of external environmental factors such as weather conditions, false obstacle perception may occur. False obstacle perception refers to the perception module detecting obstacles that do not actually exist. Identifying and addressing false obstacle perception helps the prediction module make correct decisions, which is of great significance for improving the safety and reliability of autonomous driving. Summary of the Invention

[0004] This application provides a method, apparatus, medium, and electronic device for judging whether an obstacle is a false obstacle, which is used to determine whether an obstacle perceived by the perception module in an autonomous vehicle is a real obstacle, thereby improving the safety and reliability of autonomous driving.

[0005] According to a first aspect of this application, a method for determining obstacle misperception is provided, the method comprising:

[0006] The obstacles perceived by the perception module in the autonomous vehicle are taken as target obstacles to be judged, and the feature description data of the target obstacles and the vehicle state data associated with the target obstacles are determined.

[0007] Based on the feature description data and the vehicle state data, the probability of misperception of the target obstacle is determined;

[0008] Based on the false perception probability, the feature description data, and the vehicle state data, it is determined whether the target obstacle is an obstacle that the perception module has mistakenly perceived.

[0009] According to a second aspect of this application, an apparatus for determining false obstacle perception is provided, the apparatus comprising:

[0010] The obstacle data acquisition module is used to take the obstacles perceived by the perception module in the autonomous vehicle as target obstacles to be judged, determine the feature description data of the target obstacles, and the vehicle state data associated with the target obstacles.

[0011] The false perception probability determination module is used to determine the false perception probability of the target obstacle based on the feature description data and the vehicle state data.

[0012] The obstacle type determination module is used to determine whether the target obstacle is an obstacle that the perception module has mistakenly perceived, based on the false perception probability, the feature description data, and the vehicle state data.

[0013] According to a third aspect of the present invention, embodiments of this application provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method for determining false obstacle perception as described in embodiments of this application.

[0014] According to a fourth aspect of the present invention, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the method for determining obstacle misperception as described in the embodiments of the present application.

[0015] This application's technical solution uses obstacles perceived by the perception module in an autonomous vehicle as target obstacles to be judged. It determines the feature description data of the target obstacle and the vehicle state data associated with it. Based on the feature description data and the vehicle state data, it determines the probability of misperception of the target obstacle. Based on the misperception probability, the feature description data, and the vehicle state data, it determines whether the target obstacle is a misperception by the perception module. This application provides data support for the prediction module to make correct decisions by determining whether the target obstacle perceived by the perception module is a misperception, which is beneficial to improving the safety and reliability of autonomous driving.

[0016] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this application, nor is it intended to limit the scope of this application. Other features of this application will become readily apparent from the following description. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a flowchart of a method for determining obstacle misperception provided in Embodiment 1;

[0019] Figure 2This is a flowchart of a method for determining false obstacle perception provided in Embodiment 2;

[0020] Figure 3 This is a flowchart of a method for determining false obstacle perception provided in Embodiment 3;

[0021] Figure 4 This is a schematic diagram of the device for judging false perception of obstacles provided in Embodiment 4 of this application;

[0022] Figure 5 This is a schematic diagram of the structure of an electronic device provided in Embodiment 5 of this application. Detailed Implementation

[0023] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0024] It should be noted that the terms "first," "second," "target," and "candidate," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0025] Example 1

[0026] Figure 1 This is a flowchart of a method for judging false obstacle perception provided in Embodiment 1. This embodiment can be applied to the situation of judging whether the obstacle perceived by the perception module in an autonomous vehicle is a real obstacle. The method can be executed by a device for judging false obstacle perception, can be implemented in hardware and / or software, and can be integrated into an electronic device running this system.

[0027] like Figure 1 As shown, the method includes:

[0028] S110. Take the obstacles perceived by the perception module in the autonomous vehicle as target obstacles to be judged, determine the feature description data of the target obstacles, and the vehicle state data associated with the target obstacles.

[0029] The perception model in an autonomous vehicle uses sensing hardware such as cameras and various radars to perceive the environment around the vehicle and detect obstacles. However, due to the influence of external environmental factors such as lighting and weather conditions like rain and snow, coupled with the inherent noise in the sensing hardware itself, the perception module may misperceive obstacles.

[0030] Obstacles detected by the perception module in an autonomous vehicle are designated as target obstacles. Further judgment is needed to determine the obstacle type of each target obstacle and whether it is a real obstacle. Optionally, obstacle types include real obstacles and falsely identified obstacles.

[0031] In this context, a real obstacle refers to an obstacle that actually exists in the environment surrounding the vehicle; conversely, a misjudged obstacle refers to an obstacle that does not exist in the environment surrounding the vehicle.

[0032] The feature description data of the target obstacle is used to describe the characteristics of the target obstacle. Optionally, the feature description data includes at least one of the following: shape description data, motion description data, position description data, environmental description data, and time description data. The time description data is used to describe the time when the perception model perceives the target obstacle. The environmental description data is used to describe the environment in which the perception model perceives the target obstacle. Optionally, the environmental description data includes weather description data and road description data.

[0033] The target obstacle is also associated with vehicle state data, which describes the state of the autonomous vehicle when the perception module detects the target obstacle. Optionally, the vehicle state data includes vehicle position data and / or vehicle motion data.

[0034] S120. Based on the feature description data and the vehicle state data, determine the probability of misperception of the target obstacle;

[0035] The false perception probability is used to quantify the probability that a target obstacle is a falsely identified obstacle. It is known that real obstacles are generally pedestrians or vehicles. Falsely identified obstacles do not possess the shape characteristics, behavior patterns, location patterns, and spatial patterns of real obstacles such as pedestrians or vehicles.

[0036] By using the vehicle status data associated with the target obstacle as a reference, and combining it with the characteristic description data of the target obstacle, the shape characteristics, behavior patterns, position patterns, and spatial patterns of the target obstacle can be determined.

[0037] The probability of misperception of the target obstacle is determined by comparing its shape features, behavior patterns, location patterns, and spatial patterns with those of the real obstacle.

[0038] S130. Based on the false perception probability, the feature description data, and the vehicle state data, determine whether the target obstacle is an obstacle that the perception module has mistakenly perceived.

[0039] It is known that the perception module in an autonomous vehicle has a certain perception range. Obstacles within this range will affect the subsequent planning decisions of the prediction module. The perception range is determined based on specific business requirements and is not limited here.

[0040] Based on feature description data and vehicle state data, the relative positional relationship between the target obstacle and the autonomous vehicle can be determined. Based on this relative positional relationship, it can be determined whether the target obstacle is within the perception range.

[0041] Based on the obtained relative positional relationship, and combined with the false perception probability of the target obstacle, it is determined whether the target obstacle is an obstacle that the perception module has mistakenly perceived.

[0042] This application's technical solution uses obstacles perceived by the perception module in an autonomous vehicle as target obstacles to be judged. It determines the feature description data of the target obstacle and the vehicle state data associated with it. Based on the feature description data and the vehicle state data, it determines the probability of misperception of the target obstacle. Based on the misperception probability, the feature description data, and the vehicle state data, it determines whether the target obstacle is a misperception by the perception module. This application provides data support for the prediction module to make correct decisions by determining whether the target obstacle perceived by the perception module is a misperception, which is beneficial to improving the safety and reliability of autonomous driving.

[0043] In an optional embodiment, determining the false perception probability of a target obstacle based on the feature description data and the vehicle state data includes: performing feature processing on the feature description data and the vehicle state data to obtain a feature vector of the target obstacle; and using a pre-trained false perception discrimination model to determine the false perception probability of the target obstacle based on the feature vector of the target obstacle; wherein the false perception discrimination model is a discrimination model in a generative adversarial model.

[0044] The feature description data of the target obstacle and the vehicle state data associated with the target obstacle are processed to convert the feature description data and vehicle state data into a vector form that can be processed by the misperception discrimination model, thus obtaining the feature vector of the target obstacle.

[0045] The feature vectors of the target obstacle are input into a pre-trained misperception discrimination model, which determines the misperception probability of the target obstacle. This misperception discrimination model is the discriminant model within the generative adversarial model (GAP). It is known that a GAP consists of a generative model and a discriminant model. The main task of the generative model is to generate simulated samples that approximate negative examples as closely as possible; the main task of the discriminant model is to discriminate the simulated samples with the highest possible accuracy. The simulated samples generated by the generative module are used to train the discriminant model within the GAP.

[0046] The above technical solution uses generative adversarial model to judge obstacle misperception, giving full play to the sample generation capability of the generative model in generative adversarial model. It is not limited by the number of samples and can still judge obstacle misperception even when the number of samples is small. It also makes full use of the scene generalization capability of the discriminative model in generative adversarial model, and can accurately judge obstacle misperception cases not included in the samples, thus improving the accuracy and reliability of obstacle misperception judgment.

[0047] Example 2

[0048] Figure 2 This is a flowchart of a method for judging obstacle misperception provided in Embodiment 2. This embodiment further optimizes the above embodiment, specifically providing a training method for a generative adversarial model.

[0049] like Figure 2 As shown, the method includes:

[0050] S210. The feature description data of the historical obstacles and the vehicle state data associated with the historical obstacles are used as training samples for the generative adversarial model.

[0051] Historical obstacles refer to obstacles perceived by the autonomous vehicle within a historical time period. Vehicle state data associated with historical obstacles describes the state of the autonomous vehicle when the perception module perceived the historical obstacle. Optionally, to improve the generalization ability of the generative adversarial model, historical obstacles under different weather conditions can be acquired. For example, historical obstacles under cloudy, rainy, snowy, or sunny weather conditions can be acquired.

[0052] Historical obstacle feature description data is used to describe the characteristics of historical obstacles. Optionally, the feature description data includes at least one of the following: shape description data, motion description data, location description data, environmental description data, and time description data. The time description data describes the time when the perception model senses the target obstacle. The environmental description data describes the environment in which the perception model senses the target obstacle. Optionally, the environmental description data includes weather description data and road description data.

[0053] The feature descriptions of historical obstacles, along with the vehicle state data associated with them, are used as training samples for the generative adversarial model. Historical obstacles may be real obstacles or misclassified obstacles, depending on the specific circumstances.

[0054] S220. Obtain the sample labels of the training samples; wherein, the sample labels are used to determine whether the historical obstacles corresponding to the training samples are real obstacles.

[0055] The training samples are labeled with both real obstacles and falsely identified obstacles. Optionally, the labels are obtained manually.

[0056] S230. Based on the training samples and the sample labels of the training samples, train the generative model and the discriminative model in the generative adversarial model.

[0057] Using training samples and their labels, the generative and discriminative models in the generative adversarial model are trained. The training processes of the generative and discriminative models are mutually reinforcing; during training, they compete against each other. The result is that both models improve their capabilities. The generative model enhances its ability to simulate the original data distribution, while the discriminative model improves its ability to distinguish between different data types.

[0058] This application provides a practical method for training a generative adversarial model (GAP). Historical obstacle feature description data and associated vehicle state data are used as training samples for the GAP. Based on these training samples and their labels, the generative and discriminative models within the GAP are trained. After the GAP is trained, the discriminative model is used to determine if obstacles are misperceived. This application provides technical support for using GAP to determine if obstacles are misperceived.

[0059] In an optional embodiment, training the generative model and the discriminative model in the generative adversarial model based on the training samples and the sample labels of the training samples includes: generating simulated samples through the generative model in the generative adversarial model, and using the simulated samples to expand the negative samples in the training samples; using the positive samples in the training samples and the expanded negative samples as input samples to train the discriminative model in the generative adversarial model; wherein the discriminative model and the generative model are initially trained using the training samples; the generative model is trained again according to the discrimination result of the discriminative model with respect to the input samples; and the discriminative model and the generative model are cyclically trained adversarially until a preset training termination condition is met.

[0060] The training samples include positive and negative samples. Positive samples are those labeled as real obstacles, while negative samples are those labeled as misidentified obstacles. Understandably, positive samples far outnumber negative samples in the training samples. This imbalance in the ratio of positive to negative samples in the training samples will lead to poor performance of the misperception discrimination model.

[0061] This study employs a generative model within generative adversarial modeling (GAP) to augment negative examples in the training samples. Specifically, the generative model is first trained using the training samples, enabling it to learn the data features of the training samples. Then, the generative model generates simulated samples based on these learned features. These simulated samples are then used to augment the negative examples in the training samples. Optionally, labels for misclassified obstacles are added to the simulated samples, which are then incorporated into the training samples as negative examples, and the ratio of positive to negative examples in the training samples is balanced. The training samples include not only positive and negative examples but also simulated examples.

[0062] The discriminative model in the generative adversarial model is trained using positive samples and expanded negative samples from the training samples as input samples. Optionally, the number of positive and negative samples in the input samples is roughly equal, and the negative samples in the input samples include simulated samples generated by the generative model.

[0063] Before training the discrimination model using input samples, the discrimination model is initially trained using training samples, so that the discrimination model has the ability to initially judge the false perception of obstacles.

[0064] The discriminative model, initially trained using input samples, is then trained to output discrimination results for the input samples. Based on these results, the generative model is trained again to update it. Simulated samples generated by the updated generative model, combined with positive samples, are then used as input samples to train the discriminative model again. This adversarial training of the discriminative and generative models is repeated until a preset training termination condition is met. This preset termination condition is determined based on actual business needs and is not limited here. For example, the preset termination condition can be the number of training rounds or a performance indicator. For instance, if the preset number of training rounds for the generative adversarial model is reached, the adversarial training between the generative and discriminative models in the generative adversarial model ends. Alternatively, if the performance indicators of the generative and discriminative models in the generative adversarial model meet the target, the adversarial training between the generative and discriminative models in the generative adversarial model ends.

[0065] Optionally, after the generative adversarial model (GAP) is trained, the discriminative model within the GAP is used as a misperception discrimination model and added between the perception and prediction modules of the autonomous vehicle to perform misperception judgments on target obstacles perceived by the perception module. Optionally, the trained GAP is deployed to the autonomous vehicle. For example, the GAP is converted from a PyTorch model to a libtorch model, and the converted libtorch model is deployed to the autonomous vehicle.

[0066] In one specific embodiment, the feature description data of real obstacles and the vehicle state data associated with the real obstacles are used as input samples for the generative model. Specifically, the road description data in the feature description data is combined with other descriptive information of real obstacles in the feature description data to form a bird's-eye view input generative model. The generative model generates simulated samples based on the input bird's-eye view. The shape description data, motion description data, and position description data of the misjudged obstacles corresponding to the simulated samples are random.

[0067] The above technical solution provides a method for training generative adversarial models. It trains the sample generation capability of the generative model in the generative adversarial model, so that the misperception discrimination model is not limited by the number of samples. Even when the number of samples is small, it can still judge the misperception of obstacles. It also trains the scene generalization capability of the discrimination model in the generative adversarial model, so that the misperception discrimination model can also accurately judge the misperception of obstacles not included in the samples, which is beneficial to the accuracy and reliability of obstacle misperception judgment.

[0068] In an optional embodiment, the discrimination model is initially trained using negative and positive samples from the training samples, including: determining negative samples from the training samples based on the sample labels of the training samples, determining the baseline number of training samples based on the number of negative samples; selecting the baseline number of positive samples from the training samples, and using the negative samples and the selected positive samples to initially train the discrimination model.

[0069] The sample labels are used to distinguish between positive and negative samples. Positive samples represent real obstacles, while negative samples represent misidentified obstacles. To ensure the discrimination model's accuracy in identifying misperceived obstacles, it needs to be initially trained using balanced training samples. In other words, the number of positive samples should be roughly equal to the number of negative samples in the training samples used for initial training.

[0070] It is known that before expanding the negative examples in the training samples, the number of negative examples in the training samples is far less than the number of positive examples. The number of negative examples in the training samples is used as the baseline. This baseline number is used to determine the number of positive examples required for the initial training of the discriminant model. The discriminant model is then initially trained using a similar number of positive and negative examples.

[0071] The above technical solution ensures the discrimination model's ability to judge false perceptions of obstacles by using a balanced training sample for initial training.

[0072] Example 3

[0073] Figure 3 This is a flowchart of a method for determining obstacle misperception provided in Embodiment 3. This embodiment further optimizes the above embodiment, specifically refining the operation of "determining whether the target obstacle is a misperception by the perception module based on the misperception probability, the feature description data, and the vehicle state data".

[0074] like Figure 3 As shown, the method includes:

[0075] S310. Take the obstacles perceived by the perception module in the autonomous vehicle as target obstacles to be judged, determine the feature description data of the target obstacles, and the vehicle state data associated with the target obstacles.

[0076] S320. Based on the feature description data and the vehicle state data, determine the probability of misperception of the target obstacle.

[0077] S330. Determine the relative distance between the target obstacle and the autonomous vehicle based on the feature description data and the vehicle status data.

[0078] Optionally, the feature description data includes location description data, used to describe the location of the target obstacle when the perception model perceives it; the vehicle state data includes vehicle position data, used to describe the location of the autonomous vehicle when the perception model perceives the target obstacle. Specifically, the relative distance between the target obstacle and the autonomous vehicle is determined based on the location description data in the feature description data and the vehicle position data in the vehicle position data.

[0079] S340. Based on the false perception probability and the relative distance between the target obstacle and the autonomous vehicle, determine the prediction confidence that the target obstacle belongs to a real obstacle.

[0080] Optionally, the prediction confidence that the target obstacle belongs to a real obstacle can be determined using the following formula.

[0081] prob=log[1+dis(obs,adc)]*model(obs)

[0082] Where prob represents the prediction confidence, dis(obs,adc) represents the relative distance between the target obstacle and the autonomous vehicle, obs represents the target obstacle, abc represents the autonomous vehicle, and model(obs) represents the probability of false perception.

[0083] The false perception probability is given by the false perception discrimination model, and its accuracy is related to the model performance of the model. Based on the relative distance between the target obstacle and the autonomous vehicle, it can be determined whether the target obstacle is within the perception range of the perception module in the autonomous vehicle.

[0084] By combining the probability of false perception and the relative distance between the target obstacle and the autonomous vehicle, the prediction confidence level can be determined to identify the target obstacle as a real obstacle. The prediction confidence level quantifies the degree to which the target obstacle is believed to be a real obstacle.

[0085] S350. Based on the predicted confidence level that the target obstacle is a real obstacle, determine whether the target obstacle is an obstacle mistakenly perceived by the perception module.

[0086] Optionally, the predicted confidence level of the target obstacle being a real obstacle is compared with a confidence threshold. Based on the comparison result, it is determined whether the target obstacle is a misperceived obstacle by the perception module. The confidence threshold is predetermined based on actual business needs and is not limited here. Specifically, if the predicted confidence level is greater than or equal to the confidence threshold, the target obstacle is determined to be a real obstacle and not a misperceived obstacle by the perception module; if the predicted confidence level is less than the confidence threshold, the target obstacle is determined to be a misperceived obstacle by the perception module.

[0087] Optionally, based on the prediction confidence that a target obstacle belongs to a real obstacle, the target obstacle is classified into real obstacles and misjudged obstacles. The feature description data of real obstacles is input into the prediction module in the autonomous vehicle so that the prediction model can make planning decisions based on the feature description data of real obstacles.

[0088] The technical solution of this application determines the relative distance between the target obstacle and the autonomous vehicle based on feature description data and vehicle state data; based on the false perception probability and the relative distance between the target obstacle and the autonomous vehicle, it determines the prediction confidence that the target obstacle is a real obstacle; based on the prediction confidence that the target obstacle is a real obstacle, it determines whether the target obstacle is a false obstacle perceived by the perception module, thus ensuring the accuracy of the judgment of false obstacle perception and providing a data basis for the subsequent prediction module to make correct decisions.

[0089] In an optional embodiment, determining whether a target obstacle is a mistakenly perceived obstacle by the perception module based on the predicted confidence level that the target obstacle belongs to a real obstacle includes: determining the perception timestamp associated with the target obstacle based on the feature description data; wherein the perception timestamp is determined based on the time when the perception model in the autonomous vehicle perceives the target obstacle; if the perception timestamps associated with the target obstacle are continuous, then determining to perform weighted processing on the predicted confidence level corresponding to the target obstacle; and determining whether the target obstacle is a mistakenly perceived obstacle by the perception module based on the weighted processing result of the predicted confidence level.

[0090] Optionally, the feature description data of the target obstacle includes temporal description data and obstacle identification. The temporal description data describes the time it takes for the perception model to detect the target obstacle; the obstacle identification is used to distinguish different obstacles.

[0091] Specifically, the perception timestamp of the target obstacle is determined based on its temporal description data and obstacle identifier. The perception timestamp is determined based on the time when the perception model in the autonomous vehicle perceives the target obstacle.

[0092] Based on the perception timestamps of the target obstacles, it can be determined whether the target obstacle was perceived by the perception model within a continuous time period. In other words, it can be determined whether the obstacles perceived by the perception module within a continuous time period are the same obstacle. Each obstacle perceived by the perception module has a corresponding prediction confidence level.

[0093] If the perception timestamps associated with the target obstacle are consecutive, it indicates that the perception module perceived obstacles as target obstacles within a continuous time period. Therefore, a weighted calculation is performed on the predicted confidence scores corresponding to the target obstacles. Specifically, a weight parameter is determined for each perception timestamp, and the predicted confidence scores for each perception timestamp are weighted using this parameter. The weighted result of the predicted confidence scores is used as the final confidence score for the target obstacle. Based on the final confidence score, it is determined whether the target obstacle was mistakenly perceived by the perception module.

[0094] Optionally, the prediction confidence scores corresponding to the target obstacle can be weighted based on the following formula.

[0095] prob t =(W t-1 *prob t-1 +W t-0.8 *prob t-0.8 +…+W t *prob t )

[0096] Among them, prob t This represents the weighted result obtained by weighting the prediction confidence scores corresponding to the target obstacle based on the timestamp dimension. t W t-0.8 and W t-1 These represent the weight parameters corresponding to different perceived timestamps. t prob t-0.8 and prob t-1 These represent the prediction confidence levels corresponding to different perception timestamps. The weighting parameters are determined based on actual business needs and are not specified here.

[0097] The above technical solution determines the perception timestamp associated with the target obstacle. If the perception timestamps associated with the target obstacle are continuous, the prediction confidence corresponding to the target obstacle is weighted. Based on the weighted result of the prediction confidence, it is determined whether the target obstacle is an obstacle misperceived by the perception module. This reduces random errors and helps to improve the accuracy and reliability of obstacle misperception judgment.

[0098] Example 4

[0099] Figure 4This is a schematic diagram of the device for determining false obstacle perception provided in Embodiment 4 of this application. This embodiment can be applied to determining whether an obstacle perceived by the perception module in an autonomous vehicle is a real obstacle. The device can be implemented by software and / or hardware and can be integrated into electronic devices such as smart terminals.

[0100] like Figure 4 As shown, the device may include: an obstacle data acquisition module 410, a false perception probability determination module 420, and an obstacle type determination module 430.

[0101] The obstacle data acquisition module 410 is used to take the obstacles perceived by the perception module in the autonomous vehicle as target obstacles to be judged, determine the feature description data of the target obstacles, and the vehicle state data associated with the target obstacles.

[0102] The false perception probability determination module 420 is used to determine the false perception probability of the target obstacle based on the feature description data and the vehicle state data.

[0103] The obstacle type determination module 430 is used to determine whether the target obstacle is an obstacle that the perception module has mistakenly perceived, based on the false perception probability, the feature description data, and the vehicle state data.

[0104] This application's technical solution uses obstacles perceived by the perception module in an autonomous vehicle as target obstacles to be judged. It determines the feature description data of the target obstacle and the vehicle state data associated with it. Based on the feature description data and the vehicle state data, it determines the probability of misperception of the target obstacle. Based on the misperception probability, the feature description data, and the vehicle state data, it determines whether the target obstacle is a misperception by the perception module. This application provides data support for the prediction module to make correct decisions by determining whether the target obstacle perceived by the perception module is a misperception, which is beneficial to improving the safety and reliability of autonomous driving.

[0105] Optionally, the misperception probability determination module 420 includes: a data featureization submodule, used to perform featureization processing on the feature description data and the vehicle state data to obtain the feature vector of the target obstacle; and a misperception probability determination submodule, used to determine the misperception probability of the target obstacle based on the feature vector of the target obstacle using a pre-trained misperception discrimination model; wherein the misperception discrimination model is a discrimination model in a generative adversarial model.

[0106] Optionally, the obstacle type determination module 430 includes: a distance determination submodule, used to determine the relative distance between the target obstacle and the autonomous vehicle based on the feature description data and the vehicle state data; a prediction confidence determination submodule, used to determine the prediction confidence that the target obstacle belongs to a real obstacle based on the false perception probability and the relative distance between the target obstacle and the autonomous vehicle; and an obstacle type determination submodule, used to determine whether the target obstacle is an obstacle falsely perceived by the perception module based on the prediction confidence that the target obstacle belongs to a real obstacle.

[0107] Optionally, the obstacle type determination submodule includes: a timestamp determination unit, used to determine the perception timestamp associated with the target obstacle based on the feature description data; wherein the perception timestamp is determined based on the time when the perception model in the autonomous vehicle perceives the target obstacle; a weighted processing unit, used to determine to perform weighted processing on the prediction confidence corresponding to the target obstacle if the perception timestamps associated with the target obstacle are continuous; and an obstacle type determination unit, used to determine whether the target obstacle is an obstacle mistakenly perceived by the perception module based on the weighted processing result of the prediction confidence.

[0108] Optionally, the device further includes a model training module for training the generative adversarial model; the model training module includes: a training sample determination submodule, used to use feature description data of historical obstacles and vehicle state data associated with the historical obstacles as training samples for the generative adversarial model; a sample label acquisition submodule, used to acquire sample labels for the training samples; wherein, the sample labels are used to determine whether the historical obstacles corresponding to the training samples are real obstacles; and a model training submodule, used to train the generative model and the discriminative model in the generative adversarial model based on the training samples and the sample labels of the training samples.

[0109] Optionally, the model training submodule includes: a sample expansion unit, used to generate simulated samples through the generative model in the generative adversarial model, and to expand the negative samples in the training samples using the simulated samples; a discriminative model training unit, used to train the discriminative model in the generative adversarial model using the positive samples in the training samples and the expanded negative samples as input samples; wherein the discriminative model and the generative model are initially trained using the training samples; a generative model training unit, used to train the generative model again based on the discrimination result of the discriminative model with respect to the input samples; and a loop training unit, used to cyclically adversarially train the discriminative model and the generative model until a preset training termination condition is met.

[0110] Optionally, the discriminant model training unit includes: a baseline quantity determination subunit, used to determine negative samples in the training samples based on the sample labels of the training samples, and to determine the baseline quantity of the training samples based on the number of negative samples; and a preliminary training unit, used to select the baseline quantity of positive samples from the training samples, and to perform preliminary training on the discriminant model using the negative samples and the selected positive samples.

[0111] The apparatus for determining obstacle misperception provided in the embodiments of the invention can execute the method for determining obstacle misperception provided in any embodiment of this application, and has the corresponding performance modules and beneficial effects for executing the method for determining obstacle misperception.

[0112] The collection, storage, use, processing, transmission, provision, and disclosure of user data involved in the technical solution disclosed herein all comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0113] Example 5

[0114] Figure 5 A schematic diagram of an electronic device 510, which can be implemented using an embodiment, is shown. The electronic device 510 includes at least one processor 511 and a memory, such as a read-only memory (ROM) 512, a random access memory (RAM) 513, etc., communicatively connected to the at least one processor 511. The memory stores computer programs executable by the at least one processor. The processor 511 can perform various appropriate actions and processes based on the computer program stored in the ROM 512 or loaded from storage unit 418 into the RAM 513. The RAM 513 may also store various programs and data required for the operation of the electronic device 510. The processor 511, ROM 512, and RAM 513 are interconnected via a bus 514. An input / output (I / O) interface 515 is also connected to the bus 514.

[0115] Multiple components in electronic device 510 are connected to I / O interface 515, including: input unit 516, such as keyboard, mouse, etc.; output unit 517, such as various types of displays, speakers, etc.; storage unit 518, such as disk, optical disk, etc.; and communication unit 519, such as network card, modem, wireless transceiver, etc. Communication unit 519 allows electronic device 510 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0116] Processor 511 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 511 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, digital signal processors (DSPs), and any suitable processor, controller, microcontroller, etc. Processor 511 performs the various methods and processes described above, such as methods for determining obstacle misperception.

[0117] In some embodiments, the method for determining obstacle misperception may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 518. In some embodiments, part or all of the computer program may be loaded and / or installed on electronic device 510 via ROM 512 and / or communication unit 519. When the computer program is loaded into RAM 513 and executed by processor 511, one or more steps of the method for determining obstacle misperception described above may be performed. Alternatively, in other embodiments, processor 511 may be configured to perform the method for determining obstacle misperception by any other suitable means (e.g., by means of firmware).

[0118] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0119] Computer programs used to implement the methods of this application may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0120] In the context of this application, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium can be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0121] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0122] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data processing servers), or computing systems that include middleware components (e.g., application servers), or computing systems that include frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0123] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0124] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this application can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this application can be achieved, and this is not limited herein.

[0125] The specific embodiments described above do not constitute a limitation on the scope of protection of this application. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A method for determining false perception of an obstacle, characterized in that, The method comprises: obtaining an obstacle perceived by a perception module in an autonomous vehicle as a target obstacle to be determined, determining feature description data of the target obstacle, and vehicle state data associated with the target obstacle; determining a false perception probability of the target obstacle based on the feature description data and the vehicle state data; determining whether the target obstacle is an obstacle falsely perceived by the perception module according to the false perception probability, the feature description data and the vehicle state data; wherein the false perception probability of the target obstacle is determined based on the feature description data and the vehicle state data, comprising: performing feature processing on the feature description data and the vehicle state data to obtain a feature vector of the target obstacle; and determining the false perception probability of the target obstacle based on the feature vector of the target obstacle by using a pre-trained false perception discrimination model, wherein the false perception discrimination model is a discrimination model in a generative adversarial model; wherein the determination of whether the target obstacle is an obstacle falsely perceived by the perception module according to the false perception probability, the feature description data and the vehicle state data comprises: determining a relative distance between the target obstacle and the autonomous vehicle according to the feature description data and the vehicle state data; determining a prediction confidence that the target obstacle belongs to a real obstacle based on the false perception probability and the relative distance between the target obstacle and the autonomous vehicle; and determining whether the target obstacle is an obstacle falsely perceived by the perception module according to the prediction confidence that the target obstacle belongs to a real obstacle; wherein the determination of whether the target obstacle is an obstacle falsely perceived by the perception module according to the prediction confidence that the target obstacle belongs to a real obstacle comprises: determining a perception timestamp associated with the target obstacle according to the feature description data, wherein the perception timestamp is determined based on a time at which the target obstacle is perceived by a perception model in the autonomous vehicle; if the perception timestamp associated with the target obstacle is continuous, performing weighted processing on the prediction confidence corresponding to the target obstacle; and determining whether the target obstacle is an obstacle falsely perceived by the perception module according to the weighted processing result of the prediction confidence; wherein the generative adversarial model is trained based on the following manner: taking feature description data of historical obstacles and vehicle state data associated with the historical obstacles as training samples of the generative adversarial model; obtaining sample labels of the training samples; wherein the sample labels are used to determine whether the historical obstacles corresponding to the training samples are real obstacles; and training a generative model and a discrimination model in the generative adversarial model based on the training samples and the sample labels of the training samples.

2. The method of claim 1, wherein, The training of the generative model and the discrimination model in the generative adversarial model based on the training samples and the sample labels of the training samples comprises: generating simulation samples by the generative model in the generative adversarial model, and expanding negative example samples in the training samples by using the simulation samples; The positive example samples and the expanded negative example samples in the training samples are taken as input samples to train a discriminant model in a generative adversarial model; wherein the discriminant model and the generative model are preliminarily trained by using the training samples; The generative model is trained again according to a discriminant result of the discriminant model on the input samples; The discriminant model and the generative model are trained in a loop until a preset training end condition is met.

3. The method of claim 2, wherein, The discriminant model is preliminarily trained by using the training samples, including: The negative example samples in the training samples are determined according to sample labels of the training samples, and a reference quantity of the training samples is determined based on a sample quantity of the negative example samples; The reference quantity of the positive example samples is selected from the training samples, and the discriminant model is preliminarily trained by using the negative example samples and the selected positive example samples.

4. A device for detecting false obstacle perception, characterized in that, The device includes: The obstacle data acquisition module is configured to determine, as a target obstacle to be judged, an obstacle perceived by a perception module in an autonomous vehicle, determine feature description data of the target obstacle, and determine vehicle state data associated with the target obstacle; The false perception probability determination module is configured to determine, based on the feature description data and the vehicle state data, a false perception probability of the target obstacle; The obstacle type determination module is configured to determine, based on the false perception probability, the feature description data, and the vehicle state data, whether the target obstacle is an obstacle falsely perceived by the perception module; The false perception probability determination module includes a data feature extraction submodule configured to perform feature extraction processing on the feature description data and the vehicle state data to obtain a feature vector of the target obstacle, and a false perception probability determination submodule configured to determine, based on the feature vector of the target obstacle, the false perception probability of the target obstacle by using a pre-trained false perception discriminant model; wherein the false perception discriminant model is a discriminant model in a generative adversarial model; The obstacle type determination module includes a distance determination submodule configured to determine, based on the feature description data and the vehicle state data, a relative distance between the target obstacle and the autonomous vehicle, a prediction confidence determination submodule configured to determine, based on the false perception probability and the relative distance between the target obstacle and the autonomous vehicle, a prediction confidence that the target obstacle belongs to a real obstacle, and an obstacle type determination submodule configured to determine, based on the prediction confidence that the target obstacle belongs to a real obstacle, whether the target obstacle is an obstacle falsely perceived by the perception module. The obstacle type determination sub-module comprises: a timestamp determination unit configured to determine a perception timestamp associated with the target obstacle according to the feature description data; the perception timestamp is determined based on a time at which the target obstacle is perceived by a perception model in the autonomous vehicle; a weighting processing unit configured to determine to perform weighting processing on a prediction confidence corresponding to the target obstacle if the perception timestamp associated with the target obstacle is continuous; and an obstacle type determination unit configured to determine whether the target obstacle is an obstacle mis-perceived by the perception module according to a result of the weighting processing on the prediction confidence. The device further comprises a model training module configured to train the generative adversarial model; the model training module comprises: a training sample determination sub-module configured to use feature description data of a historical obstacle and vehicle state data associated with the historical obstacle as training samples of the generative adversarial model; a sample label acquisition sub-module configured to acquire sample labels of the training samples; the sample labels are used to determine whether the historical obstacle corresponding to the training sample is a real obstacle; and a model training sub-module configured to train a generative model and a discriminative model in the generative adversarial model based on the training samples and the sample labels of the training samples.

5. A computer-readable storage medium having stored thereon a computer program, characterized in that, The program, when executed by a processor, implements the method for determining obstacle mis-perception according to any one of claims 1-3.

6. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor, when executing the computer program, implements the method for determining obstacle mis-perception according to any one of claims 1-3.

Images