Obstacle recognition model training method and device, storage medium and electronic equipment

Abstract

The invention provides an obstacle recognition model training method and device, a storage medium and electronic equipment, and belongs to the technical field of automatic driving. According to the embodiment of the invention, the method comprises the steps: obtaining original data collected by a millimeter-wave radar and truth value data synchronously collected by a truth value system; based on the truth value data, determining a valid target and an invalid target in the original data, and adding corresponding truth value labels to the valid target and the invalid target to obtain labeled data; and training the initial obstacle recognition model based on the labeled data to obtain a target obstacle recognition model. The target obstacle recognition model obtained through training in the embodiment of the invention can accurately output a derivative variable with high confidence, namely the probability that an object is a movable obstacle, so as to replace the most original recognition result of the millimeter-wave radar, and then a subsequent algorithm can be informed of the probability that the object recognized by the millimeter-wave radar is the movable obstacle. And a subsequent algorithm is assisted to make a more efficient and accurate decision.

Description

Technical Field

[0001] The present application relates to the field of autonomous driving technology, and in particular to an obstacle recognition model training method, device, storage medium and electronic device. Background Art

[0002] Autonomous Driver Assistance Systems (ADAS) use a variety of sensors installed on the vehicle (millimeter wave radar, lidar, monocular / binocular cameras, and satellite navigation) to sense the surrounding environment at any time while the car is driving, collect data, identify, detect, and track static and dynamic objects, and combine navigation map data to perform systematic calculations and analysis, so that the driver can be aware of possible dangers in advance, effectively increasing the comfort and safety of car driving.

[0003] The automated driving assistance system can be roughly divided into four modules: perception, fusion, decision-making, and execution. The perception module mainly uses cameras and millimeter-wave radars to identify obstacles around the vehicle, and then uses the fusion module to fuse the data collected by different on-board sensors. The subsequent decision-making module and execution module will fully rely on the fusion results to achieve control of the vehicle.

[0004] At present, whether an object is an obstacle depends mainly on the recognition result of the camera. In essence, this is because the millimeter-wave radar point cloud is too sparse, resulting in low accuracy of clustering results. The confidence level of the "probability of an object becoming a movable obstacle" output by the millimeter-wave radar is not high, resulting in the need to use more complex algorithms to realize the recognition of movable obstacles, which affects the efficiency and accuracy of subsequent fusion algorithms and decision algorithms. Summary of the invention

[0005] The present application provides an obstacle recognition model training method, device, storage medium and electronic device. Based on the original data collected by the millimeter wave radar, a target obstacle recognition model is trained to output a derived variable of "the probability that an object is a movable obstacle" with high confidence, so as to inform the subsequent algorithm of the possibility that the object recognized by the millimeter wave radar is a movable obstacle, thereby assisting the subsequent algorithm to make more efficient and accurate decisions.

[0006] In order to solve the above problems, this application adopts the following technical solutions:

[0007] In a first aspect, an embodiment of the present application provides an obstacle recognition model training method, the method comprising:

[0008] Obtain the original data collected by the millimeter-wave radar and the true value data collected synchronously by the true value system;

[0009] Based on the true value data, determining valid targets and invalid targets in the original data, and adding corresponding true value labels to the valid targets and the invalid targets to obtain labeled data;

[0010] Based on the labeled data, an initial obstacle recognition model is trained to obtain a target obstacle recognition model, where the target obstacle recognition model is used to determine the probability that an object around the vehicle is a movable obstacle.

[0011] In an embodiment of the present application, the step of determining valid targets and invalid targets in the original data based on the true value data includes:

[0012] Time-aligning the true value data and the original data to obtain a plurality of data groups; each data group has a one-to-one corresponding true value data frame and an original data frame;

[0013] For any data group, determine first position information of the target object in the true value data frame in the data group from the vehicle and second position information of the target object in the original data frame in the data group from the vehicle;

[0014] When a position deviation between the first position information and the second position information is within a threshold range, determining that the target object in the original data frame is a valid target;

[0015] When the position deviation between the first position information and the second position information exceeds a threshold range, it is determined that the target object in the original data frame is an invalid target.

[0016] In one embodiment of the present application, the step of time-aligning the true value data and the original data to obtain multiple data groups includes:

[0017] Obtaining a first timestamp corresponding to each true value data frame in the true value data and a second timestamp corresponding to an original data frame in the original data;

[0018] Based on the first timestamp and the second timestamp, time-adjacent true value data frames and original data frames are matched to obtain multiple data groups.

[0019] In one embodiment of the present application, the step of training the initial obstacle recognition model based on the labeled data to obtain the target obstacle recognition model includes:

[0020] Preprocessing the labeled data to obtain target data; the target data includes model training samples and model testing samples;

[0021] Using the model training samples as a training set, training the initial obstacle recognition model to obtain a first obstacle recognition model;

[0022] Identify the model test sample using the first obstacle recognition model to obtain a first recognition result;

[0023] When the first recognition result meets the preset recognition index, the target obstacle recognition model is obtained.

[0024] In one embodiment of the present application, the step of preprocessing the labeled data to obtain target data includes:

[0025] Performing balanced sampling on different types of objects in the labeled data to obtain initial data;

[0026] Feature processing is performed on the initial data to obtain target data.

[0027] In one embodiment of the present application, the step of performing balanced sampling on different types of objects in the labeled data to obtain initial data includes:

[0028] Acquiring type attribute information of objects in the labeled data; the type attribute information includes vulnerable road users, vehicles, and invalid targets;

[0029] The labeled data is evenly sampled according to the sampling weights corresponding to the vulnerable road users, the vehicles and the invalid targets to obtain the initial data.

[0030] In one embodiment of the present application, the step of performing feature processing on the initial data to obtain target data includes:

[0031] Performing different feature processing on different signals in the initial data to obtain feature data;

[0032] Invalid features in the feature data are cleared to obtain the target data.

[0033] In an embodiment of the present application, after the step of training the initial obstacle recognition model based on the labeled data to obtain the target obstacle recognition model, the method further includes:

[0034] Deploying the target obstacle recognition model into a target vehicle;

[0035] Acquire second raw data collected by a millimeter-wave radar provided on the target vehicle;

[0036] The second original data is input into the target obstacle recognition model, and the probability that the objects around the target vehicle are movable obstacles is obtained as an output.

[0037] In a second aspect, based on the same inventive concept, an embodiment of the present application provides an obstacle recognition model training device, the device comprising:

[0038] A first acquisition module is used to acquire the original data collected by the millimeter wave radar and the true value data synchronously collected by the true value system;

[0039] A labeling module, used to determine valid targets and invalid targets in the original data based on the true value data, and add corresponding true value labels to the valid targets and the invalid targets to obtain labeled data;

[0040] A training module is used to train an initial obstacle recognition model based on the labeled data to obtain a target obstacle recognition model, wherein the target obstacle recognition model is used to determine the probability that an object around the vehicle is a movable obstacle.

[0041] In one embodiment of the present application, the marking module includes:

[0042] The time alignment submodule is used to time-align the true value data and the original data to obtain multiple data groups; each data group has a one-to-one corresponding true value data frame and original data frame

[0043] A position information determination submodule, for determining, for any data group, first position information of a target object in a true value data frame in the data group from the vehicle and second position information of a target object in an original data frame in the data group from the vehicle;

[0044] a valid target determination submodule, configured to determine that the target object in the original data frame is a valid target when a position deviation between the first position information and the second position information is within a threshold range;

[0045] The invalid target determination submodule is used to determine that the target object in the original data frame is an invalid target when the position deviation between the first position information and the second position information exceeds a threshold range.

[0046] In an embodiment of the present application, the time alignment submodule includes:

[0047] A timestamp acquisition unit, used to acquire a first timestamp corresponding to each true value data frame in the true value data and a second timestamp corresponding to an original data frame in the original data;

[0048] A matching unit is used to match temporally adjacent true value data frames and original data frames based on the first timestamp and the second timestamp to obtain multiple data groups.

[0049] In one embodiment of the present application, the training module includes:

[0050] A preprocessing submodule is used to preprocess the labeled data to obtain target data; the target data includes model training samples and model testing samples;

[0051] A training submodule, configured to train the initial obstacle recognition model using the model training samples as a training set to obtain a first obstacle recognition model;

[0052] A testing submodule, configured to identify the model test sample by using the first obstacle recognition model to obtain a first recognition result;

[0053] The obtaining submodule is used to obtain the target obstacle recognition model when the first recognition result meets the preset recognition index.

[0054] In one embodiment of the present application, the preprocessing submodule includes:

[0055] A balanced sampling unit, used for performing balanced sampling on different types of objects in the labeled data to obtain initial data;

[0056] The feature processing unit is used to perform feature processing on the initial data to obtain target data.

[0057] In an embodiment of the present application, the balanced sampling unit includes:

[0058] A type information acquisition subunit, used to acquire type attribute information of objects in the labeled data; the type attribute information includes vulnerable road users, vehicles and invalid targets;

[0059] The sampling subunit is used to perform balanced sampling on the labeled data according to the sampling weights corresponding to the vulnerable road users, the vehicles and the invalid targets, so as to obtain the initial data.

[0060] In one embodiment of the present application, the feature processing unit includes:

[0061] A feature processing subunit, used for performing different feature processing on different signals in the initial data to obtain feature data;

[0062] The clearing subunit is used to clear invalid features in the feature data to obtain the target data.

[0063] In one embodiment of the present application, the obstacle recognition model training device further includes:

[0064] A deployment module, used to deploy the target obstacle recognition model into a target vehicle;

[0065] A second acquisition module is used to acquire second raw data collected by the millimeter wave radar set on the target vehicle;

[0066] The output module is used to input the second original data into the target obstacle recognition model and output the probability that the objects around the target vehicle are movable obstacles.

[0067] In the third aspect, based on the same inventive concept, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the obstacle recognition model training method proposed in the first aspect of the present application.

[0068] In the fourth aspect, based on the same inventive concept, an embodiment of the present application provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the obstacle recognition model training method proposed in the first aspect of the present application is implemented.

[0069] Compared with the prior art, this application has the following advantages:

[0070] The obstacle recognition model training method provided in the embodiment of the present application first obtains the original data collected by the millimeter wave radar and the true value data collected synchronously by the truth value system; then, based on the true value data, the valid targets and invalid targets in the original data are determined, and corresponding true value labels are added to the valid targets and invalid targets to obtain the labeled data; finally, based on the labeled data, the initial obstacle recognition model is trained to obtain the target obstacle recognition model. The target obstacle recognition model trained in the embodiment of the present application can accurately output the derived variable "the probability that the object is a movable obstacle" with high confidence to replace the original recognition result of the millimeter wave radar, and then can inform the subsequent algorithm of the probability that the object recognized by the millimeter wave radar is a movable obstacle, and assist the subsequent algorithm to make more efficient and accurate decisions. BRIEF DESCRIPTION OF THE DRAWINGS

[0071] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0072] Figure 1 It is a flowchart of the steps of an obstacle recognition model training method in one embodiment of the present application.

[0073] Figure 2 It is a schematic diagram of an AEB module misoperation scenario in an embodiment of the present application.

[0074] Figure 3 It is a schematic diagram of the functional modules of an obstacle recognition model training device in one embodiment of the present application.

[0075] Figure numerals: 300 - obstacle recognition model training device; 301 - first acquisition module; 302 - labeling module; 303 - training module. DETAILED DESCRIPTION

[0076] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0077] It should be noted that the decision-making module and execution module in the automatic driving assistance system fully rely on the results of the front-end fusion module, and the results of the fusion module are highly dependent on the results of the perception module. The perception module and the fusion module mainly rely on the recognition results of the camera and the millimeter-wave radar, and currently face the following problems: 1) The camera is greatly affected by the environment and is prone to false detection, missed detection and large range deviation; 2) The radar has a low clustering result accuracy due to the sparse point cloud, so it cannot well identify whether objects such as people and vehicles are autonomously movable obstacles; 3) The current fusion matching algorithm combines the camera and radar information, although it has the advantages and disadvantages, but the confidence of the radar in judging the "probability of an object becoming a movable obstacle" is not high. Based on the chain reaction, subsequent algorithms will adopt more rules to help determine whether the object is a threat to the vehicle, so as to determine whether to take corresponding protective measures. In this way, since the link is too long, a very cautious strategy has to be implemented when optimizing the existing algorithm. Any carelessness will affect other links. For example, if this link is optimized, other links may have a negative impact and need corresponding adjustments. In this way, every time an adjustment is made, the entire link must be continuously tested, and the efficiency is relatively low.

[0078] In view of the defects of the prior art, the present application aims to provide an obstacle recognition model training method. Through machine learning modeling, a target obstacle recognition model is trained to output a derived variable "the probability that an object is a movable obstacle" with high confidence. The target obstacle recognition model can use the original signal collected by the sensor as input and the "probability that an object is a movable obstacle" as output information. The output information is passed to the subsequent algorithm module as a derived variable to assist the subsequent algorithm in making more efficient and accurate decisions.

[0079] Reference Figure 1 , shows an obstacle recognition model training method of the present application, which may include the following steps:

[0080] S101: Acquire raw data collected by the millimeter wave radar and true value data collected synchronously by the true value system.

[0081] It should be noted that the true value system is a highly configured and mature sensor system, which is a data acquisition system composed of vehicle-mounted sensors such as millimeter-wave radar, laser radar, high-precision combined inertial navigation, and efficient data recording equipment. Since the true value system can process data and intelligently output accurate true value data, the true value system is often used to evaluate the performance of the sensor being tested.

[0082] In this embodiment, the millimeter-wave radar to be tested and the truth value system can be deployed on the data collection vehicle at the same time, thereby realizing the synchronous collection of the same road scene. The collected data includes the original data collected by the millimeter-wave radar and the truth value data collected synchronously by the truth value system.

[0083] S102: Based on the true value data, determine the valid targets and invalid targets in the original data, and add corresponding true value labels to the valid targets and invalid targets to obtain labeled data.

[0084] It should be noted that by engineering the data collected by the truth system, the truth data at the truth target level (hereinafter referred to as truth GT) can be obtained. The truth GT is accurate, reliable and rich in types. It can include dynamic information such as the speed, position, size, category and ID of obstacles, as well as static information such as lane lines, road boundaries, and road curvature. The truth label is a Boolean variable that states whether the given object attribute is true or false.

[0085] In this embodiment, by performing target-level matching on the target object in the true value GT and the target object in the original data collected by the millimeter-wave radar (hereinafter referred to as millimeter-wave radar mrr), the target object in the millimeter-wave radar mrr can be automatically labeled according to the matching results.

[0086] Specifically, if the target object in the millimeter-wave radar mrr matches the target object in the true value gt, the target object in the millimeter-wave radar mrr is determined to be a valid target, and then a corresponding true value label is added to it, that is, Truth label = 1, indicating a valid target; if the target object in the millimeter-wave radar mrr does not match the target object in the true value gt, the target object in the millimeter-wave radar mrr is determined to be an invalid target, and then a corresponding true value label is added to it, that is, Truth label = 0, indicating an invalid target. After the matching is completed, the labeled data corresponding to the millimeter-wave radar mrr is obtained.

[0087] S103: Based on the labeled data, the initial obstacle recognition model is trained to obtain a target obstacle recognition model, where the target obstacle recognition model is used to determine the probability that the objects around the vehicle are movable obstacles.

[0088] In this embodiment, supervised learning is performed based on labeled data by machine learning modeling, and the initial obstacle recognition model is used to predict the objects in the labeled data, and the prediction result is a 0-1 floating point value of the probability that the object is a movable obstacle. After the recognition result of the initial obstacle recognition model meets the corresponding evaluation index, the model parameters in the initial obstacle recognition model can be fixed, and finally the required target obstacle recognition model is obtained.

[0089] In the specific implementation, the initial obstacle recognition model can be constructed based on the GBDT model, which is an integrated algorithm based on a decision tree, with few parameters, stable training results and good effects, and can autonomously discover high-order relationships between features. Compared with the more commonly used logistic regression model, the GBDT model has better expression ability and can process linear and nonlinear data, such as signals with linear relationships such as speed and distance in millimeter wave radar mrr, and nonlinear signals such as movement direction, object category, and whether it is being tracked.

[0090] In this embodiment, based on the target obstacle recognition model, the derived variable "the probability that the object is a movable obstacle" with high confidence can be accurately output and passed to the subsequent algorithm module for use. The beneficial effects brought about by replacing the original recognition result of the millimeter wave radar with the output result of the target obstacle recognition model include:

[0091] 1) It utilizes big data and machine learning modeling methods, comprehensively considers the characteristics of the collected data, and performs data fitting, without the need to optimize the ADAS algorithm through targeted patches with narrow universality.

[0092] 2) Improve the adaptability of ADAS algorithms. Currently, the same set of algorithms will be configured with different parameters according to different projects and software integration versions, and the data information used is also different, so a lot of time will be spent on algorithm debugging. The target obstacle recognition model provided in the embodiment of the present application can be effectively applied to different ADAS algorithms, and subsequent algorithm groups including fusion algorithm groups, decision algorithm groups, etc. only need to directly call the output results of the target obstacle recognition model without making any adjustments.

[0093] 3) Improved the accuracy of the property of whether the perceived target object is a movable obstacle. Using the target obstacle recognition model, the probability of objects around the vehicle being movable obstacles can be output based on the original data collected by the millimeter-wave radar. It can even directly replace the original recognition result of the millimeter-wave radar without changing the subsequent algorithm API (Application Program Interface), which can effectively avoid the inefficiency problem of the existing technology that requires continuous testing of the entire ADAS algorithm chain.

[0094] In a feasible implementation manner, the step of determining valid targets and invalid targets in the original data based on the true value data in S102 may specifically include the following sub-steps:

[0095] S102-1: Time-align the true value data and the original data to obtain multiple data groups; each data group has a one-to-one corresponding true value data frame and original data frame.

[0096] In this embodiment, since the cycles of the true value GT and the millimeter-wave radar MRR return data are inconsistent, the return interval of the true value GT is usually 0.1 seconds, and the return interval of the millimeter-wave radar MRR is 0.02 seconds, that is, the timestamps are not completely aligned, and there is a certain time difference between the two. Therefore, it is necessary to time-align the true value GT and the millimeter-wave radar MRR to match the true value data frame in the true value GT with the original data frame of the millimeter-wave radar MRR one-to-one. It should be noted that the true value data frame refers to the image frame collected by the true value system at a certain moment, and the original data frame refers to the image frame collected by the millimeter-wave radar at a certain moment.

[0097] In this embodiment, the first timestamp corresponding to each true value data frame in the true value data and the second timestamp corresponding to the original data frame in the original data can be first obtained; then, based on the first timestamp and the second timestamp, the true value data frames and original data frames that are adjacent in time are matched to obtain multiple data groups. In a specific implementation, since the return interval of the true value GT is longer than the return interval of the millimeter wave radar mrr, the original data frames in the millimeter wave radar mrr can be matched based on the first timestamp corresponding to the true value data frame and the principle of the nearest match to achieve frame alignment matching. The result of the frame alignment matching is the set of targets contained in the true value data frame and the original data frame at a certain timestamp of a certain packet of data.

[0098] S102 - 2 : For any data group, determine the first position information of the target object from the vehicle in the true value data frame in the data group and the second position information of the target object from the vehicle in the original data frame in the data group.

[0099] In this embodiment, based on the result of frame alignment, the target object in each true value data frame is matched with the target object in the original data frame, and a true value label is added to the target object in the original data frame according to the matching result.

[0100] In this embodiment, the first position information includes the first longitudinal distance and the first lateral distance of the target object from the vehicle in the true value data frame; the second position information includes the second longitudinal distance and the second lateral distance of the target object from the vehicle in the original data frame. The longitudinal direction refers to the direction in which the vehicle is moving, and the lateral direction refers to the direction perpendicular to the direction in which the vehicle is moving.

[0101] S102-3: When the position deviation between the first position information and the second position information is within a threshold range, determining that the target object in the original data frame is a valid target.

[0102] S102-4: When the position deviation between the first position information and the second position information exceeds a threshold range, determining that the target object in the original data frame is an invalid target.

[0103] In this embodiment, the threshold range specifically includes a longitudinal threshold and a transverse threshold. That is, when the longitudinal difference between the first longitudinal distance and the second longitudinal distance is less than or equal to the longitudinal threshold, and the transverse difference between the first transverse distance and the second transverse distance is less than or equal to the transverse threshold, it indicates that the target object in the original data frame is a valid target; when the longitudinal difference between the first longitudinal distance and the second longitudinal distance is greater than the longitudinal threshold, or the transverse difference between the first transverse distance and the second transverse distance is greater than the transverse threshold, it indicates that the target object in the original data frame is an invalid target.

[0104] In this embodiment, the longitudinal threshold and the lateral threshold can be set according to the detection accuracy of the millimeter wave radar. For example, when the detection accuracy of the millimeter wave radar is 0.11-0.15 meters, the lateral threshold can be set to 0.2 meters and the longitudinal threshold can be set to 2 meters. It should be noted that this embodiment does not make specific restrictions on the longitudinal threshold and the lateral threshold.

[0105] In a feasible implementation, S103 may specifically include the following sub-steps:

[0106] S103-1: Preprocess the labeled data to obtain target data; the target data includes model training samples and model testing samples.

[0107] In this embodiment, the preprocessing mainly includes three parts: cleaning abnormal samples, sampling and feature processing. Among them, cleaning abnormal samples refers to removing the data with missing attribute information and the erroneous data that do not conform to common sense, such as time error, negative absolute speed, etc. in the labeled data; sampling is to select the required data samples from a large amount of labeled data; feature processing is to perform feature processing on the signals in the labeled data so that the initial obstacle recognition model can be better trained and the model training effect can be improved.

[0108] S103-2: Using the model training samples as a training set, training the initial obstacle recognition model to obtain a first obstacle recognition model.

[0109] S103-3: Identify the model test sample using the first obstacle recognition model to obtain a first recognition result.

[0110] In this embodiment, after the first obstacle recognition model is obtained by training the model training samples, the first obstacle recognition model is used to perform recognition test on the model test samples, so as to effectively evaluate the actual application effect of the first obstacle recognition model.

[0111] S103-4: When the first recognition result meets the preset recognition index, a target obstacle recognition model is obtained.

[0112] In this embodiment, the preset recognition indicators may include at least the following two categories: 1) Model evaluation indicators: AUC and PR under the curve area. AUC is the area under the ROC curve, which is independent of the model classification results and thresholds. It describes the model classification ability. When the threshold is constantly changing, the probability of the positive example can be ranked before the probability of the negative example as much as possible. The AUC value is between 0-1, 1 indicates an ideal situation, and 0.85 or above indicates an excellent model. The area under the PR curve describes the related change curve of Precision and Recall when the threshold is constantly changing. Similar to AUC, the larger the area under the PR curve, the higher the average precision; 2) Evaluation indicators of classification results. The evaluation indicators can include the three most commonly used typical indicators of accuracy, precision, and recall, and their specific meanings will not be elaborated here.

[0113] In this embodiment, by using the preset recognition index as the evaluation criterion, the recognition ability of the first obstacle recognition model can be comprehensively and effectively evaluated. After the first obstacle recognition model meets the preset recognition index, the model parameters at this time can be fixed to finally obtain the desired target obstacle recognition model.

[0114] In a feasible implementation, S103-1 may specifically include the following sub-steps:

[0115] S103-1-1: Perform balanced sampling on different types of objects in the labeled data to obtain initial data.

[0116] In this embodiment, since most of the labeled data are vehicles including cars and trucks, and the samples of vulnerable road users (VRUs), i.e. pedestrians and bicycles, are relatively small, for the subsequent model effect, it is necessary to first balance the samples to ensure that the model can learn the characteristics of various samples.

[0117] In a specific implementation, the type attribute information of objects in the labeled data can be first obtained, and the type attribute information includes vulnerable road users, vehicles and invalid targets; then, the labeled data is evenly sampled according to the sampling weights corresponding to the vulnerable road users, vehicles and invalid targets to obtain the initial data.

[0118] In this embodiment, by performing balanced sampling on the labeled data, the number of different types of targets in the sampled initial data can be the same or similar, so as to effectively ensure the model training effect.

[0119] S103-1-2: Perform feature processing on the initial data to obtain target data.

[0120] In this embodiment, considering that the initial data contains a large amount of linear and nonlinear data, such as linear signals such as position information, speed information, acceleration information, and nonlinear signals such as motion direction and object category, if the initial data is directly input into the initial obstacle recognition model for training, it will not be conducive to improving the training effect of the initial obstacle recognition model. Therefore, the initial data will be feature processed so that the initial obstacle recognition model can better learn the feature information contained in the target data.

[0121] In the specific implementation, different feature processing is performed on different signals in the initial data to obtain feature data. The feature processing methods can specifically include one-hot encoding processing, binning processing, precision processing and the like. Among them, one-hot encoding processing can encode nonlinear signals representing categories such as motion direction and object category into a form that is easy to process in the machine learning algorithm; binning processing is to divide the signals with continuous values ​​such as velocity information and acceleration information into several intervals; precision processing is to set the corresponding precision values ​​for data such as relative longitudinal velocity, relative longitudinal acceleration, relative longitudinal acceleration variance, relative lateral velocity, relative lateral acceleration, and relative lateral acceleration variance, such as the number of decimal places that the value needs to retain. After obtaining the feature data, feature screening will be performed to remove invalid features in the feature data to obtain the target data. Feature screening includes: removing the package name and ID of the data packet, removing the time, removing the obstacle size information, and removing the original "obstacle probability" signal information of the millimeter wave radar.

[0122] In this embodiment, by performing balanced sampling and feature processing on the labeled data, the target data can be made more conducive to model training, the efficiency and effect of model training can be improved, and the trained target obstacle recognition model can have better practical use effect.

[0123] In a feasible implementation, after S103, the obstacle recognition model training may further include the following steps:

[0124] S104: Deploy the target obstacle recognition model to the target vehicle.

[0125] In this embodiment, before the target obstacle recognition model is formally deployed on a real vehicle, a logsim offline simulation test may be performed. For example, the target obstacle recognition model may be applied to an AEB (Autonomous Emergency Braking) algorithm.

[0126] It should be noted that the AEB algorithm uses the following logic flow for the "probability of an object being a movable obstacle" information output by the millimeter-wave radar:

[0127] 1) Obtain the signal output from the original sensor, including the first probability signal output by the millimeter-wave radar indicating the "probability that the object is a movable obstacle" and the second probability signal output by the camera indicating the "probability that an obstacle exists".

[0128] 2) The fusion module processes the first probability signal and the second probability signal. The first probability signal remains unchanged, and the second probability signal is post-processed according to the target matching results of the millimeter-wave radar and the camera. Specifically, the second probability signal is assigned a value of 1 if it matches, otherwise it remains unchanged.

[0129] 3) The decision module selects the target list that poses the greatest threat to the vehicle based on the information (without changing the variable signal value output by the fusion module) and passes it to the AEB module.

[0130] 4) The AEB module calculates the product between the first probability signal and the second probability signal according to the AEB algorithm. If the product exceeds the threshold, the AEB function is triggered to perform emergency braking.

[0131] For example, refer to Figure 2 , which shows a schematic diagram of the AEB module misoperation scenario. The direction of the vehicle's movement on the curve is as follows Figure 2 As shown in the figure, there are two stationary cars obj 1 and obj 2 parked on the roadside. Due to the problem of the fusion module, "obj 1's dx and obj 2's dy" are fused together, resulting in a fusion result of a false target very close to the vehicle, which causes AEB to be falsely triggered and emergency braking. In this scenario, the value of "first probability signal * second probability signal" of the synthesized false target calculated by the AEB algorithm is large enough to exceed the threshold, so the AEB function is triggered. Therefore, after the target obstacle recognition model is introduced, the "probability of the object being a movable obstacle" output by the target obstacle recognition model replaces the original first probability signal of the millimeter-wave radar, and the "probability of the object being a movable obstacle" of the synthesized false target is low enough, which can effectively avoid the false triggering of AEB.

[0132] Through simulation tests, it can be learned that compared with the 65.7% accuracy of the original recognition result signal output by the millimeter-wave radar, the accuracy of the derived signal output by the target obstacle recognition model for cars, trucks, pedestrians, bicycles and other objects has been significantly improved, reaching 84.700%, 74.505%, 85.506% and 83.171% respectively. The overall accuracy rate reaches 84.5%, which is 18.8% higher than the original recognition result signal of 65.7%.

[0133] In this embodiment, after completing the simulation test for the target obstacle recognition model, the target obstacle recognition model can be converted into structured data and deployed to the target vehicle equipped with the same millimeter-wave radar for actual use.

[0134] S105: Acquire second original data collected by the millimeter-wave radar provided on the target vehicle.

[0135] S106: Input the second original data into the target obstacle recognition model, and output the probability that the objects around the target vehicle are movable obstacles.

[0136] In this embodiment, in the actual application stage, after obtaining the second original data collected by the input millimeter-wave radar, the target obstacle recognition model will identify the target object in the second original data and output the probability that the objects around the target vehicle are movable obstacles, so as to assist subsequent algorithms in making more efficient and accurate decisions.

[0137] In the second aspect, based on the same inventive concept, refer to Figure 3 The embodiment of the present application provides an obstacle recognition model training device 300, and the obstacle recognition model training device 300 includes:

[0138] The first acquisition module 301 is used to acquire the original data collected by the millimeter wave radar and the true value data synchronously collected by the true value system;

[0139] A labeling module 302 is used to determine valid targets and invalid targets in the original data based on the true value data, and add corresponding true value labels to the valid targets and invalid targets to obtain labeled data;

[0140] The training module 303 is used to train the initial obstacle recognition model based on the labeled data to obtain a target obstacle recognition model, and the target obstacle recognition model is used to determine the probability that the objects around the vehicle are movable obstacles.

[0141] In a feasible implementation, the marking module 302 includes:

[0142] The time alignment submodule is used to time-align the true value data and the original data to obtain multiple data groups; each data group has a one-to-one corresponding true value data frame and original data frame

[0143] A position information determination submodule, for determining, for any data group, first position information of a target object in a true value data frame in the data group from the vehicle and second position information of a target object in an original data frame in the data group from the vehicle;

[0144] A valid target determination submodule, configured to determine that a target object in an original data frame is a valid target when a position deviation between the first position information and the second position information is within a threshold range;

[0145] The invalid target determination submodule is used to determine that the target object in the original data frame is an invalid target when the position deviation between the first position information and the second position information exceeds a threshold range.

[0146] In a feasible implementation, the time alignment submodule includes:

[0147] A timestamp acquisition unit, used to acquire a first timestamp corresponding to each true value data frame in the true value data and a second timestamp corresponding to the original data frame in the original data;

[0148] The matching unit is used to match the true value data frame and the original data frame with adjacent time based on the first timestamp and the second timestamp to obtain multiple data groups.

[0149] In one feasible implementation, the training module 302 includes:

[0150] The preprocessing submodule is used to preprocess the labeled data to obtain the target data; the target data includes model training samples and model testing samples;

[0151] A training submodule, used to train the initial obstacle recognition model using the model training samples as a training set to obtain a first obstacle recognition model;

[0152] A testing submodule, used to identify the model test sample through the first obstacle recognition model to obtain a first recognition result;

[0153] The obtaining submodule is used to obtain a target obstacle recognition model when the first recognition result meets a preset recognition index.

[0154] In a feasible implementation, the preprocessing submodule includes:

[0155] A balanced sampling unit is used to perform balanced sampling on different types of objects in the labeled data to obtain initial data;

[0156] The feature processing unit is used to perform feature processing on the initial data to obtain target data.

[0157] In a feasible implementation, the balanced sampling unit includes:

[0158] A type information acquisition subunit is used to acquire type attribute information of objects in the labeled data; the type attribute information includes vulnerable road users, vehicles and invalid targets;

[0159] The sampling subunit is used to perform balanced sampling on the labeled data according to the sampling weights corresponding to the vulnerable road users, vehicles and invalid targets, so as to obtain initial data.

[0160] In a feasible implementation, the feature processing unit includes:

[0161] The feature processing subunit is used to perform different feature processing on different signals in the initial data to obtain feature data;

[0162] The clearing subunit is used to clear invalid features in the feature data to obtain target data.

[0163] In a feasible implementation manner, the obstacle recognition model training device 300 further includes:

[0164] A deployment module, used to deploy the target obstacle recognition model to the target vehicle;

[0165] A second acquisition module is used to acquire second raw data collected by a millimeter-wave radar set on the target vehicle;

[0166] The output module is used to input the second original data into the target obstacle recognition model and output the probability that the objects around the target vehicle are movable obstacles.

[0167] It should be noted that the specific implementation of the obstacle recognition model training device 300 of the embodiment of the present application refers to the specific implementation of the obstacle recognition model training method proposed in the first aspect of the embodiment of the present application, which will not be repeated here.

[0168] In the third aspect, based on the same inventive concept, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the obstacle recognition model training method proposed in the first aspect of the present application is implemented.

[0169] It should be noted that the specific implementation of the computer-readable storage medium in the embodiment of the present application refers to the specific implementation of the obstacle recognition model training method proposed in the first aspect of the embodiment of the present application, and will not be repeated here.

[0170] In the fourth aspect, based on the same inventive concept, an embodiment of the present application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the obstacle recognition model training method proposed in the first aspect of the present application is implemented.

[0171] It should be noted that the specific implementation of the electronic device in the embodiment of the present application refers to the specific implementation of the obstacle recognition model training method proposed in the first aspect of the embodiment of the present application, and will not be repeated here.

[0172] It will be appreciated by those skilled in the art that the embodiments of the present invention may be provided as methods, devices, or computer program products. Therefore, the embodiments of the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the embodiments of the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0173] The embodiments of the present invention are described with reference to the flowcharts and / or block diagrams of the methods, terminal devices (systems), and computer program products according to the embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0174] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0175] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable terminal device to implement the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0176] Although the preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present invention.

[0177] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the elements.

[0178] The above is a detailed introduction to an obstacle recognition model training method, device, storage medium and electronic device provided by the present invention. Specific examples are used in this article to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea; at the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. An obstacle recognition model training method, characterized in that: The method comprises: Obtain the original data collected by the millimeter-wave radar and the true value data collected synchronously by the true value system; Based on the true value data, determining valid targets and invalid targets in the original data, and adding corresponding true value labels to the valid targets and the invalid targets to obtain labeled data; Based on the labeled data, an initial obstacle recognition model is trained to obtain a target obstacle recognition model, where the target obstacle recognition model is used to determine the probability that an object around the vehicle is a movable obstacle.

2. The obstacle recognition model training method according to claim 1, characterized in that: The step of determining valid targets and invalid targets in the original data based on the true value data comprises: Time-aligning the true value data and the original data to obtain a plurality of data groups; each data group has a one-to-one corresponding true value data frame and an original data frame; For any data group, determine first position information of the target object in the true value data frame in the data group from the vehicle and second position information of the target object in the original data frame in the data group from the vehicle; When a position deviation between the first position information and the second position information is within a threshold range, determining that the target object in the original data frame is a valid target; When the position deviation between the first position information and the second position information exceeds a threshold range, it is determined that the target object in the original data frame is an invalid target.

3. The obstacle recognition model training method according to claim 2, characterized in that: The step of time-aligning the true value data and the original data to obtain multiple data groups includes: Obtaining a first timestamp corresponding to each true value data frame in the true value data and a second timestamp corresponding to an original data frame in the original data; Based on the first timestamp and the second timestamp, time-adjacent true value data frames and original data frames are matched to obtain multiple data groups.

4. The obstacle recognition model training method according to claim 1, characterized in that: The step of training the initial obstacle recognition model based on the labeled data to obtain the target obstacle recognition model includes: Preprocessing the labeled data to obtain target data; the target data includes model training samples and model testing samples; Using the model training samples as a training set, training the initial obstacle recognition model to obtain a first obstacle recognition model; Identify the model test sample using the first obstacle recognition model to obtain a first recognition result; When the first recognition result meets the preset recognition index, the target obstacle recognition model is obtained.

5. The obstacle recognition model training method according to claim 4, characterized in that: The step of preprocessing the labeled data to obtain target data includes: Performing balanced sampling on different types of objects in the labeled data to obtain initial data; Perform feature processing on the initial data to obtain the target data.

6. The obstacle recognition model training method according to claim 5, characterized in that: The step of performing balanced sampling on different types of objects in the labeled data to obtain initial data includes: Acquiring type attribute information of objects in the labeled data; the type attribute information includes vulnerable road users, vehicles, and invalid targets; The labeled data is evenly sampled according to the sampling weights corresponding to the vulnerable road users, the vehicles and the invalid targets to obtain the initial data.

7. The obstacle recognition model training method according to claim 1, characterized in that: After the step of training the initial obstacle recognition model based on the labeled data to obtain the target obstacle recognition model, the method further includes: Deploying the target obstacle recognition model into a target vehicle; Acquire second raw data collected by a millimeter-wave radar provided on the target vehicle; The second original data is input into the target obstacle recognition model, and the probability that the objects around the target vehicle are movable obstacles is obtained as an output.

8. An obstacle recognition model training device, characterized in that: The device comprises: A first acquisition module is used to acquire the original data collected by the millimeter wave radar and the true value data synchronously collected by the true value system; A labeling module, used to determine valid targets and invalid targets in the original data based on the true value data, and add corresponding true value labels to the valid targets and the invalid targets to obtain labeled data; A training module is used to train an initial obstacle recognition model based on the labeled data to obtain a target obstacle recognition model, wherein the target obstacle recognition model is used to determine the probability that an object around the vehicle is a movable obstacle.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the obstacle recognition model training method according to any one of claims 1 to 7 is implemented.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the obstacle recognition model training method according to any one of claims 1 to 7 is implemented.

Images