Method, device and equipment for determining confidence threshold, and storage medium
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- CHINA FAW CO LTD
- Filing Date
- 2023-05-05
- Publication Date
- 2026-06-19
Smart Images
Figure CN116524471B_ABST
Abstract
Description
Technical Field
[0001] This disclosure relates to the field of artificial intelligence technology, and in particular to a method, apparatus, device, and storage medium for determining a confidence threshold. Background Technology
[0002] In real-world traffic environments, automotive radar operates in complex conditions. To ensure that autonomous vehicles can make correct judgments in various scenarios, real-time dynamic acquisition and identification of surrounding environmental information is required through autonomous driving detection algorithms to meet the needs of the vehicle's backend decision-making system. The identification of surrounding environmental information by autonomous driving detection algorithms often requires threshold filtering; if the threshold condition is met, the identification result is output.
[0003] Currently, thresholds are set based on engineers' experience. If the threshold is set too high, it will cause missed detections in the surrounding environment perception; if it is set too low, it will cause false detections. Therefore, existing technologies that set thresholds based on experience suffer from inaccurate threshold settings, and adjusting filtering parameters is also time-consuming and labor-intensive. Summary of the Invention
[0004] This disclosure provides a method, apparatus, device, and storage medium for determining a confidence threshold, which can improve the accuracy of determining the confidence threshold.
[0005] In a first aspect, embodiments of this disclosure provide a method for determining a confidence threshold, comprising: acquiring training sample information; wherein the training sample information includes vehicle surrounding environment information and corresponding annotation information; the annotation information includes first center point location information, first size information, first category information, and obstacle serial number of an obstacle; the vehicle surrounding environment information is point cloud data acquired by vehicle-mounted radar within a set time period; filtering the annotation information according to the first center point location information and / or the first size information to obtain filtered annotation information; identifying the vehicle surrounding environment information based on a set detection model and a set segmentation model respectively to obtain first obstacle information and second obstacle information; determining target obstacle information according to the filtered annotation information, the first obstacle information, and the second obstacle information; and determining a confidence threshold for each obstacle category according to the confidence level corresponding to the target obstacle information.
[0006] Secondly, this disclosure also provides a confidence threshold determination device, comprising: a training sample information acquisition module, used to acquire training sample information; wherein the training sample information includes vehicle surrounding environment information and corresponding annotation information; the annotation information includes first center point location information, first size information, first category information and obstacle serial number of the labeled obstacle; the vehicle surrounding environment information is point cloud data within a set time period obtained by vehicle-mounted radar; an annotation information filtering module, used to filter the annotation information according to the first center point location information and / or the first size information to obtain filtered annotation information; an identification module, used to identify the vehicle surrounding environment information based on a set detection model and a set segmentation model respectively to obtain first obstacle information and second obstacle information; a target obstacle information determination module, used to determine target obstacle information according to the filtered annotation information, the first obstacle information and the second obstacle information; and a confidence threshold module, used to determine a confidence threshold for each obstacle category according to the confidence level corresponding to the target obstacle information.
[0007] Thirdly, embodiments of this disclosure also provide an electronic device, the electronic device comprising:
[0008] One or more processors;
[0009] Storage device for storing one or more programs.
[0010] When the one or more programs are executed by the one or more processors, the one or more processors implement the confidence threshold determination method as described in the embodiments of this disclosure.
[0011] Fourthly, embodiments of this disclosure also provide a storage medium containing computer-executable instructions, which, when executed by a computer processor, are used to perform the confidence threshold determination method as described in embodiments of this disclosure.
[0012] The technical solution of this disclosure embodiment involves obtaining training sample information; wherein, the training sample information includes vehicle surrounding environment information and corresponding annotation information; the annotation information includes the first center point location information, first size information, first category information, and obstacle serial number of the marked obstacle; the vehicle surrounding environment information is point cloud data within a set time period obtained by vehicle-mounted radar; the annotation information is filtered according to the first center point location information and / or the first size information to obtain filtered annotation information; the vehicle surrounding environment information is identified based on a set detection model and a set segmentation model to obtain first obstacle information and second obstacle information; target obstacle information is determined according to the filtered annotation information, the first obstacle information, and the second obstacle information; and a confidence threshold for each obstacle category is determined according to the confidence level corresponding to the target obstacle information. In this embodiment, the annotation information is filtered based on the first center point location information and / or the first size information to obtain filtered annotation information. Target obstacle information is determined based on the filtered annotation information, the first obstacle information, and the second obstacle information. The confidence threshold for each obstacle category is determined based on the confidence level corresponding to the target obstacle information. This method can improve the accuracy of the confidence threshold, thereby reducing missed detections and false detections caused by vehicle-mounted radar perception. Attached Figure Description
[0013] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and the originals and elements are not necessarily drawn to scale.
[0014] Figure 1 This is a schematic flowchart illustrating the method for determining the confidence threshold provided in an embodiment of this disclosure;
[0015] Figure 2 This is a schematic diagram illustrating the effect of implementing orientation information according to an embodiment of the present invention;
[0016] Figure 3 This is a schematic diagram illustrating the effect of merging obstacles provided in an embodiment of the present invention;
[0017] Figure 4 A schematic diagram of a confidence threshold determination device provided in an embodiment of this disclosure;
[0018] Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this disclosure. Detailed Implementation
[0019] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.
[0020] It should be understood that the steps described in the method embodiments of this disclosure may be performed in different orders and / or in parallel. Furthermore, the method embodiments may include additional steps and / or omit the steps shown. The scope of this disclosure is not limited in this respect.
[0021] The term "comprising" and its variations as used herein are open-ended inclusions, meaning "including but not limited to". The term "based on" means "at least partially based on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". Definitions of other terms will be given in the description below.
[0022] It should be noted that the concepts of "first" and "second" mentioned in this disclosure are used only to distinguish different devices, modules or units, and are not used to limit the order of functions performed by these devices, modules or units or their interdependencies.
[0023] It should be noted that the terms "a" and "a plurality of" used in this disclosure are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".
[0024] It is understood that the data involved in this technical solution (including but not limited to the data itself, the acquisition or use of the data) shall comply with the requirements of relevant laws, regulations and related provisions.
[0025] Figure 1 This is a schematic diagram of the confidence threshold determination method provided in the embodiments of this disclosure. The embodiments of this disclosure are applicable to the situation where the confidence threshold corresponding to the obstacle category is determined during the process of identifying the obstacle category of the surrounding environment by setting a detection model. The method can be executed by a confidence threshold determination device, which can be implemented in the form of software and / or hardware, and optionally, by an electronic device, such as a mobile terminal, a PC, or a server.
[0026] like Figure 1 As shown, the method includes:
[0027] S110, Obtain training sample information.
[0028] The training sample information includes information about the vehicle's surrounding environment and corresponding annotations. The annotations include the location of the first center point of each obstacle, its first size, first category, first orientation, and obstacle number. The vehicle's surrounding environment information is point cloud data acquired by onboard radar over a set time period.
[0029] The center point location information can be represented by three-dimensional coordinates. The first dimension information can be the length, width, and height of the marked obstacle. The obstacle number is used to distinguish different types of obstacles and can also be used to track obstacles within a set time period.
[0030] In this embodiment, point cloud data can be acquired in real time using vehicle-mounted radar. The point cloud data within a set time period can be understood as information about the vehicle's surrounding environment, and obstacle information within this environment can be labeled to obtain annotation information. The vehicle's surrounding environment information and its corresponding annotation information can be used as training samples. For the annotation, it can be done every set time interval to obtain multiple annotation information; that is, the vehicle's surrounding environment information can have multiple annotation information. For example, if the set time period is 15 seconds and the set time interval is 0.1 seconds, the vehicle's surrounding environment information within 0 to 15 seconds can be labeled every 0.1 seconds to obtain multiple annotation information.
[0031] S120. Filter the annotation information according to the first center point position information and / or the first dimension information to obtain the filtered annotation information.
[0032] In this embodiment, the accuracy of the annotation information is difficult to achieve due to the complexity of the data and the annotation requirements. For autonomous driving radar data, frame-by-frame annotation is usually performed. The existence of time intervals often prevents annotators from obtaining information between consecutive frames, leading to errors in obstacle information matching and ultimately resulting in inaccurate annotation information. Therefore, it is necessary to process the annotation information.
[0033] In this embodiment, firstly, the first orientation information can be determined based on the first center point position information of the marked obstacles at multiple consecutive time points. The marked information is then filtered based on the first orientation information to obtain first filtered marked information. Next, the area of the obstacles at adjacent time points is determined based on the first size information. The first filtered marked information is then filtered based on the ratio of the two obstacle areas to obtain filtered marked information. Alternatively, the first orientation information can be determined based on the first center point position information of the marked obstacles at multiple consecutive time points, and the marked information can be filtered based on the first orientation information to obtain filtered marked information. Alternatively, the area of the obstacles at adjacent time points can be determined based on the first size information, and the marked information can be filtered based on the ratio of the two obstacle areas to obtain filtered marked information.
[0034] Optionally, the annotation information is filtered based on the first center point location information and / or the first size information to obtain filtered annotation information, including: determining orientation information based on the first center point location information; filtering the annotation information based on the orientation information to obtain filtered annotation information; wherein the annotation information is information obtained by annotating the vehicle's surrounding environment information at set intervals; and / or, determining the obstacle area ratio of any adjacent time interval within a set time period based on the first size information; filtering the annotation information based on the obstacle area ratio to obtain filtered annotation information.
[0035] In this example, firstly, orientation information is determined based on multiple consecutive first center position information. The annotation information is then filtered based on this orientation information to obtain first filtered annotation information. Next, the obstacle area ratio at any adjacent time within a set time period is determined based on the first size information. The first filtered annotation information is then filtered based on this obstacle area ratio to obtain filtered annotation information. Alternatively, the orientation information is determined based on multiple consecutive first center position information. The annotation information is then filtered based on this orientation information to obtain filtered annotation information. The obstacle area ratio at any adjacent time within a set time period is then determined based on the first size information. The annotation information is then filtered based on this obstacle area ratio to obtain filtered annotation information.
[0036] Optionally, the orientation information is determined based on the first center position information, including: for each obstacle, obtaining the first center point position information of the obstacle at multiple consecutive moments; connecting the multiple first center point position information in chronological order to obtain a broken line formed by the center points; determining the included angle of the broken line; and determining the included angle as the orientation information.
[0037] For example, Figure 2 This is a schematic diagram illustrating the effect of implementing orientation information according to an embodiment of the present invention. Figure 2As shown, the first step is to obtain the position information of the first center point of the marked obstacle at any three consecutive moments within a set time period. The first moment can be represented by moment k-1, the second moment by moment k, and the third moment by moment k+1. The position information of the first center point at moment k-1 is connected to the position information of the first center point at moment k, and the position information of the first center point at moment k is connected to the position information of the first center point at moment k+1, resulting in a polyline composed of multiple center point positions. The included angle α of the polyline is then determined, where α can represent orientation information.
[0038] Optionally, the annotation information is filtered based on the orientation information to obtain the filtered annotation information, including: comparing the orientation information with a set orientation angle threshold to obtain a first comparison result; if the first comparison result is that the orientation information is greater than the set orientation angle threshold, then the annotation information of the obstacle at multiple consecutive times is deleted; if the first comparison result is that the orientation information is less than or equal to the set orientation angle threshold, then the annotation information of the obstacle at multiple consecutive times is retained; and the retained annotation information is used as the filtered annotation information.
[0039] In this embodiment, since the vehicle will not undergo a significant angular change in a short period of time, the orientation information can be determined and compared with a set orientation angle threshold. If the orientation information is greater than the set orientation angle threshold, the annotation information is filtered. For example, if the first comparison result is that the orientation information is greater than the set orientation angle threshold, the annotation information corresponding to the obstacle at times k-1, k, and k+1 is deleted. If the first comparison result is that the orientation information is less than or equal to the set orientation angle threshold, the annotation information corresponding to the obstacle at times k-1, k, and k+1 is retained; the retained annotation information is used as the filtered annotation information.
[0040] Optionally, determining the obstacle area ratio at any adjacent moment within a set time period based on the first size information includes: determining the areas of two obstacles at any adjacent moment within a set time period based on the first size information; and taking the ratio of the smaller obstacle area to the larger obstacle area as the obstacle area ratio.
[0041] In this embodiment, during the annotation process, due to obstacles being occluded, information about the vehicle's surrounding environment is missing, leading to annotation errors. This results in incorrect association between obstacles from the previous frame and obstacles in the current frame. For example, for the same obstacle number, it might be labeled as a medium-sized obstacle at time K-1, but incorrectly associated as a small obstacle at time K. Therefore, the annotation information can be filtered using the obstacle area ratio. Specifically, the obstacle area can be determined using the first size information. The formula for calculating the obstacle area ratio at any adjacent time within a set time period is as follows:
[0042]
[0043] Among them, for any adjacent time, the first time is represented by time k-1, and the second time is represented by time K. Area k Area represents the area of the obstacle at the second moment. k-1 Min(Area) represents the area of the obstacle at the first moment. k Area k-1 ) represents the smaller obstacle area between the areas at the first and second time points, max(Area) k Area k-1 The area represents the larger obstacle area between the first and second time points, and Area_ratio represents the ratio of obstacle areas.
[0044] Optionally, the annotation information is filtered based on the obstacle area ratio to obtain filtered annotation information, including: setting a set area threshold for different obstacle types; determining the obstacle type based on the first category information of the labeled obstacles; comparing the obstacle area ratio with the set area threshold corresponding to the obstacle type to obtain a second comparison result; if the second comparison result is that the obstacle area ratio is greater than the set area threshold corresponding to the obstacle type, then the annotation information corresponding to the adjacent time is retained; if the second comparison result is that the obstacle area ratio is less than or equal to the set area threshold corresponding to the obstacle type, then the annotation information corresponding to the adjacent time is deleted from the annotation information; and the retained annotation information is used as the filtered annotation information.
[0045] In this embodiment, the obstacle type can be determined based on the first obstacle category. The obstacle categories include large obstacles, medium obstacles, and large obstacles, and different area thresholds can be set for different obstacle types. The obstacle area ratio is compared with the set area threshold corresponding to the obstacle type to obtain a second comparison result. If the second comparison result shows that the obstacle area ratio is greater than the set area threshold corresponding to the obstacle type, it indicates that the obstacle area changes little between adjacent time points, and the corresponding annotation information for adjacent time points is retained. If the second comparison result shows that the obstacle area ratio is less than or equal to the set area threshold corresponding to the obstacle type, it indicates that the obstacle area changes too much between adjacent time points, and the corresponding annotation information for adjacent time points can be deleted from the annotation information. The retained annotation information is used as the filtered annotation information. It should be noted that in filtering annotation information based on obstacle area ratio, the annotation information can be unfiltered annotation information or annotation information filtered by orientation information.
[0046] S130. Based on the set detection model and the set segmentation model, the environmental information around the vehicle is identified to obtain the first obstacle information and the second obstacle information.
[0047] The training samples for the detection model and the segmentation model can be the same or different, or they can be the vehicle's surrounding environment information in this embodiment. The segmentation model can be any algorithm for segmenting obstacles, and the detection model can be any algorithm for detecting obstacles.
[0048] S140. Determine the target obstacle information based on the filtered annotation information, the first obstacle information, and the second obstacle information.
[0049] In this embodiment, the first obstacle information and the second obstacle information can be merged to obtain merged obstacle information, and the target obstacle information can be determined based on the filtered annotation information and the merged obstacle information.
[0050] Optionally, the target obstacle information is determined based on the filtered annotation information, the first obstacle information, and the second obstacle information, including: merging the first obstacle information and the second obstacle information to obtain merged obstacle information; and determining the target obstacle information based on the filtered annotation information and the merged obstacle information.
[0051] The first obstacle information includes the second center point location information, second size information, second orientation information, second category information, and the confidence level information corresponding to the second category information. The second obstacle information includes the third center point location information, third size information, third orientation information, third category information, and the confidence level information corresponding to the third category information.
[0052] Optionally, merging the first obstacle information and the second obstacle information to obtain merged obstacle information includes: determining the intersection obstacle information between the first obstacle information and the second obstacle information; determining the area of the intersection obstacle and the area of the first obstacle based on the second size information; determining the area of the second obstacle based on the third size information; determining the intersection area ratio information based on the area of the intersection obstacle, the area of the first obstacle, and the area of the second obstacle; comparing the intersection area ratio information with a set threshold to obtain a third comparison result; wherein the set threshold includes a first set threshold and a second set threshold; if the third comparison result is that the intersection area ratio information is equal to the first set threshold or less than or equal to the second set threshold, then the first obstacle information is used as the merged obstacle information; if the third comparison result is that the intersection area ratio information is greater than the second set threshold, then the first obstacle information and the second obstacle information are merged to obtain merged obstacle information.
[0053] In this embodiment, due to detection errors, the detection bounding box obtained by the set detection model may not completely contain the real point cloud data. The real point cloud data that is not contained will be identified as small obstacles by the set segmentation model (point cloud segmentation algorithm), but in reality, the two targets are the same obstacle. Therefore, it is necessary to merge the first obstacle and the second obstacle.
[0054] The formula for the intersection area ratio is as follows:
[0055]
[0056] Where Inter section(Det,Seg) is the area of the intersection obstacle, and Area Det Area is the area of the first obstacle. Seg Let the area be the area of the second obstacle. min(Area) Det Area Seg ) represents the minimum area between the areas of the first obstacle and the second obstacle, and IOU represents the intersection area ratio information.
[0057] After obtaining the intersection area ratio information, it is compared with a set threshold to obtain a third comparison result. If the third comparison result shows that the intersection area ratio is equal to the first set threshold, the first obstacle information is used as the merged obstacle information, and the second obstacle information is discarded. If the intersection area ratio is not equal to the first set threshold, the next judgment is performed. The first set threshold can be 0, and the second set threshold is an empirical value. If the third comparison result shows that the intersection area ratio is greater than the second set threshold, the first obstacle information and the second obstacle information are merged using a convex hull algorithm to obtain merged obstacle information. If the third comparison result shows that the intersection area ratio is less than or equal to the second set threshold, the first obstacle information is used as the merged obstacle information, and the second obstacle information is discarded. Figure 3 As shown, Figure 3 This is a schematic diagram illustrating the effect of merging obstacles according to an embodiment of the present invention. Figure 3 The center dot represents radar point cloud data. The square detection box is the first obstacle detection box, and the irregular detection box is the second obstacle detection box. After merging the first obstacle information and the second obstacle information, the merged obstacle detection box is obtained.
[0058] Optionally, the target obstacle information is determined based on the filtered annotation information and the merged obstacle information, including: obtaining the first category information corresponding to the filtered annotation information; obtaining the second category information corresponding to the merged obstacle information; grouping obstacles corresponding to the same category in the first category information and the second category information into one category to obtain multiple categories of obstacles; for each category of obstacle, determining the intersection area ratio information between the labeled obstacle and the merged obstacle corresponding to the filtered annotation information; determining whether the labeled obstacle and the merged obstacle match based on the intersection area ratio information; if the labeled obstacle and the merged obstacle match, then the labeled obstacle and the merged obstacle are taken as a matching pair, and the matching pair is taken as the target obstacle information.
[0059] In this embodiment, the first category information corresponding to the filtered annotation information is obtained; the second category information corresponding to the merged obstacle information is obtained (the second category information is used as the category information of the merged obstacle); obstacles corresponding to the same category as the first category information and the second category information are grouped into one category to obtain multiple obstacle categories. For each obstacle category, the intersection area ratio information between the labeled obstacle and the merged obstacle corresponding to the filtered annotation information is determined. Since there are multiple intersection area ratio information in each obstacle category, each intersection area ratio information is obtained from the labeled obstacle and the merged obstacle. Among them, the merged obstacle is the merged obstacle corresponding to the merged obstacle information. For each obstacle category, the metric matrix between the labeled obstacle and the merged obstacle can be obtained according to the multiple intersection area ratio information. The formula for the metric matrix M is as follows:
[0060]
[0061] Where, m ij This represents the ratio of the intersection area of the i-th first obstacle and the j-th labeled obstacle.
[0062] The area ratio information of each intersection in the metric matrix M is input into the data association Hungarian matching algorithm. The data association Hungarian matching algorithm can determine whether the labeled obstacle matches the merged obstacle. If the labeled obstacle matches the merged obstacle (i.e. the labeled obstacle is the same as the merged obstacle), the labeled obstacle and the merged obstacle are taken as a matching pair, and the matching pair is taken as the target obstacle information. Multiple matching pairs can be obtained.
[0063] S150. Determine the confidence threshold for each obstacle category based on the confidence level corresponding to the target obstacle information.
[0064] In this embodiment, for each type of obstacle, since multiple matching pairs can be obtained, the confidence threshold of the obstacle category can be determined based on the confidence levels corresponding to multiple target obstacle information. Taking one obstacle category as an example, the median, mean, etc., of the multiple confidence levels corresponding to multiple target obstacle information can be used as the confidence threshold of the obstacle category. The confidence level corresponding to the target obstacle information can be the confidence level corresponding to the first obstacle information.
[0065] Optionally, a confidence threshold for each obstacle category is determined based on the confidence level corresponding to the target obstacle information, including: for each obstacle category, sorting the merged obstacle information from multiple target obstacle information of the obstacle category; and determining the confidence threshold for the obstacle category based on the confidence level corresponding to the sorted multiple merged obstacle information.
[0066] Specifically, the confidence level of each obstacle category can be obtained. For each obstacle category, multiple confidence levels can be obtained. The median of the multiple confidence levels is calculated, and the confidence level corresponding to the median is used as the confidence threshold of the corresponding obstacle category.
[0067] Specifically, for each obstacle category, the confidence scores corresponding to the merged obstacle information among multiple target obstacle information for that obstacle category are sorted to obtain multiple sorted confidence scores. Among these multiple sorted confidence scores, the confidence score corresponding to the median is used as the confidence threshold for the corresponding obstacle category. The confidence score corresponding to the merged obstacle information is the confidence score corresponding to the first obstacle information.
[0068] In this embodiment, after determining the confidence threshold for each obstacle category, the detection model is configured to filter obstacles based on the confidence threshold for each obstacle category.
[0069] The technical solution of this disclosure embodiment obtains training sample information; wherein, the training sample information includes vehicle surrounding environment information and corresponding annotation information; the annotation information includes the first center point location information, first size information, first category information and obstacle number of the marked obstacle; the vehicle surrounding environment information is point cloud data within a set time period obtained by vehicle-mounted radar; the annotation information is filtered according to the first center point location information and / or the first size information to obtain filtered annotation information; the vehicle surrounding environment information is identified based on a set detection model and a set segmentation model to obtain first obstacle information and second obstacle information; the target obstacle information is determined according to the filtered annotation information, the first obstacle information and the second obstacle information; and the confidence threshold of each obstacle category is determined according to the confidence level corresponding to the target obstacle information. In this embodiment, the annotation information is filtered based on the first center point location information and / or the first size information to obtain filtered annotation information. The target obstacle information is determined based on the filtered annotation information, the first obstacle information, and the second obstacle information. The confidence threshold for each obstacle category is determined based on the confidence level corresponding to the target obstacle information. This method can improve the accuracy of the confidence threshold, thereby reducing missed detections and false detections caused by vehicle radar perception.
[0070] Figure 4 This is a schematic diagram of a confidence threshold determination device provided in an embodiment of the present disclosure, as shown below. Figure 4 As shown, the device includes: a training sample information acquisition module 410, a label information filtering module 420, a recognition module 430, a target obstacle information determination module 440, and a confidence threshold module 450;
[0071] The training sample information acquisition module 410 is used to acquire training sample information; wherein, the training sample information includes vehicle surrounding environment information and corresponding annotation information; the annotation information includes the first center point position information, first size information, first category information and obstacle number of the marked obstacle; the vehicle surrounding environment information is point cloud data within a set time period obtained by vehicle radar;
[0072] The annotation information filtering module 420 is used to filter the annotation information according to the first center point position information and / or the first size information to obtain the filtered annotation information;
[0073] The recognition module 430 is used to recognize the environmental information around the vehicle based on a set detection model and a set segmentation model, respectively, to obtain first obstacle information and second obstacle information;
[0074] The target obstacle information determination module 440 is used to determine target obstacle information based on the filtered annotation information, the first obstacle information, and the second obstacle information;
[0075] The confidence threshold module 450 is used to determine the confidence threshold for each obstacle category based on the confidence level corresponding to the target obstacle information.
[0076] The technical solution of this disclosure embodiment acquires training sample information through a training sample information acquisition module. The training sample information includes vehicle surrounding environment information and corresponding annotation information. The annotation information includes the first center point location information, first size information, first category information, and obstacle serial number of the labeled obstacle. The vehicle surrounding environment information is point cloud data acquired by vehicle-mounted radar within a set time period. An annotation information filtering module filters the annotation information based on the first center point location information and / or the first size information to obtain filtered annotation information. An identification module identifies the vehicle surrounding environment information based on a set detection model and a set segmentation model to obtain first obstacle information and second obstacle information. A target obstacle information determination module determines target obstacle information based on the filtered annotation information, the first obstacle information, and the second obstacle information. A confidence threshold module determines a confidence threshold for each obstacle category based on the confidence level corresponding to the target obstacle information. In this embodiment, the annotation information is filtered based on the first center point location information and / or the first size information to obtain filtered annotation information. Target obstacle information is determined based on the filtered annotation information, the first obstacle information, and the second obstacle information. The confidence threshold for each obstacle category is determined based on the confidence level corresponding to the target obstacle information. This method can improve the accuracy of the confidence threshold, thereby reducing missed detections and false detections caused by vehicle-mounted radar perception.
[0077] Optionally, the annotation information filtering module is specifically used for: determining orientation information based on the first center position information; filtering the annotation information based on the orientation information to obtain filtered annotation information; wherein the annotation information is information obtained by annotating the environmental information around the vehicle at set intervals; and / or, determining the obstacle area ratio of the obstacle at any adjacent time within the set time period based on the first size information; filtering the annotation information based on the obstacle area ratio to obtain filtered annotation information.
[0078] Optionally, the annotation information filtering module is further configured to: for each obstacle, obtain the position information of the first center point of the obstacle at multiple consecutive moments; connect the multiple first center point position information sequentially in chronological order to obtain a broken line formed by the center points; determine the included angle of the broken line; and determine the included angle as orientation information.
[0079] Optionally, the annotation information filtering module is further configured to: compare the orientation information with a set orientation angle threshold to obtain a first comparison result; if the first comparison result indicates that the orientation information is greater than the set orientation angle threshold, then delete the annotation information of the obstacle at the consecutive multiple time points; if the first comparison result indicates that the orientation information is less than or equal to the set orientation angle threshold, then retain the annotation information of the obstacle at the consecutive multiple time points; and use the retained annotation information as the filtered annotation information.
[0080] Optionally, the annotation information filtering module is further configured to: determine the areas of two obstacles at any adjacent time within the set time period based on the first size information; and take the ratio of the smaller obstacle area to the larger obstacle area as the obstacle area ratio.
[0081] Optionally, the annotation information filtering module is further configured to: set a predetermined area threshold for different obstacle types; determine the obstacle type based on the first category information of the labeled obstacles; compare the obstacle area ratio with the predetermined area threshold corresponding to the obstacle type to obtain a second comparison result; if the second comparison result is that the obstacle area ratio is greater than the predetermined area threshold corresponding to the obstacle type, then retain the annotation information corresponding to the adjacent time; if the second comparison result is that the obstacle area ratio is less than or equal to the predetermined area threshold corresponding to the obstacle type, then delete the annotation information corresponding to the adjacent time from the annotation information; and use the retained annotation information as the filtered annotation information.
[0082] Optionally, the target obstacle information determination module is specifically used to: merge the first obstacle information and the second obstacle information to obtain merged obstacle information; wherein, the first obstacle information includes the second center point location information, second size information, second category information, and confidence information corresponding to the second category information of the first obstacle; the second obstacle information includes the third center point location information, third size information, third category information, and confidence information corresponding to the third category information of the second obstacle; and determine the target obstacle information based on the filtered annotation information and the merged obstacle information.
[0083] Optionally, the target obstacle information determination module is further configured to: determine the intersection obstacle information between the first obstacle information and the second obstacle information; determine the area of the intersection obstacle and the area of the first obstacle based on the second size information; determine the area of the second obstacle based on the third size information; determine the intersection area ratio information based on the area of the intersection obstacle, the area of the first obstacle, and the area of the second obstacle; compare the intersection area ratio information with a set threshold to obtain a third comparison result; wherein the set threshold includes a first set threshold and a second set threshold; if the third comparison result is that the intersection area ratio information is equal to the first set threshold or less than or equal to the second set threshold, then the first obstacle information is used as merged obstacle information; if the third comparison result is that the intersection area ratio information is greater than the second set threshold, then the first obstacle information and the second obstacle information are merged to obtain merged obstacle information.
[0084] Optionally, the target obstacle information determination module is further configured to: obtain first category information corresponding to the filtered annotation information; obtain second category information corresponding to the merged obstacle information; group obstacles of the same category as the first category information and the second category information into one category to obtain multiple categories of obstacles; for each category of obstacle, determine the intersection area ratio information between the labeled obstacle and the merged obstacle corresponding to the filtered annotation information; determine whether the labeled obstacle and the merged obstacle match based on the intersection area ratio information; if the labeled obstacle and the merged obstacle match, then the labeled obstacle and the merged obstacle are taken as a matching pair, and the matching pair is taken as the target obstacle information.
[0085] Optionally, the confidence threshold module is specifically used to: for each obstacle category, sort the merged obstacle information among the multiple target obstacle information of the obstacle category; and determine the confidence threshold of the obstacle category based on the confidence level corresponding to the sorted multiple merged obstacle information.
[0086] The confidence threshold determination device provided in this disclosure can execute the confidence threshold determination method provided in any embodiment of this disclosure, and has the corresponding functional modules and beneficial effects of the execution method.
[0087] It is worth noting that the various units and modules included in the above-mentioned device are only divided according to functional logic, but are not limited to the above division, as long as the corresponding functions can be realized; in addition, the specific names of each functional unit are only for easy differentiation and are not used to limit the protection scope of the embodiments of this disclosure.
[0088] Figure 5This is a schematic diagram of the structure of an electronic device provided in an embodiment of this disclosure. Reference is made below. Figure 5 It illustrates an electronic device suitable for implementing embodiments of the present disclosure (e.g., Figure 5 The diagram below shows the structure of the terminal device or server 500. The terminal device in this embodiment may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), and vehicle terminals (e.g., vehicle navigation terminals), as well as fixed terminals such as digital TVs and desktop computers. Figure 5 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments disclosed herein.
[0089] like Figure 5 As shown, electronic device 500 may include a processing unit (e.g., central processing unit, graphics processor, etc.) 501, which can perform various appropriate actions and processes according to a program stored in read-only memory (ROM) 502 or a program loaded from storage device 508 into random access memory (RAM) 503. The RAM 503 also stores various programs and data required for the operation of electronic device 500. The processing unit 501, ROM 502, and RAM 503 are interconnected via bus 504. An edit / output (I / O) interface 505 is also connected to bus 504.
[0090] Typically, the following devices can be connected to I / O interface 505: input devices 506 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 507 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 508 including, for example, magnetic tapes, hard disks, etc.; and communication devices 509. Communication device 509 allows electronic device 500 to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 5 An electronic device 500 with various devices is shown; however, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively.
[0091] In particular, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device 509, or installed from a storage device 508, or installed from a ROM 502. When the computer program is executed by the processing device 501, it performs the functions defined in the methods of embodiments of this disclosure.
[0092] The names of messages or information exchanged between multiple devices in the embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.
[0093] The electronic device provided in this embodiment and the confidence threshold determination method provided in the above embodiments belong to the same inventive concept. Technical details not described in detail in this embodiment can be found in the above embodiments, and this embodiment has the same beneficial effects as the above embodiments.
[0094] This disclosure provides a computer storage medium storing a computer program that, when executed by a processor, implements the confidence threshold determination method provided in the above embodiments.
[0095] It should be noted that the computer-readable medium described in this disclosure can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this disclosure, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in connection with an instruction execution system, apparatus, or device. In this disclosure, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.
[0096] In some implementations, clients and servers can communicate using any currently known or future-developed network protocol such as HTTP (Hypertext Transfer Protocol) and can interconnect with digital data communication (e.g., communication networks) of any form or medium. Examples of communication networks include local area networks (“LANs”), wide area networks (“WANs”), the Internet (e.g., the Internet of Things), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future-developed networks.
[0097] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device.
[0098] The aforementioned computer-readable medium carries one or more programs. When the electronic device executes one or more of these programs, the electronic device causes the following actions: acquiring training sample information; wherein the training sample information includes vehicle surrounding environment information and corresponding annotation information; the annotation information includes the location information of a first center point, first size information, first category information, and obstacle serial number of an obstacle; the vehicle surrounding environment information is point cloud data acquired by vehicle-mounted radar within a set time period; filtering the annotation information based on the first center point location information and / or the first size information to obtain filtered annotation information; identifying the vehicle surrounding environment information based on a set detection model and a set segmentation model to obtain first obstacle information and second obstacle information; determining target obstacle information based on the filtered annotation information, the first obstacle information, and the second obstacle information; and determining a confidence threshold for each obstacle category based on the confidence level corresponding to the target obstacle information.
[0099] Computer program code for performing the operations of this disclosure can be written in one or more programming languages or a combination thereof, including but not limited to object-oriented programming languages such as Java, Smalltalk, and C++, as well as conventional procedural programming languages such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).
[0100] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.
[0101] The units described in the embodiments of this disclosure can be implemented in software or in hardware. The name of a unit does not necessarily limit the unit itself; for example, the first acquisition unit can also be described as "a unit that acquires at least two Internet Protocol addresses".
[0102] The functions described above in this document can be performed, at least in part, by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: Field Programmable Gate Arrays (FPGAs), Application-Specific Integrated Circuits (ASICs), Application Standard Products (ASSPs), System-on-Chip (SoCs), Complex Programmable Logic Devices (CPLDs), and so on.
[0103] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.
[0104] The above description is merely a preferred embodiment of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features disclosed in this disclosure that have similar functions.
[0105] Furthermore, while the operations are described in a specific order, this should not be construed as requiring these operations to be performed in the specific order shown or in a sequential order. In certain environments, multitasking and parallel processing may be advantageous. Similarly, while several specific implementation details are included in the above discussion, these should not be construed as limiting the scope of this disclosure. Certain features described in the context of individual embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented individually or in any suitable sub-combination in multiple embodiments.
[0106] Although the subject matter has been described using language specific to structural features and / or methodological logic, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. Rather, the specific features and actions described above are merely illustrative examples of implementing the claims.
Claims
1. A method for determining a confidence threshold, characterized in that, include: Acquire training sample information; wherein, the training sample information includes vehicle surrounding environment information and corresponding annotation information; the annotation information includes the first center point location information, first size information, first category information and obstacle number of the marked obstacle; the vehicle surrounding environment information is point cloud data within a set time period obtained by vehicle-mounted radar; The annotation information is filtered based on the first center point position information and / or the first size information to obtain the filtered annotation information; The vehicle's surrounding environment information is identified based on a set detection model and a set segmentation model to obtain first obstacle information and second obstacle information. The target obstacle information is determined based on the filtered annotation information, the first obstacle information, and the second obstacle information; Determine the confidence threshold for each obstacle category based on the confidence level corresponding to the target obstacle information; The step of filtering the annotation information based on the first center point position information and / or the first size information to obtain filtered annotation information includes: Determine the orientation information based on the first center location information; The annotation information is filtered based on orientation information to obtain filtered annotation information; wherein, the annotation information is obtained by annotating the environmental information around the vehicle at set time intervals; and / or, The obstacle area ratio at any adjacent time within the set time period is determined based on the first size information; The annotation information is filtered based on the obstacle area ratio to obtain filtered annotation information; The step of determining the orientation information based on the first center position information includes: For each obstacle, the position information of the first center point of the obstacle at multiple consecutive moments is obtained; wherein, the position information of the first center point at multiple consecutive moments includes the position information of the first center point at any three consecutive moments within a set time period; Connect the multiple first center point position information sequentially in chronological order to obtain a broken line formed by the center points; Determine the included angle of the broken line; The included angle is determined as orientation information; The step of filtering the annotation information based on orientation information to obtain filtered annotation information includes: The orientation information is compared with a set orientation angle threshold to obtain a first comparison result; If the first comparison result indicates that the orientation information is greater than the set orientation angle threshold, then the labeling information of the obstacle at the consecutive multiple time points will be deleted; If the first comparison result is that the orientation information is less than or equal to the set orientation angle threshold, then the labeling information of the obstacle at the consecutive multiple times is retained; The retained annotation information will be used as the filtered annotation information.
2. The method according to claim 1, characterized in that, Determining the obstacle area ratio at any adjacent time within the set time period based on the first size information includes: Based on the first size information, determine the area of the obstacle at any two adjacent moments within the set time period; The ratio of the area of the smaller obstacle to the area of the larger obstacle is called the obstacle area ratio.
3. The method according to claim 2, characterized in that, The annotation information is filtered based on the obstacle area ratio to obtain filtered annotation information, including: Set area thresholds for different obstacle types; The obstacle type is determined based on the first category information of the marked obstacles; The obstacle area ratio is compared with a set area threshold corresponding to the obstacle type to obtain a second comparison result; If the second comparison result is that the obstacle area ratio is greater than the set area threshold corresponding to the obstacle type, then the annotation information corresponding to the adjacent time moments is retained; If the second comparison result is that the obstacle area ratio is less than or equal to the set area threshold corresponding to the obstacle type, then the annotation information corresponding to the adjacent time moment is deleted from the annotation information; The retained annotation information will be used as the filtered annotation information.
4. The method according to claim 1, characterized in that, Determining target obstacle information based on the filtered annotation information, the first obstacle information, and the second obstacle information includes: The first obstacle information and the second obstacle information are merged to obtain merged obstacle information; wherein, the first obstacle information includes the second center point location information, second size information, second category information, and confidence information corresponding to the second category information of the first obstacle; the second obstacle information includes the third center point location information, third size information, third category information, and confidence information corresponding to the third category information of the second obstacle; The target obstacle information is determined based on the filtered annotation information and the merged obstacle information.
5. The method according to claim 4, characterized in that, The first obstacle information and the second obstacle information are merged to obtain merged obstacle information, including: Determine the intersection obstacle information between the first obstacle information and the second obstacle information; The areas of the intersection obstacles and the first obstacle are determined based on the second size information; The area of the second obstacle is determined based on the third size information; The intersection area ratio information is determined based on the area of the intersection obstacle, the area of the first obstacle, and the area of the second obstacle; The intersection area ratio information is compared with a set threshold to obtain a third comparison result; wherein the set threshold includes a first set threshold and a second set threshold; If the third comparison result is that the intersection area ratio information is equal to the first set threshold or less than or equal to the second set threshold, then the first obstacle information is used as the merged obstacle information; If the third comparison result is that the intersection area ratio information is greater than the second set threshold, then the first obstacle information and the second obstacle information are merged to obtain merged obstacle information.
6. The method according to claim 4, characterized in that, The target obstacle information is determined based on the filtered annotation information and the merged obstacle information, including: Obtain the first category information corresponding to the filtered annotation information; Obtain the second category information corresponding to the merged obstacle information; Obstacles corresponding to the same category in the first category information and the second category information are grouped into one category to obtain multiple categories of obstacles; For each type of obstacle, determine the intersection area ratio between the labeled obstacle and the merged obstacle corresponding to the filtered labeling information; Based on the intersection area ratio information, determine whether the labeled obstacle matches the merged obstacle; If the marked obstacle matches the merged obstacle, then the marked obstacle and the merged obstacle are considered a matching pair, and the matching pair is used as the target obstacle information.
7. The method according to claim 6, characterized in that, Determine the confidence threshold for each obstacle category based on the confidence level corresponding to the target obstacle information, including: For each obstacle category, the merged obstacle information from multiple target obstacle information for that obstacle category is sorted; The confidence threshold of the obstacle category is determined based on the confidence scores corresponding to the sorted and merged obstacle information.
8. A device for determining a confidence threshold, characterized in that, include: The training sample information acquisition module is used to acquire training sample information; wherein, the training sample information includes vehicle surrounding environment information and corresponding annotation information; the annotation information includes the first center point location information, first size information, first category information and obstacle number of the marked obstacle; the vehicle surrounding environment information is point cloud data within a set time period obtained by vehicle-mounted radar; The annotation information filtering module is used to filter the annotation information based on the first center point position information and / or the first size information to obtain the filtered annotation information; The identification module is used to identify the environmental information around the vehicle based on a set detection model and a set segmentation model, respectively, to obtain first obstacle information and second obstacle information; The target obstacle information determination module is used to determine target obstacle information based on the filtered annotation information, the first obstacle information, and the second obstacle information; The confidence threshold module is used to determine the confidence threshold for each obstacle category based on the confidence level corresponding to the target obstacle information. The annotation information filtering module is specifically used for: Determine the orientation information based on the first center location information; The annotation information is filtered based on orientation information to obtain filtered annotation information; wherein, the annotation information is obtained by annotating the environmental information around the vehicle at set time intervals; and / or, Based on the first size information, determine the obstacle area ratio at any adjacent time within the set time period; filter the annotation information based on the obstacle area ratio to obtain filtered annotation information; The annotation information filtering module is also used for: For each obstacle, the position information of the first center point of the obstacle at multiple consecutive moments is obtained; wherein, the position information of the first center point at multiple consecutive moments includes the position information of the first center point at any three consecutive moments within a set time period; Connect the multiple first center point position information sequentially in chronological order to obtain a broken line formed by the center points; Determine the included angle of the broken line; The included angle is determined as orientation information; The annotation information filtering module is also used for: The orientation information is compared with a set orientation angle threshold to obtain a first comparison result; If the first comparison result indicates that the orientation information is greater than the set orientation angle threshold, then the labeling information of the obstacle at the consecutive multiple time points will be deleted; If the first comparison result is that the orientation information is less than or equal to the set orientation angle threshold, then the labeling information of the obstacle at the consecutive multiple times is retained; The retained annotation information will be used as the filtered annotation information.
9. An electronic device, characterized in that, The electronic device includes: One or more processors; Storage device for storing one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors implement the method for determining the confidence threshold as described in any one of claims 1-7.
10. A storage medium comprising computer-executable instructions, which, when executed by a computer processor, are used to perform the method for determining a confidence threshold as described in any one of claims 1-7.