Image dataset processing method and apparatus, and device and storage medium

Abstract

Disclosed in the present application are an image dataset processing method and apparatus, and a device and a storage medium. The method comprises: acquiring an initial autonomous driving image dataset; inputting the initial image dataset into a first autonomous driving detection model and a second autonomous driving detection model, respectively, so as to obtain a first image detection result and a second image detection result, respectively; comparing the first image detection result with the second image detection result, so as to obtain a comparison result; determining a comparison type on the basis of the comparison result; on the basis of the comparison type, determining evaluation scores of a plurality of initial autonomous driving images; and on the basis of the evaluation scores of the plurality of initial autonomous driving images, screening out a target autonomous driving image dataset from the initial autonomous driving image dataset, and adjusting labels in the target autonomous driving image dataset. The embodiments of the present application can improve the accuracy of image dataset processing.

Description

Image data set processing method, device, equipment and storage medium

[0001] This application claims priority to the Chinese patent application filed with the China Patent Office on July 26, 2023, with application number 202310922726.X. The entire contents of the above application are incorporated by reference into this application. Technical Field

[0002] The embodiments of the present application relate to the field of image processing technology, and in particular to an image data set processing method, apparatus, device, and storage medium. Background Art

[0003] As we all know, the development of autonomous driving is inextricably linked to advancements in artificial intelligence (AI). Data, computing power, and algorithms, the three key elements of AI, significantly impact the maturity of autonomous driving technology. Of these three elements, data plays a crucial role. Only through extensive training with data can AI discern patterns. In practical AI applications, if scenarios never before seen in the training set are encountered, the AI ​​essentially makes blind guesses, leading to prediction errors. Therefore, reliable, high-quality data plays an extremely important role in the development of autonomous driving.

[0004] Although high-level autonomous driving test vehicles collect terabytes (TB) of data daily and require petabytes (PB) of storage space, less than 5% of this data is valuable for autonomous driving training. Finding the most valuable difficult examples within this massive amount of data and prioritizing their labeling to accelerate autonomous driving model iteration and algorithm mass production is a pressing issue.

[0005] Summary of the Invention

[0006] The embodiments of the present application provide a method, apparatus, device, and storage medium for processing an image dataset, which can improve the accuracy of image dataset processing.

[0007] In a first aspect, an embodiment of the present application provides an image dataset processing method, including: obtaining an initial autonomous driving image dataset; wherein the initial autonomous driving image dataset includes multiple initial autonomous driving images and corresponding annotations; inputting the initial image dataset into a first autonomous driving detection model and a second autonomous driving detection model, respectively, to obtain a first image detection result and a second image detection result, respectively; comparing the first image detection result and the second image detection result to obtain a comparison result; determining a comparison type based on the comparison result; wherein the first image detection result is used as a benchmark detection result; the comparison type includes true positive examples, false positive examples, and false negative examples; determining evaluation scores of the multiple initial autonomous driving images based on the comparison type; screening out a target autonomous driving image dataset from the initial autonomous driving image dataset based on the evaluation scores of the multiple initial autonomous driving images, and adjusting the annotations in the target autonomous driving image dataset.

[0008] In a second aspect, an embodiment of the present application further provides an image dataset processing device, comprising: an initial autonomous driving image dataset acquisition module, for acquiring an initial autonomous driving image dataset; wherein the initial autonomous driving image dataset includes multiple initial autonomous driving images and corresponding annotations; an image detection result acquisition module, for inputting the initial image dataset into a first autonomous driving detection model and a second autonomous driving detection model, respectively, to obtain a first image detection result and a second image detection result, respectively; an image detection result comparison module, for comparing the first image detection result and the second image detection result to obtain a comparison result; a comparison type determination module, for determining a comparison type based on the comparison result; wherein the first image detection result serves as a benchmark detection result; the comparison type includes true positive examples, false positive examples, and false negative examples; an evaluation score determination module, for determining the evaluation scores of the multiple initial autonomous driving images based on the comparison type; a target autonomous driving image dataset screening module, for screening out a target autonomous driving image dataset from the initial autonomous driving image dataset based on the evaluation scores of the multiple initial autonomous driving images, and adjusting the annotations in the target autonomous driving image dataset.

[0009] In a third aspect, an embodiment of the present application further provides an electronic device, comprising:

[0010] one or more processors;

[0011] a storage device for storing one or more programs,

[0012] When the one or more programs are executed by the one or more processors, the one or more processors implement the image dataset processing method as described in the embodiment of the present application.

[0013] In a fourth aspect, an embodiment of the present application further provides a storage medium comprising computer-executable instructions, which, when executed by a computer processor, are used to execute the image dataset processing method as described in the embodiment of the present application.

[0014] The technical solution disclosed in this embodiment includes obtaining an initial autonomous driving image dataset; wherein the initial autonomous driving image dataset includes multiple initial autonomous driving images and corresponding annotations; inputting the initial image dataset into a first autonomous driving detection model and a second autonomous driving detection model, respectively, to obtain a first image detection result and a second image detection result; comparing the first image detection result and the second image detection result to obtain a comparison result; determining a comparison type based on the comparison result; wherein the first image detection result serves as a reference detection result; the comparison types include true positive examples, false positive examples, and false negative examples; determining evaluation scores for the multiple initial autonomous driving images based on the comparison type; screening a target autonomous driving image dataset from the initial autonomous driving image dataset based on the evaluation scores of the multiple initial autonomous driving images, and adjusting the annotations in the target autonomous driving image dataset. In this embodiment of the application, the accuracy of image dataset processing can be improved by determining the comparison type based on the comparison result between the first image detection result and the second image detection result, determining the evaluation scores of the initial autonomous driving images based on the comparison type, and screening the target autonomous driving image dataset based on the evaluation scores of the initial autonomous driving images. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] The above and other features, advantages, and aspects of the various embodiments of the present application will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and that the originals and elements are not necessarily drawn to scale.

[0016] FIG1 is a flow chart of a method for processing an image data set provided in an embodiment of the present application;

[0017] FIG2 is a flowchart of another image data set processing method provided in an embodiment of the present application;

[0018] FIG3 is a schematic diagram of the effect of the IoU ratio and the IoU ratio evaluation score provided in an embodiment of the present application;

[0019] FIG4 is a schematic diagram showing the effect between a confidence level and a second evaluation score provided in an embodiment of the present application;

[0020] FIG5 is a schematic diagram showing the effect between a confidence level and a first evaluation score provided in an embodiment of the present application;

[0021] FIG6 is a schematic diagram showing the effect between an area and a second evaluation score provided in an embodiment of the present application;

[0022] FIG7 is a schematic diagram showing the effect between an area and a first evaluation score provided in an embodiment of the present application;

[0023] FIG8 is a schematic diagram showing the effect of the first image detection result provided by an embodiment of the present application;

[0024] FIG9 is a schematic diagram showing the effect of the second image detection result provided by an embodiment of the present application;

[0025] FIG10 is a schematic diagram of the structure of an image data set processing device provided in an embodiment of the present application;

[0026] FIG11 is a schematic structural diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0027] The following describes embodiments of the present application in more detail with reference to the accompanying drawings. Although certain embodiments of the present application are shown in the accompanying drawings, it should be understood that the present application can be implemented in various forms and should not be construed as being limited to the embodiments described herein. Instead, these embodiments are provided to provide a more thorough and complete understanding of the present application. It should be understood that the drawings and embodiments of the present application are for illustrative purposes only and are not intended to limit the scope of protection of the present application.

[0028] It should be understood that the various steps described in the method embodiments of the present application can be performed in different orders and / or in parallel. In addition, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present application is not limited in this respect.

[0029] As used herein, the term "including" and its variations are open-ended, i.e., "including but not limited to." The term "based on" means "based, at least in part, on." The term "one embodiment" means "at least one embodiment," the term "another embodiment" means "at least one additional embodiment," and the term "some embodiments" means "at least some embodiments." Other terms are defined in the following description.

[0030] It should be noted that the concepts of "first" and "second" mentioned in this application are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0031] It should be noted that the modifications of "one" and "multiple" mentioned in this application are illustrative rather than restrictive. Those skilled in the art should understand that unless otherwise clearly indicated in the context, they should be understood as "one or more".

[0032] It is understandable that the data involved in this technical solution (including but not limited to the data itself, the acquisition or use of the data) must comply with the requirements of relevant laws, regulations and relevant provisions.

[0033] Figure 1 is a flow chart of an image dataset processing method provided in an embodiment of the present application; this embodiment is applicable to the case of screening a target autonomous driving image dataset from an initial autonomous driving image dataset. It should be noted that this embodiment is not limited to the autonomous driving front-view camera scene target detection task, but can also be other scene tasks. When switching scene tasks, the relevant configuration information is modified according to the model capabilities and project requirements. In this embodiment, the case of screening a target autonomous driving image dataset from an initial autonomous driving image dataset can also be understood as a data mining case, in order to find the most valuable difficult samples in the massive data, and give priority to labeling them to accelerate model iteration. The method can be executed by an image dataset processing device, and specifically includes the following steps:

[0034] S110: Obtain an initial autonomous driving image dataset.

[0035] The initial autonomous driving image dataset includes multiple initial autonomous driving images and their corresponding annotations. The annotations include information such as the detection box, category, confidence level, and area of ​​the initial autonomous driving images. When this embodiment is used for 3D scene tasks, the area can be a 3D target volume or projected area.

[0036] The initial autonomous driving images can be collected by a high-level autonomous driving test vehicle. The autonomous driving image dataset may or may not include annotations, and this embodiment does not impose any restrictions on this.

[0037] S120. Input the initial image data set into the first autonomous driving detection model and the second autonomous driving detection model, respectively, to obtain a first image detection result and a second image detection result, respectively.

[0038] The first autonomous driving detection model can be a server-side model, which has sufficient computing power and can be processed offline, and can be called a large-scale model. The second autonomous driving detection model can be an edge-side model, which has limited computing power and high real-time requirements, and can be called a small-scale model.

[0039] Optionally, each initial autonomous driving image includes at least one target object; the first image detection result includes first detection frame information of at least one target object, a first category corresponding to at least one target object, a first confidence level of the first category, and a first area corresponding to the first detection frame information; the second image detection result includes second detection frame information of at least one target object, a second category corresponding to at least one target object, a second confidence level corresponding to the second category, and a second area corresponding to the second detection frame information.

[0040] The target object may be a vehicle, a railing, a pedestrian, a house, etc., which is not limited in this embodiment. The detection frame information may include the length, width, and height of the detection frame.

[0041] S130 : Compare the first image detection result and the second image detection result to obtain a comparison result.

[0042] In this embodiment, the first image detection result and the second image detection result may be compared to obtain a corresponding comparison result, where the comparison result includes a consistent comparison and an inconsistent comparison.

[0043] Optionally, the comparison result includes a category comparison result, and comparing the first image detection result and the second image detection result to obtain the comparison result includes: comparing the first category and the second category to obtain a category comparison result; wherein the category comparison result includes category comparison consistency and category comparison inconsistency.

[0044] In this embodiment, a category comparison result can be obtained by comparing the first category and the second category, and the comparison type is determined based on the category comparison result.

[0045] S140: Determine the comparison type according to the comparison result.

[0046] The first image detection result is used as the baseline detection result; the comparison types include true positive (TP), false positive (FP), and false negative (FN). For example, if the category comparison is consistent, the comparison type is true positive; if the category comparison is inconsistent, the comparison type is false positive or false negative.

[0047] Optionally, the comparison type is determined according to the comparison result, including: determining the intersection-and-union (IoU) based on the first detection frame information and the second detection frame information; determining the first IoU setting value based on the first area and / or the second area; if the IoU is greater than or equal to the first IoU setting value, and the category comparison result is a category comparison consistency, then the comparison type is determined to be a true positive example; if the IoU is less than the first IoU setting value and or the category comparison result is a category comparison inconsistency, then the comparison type is determined to be a false positive example and a false negative example; or; if the first image detection result includes the first detection frame information of the target object, and the second image detection result does not include the second detection frame information of the target object, then the comparison type is determined to be a false negative example; if the first image detection result does not include the first detection frame information of the target object, and the second image detection result includes the second detection frame information of the target object, then the comparison type is determined to be a false positive example.

[0048] In this embodiment, the intersection-in-two ratio can be determined based on the first detection frame information and the second detection frame information; the size of the target object is determined based on the first area and / or the second area. If the first area and / or the second area is less than 32*32, the target object is a small target, and the corresponding first intersection-in-two ratio setting value can be 0.3; if the first area and / or the second area is greater than or equal to 32*32, the target object is a normal target, and the corresponding first intersection-in-two ratio setting value can be 0.5.

[0049] If the intersection-in-union ratio is greater than or equal to the first intersection-in-union ratio setting value, and the category comparison result is a category comparison consistency, the comparison type is determined to be a true positive example. If the intersection-in-union ratio is less than the first intersection-in-union ratio setting value and or the category comparison result is a category comparison inconsistency, the comparison type is determined to be a false positive example and a false negative example. Or; if the first image detection result includes the first detection frame information of the target object, and the second image detection result does not include the second detection frame information of the target object, the comparison type is determined to be a false negative example, that is, the first image detection result has an extra detection frame relative to the second image detection result, and the corresponding comparison types are all false negative examples. If the first image detection result does not include the first detection frame information of the target object, and the second image detection result includes the second detection frame information of the target object, the comparison type is determined to be a false positive example, that is, the second image detection result has an extra detection frame relative to the first image detection result, and the corresponding comparison types are all false positive examples.

[0050] In this embodiment, the comparison type can be accurately determined by determining the comparison type based on the comparison results and the intersection-and-union ratio, or by determining whether the first image detection result and the second image detection result include detection frame information of the same target object.

[0051] S150. Determine evaluation scores of the multiple initial autonomous driving images according to the comparison type.

[0052] In this embodiment, the evaluation score of each initial autonomous driving image is determined as follows: the evaluation scores of all target objects in the initial autonomous driving image are determined according to the comparison type, the evaluation scores of the target objects include the intersection-over-union score, the confidence score, the area score and the category score, and the evaluation score of the initial autonomous driving image is obtained according to the evaluation scores of all target objects and the scene score of the initial autonomous driving image, thereby obtaining the evaluation scores of all initial autonomous driving images.

[0053] S160. Filter out a target autonomous driving image dataset from the initial autonomous driving image dataset based on the evaluation scores of the multiple initial autonomous driving images, and adjust the annotations in the target autonomous driving image dataset.

[0054] In this embodiment, a target autonomous driving image dataset can be selected from the initial autonomous driving image dataset based on the evaluation scores of all initial autonomous driving images. The target autonomous driving image dataset can be understood as a dataset that significantly impacts the detection accuracy of the second autonomous driving detection model (or other autonomous driving detection models), or as a dataset with high target value. By adjusting the annotations in the target autonomous driving image dataset, the detection accuracy of the autonomous driving detection model can be improved.

[0055] In this embodiment, it is not limited to the comparison between two models, but can also be a pairwise comparison between multiple models. The evaluation score of an initial autonomous driving image can obtain multiple value scores, and the maximum score is taken or other fusion score algorithms are used to obtain the final value score.

[0056] In this embodiment, only one model may be used, but forward calculations in two different modes are required, such as a TTA (Test Time Augmentation) mode and a normal mode, to obtain two different sets of detection results.

[0057] The technical solution disclosed in this embodiment includes obtaining an initial autonomous driving image dataset; wherein the initial autonomous driving image dataset includes multiple initial autonomous driving images and corresponding annotations; inputting the initial image dataset into a first autonomous driving detection model and a second autonomous driving detection model, respectively, to obtain a first image detection result and a second image detection result; comparing the first image detection result and the second image detection result to obtain a comparison result; determining a comparison type based on the comparison result; wherein the first image detection result serves as a reference detection result; the comparison types include true positive examples, false positive examples, and false negative examples; determining evaluation scores for the multiple initial autonomous driving images based on the comparison type; screening a target autonomous driving image dataset from the initial autonomous driving image dataset based on the evaluation scores of the multiple initial autonomous driving images, and adjusting the annotations in the target autonomous driving image dataset. In this embodiment of the application, the accuracy of image dataset processing can be improved by determining the comparison type based on the comparison result between the first image detection result and the second image detection result, determining the evaluation scores of the initial autonomous driving images based on the comparison type, and screening the target autonomous driving image dataset based on the evaluation scores of the initial autonomous driving images.

[0058] FIG2 is a flowchart of another method for processing an image dataset provided by an embodiment of the present application. The present embodiment is a refinement of the above-mentioned invention embodiment. Referring to FIG2 , the method provided by the present embodiment specifically includes the following steps:

[0059] S201. Obtain an initial autonomous driving image dataset.

[0060] S202: Input the initial image data set into the first autonomous driving detection model and the second autonomous driving detection model, respectively, to obtain a first image detection result and a second image detection result.

[0061] S203 : Compare the first image detection result and the second image detection result to obtain a comparison result.

[0062] S204: Determine the comparison type according to the comparison result.

[0063] S205 . For any evaluation score of the initial autonomous driving image, determine the evaluation score of the target object in the initial autonomous driving image according to the comparison type.

[0064] The target object evaluation score includes its intersection-over-union (IoU) score, confidence score, area score, and category score. In this embodiment, different comparison types result in different target object evaluation scores. Depending on the comparison type, the evaluation scores of all or some of the target objects in the initial autonomous driving image can be determined.

[0065] Optionally, determining the evaluation score of the target object in the initial autonomous driving image based on the comparison type includes: determining at least one of the intersection-over-union score, confidence score, and area score of the target object based on the comparison type; obtaining a preset category score corresponding to the target object; and determining the evaluation score of the target object based on at least one of the intersection-over-union score, confidence score, area score, and category score.

[0066] In this embodiment, the evaluation score of each target object can be obtained as follows: the intersection-over-union (IoU) score, confidence score, and area score of the target object are obtained according to the comparison type, and the pre-set category score corresponding to the target object is obtained. The IoU score, confidence score, area score, and category score are multiplied or weighted averaged, and the result of the multiplication or weighted average is used as the evaluation score of the target object. In this embodiment, the elements for determining the evaluation score of the target object are not limited to the IoU score, confidence score, area score, and category score, and corresponding elements can also be added or subtracted.

[0067] It should be noted that each target object category is pre-assigned a corresponding category weight, also known as a category score, which ranges from [0, 1]. After predicting the target object's category, the corresponding category score is obtained. The category weights are based on the distribution information of the initial autonomous driving image dataset, assigning higher category weights to the long-tailed categories. The model's evaluation results on the test set can also be incorporated to increase the weights for categories with poor accuracy.

[0068] In this embodiment, the evaluation score of the target object is determined by using at least one of the intersection-over-union score, the confidence score, the area score, and the category score, so that the evaluation score of the target object can be accurately determined.

[0069] Optionally, determining the intersection-and-union score of the target object based on the comparison type includes: determining a second intersection-and-union setting value based on the first area and / or the second area; and determining an intersection-and-union evaluation score based on the comparison type, the first intersection-and-union setting value, the second intersection-and-union setting value, and the intersection-and-union.

[0070] The second IoU setting value is greater than the first IoU setting value; the second IoU setting value can be understood as a maximum IoU threshold, and the first IoU setting value can be understood as a minimum IoU threshold. In this embodiment, the size of the target object is determined based on the first area and / or the second area. If the first area and / or the second area is less than 32*32, the target object is a small target, and the corresponding second IoU setting value can be 0.7. If the first area and / or the second area is greater than or equal to 32*32, the target object is a normal target, and the corresponding first IoU setting value can be 0.9.

[0071] For example, the intersection-over-union threshold formula is as follows:

[0072] Where iou_thresh represents the IoU threshold. When the target object is a small object, the first IoU setting value can be 0.3, and the second IoU setting value can be 0.7. When the target object is a normal object, the first IoU setting value can be 0.5, and the second IoU setting value can be 0.9.

[0073] Specifically, if the comparison type is a false positive or false negative, and the IoU is less than the first IoU setting value, the IoU evaluation score can be directly obtained. If the comparison type is a true positive, and the IoU is greater than or equal to the second IoU setting value, the IoU evaluation score can be directly obtained. If the comparison type is a true positive, and the IoU falls within the interval formed by the first IoU setting value and the second IoU setting value, the IoU evaluation score can be determined based on the first IoU setting value, the second IoU setting value, and the IoU.

[0074] In this embodiment, the IoU evaluation score can be accurately determined by comparing the comparison type, the first IoU setting value, the second IoU setting value, and the IoU determining the IoU evaluation score.

[0075] Optionally, the intersection-and-union evaluation score is determined based on the comparison type, the first intersection-and-union setting value, the second intersection-and-union setting value, and the intersection-and-union, including: if the comparison type is a false positive or a false negative, and the intersection-and-union is less than the first intersection-and-union setting value, then the intersection-and-union evaluation score is the first setting intersection-and-union evaluation score; if the comparison type is a true positive, and the intersection-and-union is greater than or equal to the second intersection-and-union setting value, then the intersection-and-union evaluation score is the second setting intersection-and-union evaluation score; if the comparison type is a true positive, and the intersection-and-union falls within the interval formed by the first intersection-and-union setting value and the second intersection-and-union setting value, then the first intersection-and-union evaluation score is determined based on the second intersection-and-union setting value and the intersection-and-union; the second intersection-and-union evaluation score is determined based on the second intersection-and-union setting value and the first intersection-and-union setting value; and the intersection-and-union evaluation score is determined based on the first intersection-and-union evaluation score and the second intersection-and-union evaluation score.

[0076] For example, the intersection-over-union evaluation score formula is as follows:

[0077] Where iou_score is the intersection-over-union (IoU) score, min_iou is the first IoU setting value, max_iou is the second IoU setting value, and iou is the IoU. The first IoU setting score is 1, and the second IoU setting score is 0.

[0078] Among them, when the comparison type is a false positive or false negative example, and the intersection and union ratio is less than the first intersection and union ratio setting value, the value is the highest, and the intersection and union ratio evaluation score is 1; when the comparison type is a true positive example, and the intersection and union ratio is greater than or equal to the second intersection and union ratio setting value, it can be considered that the model does not need to pay attention to small marginal differences, has no value, and the intersection and union ratio evaluation score is 0; when the comparison type is a true positive example, and the intersection and union ratio falls within the interval formed by the first intersection and union ratio setting value and the second intersection and union ratio setting value, the lower the intersection and union ratio, the higher the value, the first intersection and union ratio evaluation score is max_iou-iou, the second intersection and union ratio evaluation score is max_iou-min_iou, and the intersection and union ratio evaluation score is

[0079] For example, Figure 3 illustrates the effects of the IoU and IoU evaluation scores provided in an embodiment of the present application. As shown in Figure 3 , the abscissa represents the IoU ratio (iou), and the ordinate represents the IoU evaluation score (iou_score). It can be seen that for small or normal targets, the IoU ratio and the IoU evaluation score are inversely proportional.

[0080] It should be noted that the reason for treating target objects differently according to their size is that the absolute coordinate deviation (measured in pixels) of the same target position has a greater impact on the IOU value of small targets during registration. Therefore, the IOU value of small targets will not be very high, which will result in higher evaluation scores for more small targets. This is inconsistent with the concept that "larger targets are more valuable".

[0081] In this embodiment, when the comparison type is a false positive example or a false negative example, and the intersection-and-union ratio is less than the first intersection-and-union ratio setting value, the first set intersection-and-union ratio evaluation score is used as the intersection-and-union ratio evaluation score; when the comparison type is a true positive example, and the intersection-and-union ratio is greater than or equal to the second intersection-and-union ratio setting value, the second set intersection-and-union ratio evaluation score is used as the intersection-and-union ratio evaluation score; when the comparison type is a true positive example, and the intersection-and-union ratio falls within the interval formed by the first intersection-and-union ratio setting value and the second intersection-and-union ratio setting value, the intersection-and-union ratio evaluation score is determined according to the first intersection-and-union ratio evaluation score and the second intersection-and-union ratio evaluation score, so that the intersection-and-union ratio evaluation score can be accurately determined.

[0082] Optionally, the confidence score of the target object is determined according to the comparison type, including: if the comparison type is a false positive example, determining the first evaluation score of the second confidence level, and using the first evaluation score of the second confidence level as the confidence score of the target object; if the comparison type is a false negative example, determining the first evaluation score of the first confidence level, and using the first evaluation score of the first confidence level as the confidence score of the target object; if the comparison type is a true positive example, determining the second evaluation score of the first confidence level and the second evaluation score of the second confidence level respectively, and determining the confidence score of the target object based on the second evaluation score of the first confidence level and the second evaluation score of the second confidence level.

[0083] Exemplarily, the formula for determining the confidence score of the target object is as follows:

[0084] Among them, matched_conf_score is the confidence score of the target object, conf_score small1 It is the first evaluation score of the second confidence level when the comparison type is a false positive, that is, the confidence score of the target object when the comparison type is a false positive. big1 It is the first evaluation score of the first confidence when the comparison type is a false negative example, that is, the confidence score of the target object when the comparison type is a false negative example. conf_score is the confidence score of the target object when the comparison type is a true positive. big2 The second evaluation score of the first confidence level, conf_score small2 is the second evaluation score of the second confidence level.

[0085] In this embodiment, if the comparison type is a false positive example, the first evaluation score of the second confidence level output by the second autonomous driving detection model is determined, and the first evaluation score of the second confidence level is used as the confidence score of the target object when the comparison type is a false positive example; if the comparison type is a false negative example, the first evaluation score of the first confidence level output by the first autonomous driving detection model is determined, and the first evaluation score of the first confidence level is used as the confidence score of the target object when the comparison type is a false negative example; if the comparison type is a true positive example, the second evaluation score of the first confidence level and the second evaluation score of the second confidence level are respectively determined, and the confidence score of the target object when the comparison type is a true positive example is determined based on the second evaluation score of the first confidence level and the second evaluation score of the second confidence level. In this way, the confidence score of the target object can be accurately determined.

[0086] Optionally, determining a first evaluation score of a first confidence level, or determining a first evaluation score of a second confidence level, includes: obtaining a first confidence threshold and a second confidence threshold; wherein the second confidence threshold is greater than the first confidence threshold; if the first confidence level or the second confidence level is less than the first confidence threshold, then using the first set confidence score as the first evaluation score of the first confidence level or the first evaluation score of the second confidence level; if the first confidence level is greater than or equal to the second confidence threshold, or the second confidence level is greater than or equal to the second confidence threshold, then using the second set confidence score as the first evaluation score of the first confidence level or the first evaluation score of the second confidence level; if the first confidence level falls between the first confidence threshold and If the second confidence falls within the interval formed by the first confidence threshold and the second confidence threshold, a first confidence score is determined based on the first confidence, the second confidence threshold, and the first confidence threshold; a second confidence score is determined based on the second confidence threshold and the first confidence threshold; a first evaluation score of the first confidence is determined based on the first confidence score and the second confidence score; or; if the second confidence falls within the interval formed by the first confidence threshold and the second confidence threshold, a third confidence score is determined based on the second confidence, the second confidence threshold, and the first confidence threshold; a fourth confidence score is determined based on the second confidence threshold and the first confidence threshold; and a first evaluation score of the second confidence is determined based on the third confidence score and the fourth confidence score.

[0087] Exemplarily, when the comparison type is FP, the first evaluation score formula for determining the second confidence level is as follows:

[0088] The second confidence threshold may be 0.5, the first confidence threshold may be 0.3, the first set confidence score may be 0, and the second set confidence score may be 1.

[0089] Among them, when the comparison type is a false positive, conf is the second confidence level; if the second confidence level is less than the first confidence level threshold, the first evaluation score of the second confidence level is 0, which is worthless; if the second confidence level is greater than or equal to the second confidence level threshold, the first evaluation score of the second confidence level is 1, which is the most valuable; if the second confidence level falls within the interval formed by the first confidence level threshold and the second confidence level threshold, then is the first evaluation score of the second confidence level.

[0090] In this embodiment, if the second confidence level is less than the first confidence threshold, the first set confidence score is used as the first evaluation score of the second confidence level; if the second confidence level is greater than or equal to the second confidence threshold, the second set confidence score is used as the first evaluation score of the second confidence level; if the second confidence level falls within the interval formed by the first confidence threshold and the second confidence threshold, the first evaluation score of the second confidence level is determined according to the third confidence score and the fourth confidence score, so that the first evaluation score of the second confidence level can be accurately determined.

[0091] Exemplarily, when the comparison type is FN, the first evaluation score formula for determining the first confidence level is as follows:

[0092] Among them, when the comparison type is a false negative example, conf is the first confidence level; if the first confidence level is less than the first confidence level threshold, the first evaluation score of the first confidence level is 0, which is worthless; if the first confidence level is greater than or equal to the second confidence level threshold, the first evaluation score of the first confidence level is 1, which is the most valuable; if the first confidence level falls within the interval formed by the first confidence level threshold and the second confidence level threshold, then is the first evaluation score of the first confidence level.

[0093] In this embodiment, if the first confidence is less than the first confidence threshold, the first set confidence score is used as the first evaluation score of the first confidence; if the first confidence is greater than or equal to the second confidence threshold, the second set confidence score is used as the first evaluation score of the first confidence; if the first confidence falls within the interval formed by the first confidence threshold and the second confidence threshold, the first evaluation score of the first confidence is determined according to the first confidence score and the second confidence score, so that the first evaluation score of the first confidence can be accurately determined.

[0094] Optionally, determining the second evaluation score of the first confidence level, or determining the second evaluation score of the second confidence level, includes: if the first confidence level or the second confidence level is less than the first confidence level threshold, using the first set confidence level score as the second evaluation score of the first confidence level or the second evaluation score of the second confidence level; if the first confidence level is greater than or equal to the first confidence level threshold, or the second confidence level is greater than or equal to the second confidence level threshold, using the first set confidence level score as the second evaluation score of the first confidence level or the second evaluation score of the second confidence level; if the first confidence level falls within the range formed by the first confidence level threshold and the second confidence level threshold, If the second confidence falls within the interval formed by the first confidence threshold and the second confidence threshold, a fifth confidence score is determined according to the first confidence level and the second confidence threshold; a sixth confidence score is determined according to the second confidence threshold and the first confidence threshold; a second evaluation score of the first confidence level is determined according to the fifth confidence score and the sixth confidence score; or, if the second confidence level falls within the interval formed by the first confidence threshold and the second confidence threshold, a seventh confidence score is determined according to the second confidence level and the second confidence threshold; an eighth confidence score is determined according to the second confidence threshold and the first confidence threshold; and a second evaluation score of the second confidence level is determined according to the seventh confidence score and the eighth confidence score.

[0095] Exemplarily, when the comparison type is a true positive, the formula for determining the second evaluation score of the second confidence is as follows:

[0096] Among them, when the comparison type is a true positive, conf1 is the second confidence level; if the second confidence level is less than the first confidence level threshold, the second evaluation score of the second confidence level is 0, which is worthless; if the second confidence level is greater than or equal to the second confidence level threshold, the second evaluation score of the second confidence level is 0, which is worthless; if the second confidence level falls within the interval formed by the first confidence level threshold and the second confidence level threshold, then is the second evaluation score of the second confidence level.

[0097] In this embodiment, if the second confidence level is less than the first confidence threshold, the first set confidence score is used as the second evaluation score of the second confidence level; if the second confidence level is greater than or equal to the second confidence threshold, the first set confidence score is used as the second evaluation score of the second confidence level; if the second confidence level falls within the interval formed by the first confidence threshold and the second confidence threshold, the second evaluation score of the second confidence level is determined according to the seventh confidence score and the eighth confidence score, so that the second evaluation score of the second confidence level can be accurately determined.

[0098] Exemplarily, when the comparison type is a true positive, the formula for determining the second evaluation score of the first confidence is as follows:

[0099] Among them, when the comparison type is a true positive, conf2 is the first confidence level; if the first confidence level is less than the first confidence level threshold, the second evaluation score of the first confidence level is 0; if the first confidence level is greater than or equal to the first confidence level threshold, the second evaluation score of the first confidence level is 0; if the first confidence level falls within the interval formed by the first confidence level threshold and the second confidence level threshold, then is the second evaluation score of the first confidence.

[0100] In this embodiment, if the first confidence is less than the first confidence threshold, the first set confidence score is used as the second evaluation score of the first confidence; if the first confidence is greater than or equal to the first confidence threshold, the first set confidence score is used as the second evaluation score of the first confidence; if the first confidence falls within the interval formed by the first confidence threshold and the second confidence threshold, the second evaluation score of the first confidence is determined according to the fifth confidence score and the sixth confidence score, so that the second evaluation score of the first confidence can be accurately determined.

[0101] Exemplarily, Figure 4 is a schematic diagram of the effect between a confidence level and a second evaluation score provided in an embodiment of the present application. As shown in Figure 4, when the comparison type is a true example, the horizontal axis is conf, conf is the first confidence level or the second confidence level, min_conf is the first confidence threshold, and max_conf is the second confidence threshold. The vertical axis is conf_score, and conf_score is the second evaluation score of the first confidence level or the second confidence level. When the first confidence level or the second confidence level is less than the first confidence threshold, or is greater than or equal to the second confidence threshold, the second evaluation score is 0 and has no value. When conf is between the first confidence threshold and the second confidence threshold, the second evaluation score (which can also be understood as value) is inversely proportional to the confidence level.

[0102] Exemplarily, FIG5 is a schematic diagram of the effect between a confidence level and a first evaluation score provided in an embodiment of the present application. As shown in FIG5 , when the comparison type is a false negative example or a false positive example, the horizontal axis is conf, conf is the first confidence level or the second confidence level, min_conf is the first confidence level threshold, max_conf is the second confidence level threshold, and the vertical axis is conf_score, conf_score is the first evaluation score of the first confidence level or the second confidence level. When the first confidence level or the second confidence level is less than the first confidence level threshold, the first evaluation scores are all 0 and have no value. When the first confidence level or the second confidence level is greater than or equal to the second confidence level threshold, the value is the largest, and the first evaluation scores are all 1. When conf is between the first confidence level threshold and the second confidence level threshold, the first evaluation score (which can also be understood as value) is directly proportional to the confidence level.

[0103] Optionally, the area score of the target object is determined according to the comparison type, including: if the comparison type is a false positive example, determining the first evaluation score of the second area, and using the first evaluation score of the second area as the area score of the target object; if the comparison type is a false negative example, determining the first evaluation score of the first area, and using the first evaluation score of the first area as the area score of the target object; if the comparison type is a true positive example, determining the second evaluation score of the first area and the second evaluation score of the second area respectively, and determining the area score of the target object based on the second evaluation score of the first area and the second evaluation score of the second area.

[0104] Exemplarily, the formula for determining the area score of the target object is as follows:

[0105] Among them, matched_area_score is the area score of the target object, area_score small1 It is the first evaluation score of the second area when the comparison type is a false positive, that is, the area score of the target object when the comparison type is a false positive. big1 It is the first evaluation score of the first area when the comparison type is a false negative example, that is, the area score of the target object when the comparison type is a false negative example. The area score of the target object when the comparison type is a true positive. area_score big2 The second evaluation score for the first area, area_score small2 A second assessment score is given for the second area.

[0106] In this embodiment, if the comparison type is a false positive example, the first evaluation score of the second area is determined, and the first evaluation score of the second area is used as the area score of the target object when the comparison type is a false positive example; if the comparison type is a false negative example, the first evaluation score of the first area is determined, and the first evaluation score of the first area is used as the area score of the target object when the comparison type is a false negative example; if the comparison type is a true positive example, the area score of the target object when the comparison type is a true positive example is determined according to the second evaluation score of the first area and the second evaluation score of the second area, so that the area score of the target object can be accurately determined.

[0107] Optionally, determining the first evaluation score of the first area, or determining the first evaluation score of the second area, includes: obtaining a first area threshold and a second area threshold; wherein the second area threshold is greater than the first area threshold; if the first area or the second area is less than the first area threshold, then using the first set area score as the first evaluation score of the first area or the first evaluation score of the second area; if the first area is greater than or equal to the second area threshold, or the second area is greater than or equal to the second area threshold, then using the second set area score as the first evaluation score of the first area or the first evaluation score of the second area; if the first area falls within the interval formed by the first area threshold and the second area threshold, then determining the first area score based on the first area, the second area threshold and the first area threshold; determining the second area score based on the second area threshold and the first area threshold; determining the first evaluation score of the first area based on the first area score and the second area score; or; if the second area falls within the interval formed by the first area threshold and the second area threshold, then determining the third area score based on the second area, the second area threshold and the first area threshold; determining the fourth area score based on the second area threshold and the first area threshold; and determining the first evaluation score of the second area based on the third area score and the fourth area score.

[0108] The first area threshold may be 30*30, the second area threshold may be 500*500; the first set area score may be 0; and the second set area score may be 1.

[0109] Exemplarily, when the alignment type is FN, the formula for determining the first evaluation score of the first area is as follows:

[0110] Among them, when the comparison type is false negative example, area is the first area. If the first area is less than the first area threshold, the first evaluation score of the first area is 0; if the first area is greater than or equal to the second area threshold, the first evaluation score of the first area is 1; if the first area falls within the interval formed by the first area threshold and the second area threshold, then A first assessment score is given for the first area.

[0111] In this embodiment, if the first area is less than the first area threshold, the first set area score is used as the first evaluation score of the first area; if the first area is greater than or equal to the second area threshold, the second set area score is used as the first evaluation score of the first area; if the first area falls within the interval formed by the first area threshold and the second area threshold, the first evaluation score of the first area is determined according to the first area score and the second area score, so that the first evaluation score of the first area can be accurately determined.

[0112] Exemplarily, when the comparison type is FP, the formula for determining the first evaluation score of the second area is as follows:

[0113] Among them, when the comparison type is a false positive, area is the second area; if the second area is less than the first area threshold, the first evaluation score of the second area is 0; if the second area is greater than or equal to the second area threshold, the first evaluation score of the second area is 1; if the second area falls within the interval formed by the first area threshold and the second area threshold, then A score is given for the first assessment of the second area.

[0114] In this embodiment, if the second area is smaller than the first area threshold, the first set area score is used as the first evaluation score of the second area; if the second area is greater than or equal to the second area threshold, the second set area score is used as the first evaluation score of the second area; if the second area falls within the interval formed by the first area threshold and the second area threshold, the first evaluation score of the second area is determined according to the third area score and the fourth area score, thereby accurately determining the first evaluation score of the second area.

[0115] Optionally, determining the second evaluation score of the first area, or determining the second evaluation score of the second area, includes: if the first area or the second area is less than the first area threshold, using the first set area score as the second evaluation score of the first area or the second evaluation score of the second area; if the first area is greater than or equal to the first area threshold, or if the second area is greater than or equal to the first area threshold, using the second set area score as the second evaluation score of the first area or the second evaluation score of the second area; if the first area falls within the interval formed by the first area threshold and the second area value, determining the fifth area score based on the first area and the first area threshold; determining the sixth area score based on the second area threshold and the first area threshold; determining the second evaluation score of the first area based on the fifth area score and the sixth area score; or; if the second area falls within the interval formed by the first area threshold and the second area value, determining the seventh area score based on the second area and the first area threshold; determining the eighth area score based on the second area threshold and the first area threshold; and determining the second evaluation score of the second area based on the seventh area score and the eighth area score.

[0116] Exemplarily, when the comparison type is a true positive, the formula for determining the second evaluation score of the first area is as follows:

[0117] Among them, when the comparison type is a true example, conf2 is the first area; if the first area is less than the first area threshold, the second evaluation score of the first area is 0; if the first area is greater than or equal to the first area threshold, the second evaluation score of the first area is 1; if the first area falls within the interval formed by the first area threshold and the second area value, then A second evaluation score is given for the first area.

[0118] In this embodiment, if the first area is less than the first area threshold, the first set area score is used as the second evaluation score of the first area; if the first area is greater than or equal to the first area threshold, the second set area score is used as the second evaluation score of the first area; if the first area falls within the interval formed by the first area threshold and the second area value, the second evaluation score of the first area is determined according to the fifth area score and the sixth area score, so that the second evaluation score of the first area can be accurately determined.

[0119] Exemplarily, when the comparison type is a true positive, the formula for determining the second evaluation score of the second area is as follows:

[0120] Among them, when the comparison type is a true example, area2 is the second area; if the second area is less than the first area threshold, the second evaluation score of the second area is 0; if the second area is greater than or equal to the first area threshold, the second evaluation score of the second area is 1; if the second area falls within the interval formed by the first area threshold and the second area value, then A second assessment score is given for the second area.

[0121] In this embodiment, if the second area is less than the first area threshold, the first set area score is used as the second evaluation score of the second area; if the second area is greater than or equal to the first area threshold, the second set area score is used as the second evaluation score of the second area; if the second area falls within the interval formed by the first area threshold and the second area value, the second evaluation score of the second area is determined according to the seventh area score and the eighth area score, so that the second evaluation score of the second area can be accurately determined.

[0122] Exemplarily, Figure 6 is a schematic diagram of the effect between an area and a second evaluation score provided in an embodiment of the present application. As shown in Figure 6, when the comparison type is a true example, the horizontal axis is area, area is the first area or the second area, min_area is the first area threshold, and max_conf is the second area threshold. The vertical axis is area_score, and area_score is the second evaluation score. When the first area or the second area is less than the first area threshold, the second evaluation score is 0 and has no value. When the first area or the second area is greater than or equal to the second area threshold, the value is the largest, and the second evaluation scores are all 1. When area is between the first area threshold and the second area threshold, the second evaluation score (which can also be understood as the value) is proportional to the area.

[0123] Exemplarily, Figure 7 is a schematic diagram of the effect between an area and a first evaluation score provided in an embodiment of the present application. As shown in Figure 7, when the comparison type is a false negative example or a false positive example, the horizontal axis is area, area is the first area or the second area, min_area is the first area threshold, max_area is the second area threshold, and the vertical axis is area_score, which is the first evaluation score of the area. When the first area or the second area is less than the first area threshold, the first evaluation scores are all 0 and have no value. When the first area or the second area is greater than or equal to the second area threshold, the value is the largest, and the first evaluation scores are all 1. When area is between the first area threshold and the second area threshold, the first evaluation score (which can also be understood as the value) is proportional to the area.

[0124] S206: Determine a scene score for the initial autonomous driving image.

[0125] In this embodiment, there is no restriction on the scene of the initial autonomous driving image, such as rainy scenes, snowy scenes, traffic jam scenes, off-road scenes, etc.

[0126] In this embodiment, actual scene information can be obtained based on the initial autonomous driving image, the initial autonomous driving image is input into the autonomous driving scene model, the predicted scene information is output, and the scene score is obtained based on the similarity between the predicted scene information and the actual scene information.

[0127] Optionally, determining the scene score of the initial autonomous driving image includes: inputting the initial autonomous driving image into an autonomous driving scene model and outputting predicted scene information; obtaining set scene information corresponding to the initial autonomous driving image; determining scene similarity between the predicted scene information and the set scene information; and determining the scene score of the initial autonomous driving image based on the scene similarity.

[0128] The autonomous driving scene model can be a scene model based on any deep learning algorithm. The set scene information can be the real scene information corresponding to the initial autonomous driving image. In this embodiment, the initial autonomous driving image is input into the autonomous driving scene model, and the corresponding predicted scene information is output; the set scene information corresponding to the initial autonomous driving image is obtained, and the scene similarity between the predicted scene information and the set scene information is calculated based on any similarity algorithm; and a scene score for the initial autonomous driving image is determined based on the scene similarity. The scene similarity range is [0, 1], which means that the scene score range is [0, 1].

[0129] In this embodiment, the scene score of the initial autonomous driving image is determined by the scene similarity between the predicted scene information and the set scene information, so that the scene score of the initial autonomous driving image can be accurately determined.

[0130] S207: Determine an evaluation score for the initial autonomous driving image based on the evaluation score of the target object and the scene score.

[0131] In this embodiment, the evaluation score of any initial autonomous driving image is calculated as follows: the evaluation scores of all target objects in the current initial autonomous driving image are accumulated to obtain an accumulated result, and then the accumulated result is multiplied by the scene score to obtain the evaluation score of the current initial autonomous driving image.

[0132] Exemplarily, the formula for determining the evaluation score of the initial autonomous driving image is as follows:

[0133] Among them, img_value_score represents the evaluation score of the initial autonomous driving image, scene_score represents the scene score, and bbox_value_score represents the evaluation score of the target object.

[0134] S208. Filter out a target autonomous driving image dataset from the initial autonomous driving image dataset based on the evaluation scores of the multiple initial autonomous driving images, and adjust the annotations in the target autonomous driving image dataset.

[0135] In this embodiment, the initial autonomous driving images in the initial autonomous driving image data set can be sorted from high to low according to the evaluation score of each initial autonomous driving image, and a set number of initial autonomous driving images with the highest rankings can be extracted as a set number of target autonomous driving images, and the labels corresponding to the set number of target autonomous driving images can be adjusted.

[0136] Optionally, a target autonomous driving image dataset is screened out from the initial autonomous driving image dataset based on the evaluation scores of the multiple initial autonomous driving images, including: sorting the multiple initial autonomous driving images according to the evaluation scores of the multiple initial autonomous driving images to obtain a sorted initial autonomous driving image dataset; and screening out a set number of target autonomous driving image datasets from the sorted initial autonomous driving image dataset.

[0137] Specifically, the multiple initial autonomous driving images are sorted from high to low according to their evaluation scores to obtain a sorted initial autonomous driving image dataset; a set number of target autonomous driving image datasets with the highest rankings are screened out from the sorted initial autonomous driving image dataset, that is, the first set number of initial autonomous driving images are extracted as the set number of target autonomous driving images. Alternatively, the multiple initial autonomous driving images are sorted from low to high according to their evaluation scores to obtain a sorted initial autonomous driving image dataset; a set number of target autonomous driving image datasets with the lowest rankings are screened out from the sorted initial autonomous driving image dataset, that is, the last set number of initial autonomous driving images are extracted as the set number of target autonomous driving images.

[0138] In this embodiment, by screening out a set number of target autonomous driving image data sets from the sorted initial autonomous driving image data set, the target autonomous driving images can be accurately screened out.

[0139] Exemplarily, FIG8 is a schematic diagram of the effect of the first image detection result provided by an embodiment of the present application; FIG9 is a schematic diagram of the effect of the second image detection result provided by an embodiment of the present application. FIG8 is the first image detection result obtained by the first autonomous driving detection model, and FIG9 is the second image detection result obtained by the second autonomous driving detection model. As can be seen from FIG8 and FIG9, the first autonomous driving detection model detects a group of bicycles with only one frame, while the second autonomous driving detection model detects each bicycle in a separate frame, so the frames of all bicycles are not matched, and the comparison type is FN or FP; the last column in FIG8 or FIG9 is detected by the first autonomous driving detection model, but not by the second autonomous driving detection model, and the comparison type is FN, with a high value score; therefore, the evaluation score of the target object is mostly the evaluation score of FN and FP, which is very high.

[0140] FIG10 is a schematic diagram of the structure of an image dataset processing device provided in an embodiment of the present application. As shown in FIG10 , the device includes: an initial autonomous driving image dataset acquisition module 1001, an image detection result acquisition module 1002, an image detection result comparison module 1003, a comparison type determination module 1004, an evaluation score determination module 1005, and a target autonomous driving image dataset screening module 1006;

[0141] An initial autonomous driving image dataset acquisition module 1001 is configured to acquire an initial autonomous driving image dataset, wherein the initial autonomous driving image dataset includes a plurality of initial autonomous driving images and corresponding annotations;

[0142] An image detection result obtaining module 1002 is configured to input the initial image data set into a first autonomous driving detection model and a second autonomous driving detection model, respectively, to obtain a first image detection result and a second image detection result;

[0143] An image detection result comparison module 1003 is configured to compare the first image detection result with the second image detection result to obtain a comparison result;

[0144] A comparison type determination module 1004 is configured to determine a comparison type based on the comparison result; wherein the first image detection result is used as a reference detection result; and the comparison types include true positives, false positives, and false negatives;

[0145] an evaluation score determination module 1005, configured to determine evaluation scores of the plurality of initial autonomous driving images according to the comparison type;

[0146] The target autonomous driving image dataset screening module 1006 is used to screen out a target autonomous driving image dataset from the initial autonomous driving image dataset based on the evaluation scores of the multiple initial autonomous driving images, and adjust the annotations in the target autonomous driving image dataset.

[0147] The technical solution disclosed in this embodiment is to obtain an initial autonomous driving image dataset through an initial autonomous driving image dataset acquisition module; wherein the initial autonomous driving image dataset includes multiple initial autonomous driving images and corresponding annotations; the initial image dataset is input into a first autonomous driving detection model and a second autonomous driving detection model respectively through an image detection result acquisition module to obtain a first image detection result and a second image detection result respectively; the first image detection result and the second image detection result are compared through an image detection result comparison module to obtain a comparison result; a comparison type is determined according to the comparison result through a comparison type determination module; wherein the first image detection result is used as a benchmark detection result; the comparison type includes true positive examples, false positive examples and false negative examples; an evaluation score determination module determines the evaluation scores of multiple initial autonomous driving images according to the comparison type; a target autonomous driving image dataset is screened out from the initial autonomous driving image dataset according to the evaluation scores of the multiple initial autonomous driving images through a target autonomous driving image dataset screening module, and the annotations in the target autonomous driving image dataset are adjusted. In an embodiment of the present application, the comparison type is determined based on the comparison result between the first image detection result and the second image detection result, and the evaluation score of the initial autonomous driving image is determined based on the comparison type; and the target autonomous driving image dataset is screened out based on the evaluation score of the initial autonomous driving image, so as to improve the accuracy of image dataset processing.

[0148] Optionally, each initial autonomous driving image includes at least one target object; the first image detection result includes first detection frame information of at least one target object, a first category corresponding to the at least one target object, a first confidence level of the first category, and a first area corresponding to the first detection frame information; the second image detection result includes second detection frame information of at least one target object, a second category corresponding to the at least one target object, a second confidence level corresponding to the second category, and a second area corresponding to the second detection frame information; the comparison result includes a category comparison result. Optionally, the image detection result comparison module is specifically used to: compare the first category and the second category to obtain a category comparison result; wherein the category comparison result includes category comparison consistency and category comparison inconsistency.

[0149] Optionally, the comparison type determination module is specifically used to: determine the intersection-and-union (IoU) based on the first detection frame information and the second detection frame information; determine a first IoU setting value based on the first area and / or the second area; if the IoU is greater than or equal to the first IoU setting value, and the category comparison result is a category comparison consistency, then determine the comparison type as a true positive example; if the IoU is less than the first IoU setting value and or the category comparison result is a category comparison inconsistency, then determine the comparison type as a false positive example and a false negative example; or; if the first image detection result includes the first detection frame information of the target object, and the second image detection result does not include the second detection frame information of the target object, then determine the comparison type as a false negative example; if the first image detection result does not include the first detection frame information of the target object, and the second image detection result includes the second detection frame information of the target object, then determine the comparison type as a false positive example.

[0150] Optionally, the evaluation score determination module is specifically used to: for the evaluation score of any initial autonomous driving image, determine the evaluation score of the target object in the initial autonomous driving image according to the comparison type; determine the scene score of the initial autonomous driving image; and determine the evaluation score of the initial autonomous driving image according to the evaluation score of the target object and the scene score.

[0151] Optionally, the evaluation score determination module is also used to: determine at least one of the intersection-over-union score, confidence score and area score of the target object based on the comparison type; obtain a preset category score corresponding to the target object; and determine the evaluation score of the target object based on at least one of the intersection-over-union score, the confidence score, the area score and the category score.

[0152] Optionally, the evaluation score determination module is also used to: determine a second intersection-and-union ratio setting value based on the first area and / or the second area; wherein the second intersection-and-union ratio setting value is greater than the first intersection-and-union ratio setting value; and determine the intersection-and-union ratio evaluation score based on the comparison type, the first intersection-and-union ratio setting value, the second intersection-and-union ratio setting value, and the intersection-and-union ratio.

[0153] Optionally, the evaluation score determination module is also used to: if the comparison type is a false positive example or a false negative example, and the intersection-and-union ratio is less than the first intersection-and-union ratio setting value, then the intersection-and-union ratio evaluation score is the first setting intersection-and-union ratio evaluation score; if the comparison type is a true positive example, and the intersection-and-union ratio is greater than or equal to the second intersection-and-union ratio setting value, then the intersection-and-union ratio evaluation score is the second setting intersection-and-union ratio evaluation score; if the comparison type is a true positive example, and the intersection-and-union ratio falls within the interval formed by the first intersection-and-union ratio setting value and the second intersection-and-union ratio setting value, then determine the first intersection-and-union ratio evaluation score based on the second intersection-and-union ratio setting value and the intersection-and-union ratio; determine the second intersection-and-union ratio evaluation score based on the second intersection-and-union ratio setting value and the first intersection-and-union ratio setting value; determine the intersection-and-union ratio evaluation score based on the first intersection-and-union ratio evaluation score and the second intersection-and-union ratio evaluation score.

[0154] Optionally, the evaluation score determination module is also used to: if the comparison type is a false positive example, determine the first evaluation score of the second confidence level, and use the first evaluation score of the second confidence level as the confidence score of the target object; if the comparison type is a false negative example, determine the first evaluation score of the first confidence level, and use the first evaluation score of the first confidence level as the confidence score of the target object; if the comparison type is a true positive example, respectively determine the second evaluation score of the first confidence level and the second evaluation score of the second confidence level, and determine the confidence score of the target object based on the second evaluation score of the first confidence level and the second evaluation score of the second confidence level.

[0155] Optionally, the evaluation score determination module is further used to: obtain a first confidence threshold and a second confidence threshold; wherein, the second confidence threshold is greater than the first confidence threshold; if the first confidence or the second confidence is less than the first confidence threshold, the first set confidence score is used as the first evaluation score of the first confidence or the first evaluation score of the second confidence; if the first confidence is greater than or equal to the second confidence threshold, or the second confidence is greater than or equal to the second confidence threshold, the second set confidence score is used as the first evaluation score of the first confidence or the first evaluation score of the second confidence; if the first confidence falls within the interval formed by the first confidence threshold and the second confidence threshold, the confidence score is determined according to the first confidence threshold. Determine a first confidence score based on the first confidence level, the second confidence threshold, and the first confidence threshold; determine a second confidence score based on the second confidence threshold and the first confidence threshold; determine a first evaluation score of the first confidence level based on the first confidence score and the second confidence score; or; if the second confidence level falls within the interval formed by the first confidence threshold and the second confidence threshold, determine a third confidence score based on the second confidence level, the second confidence threshold, and the first confidence threshold; determine a fourth confidence score based on the second confidence threshold and the first confidence threshold; and determine a first evaluation score of the second confidence level based on the third confidence score and the fourth confidence score.

[0156] Optionally, the evaluation score determination module is further used to: if the first confidence level or the second confidence level is less than a first confidence threshold, use the first set confidence score as the second evaluation score of the first confidence level or the second evaluation score of the second confidence level; if the first confidence level is greater than or equal to the first confidence threshold, or the second confidence level is greater than or equal to the second confidence threshold, use the first set confidence score as the second evaluation score of the first confidence level or the second evaluation score of the second confidence level; if the first confidence level falls within the interval formed by the first confidence threshold and the second confidence threshold, use the first set confidence score as the second evaluation score of the first confidence level or the second evaluation score of the second confidence level according to the first confidence level. and the second confidence threshold; determine a fifth confidence score according to the second confidence threshold and the first confidence threshold; determine a sixth confidence score according to the second confidence threshold and the first confidence threshold; determine a second evaluation score of the first confidence according to the fifth confidence score and the sixth confidence score; or; if the second confidence falls within the interval formed by the first confidence threshold and the second confidence threshold, determine a seventh confidence score according to the second confidence threshold and the second confidence threshold; determine an eighth confidence score according to the second confidence threshold and the first confidence threshold; and determine a second evaluation score of the second confidence according to the seventh confidence score and the eighth confidence score.

[0157] Optionally, the evaluation score determination module is also used to: if the comparison type is a false positive example, determine the first evaluation score of the second area, and use the first evaluation score of the second area as the area score of the target object; if the comparison type is a false negative example, determine the first evaluation score of the first area, and use the first evaluation score of the first area as the area score of the target object; if the comparison type is a true positive example, respectively determine the second evaluation score of the first area and the second evaluation score of the second area, and determine the area score of the target object based on the second evaluation score of the first area and the second evaluation score of the second area.

[0158] Optionally, the evaluation score determination module is also used to: obtain a first area threshold and a second area threshold; wherein the second area threshold is greater than the first area threshold; if the first area or the second area is less than the first area threshold, the first set area score is used as the first evaluation score of the first area or the first evaluation score of the second area; if the first area is greater than or equal to the second area threshold, or the second area is greater than or equal to the second area threshold, the second set area score is used as the first evaluation score of the first area or the first evaluation score of the second area; if the first area falls within the interval formed by the first area threshold and the second area threshold, the first area score is determined according to the first area, the second area threshold and the first area threshold; the second area score is determined according to the second area threshold and the first area threshold; the first evaluation score of the first area is determined according to the first area score and the second area score; or; if the second area falls within the interval formed by the first area threshold and the second area threshold, the third area score is determined according to the second area, the second area threshold and the first area threshold; the fourth area score is determined according to the second area threshold and the first area threshold; the first evaluation score of the second area is determined according to the third area score and the fourth area score.

[0159] Optionally, the evaluation score determination module is also used to: if the first area or the second area is smaller than the first area threshold, use the first set area score as the second evaluation score of the first area or the second evaluation score of the second area; if the first area is greater than or equal to the first area threshold, or if the second area is greater than or equal to the first area threshold, use the second set area score as the second evaluation score of the first area or the second evaluation score of the second area; if the first area falls within the interval formed by the first area threshold and the second area value, determine the fifth area score based on the first area and the first area threshold; determine the sixth area score based on the second area threshold and the first area threshold; determine the second evaluation score of the first area based on the fifth area score and the sixth area score; or; if the second area falls within the interval formed by the first area threshold and the second area value, determine the seventh area score based on the second area and the first area threshold; determine the eighth area score based on the second area threshold and the first area threshold; determine the second evaluation score of the second area based on the seventh area score and the eighth area score.

[0160] Optionally, the evaluation score determination module is also used to: input the initial autonomous driving image into the autonomous driving scene model and output predicted scene information; obtain the set scene information corresponding to the initial autonomous driving image; determine the scene similarity between the predicted scene information and the set scene information; and determine the scene score of the initial autonomous driving image based on the scene similarity.

[0161] Optionally, the target autonomous driving image dataset screening module is specifically used to: sort the multiple initial autonomous driving images according to their evaluation scores to obtain a sorted initial autonomous driving image dataset; and screen out a set number of target autonomous driving image datasets from the sorted initial autonomous driving image dataset.

[0162] The image dataset processing device provided in the embodiments of the present application can execute the image dataset processing method provided in any embodiment of the present application, and has the corresponding functional modules and beneficial effects of the execution method.

[0163] It is worth noting that the various units and modules included in the above-mentioned device are only divided according to functional logic, but are not limited to the above-mentioned division, as long as the corresponding functions can be achieved; in addition, the specific names of the functional units are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of the embodiments of this application.

[0164] Figure 11 is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Referring to Figure 11 below, it shows a schematic diagram of the structure of an electronic device (such as the terminal device or server in Figure 11) 1100 suitable for implementing an embodiment of the present application. The terminal device in the embodiment of the present application may include but is not limited to mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. The electronic device shown in Figure 11 is merely an example and should not impose any restrictions on the functions and scope of use of the embodiments of the present application.

[0165] As shown in FIG11 , the electronic device 1100 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 1101, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1102 or a program loaded from a storage device 1108 into a random access memory (RAM) 1103. Various programs and data required for the operation of the electronic device 1100 are also stored in the RAM 1103. The processing device 1101, the ROM 1102, and the RAM 1103 are connected to each other via a bus 1104. An edit / output (I / O) interface 1105 is also connected to the bus 1104.

[0166] Typically, the following devices may be connected to the I / O interface 1105: an input device 1106 including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 1107 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 1108 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1109. The communication device 1109 may allow the electronic device 1100 to communicate with other devices wirelessly or by wire to exchange data. Although FIG. 11 illustrates the electronic device 1100 with various devices, it should be understood that not all of the devices shown are required to be implemented or present. More or fewer devices may alternatively be implemented or present.

[0167] In particular, according to an embodiment of the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present application includes a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication device 1109, or installed from the storage device 1108, or installed from the ROM 1102. When the computer program is executed by the processing device 1101, the above-mentioned functions defined in the method of the embodiment of the present application are performed.

[0168] The names of the messages or information exchanged between multiple devices in the embodiments of the present application are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0169] The electronic device provided in the embodiment of the present application and the image data set processing method provided in the above embodiment belong to the same inventive concept. For technical details not fully described in this embodiment, please refer to the above embodiment, and this embodiment has the same beneficial effects as the above embodiment.

[0170] An embodiment of the present application provides a computer storage medium having a computer program stored thereon. When the program is executed by a processor, the image data set processing method provided in the above embodiment is implemented.

[0171] It should be noted that the computer-readable medium mentioned above in this application can be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. The computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any combination of the above. More specific examples of computer-readable storage media can include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this application, a computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, device, or device. In this application, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, which carries computer-readable program code. This propagated data signal can take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium may be transmitted using any suitable medium, including but not limited to wires, optical cables, RF (radio frequency), etc., or any suitable combination thereof.

[0172] In some embodiments, the client and server can communicate using any currently known or later developed network protocol, such as HTTP (HyperText Transfer Protocol), and can be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or later developed network.

[0173] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.

[0174] The above-mentioned computer-readable medium carries one or more programs. When the above-mentioned one or more programs are executed by the electronic device, the electronic device is enabled to: obtain an initial autonomous driving image dataset; wherein, the initial autonomous driving image dataset includes multiple initial autonomous driving images and corresponding annotations; input the initial image dataset into a first autonomous driving detection model and a second autonomous driving detection model respectively to obtain a first image detection result and a second image detection result respectively; compare the first image detection result and the second image detection result to obtain a comparison result; determine a comparison type based on the comparison result; wherein, the first image detection result is used as a benchmark detection result; the comparison type includes true positive examples, false positive examples and false negative examples; determine the evaluation scores of the multiple initial autonomous driving images based on the comparison type; filter out a target autonomous driving image dataset from the initial autonomous driving image dataset based on the evaluation scores of the multiple initial autonomous driving images, and adjust the annotations in the target autonomous driving image dataset.

[0175] Computer program code for performing the operations of the present application may be written in one or more programming languages, or a combination thereof, including, but not limited to, object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0176] The flow charts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present application. In this regard, each box in the flow chart or block diagram can represent a module, program segment or a part of code, and the module, program segment or a part of code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order than that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented by a dedicated hardware-based system that performs the specified function or operation, or can be implemented by a combination of dedicated hardware and computer instructions.

[0177] The units involved in the embodiments described in this application may be implemented in software or hardware. The name of a unit does not necessarily limit the unit itself. For example, the first acquisition unit may also be described as a "unit for acquiring at least two Internet Protocol addresses."

[0178] The functions described above herein may be performed, at least in part, by one or more hardware logic components. For example, and without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chip (SOCs), complex programmable logic devices (CPLDs), and the like.

[0179] In the context of the present application, a machine-readable medium can be a tangible medium that can contain or store a program for use by an instruction execution system, device or equipment or used in combination with an instruction execution system, device or equipment. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared or semiconductor system, device or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium can include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0180] The above description is merely a preferred embodiment of the present application and an illustration of the technical principles employed. Those skilled in the art should understand that the scope of disclosure in this application is not limited to the technical solutions formed by a specific combination of the above-mentioned technical features, but also encompasses other technical solutions formed by any combination of the above-mentioned technical features or their equivalents without departing from the above-mentioned disclosed concepts. For example, a technical solution formed by replacing the above-mentioned features with (but not limited to) technical features with similar functions disclosed in this application.

[0181] In addition, although adopting specific order to describe each operation, this should not be interpreted as requiring these operations to be executed in the specific order shown or in sequential order.Under certain environment, multitasking and parallel processing may be advantageous.Similarly, although comprising some specific implementation details in the above discussion, these should not be interpreted as limiting the scope of the application.Some features described in the context of separate embodiment can also be implemented in a single embodiment in combination.On the contrary, the various features described in the context of a single embodiment also can be implemented in multiple embodiments individually or in the mode of any suitable subcombination.

[0182] Although the subject matter has been described in language specific to structural features and / or methodological logical acts, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are merely example forms of implementing the claims.

Claims

1. A method for processing an image data set, comprising: Acquire an initial autonomous driving image dataset; wherein the initial autonomous driving image dataset includes a plurality of initial autonomous driving images and corresponding annotations; Inputting the initial image data set into a first autonomous driving detection model and a second autonomous driving detection model respectively, to obtain a first image detection result and a second image detection result respectively; Comparing the first image detection result and the second image detection result to obtain a comparison result; Determine the comparison type according to the comparison result; wherein the first image detection result is used as the reference detection result; the comparison type includes true positive examples, false positive examples and false negative examples; determining evaluation scores of the plurality of initial autonomous driving images according to the comparison type; A target autonomous driving image dataset is screened out from the initial autonomous driving image dataset according to the evaluation scores of the multiple initial autonomous driving images, and the annotations in the target autonomous driving image dataset are adjusted.

2. The method according to claim 1, wherein: Each initial autonomous driving image includes at least one target object; the first image detection result includes first detection frame information of at least one target object, a first category corresponding to the at least one target object, a first confidence level of the first category, and a first area corresponding to the first detection frame information; the second image detection result includes second detection frame information of at least one target object, a second category corresponding to the at least one target object, a second confidence level corresponding to the second category, and a second area corresponding to the second detection frame information; The comparison result includes a category comparison result, and the comparison result is obtained by comparing the first image detection result and the second image detection result, including: Compare the first category and the second category to obtain a category comparison result; wherein the category comparison result includes a consistent category comparison and an inconsistent category comparison.

3. The method according to claim 2, wherein: Determining the comparison type according to the comparison result includes: Determine an intersection-over-union ratio according to the first detection frame information and the second detection frame information; Determine a first intersection-over-joint ratio setting value according to the first area and / or the second area; If the intersection-over-union ratio is greater than or equal to the first intersection-over-union ratio setting value, and the category comparison result is a category comparison consistency, the comparison type is determined as a true positive; If the intersection-over-union ratio is less than the first intersection-over-union ratio setting value and or the category comparison result is category comparison inconsistency, the comparison type is determined as a false positive example and a false negative example; or; If the first image detection result includes first detection frame information of the target object, and the second image detection result does not include second detection frame information of the target object, the comparison type is determined as a false negative example; If the first image detection result does not include first detection box information of the target object, and the second image detection result includes second detection box information of the target object, the comparison type is determined as a false positive example.

4. The method according to claim 3, wherein: Determining evaluation scores of the plurality of initial autonomous driving images according to the comparison type includes: For any evaluation score of an initial autonomous driving image, determining an evaluation score of a target object in the initial autonomous driving image according to a comparison type; Determining a scene score for the initial autonomous driving image; An evaluation score of an initial autonomous driving image is determined based on the evaluation score of the target object and the scene score.

5. The method according to claim 4, wherein: The evaluation score of the target object in the initial autonomous driving image is determined based on the comparison type, including: Determine at least one of an intersection-over-union score, a confidence score, and an area score of the target object according to the comparison type; Get the preset category score corresponding to the target object; An evaluation score of the target object is determined according to at least one of the intersection-over-union score, the confidence score, the area score, and the category score.

6. The method according to claim 5, wherein: Determine the intersection-over-union score of the target object based on the comparison type, including: Determine a second intersection-and-joint ratio setting value according to the first area and / or the second area; wherein the second intersection-and-joint ratio setting value is greater than the first intersection-and-joint ratio setting value; An IoU evaluation score is determined according to the comparison type, the first IoU setting value, the second IoU setting value, and the IoU.

7. The method according to claim 6, wherein: Determining an IoU evaluation score according to the comparison type, the first IoU setting value, the second IoU setting value, and the IoU includes: If the comparison type is a false positive example or a false negative example, and the intersection-over-union ratio is less than the first intersection-over-union ratio setting value, the intersection-over-union ratio evaluation score is the first setting intersection-over-union ratio evaluation score; If the comparison type is a true example, and the intersection-and-union ratio is greater than or equal to the second intersection-and-union ratio setting value, the intersection-and-union ratio evaluation score is the second set intersection-and-union ratio evaluation score; If the comparison type is a true example, and the IoU falls within the interval formed by the first IoU setting value and the second IoU setting value, determining a first IoU evaluation score according to the second IoU setting value and the IoU; Determine a second intersection-over-union evaluation score according to the second intersection-over-union setting value and the first intersection-over-union setting value; An intersection-over-union evaluation score is determined according to the first intersection-over-union evaluation score and the second intersection-over-union evaluation score.

8. The method according to claim 5, wherein: Determines the confidence score of the target object based on the comparison type, including: If the comparison type is a false positive example, determining a first evaluation score of a second confidence level, and using the first evaluation score of the second confidence level as a confidence score of the target object; If the comparison type is a false negative example, determining a first evaluation score of a first confidence level, and using the first evaluation score of the first confidence level as a confidence score of the target object; If the comparison type is a true positive, a second evaluation score of the first confidence and a second evaluation score of the second confidence are determined respectively, and a confidence score of the target object is determined according to the second evaluation score of the first confidence and the second evaluation score of the second confidence.

9. The method according to claim 8, wherein: Determining a first evaluation score of a first confidence level, or determining a first evaluation score of a second confidence level, comprises: Obtaining a first confidence threshold and a second confidence threshold; wherein the second confidence threshold is greater than the first confidence threshold; If the first confidence level or the second confidence level is less than a first confidence level threshold, using the first set confidence level score as a first evaluation score of the first confidence level or a first evaluation score of the second confidence level; If the first confidence is greater than or equal to a second confidence threshold, or the second confidence is greater than or equal to a second confidence threshold, taking the second set confidence score as the first evaluation score of the first confidence or the first evaluation score of the second confidence; If the first confidence falls within the interval formed by the first confidence threshold and the second confidence threshold, determining a first confidence score according to the first confidence, the second confidence threshold and the first confidence threshold; determining a second confidence score based on the second confidence threshold and the first confidence threshold; Determining a first assessment score of a first confidence level according to the first confidence score and the second confidence score; or; If the second confidence falls within the interval formed by the first confidence threshold and the second confidence threshold, determining a third confidence score according to the second confidence, the second confidence threshold and the first confidence threshold; determining a fourth confidence score based on the second confidence threshold and the first confidence threshold; A first evaluation score of a second confidence is determined based on the third confidence score and the fourth confidence score.

10. The method according to claim 9, wherein: Determining a second evaluation score of the first confidence, or determining a second evaluation score of the second confidence, comprises: If the first confidence level or the second confidence level is less than a first confidence level threshold, using the first set confidence level score as a second evaluation score of the first confidence level or a second evaluation score of the second confidence level; If the first confidence is greater than or equal to a first confidence threshold, or the second confidence is greater than or equal to a second confidence threshold, the first set confidence score is used as a second evaluation score of the first confidence or a second evaluation score of the second confidence; If the first confidence falls within the interval formed by the first confidence threshold and the second confidence threshold, determining a fifth confidence score according to the first confidence and the second confidence threshold; determining a sixth confidence score based on the second confidence threshold and the first confidence threshold; determining a second evaluation score of the first confidence based on the fifth confidence score and the sixth confidence score; or; If the second confidence falls within the interval formed by the first confidence threshold and the second confidence threshold, determining a seventh confidence score according to the second confidence and the second confidence threshold; determining an eighth confidence score based on the second confidence threshold and the first confidence threshold; A second assessment score of a second confidence is determined based on the seventh confidence score and the eighth confidence score.

11. The method according to claim 5, wherein: Determines the area score of the target object based on the comparison type, including: If the comparison type is a false positive, determining a first evaluation score for the second area, and using the first evaluation score for the second area as the area score of the target object; If the comparison type is a false negative example, determining a first evaluation score of the first area, and using the first evaluation score of the first area as the area score of the target object; If the comparison type is a true positive, a second evaluation score of the first area and a second evaluation score of the second area are determined respectively, and an area score of the target object is determined according to the second evaluation score of the first area and the second evaluation score of the second area.

12. The method according to claim 11, wherein: Determining a first evaluation score for a first area, or determining a first evaluation score for a second area, includes: Acquire a first area threshold and a second area threshold; wherein the second area threshold is greater than the first area threshold; If the first area or the second area is smaller than a first area threshold, taking the first set area score as the first evaluation score of the first area or the first evaluation score of the second area; If the first area is greater than or equal to the second area threshold, or the second area is greater than or equal to the second area threshold, the second set area score is used as the first evaluation score of the first area or the first evaluation score of the second area; If the first area falls within the interval formed by the first area threshold and the second area threshold, determining a first area score according to the first area, the second area threshold and the first area threshold; determining a second area score based on the second area threshold and the first area threshold; determining a first assessment score for the first area based on the first area score and the second area score; or; If the second area falls within the interval formed by the first area threshold and the second area threshold, determining a third area score according to the second area, the second area threshold and the first area threshold; determining a fourth area score based on the second area threshold and the first area threshold; A first evaluation score for the second area is determined according to the third area score and the fourth area score.

13. The method according to claim 12, wherein: Determining a second evaluation score for the first area, or determining a second evaluation score for the second area, includes: If the first area or the second area is smaller than a first area threshold, the first set area score is used as a second evaluation score of the first area or a second evaluation score of the second area; If the first area is greater than or equal to a first area threshold, or if the second area is greater than or equal to the first area threshold, using a second set area score as a second evaluation score of the first area or a second evaluation score of the second area; If the first area falls within the interval formed by the first area threshold and the second area value, determining a fifth area score according to the first area and the first area threshold; determining a sixth area score based on the second area threshold and the first area threshold; determining a second evaluation score for the first area based on the fifth area score and the sixth area score; or; If the second area falls within the interval formed by the first area threshold and the second area value, determining a seventh area score according to the second area and the first area threshold; determining an eighth area score based on the second area threshold and the first area threshold; A second evaluation score for a second area is determined based on the seventh area score and the eighth area score.

14. The method according to claim 4, wherein: Determining a scene score of the initial autonomous driving image includes: Inputting the initial autonomous driving image into an autonomous driving scene model and outputting predicted scene information; Acquire setting scene information corresponding to the initial automatic driving image; Determine the predicted scene information and the set scene information to determine scene similarity; A scene score of the initial autonomous driving image is determined according to the scene similarity.

15. The method according to claim 1, wherein: Filtering a target autonomous driving image dataset from the initial autonomous driving image dataset according to the evaluation scores of the plurality of initial autonomous driving images includes: sorting the multiple initial autonomous driving images according to the evaluation scores of the multiple initial autonomous driving images to obtain a sorted initial autonomous driving image dataset; A set number of target autonomous driving image data sets are screened out from the sorted initial autonomous driving image data set.

16. An image data set processing device, comprising: An initial autonomous driving image data set acquisition module, configured to acquire an initial autonomous driving image data set; wherein the initial autonomous driving image data set includes a plurality of initial autonomous driving images and corresponding annotations; an image detection result obtaining module, configured to input the initial image data set into the first autonomous driving detection model and the second autonomous driving detection model respectively, to obtain a first image detection result and a second image detection result respectively; An image detection result comparison module, configured to compare the first image detection result and the second image detection result to obtain a comparison result; A comparison type determination module, configured to determine a comparison type according to the comparison result; wherein the first image detection result is used as a reference detection result; and the comparison type includes true positive examples, false positive examples, and false negative examples; an evaluation score determination module, configured to determine evaluation scores of the plurality of initial autonomous driving images according to the comparison type; The target autonomous driving image dataset screening module is configured to screen out a target autonomous driving image dataset from the initial autonomous driving image dataset according to the evaluation scores of the multiple initial autonomous driving images, and adjust the annotations in the target autonomous driving image dataset.

17. An electronic device comprising: at least one processor; a storage device configured to store at least one program, When the at least one program is executed by the at least one processor, the one or more processors implement the image dataset processing method as described in any one of claims 1-15.

18. A storage medium comprising computer executable instructions, wherein the computer executable instructions are used to perform the image data set processing method according to any one of claims 1 to 15 when executed by a computer processor.

Images