Image dataset processing method, device, equipment and storage medium

Abstract

The present application provides an image dataset processing method, apparatus, device, and storage medium, the method including: acquiring an initial autonomous driving image dataset; inputting the initial autonomous driving image dataset into a first autonomous driving detection model and a second autonomous driving detection model, respectively, to obtain a first image detection result and a second image detection result; comparing the first image detection result and the second image detection result to obtain a comparison result; determining a comparison type based on the comparison result; determining an evaluation score for a plurality of initial autonomous driving images based on the comparison type; selecting a target autonomous driving image dataset from the initial autonomous driving image dataset based on the evaluation scores of the plurality of initial autonomous driving images; and adjusting tags in the target autonomous driving image dataset. The present application can improve the accuracy of image dataset processing.

Description

[Technical Field]

[0001] This application claims priority to a Chinese patent application bearing application number 202310922726.X, filed with the China Patent Office on July 26, 2023, the entire contents of which are incorporated herein by reference.

[0002] FIELD Embodiments of the present application relate to the technical field of image processing, and in particular to an image dataset processing method, apparatus, device and storage medium. [Background technology]

[0003] As is well known, the development of autonomous driving is inseparable from the advancement of artificial intelligence technology. Data, computing power, and algorithms are the three elements of artificial intelligence that have a significant impact on the maturity of autonomous driving technology. Of these three elements, data plays a crucial role. AI can only compile rules after training on large amounts of data. In actual application, when AI encounters a scene that is not in the training set, it is largely guesswork, and predictions may be incorrect. Therefore, the role of reliable, high-quality data in the development of autonomous driving is particularly important.

[0004] High-level autonomous driving test vehicles collect terabytes (TB) of data daily and require petabytes (PB) of storage space, but only about 5% of this data is valuable enough to be used for autonomous driving training. How to find the most valuable and challenging samples from the vast amount of data, prioritize tagging these challenging samples, and accelerate the iteration of autonomous driving models and mass production of algorithms is an urgent problem that needs to be solved. Summary of the Invention [Problem to be solved by the invention]

[0005] SUMMARY OF THE INVENTION Embodiments of the present application provide an image dataset processing method, apparatus, device, and storage medium that can improve the accuracy of image dataset processing. [Means for solving the problem]

[0006] In aspect 1, the present embodiment comprises: obtaining an initial autonomous driving image dataset including a plurality of initial autonomous driving images and corresponding tags; Inputting the initial autonomous driving image data set into a first autonomous driving detection model and a second autonomous driving detection model, respectively, to obtain a first image detection result and a second image detection result, respectively; comparing the first image detection result with the second image detection result to obtain a comparison result, wherein the first image detection result is set as a reference detection result; determining a comparison type based on the comparison result, the comparison type including true positive examples, false positive examples, and false negative examples; determining an evaluation score for the plurality of initial autonomous driving images based on the comparison type; and selecting a target autonomous driving image dataset from the initial autonomous driving image dataset based on the evaluation scores of the plurality of initial autonomous driving images, and adjusting tags in the target autonomous driving image dataset. A method for processing an image dataset is provided.

[0007] In aspect 2, the present embodiment comprises: an initial autonomous driving image dataset acquisition module for acquiring an initial autonomous driving image dataset including a plurality of initial autonomous driving images and corresponding tags; an image detection result acquisition module for inputting the initial autonomous driving image data set into a first autonomous driving detection model and a second autonomous driving detection model, respectively, and obtaining a first image detection result and a second image detection result, respectively; an image detection result comparison module for comparing the first image detection result with the second image detection result to obtain a comparison result, the image detection result comparison module taking the first image detection result as a reference detection result; a comparison type determination module for determining a comparison type based on the comparison result, the comparison type including true positive examples, false positive examples, and false negative examples; an evaluation score determination module for determining an evaluation score of the plurality of initial autonomous driving images based on the comparison type; a target autonomous driving image dataset selection module for selecting a target autonomous driving image dataset from the initial autonomous driving image dataset based on the evaluation scores of the plurality of initial autonomous driving images and adjusting tags in the target autonomous driving image dataset; An image dataset processing device is also provided.

[0008] In aspect 3, the present embodiment comprises: one or more processors; a storage device for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement an image dataset processing method according to an embodiment of the present application. An electronic device is also provided.

[0009] In aspect 4, the present embodiment comprises: comprising computer-executable instructions for, when executed by a computer processor, performing an image dataset processing method according to an embodiment of the present application; A storage medium is also provided. [Effects of the Invention]

[0010] In the technical solution disclosed in this embodiment, an initial autonomous driving image dataset including a plurality of initial autonomous driving images and corresponding tags is obtained, the initial autonomous driving image dataset is input into a first autonomous driving detection model and a second autonomous driving detection model, a first image detection result and a second image detection result are obtained, the first image detection result is used as a reference detection result to compare the first image detection result with the second image detection result to obtain a comparison result, a comparison type is determined based on the comparison result, the comparison type including true positive examples, false positive examples, and false negative examples, evaluation scores of the plurality of initial autonomous driving images are determined based on the comparison type, a target autonomous driving image dataset is selected from the initial autonomous driving image dataset based on the evaluation scores of the plurality of initial autonomous driving images, and the tags in the target autonomous driving image dataset are adjusted. The embodiment of the present application can improve the accuracy of image dataset processing by determining the comparison type based on the comparison result between the first image detection result and the second image detection result, determining the evaluation scores of the initial autonomous driving images based on the comparison type, and selecting the target autonomous driving image dataset based on the evaluation scores of the initial autonomous driving images. [Brief explanation of the drawings]

[0011] The above and other features, advantages, and aspects of each embodiment of the present application will become more apparent with reference to the following specific embodiments in conjunction with the drawings. Throughout the drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and that the objects and elements are not necessarily made to scale.

[0012] [Figure 1] 1 is a flowchart of a method for processing an image dataset according to an embodiment of the present application; [Figure 2] 1 is a flowchart of another image dataset processing method according to an embodiment of the present application; [Figure 3] FIG. 1 is a schematic diagram illustrating the effects of IoU and IoU evaluation scores according to an embodiment of the present application. [Figure 4] FIG. 10 is a schematic diagram of the effect between reliability and second evaluation score according to an example of the present application. [Figure 5] FIG. 10 is a schematic diagram of the effect between reliability and first evaluation score according to an embodiment of the present application. [Figure 6] FIG. 10 is a schematic diagram of the effect between area and second evaluation score according to an example of the present application. [Figure 7] FIG. 10 is a schematic diagram of the effect between area and first evaluation score according to an example of the present application. [Figure 8] FIG. 10 is a schematic diagram illustrating the effect of a first image detection result according to an embodiment of the present application. [Figure 9] FIG. 10 is a schematic diagram illustrating the effect of a second image detection result according to an embodiment of the present application. [Figure 10] 1 is a structural schematic diagram of an image dataset processing device according to an embodiment of the present application; [Figure 11] 1 is a structural schematic diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION OF THE INVENTION

[0013] Hereinafter, the embodiments of the present application will be described in more detail with reference to the drawings. Although the drawings show several embodiments of the present application, it should be understood that the present application can be realized in various forms and should not be construed as being limited to the embodiments described herein, but on the contrary, these embodiments are provided for a clearer and more complete understanding of the present application. It should be understood that the drawings and embodiments of the present application are merely illustrative and are not intended to limit the scope of protection of the present application.

[0014] It should be understood that the steps described in the method embodiments of the present application may be performed in different orders and / or in parallel. Also, method embodiments may include additional steps and / or steps shown as omitted. The scope of the present application is not limited in this respect.

[0015] As used herein, the term "comprises" and variations thereof are open inclusive, i.e., "including, but not limited to." The term "based on" means "based at least in part on." The term "one embodiment" means "at least one embodiment." The term "another embodiment" means "at least one other embodiment." The term "some embodiments" means "at least some embodiments." Relevant definitions of other terms are provided below.

[0016] It should be noted that the concepts of "first," "second," etc. referred to in this application are merely intended to distinguish between devices, modules, or units, and are not intended to limit the order or interdependence of the functions performed by these devices, modules, or units.

[0017] It should be understood that the modifications "one" and "multiple" referred to in this application are exemplary and not limiting, and that those skilled in the art should understand them as "one or more" unless the context clearly indicates otherwise.

[0018] It is understood that the data related to this technical solution (including but not limited to the data itself, the acquisition or use of the data) shall comply with the requirements of corresponding laws, regulations and related provisions.

[0019] FIG. 1 is a flowchart of an image dataset processing method according to an embodiment of the present application. This embodiment can be applied to selecting a target autonomous driving image dataset from an initial autonomous driving image dataset. This embodiment is not limited to target detection tasks in scenes from an autonomous driving front camera, but can also be other scene tasks. When switching scene tasks, related configuration information is modified according to model capabilities and project needs. In this embodiment, selecting a target autonomous driving image dataset from an initial autonomous driving image dataset can be understood as a data mining case, in which the most valuable and difficult samples are found from a large amount of data, and tagged preferentially to accelerate model iteration. The method can be performed by an image dataset processing device and specifically includes the following steps:

[0020] In S110, an initial autonomous driving image dataset is acquired.

[0021] Here, the initial autonomous driving image dataset includes a plurality of initial autonomous driving images and corresponding tags, where the tags include information such as detection frame information, category, confidence level, area, etc. corresponding to the initial autonomous driving image. When this embodiment is used for a 3D scene task, the area may be the volume or projected area of ​​a 3D target.

[0022] Here, the initial autonomous driving images can be collected by a high-level autonomous driving test vehicle. The autonomous driving image dataset may or may not include tags, and this embodiment is not limited thereto.

[0023] In S120, the initial autonomous driving image data set is input into a first autonomous driving detection model and a second autonomous driving detection model, respectively, to obtain a first image detection result and a second image detection result, respectively.

[0024] Here, the first autonomous driving detection model may be a server-side model, which has sufficient computing power and is capable of offline processing and can be called a large-scale model, and the second autonomous driving detection model may be an edge-side model, which has limited computing power and high real-time requirements and can be called a small-scale model.

[0025] Preferably, here, each initial autonomous driving image includes at least one target object, the first image detection result includes first detection frame information of the at least one target object, a first category corresponding to each of the at least one target object, a first reliability of the first category, and a first area corresponding to the first detection frame information, and the second image detection result includes second detection frame information of the at least one target object, a second category corresponding to each of the at least one target object, a second reliability corresponding to the second category, and a second area corresponding to the second detection frame information.

[0026] Here, the target object may be a vehicle, a railing, a pedestrian, a house, etc., and this embodiment is not limited thereto. The detection frame information may include the length, width, and height of the detection frame.

[0027] In S130, the first image detection result is used as a reference detection result, and the first image detection result is compared with the second image detection result to obtain a comparison result.

[0028] In this embodiment, the first image detection result and the second image detection result are compared correspondingly to obtain corresponding comparison results, which include comparison match and comparison mismatch.

[0029] Preferably, the comparison result includes a category comparison result, and comparing the first image detection result with the second image detection result and obtaining the comparison result includes comparing the first category with the second category and obtaining a category comparison result including a category comparison match and a category comparison mismatch.

[0030] In this embodiment, a category comparison result can be obtained by comparing the first category with the second category, and a comparison type is determined based on the category comparison result.

[0031] In S140, a comparison type is determined based on the comparison result.

[0032] Here, the comparison type includes a true positive example (TP), a false positive example (FP), and a false negative example (FN). Illustratively, in the case of a category comparison match, the comparison type is a true positive example, and in the case of a category comparison mismatch, the comparison type is a false positive example or a false negative example.

[0033] Preferably, determining the comparison type based on the comparison result includes determining an IoU based on first detection frame information and second detection frame information; determining a first IoU setting value based on the first area and / or the second area; determining the comparison type as a true positive example if the IoU is greater than or equal to the first IoU setting value and the category comparison result is a category comparison match; determining the comparison type as a false positive example and a false negative example if the IoU is smaller than the first IoU setting value and / or the category comparison result is a category comparison mismatch; or determining the comparison type as a false negative example if the first image detection result includes first detection frame information of the target object and the second image detection result does not include second detection frame information of the target object; and determining the comparison type as a false positive example if the first image detection result does not include first detection frame information of the target object and the second image detection result includes second detection frame information of the target object.

[0034] In this embodiment, the IoU can be determined based on the first detection frame information and the second detection frame information, and the size of the target object is determined based on the first area and / or the second area. If the first area and / or the second area is smaller than 32*32, the target object is a small target, and the corresponding first IoU setting value may be 0.3. If the first area and / or the second area is equal to or greater than 32*32, the target object is a normal target, and the corresponding first IoU setting value may be 0.5.

[0035] If the IoU is greater than or equal to the first IoU setting value and the category comparison result is a category comparison match, the comparison type is determined as a true positive example. If the IoU is less than the first IoU setting value and / or the category comparison result is a category comparison mismatch, the comparison types are determined as false positive examples and false negative examples. Alternatively, if the first image detection result includes first detection frame information of the target object and the second image detection result does not include second detection frame information of the target object, the comparison type is determined as a false negative example. That is, if the first image detection result does not include first detection frame information of the target object and the second image detection result includes second detection frame information of the target object, the comparison type is determined as a false positive example. That is, if the second image detection result includes more detection frames than the first image detection result, the comparison type is determined as a false positive example.

[0036] In this embodiment, the comparison type can be accurately determined by determining the comparison type based on the comparison result and IoU, or by determining whether the first image detection result and the second image detection result contain detection frame information of the same target object.

[0037] At S150, an evaluation score is determined for the plurality of initial autonomous driving images based on the comparison type.

[0038] In this embodiment, the evaluation score of each initial autonomous driving image is determined as follows: the evaluation scores of all target objects in the initial autonomous driving image are determined based on the comparison type, and the evaluation scores of the target objects include an IoU score, a confidence score, an area score, and a category score; the evaluation score of the initial autonomous driving image is obtained based on the evaluation scores of all target objects and the scene score of the initial autonomous driving image, so that the evaluation scores of all the initial autonomous driving images can be obtained.

[0039] At S160, a target autonomous driving image dataset is selected from the initial autonomous driving image dataset based on the evaluation scores of the plurality of initial autonomous driving images, and tags in the target autonomous driving image dataset are adjusted.

[0040] In this embodiment, a target autonomous driving image dataset can be selected from the initial autonomous driving image dataset based on the evaluation scores of all the initial autonomous driving images. Here, the target autonomous driving image dataset can be understood as a dataset that has a significant impact on the detection accuracy of the second autonomous driving detection model (which may be another autonomous driving detection model), or as a dataset with a large target value. By adjusting the tags in the target autonomous driving image dataset, the detection accuracy of the autonomous driving detection model can be improved.

[0041] In this embodiment, the comparison is not limited to two models, but may be two-by-two comparisons between multiple models. Multiple value scores can be obtained from the evaluation score of one initial autonomous driving image, and the largest score is taken, or another score fusion algorithm is used to obtain the final value score.

[0042] In this embodiment, although only one model can be used, it is necessary to perform forward calculations in two different modes, such as TTA (Test Time Augmentation) mode and normal mode, and obtain two groups of different detection results.

[0043] In a technical solution disclosed in this embodiment, an initial autonomous driving image dataset including a plurality of initial autonomous driving images and corresponding tags is obtained, the initial autonomous driving image dataset is input into a first autonomous driving detection model and a second autonomous driving detection model, a first image detection result and a second image detection result are obtained, the first image detection result is used as a reference detection result to compare the first image detection result with the second image detection result, a comparison result is obtained, a comparison type is determined based on the comparison result, the comparison type including true positive examples, false positive examples, and false negative examples, evaluation scores of the plurality of initial autonomous driving images are determined based on the comparison type, a target autonomous driving image dataset is selected from the initial autonomous driving image dataset based on the evaluation scores of the plurality of initial autonomous driving images, and the tags in the target autonomous driving image dataset are adjusted. The embodiment of the present application can improve the accuracy of image dataset processing by determining the comparison type based on the comparison result between the first image detection result and the second image detection result, determining the evaluation scores of the initial autonomous driving images based on the comparison type, and selecting the target autonomous driving image dataset based on the evaluation scores of the initial autonomous driving images.

[0044] 2 is a flowchart of another image dataset processing method according to an embodiment of the present application, which is implemented based on the above-mentioned embodiment of the present invention. Referring to FIG. 2, the method according to the embodiment of the present application specifically includes the following steps:

[0045] In S201, an initial autonomous driving image dataset is acquired.

[0046] In S202, the initial autonomous driving image data set is input into a first autonomous driving detection model and a second autonomous driving detection model, respectively, to obtain a first image detection result and a second image detection result, respectively.

[0047] In S203, the first image detection result and the second image detection result are compared to obtain the comparison result.

[0048] In S204, a comparison type is determined based on the comparison result.

[0049] In S205, for the evaluation score of any of the initial autonomous driving images, the evaluation score of the target object in the initial autonomous driving image is determined based on the comparison type.

[0050] Here, the evaluation score of the target object includes the IoU score, confidence score, area score, category score, etc. of the target object. In this embodiment, the evaluation score of the target object varies depending on the comparison type. Based on the comparison type, the evaluation scores of all or some of the target objects in the initial autonomous driving image can be determined.

[0051] Preferably, determining the evaluation score of the target object in the initial autonomous driving image based on the comparison type includes determining at least one of an IoU score, a confidence score, and an area score of the target object based on the comparison type, obtaining a predetermined category score corresponding to the target object, and determining the evaluation score of the target object based on at least one of the IoU score, the confidence score, the area score, and the category score.

[0052] In this embodiment, the evaluation score of each target object is obtained in the following manner: The IoU score, confidence score, and area score of the target object are obtained based on the respective comparison types, and a preset category score corresponding to the target object is obtained. The IoU score, confidence score, area score, and category score are multiplied or weighted averaged, and the result after multiplication or weighted average is used as the evaluation score of the target object. In this embodiment, the factors used to determine the evaluation score of the target object are not limited to the IoU score, confidence score, area score, and category score, and corresponding factors may be added or subtracted.

[0053] Each target object category has a corresponding category weight preset, i.e., a category score preset, which ranges from [0, 1]. After predicting the category to which the target object belongs, the category score corresponding to the predicted category can be obtained. Regarding category weights, distribution information from the initial autonomous driving image dataset can be combined to set higher category weights for the long tail. The model evaluation results from the test set can also be combined to increase the weights for categories with poor accuracy.

[0054] In this embodiment, the evaluation score of the target object is determined based on at least one of the IoU score, the confidence score, the area score, and the category score, so that the evaluation score of the target object can be accurately determined.

[0055] Preferably, determining the IoU score of the target object based on the comparison type includes determining a second IoU setting value based on the first area and / or the second area, and determining an IoU evaluation score based on the comparison type, the first IoU setting value, the second IoU setting value, and the IoU.

[0056] Here, the second IoU setting value is greater than the first IoU setting value, and the second IoU setting value may be understood as the largest IoU threshold, and the first IoU setting value may be understood as the smallest IoU threshold. In this embodiment, the size of the target object is determined based on the first area and / or the second area. If the first area and / or the second area is smaller than 32*32, the target object is a small target, and the corresponding second IoU setting value may be 0.7. If the first area and / or the second area is equal to or greater than 32*32, the target object is a normal target, and the corresponding first IoU setting value may be 0.9.

[0057] Illustratively, the formula for the IoU threshold is:

number

[0058] Specifically, if the comparison type is a false positive example or a false negative example and the IoU is smaller than the first IoU setting value, the IoU evaluation score can be directly obtained. If the comparison type is a true positive example and the IoU is equal to or greater than the second IoU setting value, the IoU evaluation score can be directly obtained. If the comparison type is a true positive example and the IoU is within the interval defined by the first IoU setting value and the second IoU setting value, the IoU evaluation score can be determined based on the first IoU setting value, the second IoU setting value, and the IoU.

[0059] In this embodiment, the IoU evaluation score is determined based on the comparison type, the first IoU setting value, the second IoU setting value, and the IoU, so that the IoU evaluation score can be determined accurately.

[0060] Preferably, determining the IoU evaluation score based on the comparison type, the first IoU setting value, the second IoU setting value, and the IoU comprises: if the comparison type is a false positive example or a false negative example and the IoU is smaller than the first IoU setting value, the IoU evaluation score is the first setting IoU evaluation score; and if the comparison type is a true positive example and the IoU is equal to or larger than the second IoU setting value, the IoU evaluation score is the second setting IoU evaluation score. and if the comparison type is a true positive example and the IoU is within an interval configured by the first IoU setting value and the second IoU setting value, determining a first IoU evaluation score based on the second IoU setting value and the IoU; determining a second IoU evaluation score based on the second IoU setting value and the first IoU setting value; and determining an IoU evaluation score based on the first IoU evaluation score and the second IoU evaluation score.

[0061] Illustratively, the formula for the IoU evaluation score is:

number

[0062] Here, if the comparison type is a false positive example or a false negative example and the IoU is smaller than the first IoU setting value, the value is highest and the IoU evaluation score is 1. If the comparison type is a true positive example and the IoU is equal to or greater than the second IoU setting value, the model does not need to pay attention to small edge differences, so there is no value and the IoU evaluation score is 0. If the comparison type is a true positive example and the IoU is within the interval formed by the first IoU setting value and the second IoU setting value, the lower the IoU, the higher the value, and the first IoU evaluation score is max_iou-iou, the second IoU evaluation score is max_iou-min_iou, and the IoU evaluation score is (max_iou-iou) / (max_iou-min_iou).

[0063] 3 is a schematic diagram illustrating the effect of IoU and IoU evaluation score according to an embodiment of the present application. As shown in FIG. 3, the horizontal axis is IoU iou, and the vertical axis is IoU evaluation score iou_score. It can be seen that when the target object is a small target or a normal target, there is an inverse proportion between IoU and IoU evaluation score.

[0064] The reason for distinguishing target objects according to their size is that absolute coordinate deviations (measured in pixels) for the same target position have a greater impact on the IoU value of small targets during alignment. Therefore, the IoU value of small targets is generally not high, and the evaluation scores of more small targets are higher, which does not conform to the idea that "large targets are more valuable."

[0065] In this embodiment, when the comparison type is a false positive example or a false negative example and the IoU is smaller than the first IoU setting value, the first setting IoU evaluation score is used as the IoU evaluation score; when the comparison type is a true positive example and the IoU is equal to or greater than the second IoU setting value, the second setting IoU evaluation score is used as the IoU evaluation score; and when the comparison type is a true positive example and the IoU is within the interval formed by the first IoU setting value and the second IoU setting value, the IoU evaluation score is determined based on the first IoU evaluation score and the second IoU evaluation score, thereby making it possible to accurately determine the IoU evaluation score.

[0066] Preferably, determining the confidence score of the target object based on the comparison type includes: when the comparison type is a false positive example, determining a first evaluation score of a second confidence level and setting the first evaluation score of the second confidence level as the confidence score of the target object; when the comparison type is a false negative example, determining a first evaluation score of a first confidence level and setting the first evaluation score of the first confidence level as the confidence score of the target object; and when the comparison type is a true positive example, respectively determining a second evaluation score of the first confidence level and a second evaluation score of the second confidence level and setting the confidence score of the target object based on the second evaluation score of the first confidence level and the second evaluation score of the second confidence level.

[0067] Illustratively, the formula for determining the confidence score of a target object is as follows:

number

[0068] In this embodiment, when the comparison type is a false positive example, the reliability score of the target object can be accurately determined by a method in which, when the comparison type is a false positive example, a first evaluation score of the second reliability output from the second autonomous driving detection model is determined and the first evaluation score of the second reliability is used as the reliability score of the target object when the comparison type is a false positive example; a method in which, when the comparison type is a false negative example, a first evaluation score of the first reliability output from the first autonomous driving detection model is determined and the first evaluation score of the first reliability is used as the reliability score of the target object when the comparison type is a false negative example; and a method in which, when the comparison type is a true positive example, a second evaluation score of the first reliability and a second evaluation score of the second reliability are determined, and the reliability score of the target object when the comparison type is a true positive example is determined based on the second evaluation score of the first reliability and the second evaluation score of the second reliability.

[0069] Preferably, determining the first evaluation score of the first reliability or determining the first evaluation score of the second reliability includes obtaining a first reliability threshold and a second reliability threshold; if the second reliability threshold is greater than the first reliability threshold, and if the first reliability or the second reliability is less than the first reliability threshold, setting the first set reliability score as the first evaluation score of the first reliability or the first evaluation score of the second reliability; if the first reliability is greater than or equal to the second reliability threshold or the second reliability is greater than or equal to the second reliability threshold, setting the second set reliability score as the first evaluation score of the first reliability or the first evaluation score of the second reliability; if the first reliability is within an interval defined by the first reliability threshold and the second reliability threshold, The method includes determining a first reliability score based on the first reliability, the second reliability threshold, and the first reliability threshold; determining a second reliability score based on the second reliability threshold and the first reliability threshold; determining a first evaluation score for the first reliability based on the first reliability score and the second reliability score; or, if the second reliability is within an interval defined by the first reliability threshold and the second reliability threshold, determining a third reliability score based on the second reliability, the second reliability threshold, and the first reliability threshold; determining a fourth reliability score based on the second reliability threshold and the first reliability threshold; and determining a first evaluation score for the second reliability based on the third reliability score and the fourth reliability score.

[0070] Illustratively, when the comparison type is FP, the formula for determining the first evaluation score of the second reliability is as follows:

number

[0071] Here, if the comparison type is a false positive example, conf is the second confidence level, and if the second confidence level is less than the first confidence level threshold, the first evaluation score of the second confidence level is 0, which means it has no value; if the second confidence level is greater than or equal to the second confidence level threshold, the first evaluation score of the second confidence level is 1, which means it has the greatest value; and if the second confidence level is within the interval formed by the first confidence level threshold and the second confidence level threshold, {0.5*(conf+max_conf)-min_conf} / (max_conf-min_conf) is the first evaluation score of the second confidence level.

[0072] In this embodiment, when the second reliability is smaller than the first reliability threshold, the first set reliability score is used as the first evaluation score of the second reliability; when the second reliability is equal to or greater than the second reliability threshold, the second set reliability score is used as the first evaluation score of the second reliability; when the second reliability is within the interval formed by the first reliability threshold and the second reliability threshold, the first evaluation score of the second reliability is determined based on the third reliability score and the fourth reliability score, thereby accurately determining the first evaluation score of the second reliability.

[0073] Illustratively, when the comparison type is FN, the formula for determining the first evaluation score of the first reliability is as follows:

number

[0074] In this embodiment, when the first reliability is smaller than the first reliability threshold, the first set reliability score is used as the first evaluation score of the first reliability; when the first reliability is greater than or equal to the second reliability threshold, the second set reliability score is used as the first evaluation score of the first reliability; when the first reliability is within the interval formed by the first reliability threshold and the second reliability threshold, the first evaluation score of the first reliability is determined based on the first reliability score and the second reliability score, thereby accurately determining the first evaluation score of the first reliability.

[0075] Preferably, determining the second evaluation score of the first reliability or determining the second evaluation score of the second reliability includes: when the first reliability or the second reliability is smaller than a first reliability threshold, setting the first set reliability score as the second evaluation score of the first reliability or the second evaluation score of the second reliability; when the first reliability is equal to or larger than the first reliability threshold or the second reliability is equal to or larger than a second reliability threshold, setting the first set reliability score as the second evaluation score of the first reliability or the second evaluation score of the second reliability; and when the first reliability is within an interval defined by the first reliability threshold and the second reliability threshold, setting the first set reliability score as the second evaluation score of the first reliability or the second evaluation score of the second reliability. determining a fifth confidence score based on the first confidence threshold; determining a sixth confidence score based on the second confidence threshold and the first confidence threshold; determining a second evaluation score for the first confidence level based on the fifth confidence score and the sixth confidence score; or, if the second confidence level is within the interval defined by the first confidence threshold and the second confidence threshold, determining a seventh confidence score based on the second confidence level and the second confidence threshold; determining an eighth confidence score based on the second confidence threshold and the first confidence threshold; and determining a second evaluation score for the second confidence level based on the seventh confidence score and the eighth confidence score.

[0076] Illustratively, when the comparison type is true positive examples, the formula for determining the second evaluation score of the second confidence level is as follows:

number

[0077] In this embodiment, when the second reliability is smaller than the first reliability threshold, the first set reliability score is used as the second evaluation score for the second reliability; when the second reliability is equal to or greater than the second reliability threshold, the first set reliability score is used as the second evaluation score for the second reliability; when the second reliability is within the interval formed by the first reliability threshold and the second reliability threshold, the second evaluation score for the second reliability is determined based on the seventh reliability score and the eighth reliability score, thereby accurately determining the second evaluation score for the second reliability.

[0078] Illustratively, when the comparison type is true positive examples, the formula for determining the second evaluation score of the first confidence level is as follows:

number

[0079] In this embodiment, when the first reliability is smaller than the first reliability threshold, the first set reliability score is used as the second evaluation score of the first reliability; when the first reliability is equal to or greater than the first reliability threshold, the first set reliability score is used as the second evaluation score of the first reliability; when the first reliability is within the interval formed by the first reliability threshold and the second reliability threshold, the second evaluation score of the first reliability is determined based on the fifth reliability score and the sixth reliability score, thereby accurately determining the second evaluation score of the first reliability.

[0080] For example, FIG. 4 is a schematic diagram illustrating the effect between the reliability and the second evaluation score according to an embodiment of the present application. As shown in FIG. 4, when the comparison type is a true positive example, the horizontal axis is conf, where conf is the first reliability or the second reliability, min_conf is the first reliability threshold, and max_conf is the second reliability threshold. The vertical axis is conf_score, where conf_score is the second evaluation score of the first reliability or the second reliability. If the first reliability or the second reliability is less than the first reliability threshold or greater than the second reliability threshold, the second evaluation score is 0 and has no value. If conf is between the first reliability threshold and the second reliability threshold, there is an inverse relationship between the second evaluation score (which may be understood as value) and the reliability.

[0081] For example, FIG. 5 is a schematic diagram illustrating the effect between reliability and first evaluation score according to an embodiment of the present application. As shown in FIG. 5, when the comparison type is a false negative example or a false positive example, the horizontal axis is conf, where conf is the first reliability or second reliability, min_conf is the first reliability threshold, max_conf is the second reliability threshold, and the vertical axis is conf_score, where conf_score is the first evaluation score of the first reliability or second reliability. When the first reliability or second reliability is less than the first reliability threshold, the first evaluation scores are all 0 and have no value. When the first reliability or second reliability is equal to or greater than the second reliability threshold, the value is the highest and the first evaluation scores are all 1. When conf is between the first reliability threshold and the second reliability threshold, there is a proportional relationship between the first evaluation score (which may be understood as value) and reliability.

[0082] Preferably, determining the area score of the target object based on the comparison type includes: when the comparison type is a false positive example, determining a first evaluation score of the second area and setting the first evaluation score of the second area as the area score of the target object; when the comparison type is a false negative example, determining a first evaluation score of the first area and setting the first evaluation score of the first area as the area score of the target object; when the comparison type is a true positive example, respectively determining a second evaluation score of the first area and a second evaluation score of the second area, and determining the area score of the target object based on the second evaluation score of the first area and the second evaluation score of the second area.

[0083] Illustratively, the formula for determining the area score of a target object is as follows:

number

[0084] In this embodiment, when the comparison type is a false positive example, a first evaluation score for the second area is determined, and the first evaluation score for the second area is used as the area score of the target object when the comparison type is a false positive example; when the comparison type is a false negative example, a first evaluation score for the first area is determined, and the first evaluation score for the first area is used as the area score of the target object when the comparison type is a false negative example; when the comparison type is a true positive example, the area score of the target object when the comparison type is a true positive example is determined based on the second evaluation score for the first area and the second evaluation score for the second area, thereby accurately determining the area score of the target object.

[0085] Preferably, determining the first evaluation score of the first area or determining the first evaluation score of the second area includes obtaining a first area threshold and a second area threshold; if the second area threshold is greater than the first area threshold, and if the first area or the second area is smaller than the first area threshold, setting the first set area score as the first evaluation score of the first area or the first evaluation score of the second area; if the first area is equal to or greater than the second area threshold or the second area is equal to or greater than the second area threshold, setting the second set area score as the first evaluation score of the first area or the first evaluation score of the second area; if the first area is within an interval defined by the first area threshold and the second area threshold, determining a first area score based on the first area, the second area threshold, and the first area threshold; determining a second area score based on the second area threshold and the first area threshold; determining a first evaluation score for the first area based on the first area score and the second area score; or, if the second area is within an interval defined by the first area threshold and the second area threshold, determining a third area score based on the second area, the second area threshold, and the first area threshold; determining a fourth area score based on the second area threshold and the first area threshold; and determining a first evaluation score for the second area based on the third area score and the fourth area score.

[0086] Here, the first area threshold may be 30*30, the second area threshold may be 500*500, the first set area score may be 0, and the second set area score may be 1.

[0087] Illustratively, when the comparison type is FN, the formula for determining the first evaluation score of the first area is as follows:

number

[0088] In this embodiment, if the first area is smaller than the first area threshold, the first set area score is used as the first evaluation score for the first area; if the first area is equal to or greater than the second area threshold, the second set area score is used as the first evaluation score for the first area; and if the first area is within the interval defined by the first area threshold and the second area threshold, the first evaluation score for the first area is determined based on the first area score and the second area score, thereby making it possible to accurately determine the first evaluation score for the first area.

[0089] Illustratively, when the comparison type is FP, the formula for determining the first evaluation score of the second area is as follows:

number

[0090] In this embodiment, if the second area is smaller than the first area threshold, the first set area score is used as the first evaluation score for the second area; if the second area is equal to or greater than the second area threshold, the second set area score is used as the first evaluation score for the second area; and if the second area is within the interval defined by the first area threshold and the second area threshold, the first evaluation score for the second area is determined based on the third area score and the fourth area score, thereby making it possible to accurately determine the first evaluation score for the second area.

[0091] Preferably, determining the second evaluation score of the first area or determining the second evaluation score of the second area comprises: when the first area or the second area is smaller than a first area threshold, setting the first set area score as the second evaluation score of the first area or the second evaluation score of the second area; when the first area is equal to or larger than the first area threshold or the second area is equal to or larger than the first area threshold, setting the second set area score as the second evaluation score of the first area or the second evaluation score of the second area; and when the first area is within a range defined by the first area threshold and the second area threshold, setting the second set area score as the second evaluation score of the first area or the second evaluation score of the second area. determining a fifth area score based on the second area threshold and the first area threshold; determining a sixth area score based on the second area threshold and the first area threshold; determining a second evaluation score for the first area based on the fifth area score and the sixth area score; or, if the second area is within the interval defined by the first area threshold and the second area threshold, determining a seventh area score based on the second area and the first area threshold; determining an eighth area score based on the second area threshold and the first area threshold; and determining a second evaluation score for the second area based on the seventh area score and the eighth area score.

[0092] Illustratively, when the comparison type is a true positive example, the formula for determining the second evaluation score of the first area is as follows:

number

[0093] In this embodiment, when the first area is smaller than the first area threshold, the first set area score is used as the second evaluation score of the first area; when the first area is equal to or greater than the first area threshold, the second set area score is used as the second evaluation score of the first area; and when the first area is within the interval defined by the first area threshold and the second area threshold, the second evaluation score of the first area is determined based on the fifth area score and the sixth area score, thereby accurately determining the second evaluation score of the first area.

[0094] Illustratively, when the comparison type is a true positive example, the formula for determining the second evaluation score of the second area is as follows:

number

[0095] In this embodiment, when the second area is smaller than the first area threshold, the first set area score is used as the second evaluation score of the second area; when the second area is equal to or greater than the first area threshold, the second set area score is used as the second evaluation score of the second area; and when the second area is within the interval formed by the first area threshold and the second area threshold, the second evaluation score of the second area is determined based on the seventh area score and the eighth area score, thereby making it possible to accurately determine the second evaluation score of the second area.

[0096] For example, FIG. 6 is a schematic diagram illustrating the effect between area and second evaluation score according to an embodiment of the present application. As shown in FIG. 6, when the comparison type is a true positive example, the horizontal axis is area, where area is the first or second area, min_area is the first area threshold, and max_conf is the second area threshold. The vertical axis is area_score, where area_score is the second evaluation score. When the first or second area is smaller than the first area threshold, the second evaluation score is 0 and has no value. When the first or second area is equal to or greater than the second area threshold, the value is the highest and all second evaluation scores are 1. When the area is between the first and second area thresholds, the second evaluation score (which may be understood as value) is proportional to the area.

[0097] For example, FIG. 7 is a schematic diagram illustrating the effect between area and first evaluation score according to an embodiment of the present application. As shown in FIG. 7, when the comparison type is a false negative example or a false positive example, the horizontal axis is area, where area is the first area or the second area, min_area is the first area threshold, max_area is the second area threshold, and the vertical axis is area_score, where area_score is the first evaluation score of the area. When the first area or the second area is smaller than the first area threshold, the first evaluation scores are all 0 and have no value. When the first area or the second area is equal to or greater than the second area threshold, the value is the highest and the first evaluation scores are all 1. When the area is between the first area threshold and the second area threshold, the first evaluation score (which may be understood as value) and the area are proportional.

[0098] In S206, the scene score of the initial autonomous driving image is determined.

[0099] In this embodiment, the scenes of the initial autonomous driving images are not limited, and may be, for example, rainy scenes, snowy scenes, traffic jam scenes, off-road scenes, etc.

[0100] In this embodiment, actual scene information is obtained based on the initial autonomous driving image, the initial autonomous driving image is input into the autonomous driving scene model, predicted scene information is output, and a scene score can be obtained based on the similarity between the predicted scene information and the actual scene information.

[0101] Preferably, determining the scene score of the initial autonomous driving image includes inputting the initial autonomous driving image into an autonomous driving scene model and outputting predicted scene information; obtaining set scene information corresponding to the initial autonomous driving image; determining scene similarity based on the predicted scene information and the set scene information; and determining the scene score of the initial autonomous driving image based on the scene similarity.

[0102] Here, the autonomous driving scene model may be a scene model based on any deep learning algorithm. The set scene information may be real scene information corresponding to the initial autonomous driving image. In this embodiment, the initial autonomous driving image is input to the autonomous driving scene model, corresponding predicted scene information is output, and set scene information corresponding to the initial autonomous driving image is obtained. The scene similarity between the predicted scene information and the set scene information is calculated based on any similarity algorithm, and a scene score for the initial autonomous driving image is determined based on the scene similarity. Here, the range of the scene similarity is [0, 1], that is, the range of the scene score is [0, 1].

[0103] This embodiment can accurately determine the scene score of the initial autonomous driving image by determining the scene score of the initial autonomous driving image based on the scene similarity between the predicted scene information and the set scene information.

[0104] In S207, the evaluation score of the initial autonomous driving image is determined based on the evaluation score of the target object and the scene score.

[0105] In this embodiment, the calculation method for the evaluation score of any initial autonomous driving image is as follows: The evaluation scores of all target objects in the current initial autonomous driving image are accumulated to obtain the accumulated result, and then the accumulated result is multiplied by the scene score to obtain the evaluation score of the current initial autonomous driving image.

[0106] Illustratively, the formula for determining the evaluation score of the initial autonomous driving image is as follows:

number

[0107] In S208, a target autonomous driving image dataset is selected from the initial autonomous driving image dataset based on the evaluation scores of the plurality of initial autonomous driving images, and tags in the target autonomous driving image dataset are adjusted.

[0108] In this embodiment, the initial autonomous driving images in the initial autonomous driving image dataset are ranked in descending order according to the evaluation score of each initial autonomous driving image, and a set number of the top initial autonomous driving images are extracted and used as a set number of target autonomous driving images, and all tags corresponding to the set number of target autonomous driving images are adjusted.

[0109] Preferably, selecting a target autonomous driving image dataset from the initial autonomous driving image dataset based on the evaluation scores of the plurality of initial autonomous driving images includes ranking the plurality of initial autonomous driving images based on the evaluation scores of the plurality of initial autonomous driving images, obtaining a ranked initial autonomous driving image dataset, and selecting a set number of target autonomous driving image datasets from the ranked initial autonomous driving image dataset.

[0110] Specifically, the multiple initial autonomous driving images are ranked in descending order based on the evaluation scores of the multiple initial autonomous driving images, a ranked initial autonomous driving image dataset is obtained, and a set number of top target autonomous driving image datasets are selected from the ranked initial autonomous driving image dataset, i.e., a set number of initial autonomous driving images from the beginning are extracted to set as the set number of target autonomous driving images.Alternatively, the multiple initial autonomous driving images are ranked in descending order based on the evaluation scores of the multiple initial autonomous driving images, a ranked initial autonomous driving image dataset is obtained, and a set number of bottom target autonomous driving image datasets are selected from the ranked initial autonomous driving image dataset, i.e., a set number of initial autonomous driving images from the end are extracted to set as the set number of target autonomous driving images.

[0111] In this embodiment, the target autonomous driving images can be accurately selected by selecting a set number of target autonomous driving image datasets from the initial autonomous driving image datasets after ranking.

[0112] Illustratively, FIG. 8 is a schematic diagram illustrating the effect of a first image detection result according to an embodiment of the present application. FIG. 9 is a schematic diagram illustrating the effect of a second image detection result according to an embodiment of the present application. FIG. 8 shows the first image detection result detected by the first autonomous driving detection model, and FIG. 9 shows the second image detection result detected by the second autonomous driving detection model. As can be seen from FIGS. 8 and 9, One group of bicycles detected by the first autonomous driving detection model exits from only one frame, but each bicycle detected by the second autonomous driving detection model exits from its own frame individually, so the frames of all the bicycles do not match and the comparison type is FN or FP. The rearmost pillar in Figure 8 or Figure 9 was detected by the first autonomous driving detection model but not by the second autonomous driving detection model, so the comparison type is FN and the value score is high. Therefore, the evaluation scores of the target object are mostly FN and FP evaluation scores, which are high.

[0113] FIG. 10 is a structural schematic diagram of an image dataset processing device according to an embodiment of the present application. As shown in FIG. 10, the device includes an initial autonomous driving image dataset acquisition module 1001, an image detection result acquisition module 1002, an image detection result comparison module 1003, a comparison type determination module 1004, an evaluation score determination module 1005, and a target autonomous driving image dataset selection module 1006.

[0114] The initial autonomous driving image dataset acquisition module 1001 is used to acquire an initial autonomous driving image dataset including a plurality of initial autonomous driving images and corresponding tags.

[0115] The image detection result acquisition module 1002 is used to input the initial autonomous driving image dataset into a first autonomous driving detection model and a second autonomous driving detection model respectively, and obtain a first image detection result and a second image detection result respectively.

[0116] The image detection result comparison module 1003 is used to compare the first image detection result with the second image detection result to obtain a comparison result, and the first image detection result is taken as a reference detection result.

[0117] The comparison type determination module 1004 is used to determine a comparison type based on the comparison result, where the comparison type includes true positive examples, false positive examples and false negative examples.

[0118] The evaluation score determination module 1005 is used to determine evaluation scores of the plurality of initial autonomous driving images based on the comparison type.

[0119] The target autonomous driving image dataset selection module 1006 is used to select a target autonomous driving image dataset from the initial autonomous driving image dataset based on the evaluation scores of the plurality of initial autonomous driving images, and adjust tags in the target autonomous driving image dataset.

[0120] In the technical solution disclosed in this embodiment, an initial autonomous driving image dataset acquisition module acquires an initial autonomous driving image dataset including a plurality of initial autonomous driving images and corresponding tags; an image detection result acquisition module inputs the initial autonomous driving image dataset into a first autonomous driving detection model and a second autonomous driving detection model, respectively, to obtain a first image detection result and a second image detection result, respectively; an image detection result comparison module compares the first image detection result with the second image detection result using the first image detection result as a reference detection result to obtain a comparison result; a comparison type determination module determines a comparison type based on the comparison result, the comparison type including true positive examples, false positive examples, and false negative examples; an evaluation score determination module determines evaluation scores of the plurality of initial autonomous driving images based on the comparison type; and a target autonomous driving image dataset selection module selects a target autonomous driving image dataset from the initial autonomous driving image dataset based on the evaluation scores of the plurality of initial autonomous driving images, and adjusts the tags in the target autonomous driving image dataset. The embodiments of the present application can improve the accuracy of image dataset processing by determining a comparison type based on the comparison result between the first image detection result and the second image detection result, determining an evaluation score of the initial autonomous driving image based on the comparison type, and selecting a target autonomous driving image dataset based on the evaluation score of the initial autonomous driving image.

[0121] Preferably, here, each initial autonomous driving image includes at least one target object, the first image detection result includes first detection frame information of at least one target object, a first category corresponding to each of the at least one target object, a first reliability of the first category, and a first area corresponding to the first detection frame information, the second image detection result includes second detection frame information of at least one target object, a second category corresponding to each of the at least one target object, a second reliability corresponding to the second category, and a second area corresponding to the second detection frame information, and the comparison result includes a category comparison result, and preferably, the image detection result comparison module is specifically used for comparing the first category with the second category and obtaining a category comparison result including a category comparison match and a category comparison mismatch.

[0122] Preferably, the comparison type determination module is specifically used for: determining an IoU based on first detection frame information and the second detection frame information; determining a first IoU setting value based on the first area and / or the second area; determining the comparison type as a true positive example if the IoU is equal to or greater than the first IoU setting value and the category comparison result is a category comparison match; determining the comparison type as a false positive example and a false negative example if the IoU is smaller than the first IoU setting value and / or the category comparison result is a category comparison mismatch; or determining the comparison type as a false negative example if the first image detection result includes first detection frame information of a target object and the second image detection result does not include second detection frame information of the target object; and determining the comparison type as a false positive example if the first image detection result does not include first detection frame information of a target object and the second image detection result includes second detection frame information of the target object.

[0123] Preferably, the evaluation score determination module is specifically used for determining an evaluation score of a target object in an initial autonomous driving image based on a comparison type for the evaluation score of any initial autonomous driving image, determining a scene score of the initial autonomous driving image, and determining an evaluation score of the initial autonomous driving image based on the evaluation score of the target object and the scene score.

[0124] Preferably, the evaluation score determination module is further used for determining at least one of an IoU score, a confidence score, and an area score of the target object based on the comparison type; obtaining a preset category score corresponding to the target object; and determining an evaluation score of the target object based on at least one of the IoU score, the confidence score, the area score, and the category score.

[0125] Preferably, the evaluation score determination module is further used for determining a second IoU setting value based on the first area and / or the second area, wherein the second IoU setting value is greater than the first IoU setting value, and determining an IoU evaluation score based on the comparison type, the first IoU setting value, the second IoU setting value and the IoU.

[0126] Preferably, the evaluation score determination module is further configured to: determine, when the comparison type is a false positive example or a false negative example and the IoU is smaller than the first IoU setting value, the IoU evaluation score is a first setting IoU evaluation score; determine, when the comparison type is a true positive example and the IoU is equal to or greater than the second IoU setting value, the IoU evaluation score is a second setting IoU evaluation score; determine, when the comparison type is a true positive example and the IoU is within an interval defined by the first IoU setting value and the second IoU setting value, a first IoU evaluation score based on the second IoU setting value and the IoU; determine, when the comparison type is a true positive example and the IoU is within an interval defined by the first IoU setting value and the second IoU setting value, a second IoU evaluation score based on the second IoU setting value and the first IoU setting value; and determine, when the comparison type is a true positive example and the IoU is within an interval defined by the first IoU setting value and the second IoU setting value, an IoU evaluation score based on the first IoU evaluation score and the second IoU evaluation score.

[0127] Preferably, the evaluation score determination module is further used for: when the comparison type is a false positive example, determining a first evaluation score of a second reliability level and setting the first evaluation score of the second reliability level as the reliability score of the target object; when the comparison type is a false negative example, determining a first evaluation score of a first reliability level and setting the first evaluation score of the first reliability level as the reliability score of the target object; and when the comparison type is a true positive example, determining a second evaluation score of the first reliability level and a second evaluation score of the second reliability level, and determining the reliability score of the target object based on the second evaluation score of the first reliability level and the second evaluation score of the second reliability level.

[0128] Preferably, the evaluation score determination module further obtains a first reliability threshold and a second reliability threshold; and when the second reliability threshold is greater than the first reliability threshold and the first reliability or the second reliability is less than the first reliability threshold, sets the first set reliability score as the first evaluation score of the first reliability or the first evaluation score of the second reliability; when the first reliability is equal to or greater than the second reliability threshold or the second reliability is equal to or greater than the second reliability threshold, sets the second set reliability score as the first evaluation score of the first reliability or the first evaluation score of the second reliability; and when the first reliability is within an interval defined by the first reliability threshold and the second reliability threshold, sets the first reliability, the second reliability threshold and the first set reliability score as the first evaluation score of the first reliability or the first evaluation score of the second reliability. It is used for determining a first reliability score based on a reliability threshold, determining a second reliability score based on the second reliability threshold and the first reliability threshold, determining a first evaluation score of the first reliability based on the first reliability score and the second reliability score, or, if the second reliability is within an interval defined by the first reliability threshold and the second reliability threshold, determining a third reliability score based on the second reliability, the second reliability threshold and the first reliability threshold, determining a fourth reliability score based on the second reliability threshold and the first reliability threshold, and determining a first evaluation score of the second reliability based on the third reliability score and the fourth reliability score.

[0129] Preferably, the evaluation score determination module further includes: when the first reliability or the second reliability is less than a first reliability threshold, the first set reliability score is the second evaluation score of the first reliability or the second evaluation score of the second reliability; when the first reliability is equal to or greater than the first reliability threshold or the second reliability is equal to or greater than a second reliability threshold, the first set reliability score is the second evaluation score of the first reliability or the second evaluation score of the second reliability; and when the first reliability is within an interval defined by the first reliability threshold and the second reliability threshold, determine a fifth reliability score based on the first reliability and the second reliability threshold. determining a sixth confidence score based on the second confidence threshold and the first confidence threshold; determining a second evaluation score of the first confidence level based on the fifth confidence score and the sixth confidence score; or, if the second confidence level is within the interval defined by the first confidence threshold and the second confidence threshold, determining a seventh confidence score based on the second confidence level and the second confidence threshold; determining an eighth confidence score based on the second confidence threshold and the first confidence threshold; and determining a second evaluation score of the second confidence level based on the seventh confidence score and the eighth confidence score.

[0130] Preferably, the evaluation score determination module is further used for: when the comparison type is a false positive example, determining a first evaluation score for the second area and setting the first evaluation score for the second area as the area score of the target object; when the comparison type is a false negative example, determining a first evaluation score for the first area and setting the first evaluation score for the first area as the area score of the target object; and when the comparison type is a true positive example, determining a second evaluation score for the first area and a second evaluation score for the second area respectively, and determining the area score of the target object based on the second evaluation score for the first area and the second evaluation score for the second area.

[0131] Preferably, the evaluation score determination module further acquires a first area threshold and a second area threshold, and when the second area threshold is greater than the first area threshold and the first area or the second area is smaller than the first area threshold, sets a first set area score as a first evaluation score of the first area or a first evaluation score of the second area, when the first area is equal to or greater than the second area threshold or the second area is equal to or greater than the second area threshold, sets a second set area score as a first evaluation score of the first area or a first evaluation score of the second area, when the first area is within a range defined by the first area threshold and the second area threshold, and a first area threshold; determining a second area score based on a second area threshold and the first area threshold; determining a first evaluation score for the first area based on the first area score and the second area score; or, if the second area is within the interval defined by the first area threshold and the second area threshold, determining a third area score based on the second area, the second area threshold and the first area threshold; determining a fourth area score based on the second area threshold and the first area threshold; and determining a first evaluation score for the second area based on the third area score and the fourth area score.

[0132] Preferably, the evaluation score determination module further includes: when the first area or the second area is smaller than a first area threshold, a first set area score is a second evaluation score of the first area or a second evaluation score of the second area; when the first area is equal to or larger than the first area threshold or the second area is equal to or larger than the first area threshold, a second set area score is a second evaluation score of the first area or a second evaluation score of the second area; and when the first area is within an interval defined by the first area threshold and the second area threshold, a fifth area score is determined based on the first area and the first area threshold. determining a sixth area score based on the second area threshold and the first area threshold; determining a second evaluation score for the first area based on the fifth area score and the sixth area score; or, if the second area is within the interval defined by the first area threshold and the second area threshold, determining a seventh area score based on the second area and the first area threshold; determining an eighth area score based on the second area threshold and the first area threshold; and determining a second evaluation score for the second area based on the seventh area score and the eighth area score.

[0133] Preferably, the evaluation score determination module is further used to input the initial autonomous driving image into an autonomous driving scene model and output predicted scene information, obtain set scene information corresponding to the initial autonomous driving image, determine scene similarity based on the predicted scene information and the set scene information, and determine a scene score of the initial autonomous driving image based on the scene similarity.

[0134] Preferably, the target autonomous driving image dataset selection module is specifically used for ranking the plurality of initial autonomous driving images based on the evaluation scores of the plurality of initial autonomous driving images, obtaining an initial autonomous driving image dataset after ranking, and selecting a set number of target autonomous driving image datasets from the initial autonomous driving image dataset after ranking.

[0135] An image dataset processing apparatus according to an embodiment of the present application is capable of executing an image dataset processing method according to any embodiment of the present application, and comprises functional modules and beneficial effects corresponding to the execution of the method.

[0136] Note that the units and modules of the above-mentioned device are merely divided according to functional logic and are not limited to the above divisions, as long as they can realize the corresponding functions. Furthermore, the specific names of the functional units are merely used to make them easier to distinguish from one another, and do not limit the scope of protection of the embodiments of the present application.

[0137] FIG. 11 is a structural schematic diagram of an electronic device according to an embodiment of the present application. Hereinafter, referring to FIG. 11, a structural schematic diagram of an electronic device (e.g., a terminal device or server in FIG. 11) 1100 suitable for implementing the embodiment of the present application is shown. The terminal device in the embodiment of the present application may include, but is not limited to, mobile terminals such as mobile phones, notebook computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), in-car terminals (e.g., in-car navigation terminals), etc., and fixed terminals such as digital TVs and desktop computers, etc. The electronic device shown in FIG. 11 is merely an example and does not impose any limitations on the functions and scope of use of the embodiment of the present application.

[0138] 11, the electronic device 1100 may include a processing unit (e.g., a central processor, a graphics processor, etc.) 1101, which can perform various appropriate operations and processes based on programs stored in a read-only memory (ROM) 1102 or programs loaded from a storage device 1108 into a random access memory (RAM) 1103. The RAM 1103 further stores various programs and data necessary for the operation of the electronic device 1100. The processing unit 1101, the ROM 1102, and the RAM 1103 are connected to each other via a bus 1104. An input / output (I / O) interface 115 is also connected to the bus 1104.

[0139] Typically, input devices 1106 including, for example, a touch panel, touchpad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, etc., output devices 1107 including, for example, a liquid crystal display (LCD), speaker, vibrator, etc., storage devices 1108 including, for example, a magnetic tape, hard disk, etc., and communication devices 1109 can be connected to the I / O interface 1105. The communication devices 1109 enable the electronic device 1100 to communicate wirelessly or via wires with other devices to exchange data. While FIG. 11 illustrates the electronic device 1100 with various devices, it should be understood that it is not necessary for the electronic device 1100 to implement or include all of the illustrated devices. Instead, more or fewer devices may be implemented or included.

[0140] In particular, according to embodiments of the present application, the processes described with reference to the flowcharts above may be implemented as a computer software program. For example, embodiments of the present application include a computer program product including a computer program carried on a non-transitory computer-readable medium, the computer program including program code for performing the methods illustrated in the flowcharts. In such embodiments, the computer program may be downloaded and installed from a network via the communication device 1109, installed from the storage device 1108, or installed from the ROM 1102. When the computer program is executed by the processing device 1101, it performs the functions defined by the methods of the embodiments of the present application.

[0141] The names of messages or information exchanged between devices in the embodiments of the present application are for illustrative purposes only and are not intended to limit the scope of these messages or information.

[0142] The electronic device according to the embodiment of the present application belongs to the same inventive idea as the image dataset processing method according to the above embodiment, and the technical details not described in detail in this embodiment can be referred to the above embodiment, and this embodiment has the same beneficial effects as the above embodiment.

[0143] An embodiment of the present application provides a computer storage medium having stored thereon a computer program which, when executed by a processor, implements the image dataset processing method according to the embodiment.

[0144] It should be noted that the computer-readable medium described herein may be a computer-readable signal medium, a computer-readable storage medium, or any combination thereof. The computer-readable storage medium may be, for example, but is not limited to, an electric, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. Further specific examples of the computer-readable storage medium may include, but are not limited to, an electrical connection having one or more leads, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this application, a computer-readable storage medium may be any tangible medium that contains or stores a program that can be used in or in connection with an instruction execution system, apparatus, or device. In this application, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, having computer-readable program code carried therein. Such propagated data signals may take various forms, including, but not limited to, electromagnetic signals, optical signals, or any suitable combination of the foregoing. A computer-readable signal medium may be any computer-readable medium other than a computer-readable storage medium, which is capable of transmitting, propagating, or transmitting a program for use in or in connection with an instruction execution system, apparatus, or device. The program code contained in the computer-readable medium may be transmitted over any suitable medium, including, but not limited to, electrical wire, optical cable, RF (radio frequency), etc., or any suitable combination of the foregoing.

[0145] In some embodiments, the clients and servers may communicate using any now known or later developed network protocol, such as HTTP (Hyper Text Transfer Protocol), and may interconnect with any form or medium of digital data communication (e.g., a communications network). Examples of communications networks include local area networks ("LANs"), wide area networks ("WANs"), networks off networks (e.g., the Internet), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), and any now known or later developed networks.

[0146] The computer-readable medium may be included in the electronic device, or may exist independently and not be attached to the electronic device.

[0147] The computer-readable medium carries one or more programs, and when the one or more programs are executed by the electronic device, the electronic device An initial autonomous driving image dataset including a plurality of initial autonomous driving images and corresponding tags is obtained, the initial autonomous driving image dataset is input into a first autonomous driving detection model and a second autonomous driving detection model, respectively, a first image detection result and a second image detection result are obtained, the first image detection result is compared with the first image detection result and the second image detection result as a reference detection result, a comparison result is obtained, a comparison type is determined based on the comparison result, the comparison type includes true positive examples, false positive examples, and false negative examples, an evaluation score of the plurality of initial autonomous driving images is determined based on the comparison type, a target autonomous driving image dataset is selected from the initial autonomous driving image dataset based on the evaluation scores of the plurality of initial autonomous driving images, and the tags in the target autonomous driving image dataset are adjusted.

[0148] Computer program code for carrying out the operations of the present application can be written in one or more programming languages, or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may run entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or a business server. When referring to a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., connected via the Internet using an Internet Service Provider).

[0149] The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present application. In this regard, each block in the flowcharts or block diagrams may represent a module, program, or portion of code, which includes one or more executable instructions for implementing a given logical function. It should be noted that in some alternative implementations, the functions noted in the blocks may occur in a different order than that depicted in the figures. For example, two blocks shown as connected may actually execute substantially in parallel or in reverse order, depending on the functionality involved. It should be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented in a dedicated hardware-based system that executes a given function or operation, or in a combination of dedicated hardware and computer instructions.

[0150] The units according to the embodiments of the present application may be implemented in a software manner or a hardware manner, and the names of the units may not necessarily be used to limit the units themselves, for example, the first acquiring unit may be described as "a unit for acquiring at least two Internet Protocol addresses."

[0151] The functionality herein may be performed, at least in part, by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard parts (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), etc.

[0152] In this specification, a machine-readable medium may be a tangible medium that can contain or store a program used in or in connection with an instruction execution system, apparatus, or device. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the above. Further specific examples of a machine-readable storage medium include an electrical connection of one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk-read-only disk (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0153] The above description merely describes the preferred embodiments and technical principles used in the present application. Those skilled in the art should understand that the scope of the present application is not limited to a technical solution composed of a specific combination of the above technical features, but also includes other technical solutions formed by any combination of the above technical features or their equivalent features, provided that the scope does not deviate from the spirit of the above disclosure. For example, a technical solution formed by mutually replacing the above features with technical features having similar functions (including but not limited to) disclosed in the present application.

[0154] Additionally, although operations are described in a particular order, it should not be understood that these operations must be performed in the particular order shown, or in any forward order. In certain environments, multitasking and parallel processing may be advantageous. Similarly, although the above discussion includes several specific implementation details, these should not be understood as limiting the scope of the present application. Some features described in the description of a single embodiment may also be implemented in combination in a single embodiment. Conversely, various features described in the description of a single embodiment may be implemented in multiple embodiments, either alone or in any suitable subcombination.

[0155] Although the present subject matter has been described using language specific to structural features and / or methodological operations, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or operations described above. Rather, the specific features and operations described above are merely example forms of implementing the claims.

Claims

1. obtaining an initial autonomous driving image dataset including a plurality of initial autonomous driving images and corresponding tags; Inputting the initial autonomous driving image data set into a first autonomous driving detection model and a second autonomous driving detection model, respectively, to obtain a first image detection result and a second image detection result, respectively; comparing the first image detection result with the second image detection result to obtain a comparison result, and setting the first image detection result as a reference detection result; determining a comparison type based on the comparison result, the comparison type including true positive examples, false positive examples, and false negative examples; determining an evaluation score for the plurality of initial autonomous driving images based on the comparison type; and selecting a target autonomous driving image dataset from the initial autonomous driving image dataset based on the evaluation scores of the plurality of initial autonomous driving images, and adjusting tags in the target autonomous driving image dataset. Image dataset processing methods.

2. Each initial autonomous driving image includes at least one target object; the first image detection result includes first detection frame information of at least one target object, a first category corresponding to each of the at least one target object, a first reliability of the first category, and a first area corresponding to the first detection frame information; the second image detection result includes second detection frame information of at least one target object, a second category corresponding to each of the at least one target object, a second reliability corresponding to the second category, and a second area corresponding to the second detection frame information; the comparison results include category comparison results; Comparing the first image detection result with the second image detection result and obtaining a comparison result includes: comparing the first category with the second category and obtaining a category comparison result including a category comparison match and a category comparison mismatch; The method of claim 1.

3. determining a comparison type based on the comparison result; determining an IoU based on first detection window information and the second detection window information; determining a first IoU setpoint based on the first area and / or the second area; and If the IoU is equal to or greater than a first IoU setting value and the category comparison result is a category comparison match, determining the comparison type as a true positive example; If the IoU is smaller than a first IoU setting value and / or the category comparison result is a category comparison mismatch, determine the comparison type as a false positive example and a false negative example; or If the first image detection result includes first detection frame information of the target object and the second image detection result does not include second detection frame information of the target object, determine the comparison type as a false negative example; determining the comparison type as a false positive example when the first image detection result does not include first detection frame information of the target object and the second image detection result includes second detection frame information of the target object; The method of claim 2.

4. Determining an evaluation score for the plurality of initial autonomous driving images based on the comparison type includes: For the evaluation score of any of the initial autonomous driving images, determining an evaluation score of the target object in the initial autonomous driving image based on a comparison type; determining a scene score for the initial autonomous driving image; determining an evaluation score of the initial autonomous driving image based on the evaluation score of the target object and the scene score; The method of claim 3.

5. Determining an evaluation score of the target object in the initial autonomous driving image based on the comparison type includes: determining at least one of an IoU score, a confidence score, and an area score for the target object based on the comparison type; Obtaining a preset category score corresponding to the target object; determining an evaluation score for the target object based on at least one of the IoU score, the confidence score, the area score, and the category score; The method of claim 4.

6. Determining the IoU score of the target object based on the comparison type includes: determining a second IoU setpoint based on the first area and / or the second area, the second IoU setpoint being greater than the first IoU setpoint; determining an IoU evaluation score based on the comparison type, the first IoU set value, the second IoU set value, and the IoU; The method of claim 5.

7. determining an IoU evaluation score based on the comparison type, the first IoU set value, the second IoU set value, and the IoU; If the comparison type is a false positive example or a false negative example and the IoU is smaller than the first IoU setting value, the IoU evaluation score is a first setting IoU evaluation score; If the comparison type is a true positive example and the IoU is equal to or greater than the second IoU setting value, the IoU evaluation score is a second setting IoU evaluation score; If the comparison type is a true positive example and the IoU is within an interval defined by the first IoU set value and the second IoU set value, determining a first IoU evaluation score based on the second IoU set value and the IoU; determining a second IoU evaluation score based on the second IoU set value and the first IoU set value; determining an IoU evaluation score based on the first IoU evaluation score and the second IoU evaluation score; The method of claim 6.

8. Determining a confidence score for a target object based on a comparison type includes: If the comparison type is a false positive example, determine a first evaluation score with a second reliability, and set the first evaluation score with the second reliability as a reliability score of the target object; If the comparison type is a false negative example, determine a first evaluation score of a first reliability, and set the first evaluation score of the first reliability as a reliability score of a target object; If the comparison type is a true positive example, respectively determining a second evaluation score of a first reliability and a second evaluation score of a second reliability, and determining a reliability score of the target object based on the second evaluation score of the first reliability and the second evaluation score of the second reliability; The method of claim 5.

9. Determining the first assessment score of the first reliability or determining the first assessment score of the second reliability may include: obtaining a first reliability threshold and a second reliability threshold, the second reliability threshold being greater than the first reliability threshold; When the first reliability or the second reliability is smaller than a first reliability threshold, a first set reliability score is set to a first evaluation score of the first reliability or a first evaluation score of the second reliability; When the first reliability is equal to or greater than a second reliability threshold, or when the second reliability is equal to or greater than a second reliability threshold, setting the second set reliability score to the first evaluation score of the first reliability or the first evaluation score of the second reliability; if the first confidence is within an interval defined by the first confidence threshold and the second confidence threshold, determining a first confidence score based on the first confidence, the second confidence threshold, and the first confidence threshold; determining a second confidence score based on the second confidence threshold and the first confidence threshold; determining a first assessment score of a first reliability based on the first reliability score and the second reliability score; or if the second confidence is within an interval defined by the first confidence threshold and the second confidence threshold, determining a third confidence score based on the second confidence, the second confidence threshold, and the first confidence threshold; determining a fourth confidence score based on the second confidence threshold and the first confidence threshold; determining a first evaluation score of a second reliability based on the third reliability score and the fourth reliability score; The method of claim 8.

10. Determining the second assessment score of the first reliability or determining the second assessment score of the second reliability may include: When the first reliability or the second reliability is smaller than a first reliability threshold, the first set reliability score is set to the second evaluation score of the first reliability or the second evaluation score of the second reliability; When the first reliability is equal to or greater than a first reliability threshold, or when the second reliability is equal to or greater than a second reliability threshold, setting the first set reliability score to a second evaluation score of the first reliability or a second evaluation score of the second reliability; if the first confidence is within an interval defined by the first confidence threshold and the second confidence threshold, determining a fifth confidence score based on the first confidence and the second confidence threshold; determining a sixth confidence score based on the second confidence threshold and the first confidence threshold; determining a second assessment score of a first reliability based on the fifth reliability score and the sixth reliability score; or if the second confidence is within an interval defined by the first confidence threshold and the second confidence threshold, determining a seventh confidence score based on the second confidence and the second confidence threshold; determining an eighth confidence score based on the second confidence threshold and the first confidence threshold; determining a second assessment score of a second reliability based on the seventh reliability score and the eighth reliability score; 10. The method of claim 9.

11. Determining the area score of the target object based on the comparison type includes: If the comparison type is a false positive example, determine a first evaluation score of a second area, and set the first evaluation score of the second area as an area score of the target object; If the comparison type is a false negative example, determine a first evaluation score of a first area, and set the first evaluation score of the first area as an area score of the target object; If the comparison type is a true positive example, respectively determining a second evaluation score of the first area and a second evaluation score of the second area, and determining an area score of the target object based on the second evaluation score of the first area and the second evaluation score of the second area. The method of claim 5.

12. Determining the first evaluation score of the first area or determining the first evaluation score of the second area may include: obtaining a first area threshold and a second area threshold, the second area threshold being greater than the first area threshold; When the first area or the second area is smaller than a first area threshold, a first set area score is set to a first evaluation score of the first area or a first evaluation score of the second area; When the first area is equal to or greater than a second area threshold, or when the second area is equal to or greater than a second area threshold, a second set area score is set to the first evaluation score of the first area or the first evaluation score of the second area; if the first area is within an interval defined by the first area threshold and the second area threshold, determining a first area score based on the first area, the second area threshold, and the first area threshold; determining a second area score based on the second area threshold and the first area threshold; determining a first evaluation score for the first area based on the first area score and the second area score; or if the second area is within an interval defined by the first area threshold and the second area threshold, determining a third area score based on the second area, the second area threshold, and the first area threshold; determining a fourth area score based on the second area threshold and the first area threshold; determining a first assessment score for the second area based on the third area score and the fourth area score; The method of claim 11.

13. Determining the second evaluation score of the first area or determining the second evaluation score of the second area includes: When the first area or the second area is smaller than a first area threshold, a first set area score is set to a second evaluation score of the first area or a second evaluation score of the second area; When the first area is equal to or greater than a first area threshold, or when the second area is equal to or greater than a first area threshold, setting the second set area score to a second evaluation score of the first area or a second evaluation score of the second area; if the first area is within an interval defined by the first area threshold and the second area threshold, determining a fifth area score based on the first area and the first area threshold; determining a sixth area score based on the second area threshold and the first area threshold; determining a second assessment score for the first area based on the fifth area score and the sixth area score; or if the second area is within an interval defined by the first area threshold and the second area threshold, determining a seventh area score based on the second area and the first area threshold; determining an eighth area score based on the second area threshold and the first area threshold; determining a second assessment score for a second area based on the seventh area score and the eighth area score; The method of claim 12.

14. Determining a scene score of the initial autonomous driving image includes: Inputting the initial autonomous driving image into an autonomous driving scene model and outputting predicted scene information; Acquiring setting scene information corresponding to the initial autonomous driving image; determining a scene similarity based on the predicted scene information and the set scene information; determining a scene score for the initial autonomous driving image based on the scene similarity; The method of claim 4.

15. Selecting a target autonomous driving image dataset from the initial autonomous driving image dataset based on the evaluation scores of the plurality of initial autonomous driving images includes: Ranking the plurality of initial autonomous driving images based on the evaluation scores of the plurality of initial autonomous driving images, and obtaining a ranked initial autonomous driving image dataset; selecting a set number of target autonomous driving image datasets from the ranked initial autonomous driving image datasets; The method of claim 1.

16. an initial autonomous driving image dataset acquisition module configured to acquire an initial autonomous driving image dataset including a plurality of initial autonomous driving images and corresponding tags; an image detection result acquisition module configured to input the initial autonomous driving image dataset into a first autonomous driving detection model and a second autonomous driving detection model, respectively, to obtain a first image detection result and a second image detection result, respectively; an image detection result comparison module configured to compare the first image detection result with the second image detection result to obtain a comparison result, the image detection result comparison module taking the first image detection result as a reference detection result; a comparison type determination module configured to determine a comparison type based on the comparison result, the comparison type including true positive examples, false positive examples, and false negative examples; an evaluation score determination module configured to determine an evaluation score for the plurality of initial autonomous driving images based on the comparison type; a target autonomous driving image dataset selection module configured to select a target autonomous driving image dataset from the initial autonomous driving image dataset based on the evaluation scores of the plurality of initial autonomous driving images, and to adjust tags in the target autonomous driving image dataset. Image dataset processing unit.

17. at least one processor; a storage device configured to store at least one program; When said at least one program is executed by said at least one processor, said at least one processor implements the image dataset processing method according to any one of claims 1 to 15. electronic equipment.

18. comprising computer executable instructions for, when executed by a computer processor, performing the image dataset processing method of any one of claims 1 to 15, storage medium.

Images