Method, device and medium for optimizing target detection model

Abstract

The application relates to the technical field of automatic driving and discloses a target detection model optimization method and device, equipment and a medium, which comprises the following steps: acquiring a target image, processing the target image based on a first model and a second model respectively, expanding the target image if the overall difference degree is greater than a preset difference threshold, processing the expanded sample based on the second model, and calculating consistency based on the detection result; if the overall consistency is greater than a preset consistency threshold, adding the detection result of the second model to a training data set, and optimizing the first model based on the training data set. In the case that the overall difference degree of the detection result output by the first model and the detection result output by the second model is relatively large, the method expands the samples with significant differences, reduces the number of samples that need to be manually labeled, verifies the consistency of the expanded samples, ensures that the labeled results included in the training data set are reliable, and thus improves the detection precision of the first model.

Description

Technical Field

[0001] This invention relates to the field of autonomous driving technology, and specifically to methods, apparatus, equipment and media for optimizing target detection models. Background Technology

[0002] With the development of autonomous driving technology, object detection, as a core module of environmental perception, determines the safety performance of autonomous driving systems through its detection accuracy and robustness. In existing technologies, due to the complexity and diversity of autonomous driving scenarios, high-quality manual annotation requires significant manpower and time costs, and the annotation results are easily affected by subjective factors. Furthermore, the detection performance is poor for long-tail scenarios; when faced with sparse samples that appear infrequently in the road environment, the detection accuracy is easily affected, making it difficult to meet practical detection needs. Summary of the Invention

[0003] In view of this, the present invention provides an optimization method, apparatus, device and medium for target detection models to solve the problem of low target detection efficiency in autonomous driving scenarios.

[0004] In a first aspect, the present invention provides a method for optimizing a target detection model, the method comprising:

[0005] A target image is acquired and processed based on a first model and a second model respectively to obtain the detection results output by the first model and the second model. The detection results include at least the bounding box coordinates of the target object in the target image and the category of the target object in the target image. The first model is deployed on the target vehicle and the second model is deployed in the cloud.

[0006] Calculate the overall difference between the detection results output by the first model and the detection results output by the second model. If the overall difference is greater than a preset difference threshold, then the target image is augmented to obtain an augmented sample.

[0007] The expanded sample is processed based on the second model to obtain the first detection result;

[0008] Based on the first detection result, the bounding box consistency and category consistency are calculated, and the bounding box consistency and category consistency are fused to obtain the overall consistency;

[0009] If the overall consistency is greater than a preset consistency threshold, the detection results of the second model are added to the training dataset, and the first model is optimized based on the training dataset.

[0010] In one optional implementation, calculating the overall difference between the detection results output by the first model and the detection results output by the second model includes:

[0011] Based on the bounding box coordinates of the target object in the detection results output by the first model and the bounding box coordinates of the target object in the detection results output by the second model, the bounding box difference is calculated.

[0012] Based on the target object category in the detection results output by the first model and the target object category in the detection results output by the second model, calculate the category difference.

[0013] The overall difference is obtained by fusing the bounding box difference and the category difference.

[0014] In one optional implementation, calculating the bounding box difference based on the bounding box coordinates of the target object in the detection results output by the first model and the bounding box coordinates of the target object in the detection results output by the second model includes:

[0015] Calculate the intersection-union ratio (IoU) of the bounding box coordinates of the target object in the detection results output by the first model and the bounding box coordinates of the target object in the detection results output by the second model;

[0016] Calculate the sum of the intersection-union ratios (IURs) for all target objects, and determine the bounding box difference based on the sum of the IURs.

[0017] In one optional implementation, calculating the category difference based on the target object categories in the detection results output by the first model and the target object categories in the detection results output by the second model includes:

[0018] Calculate the cross-entropy between the predicted probability distribution of the target object's category in the detection results output by the first model and the predicted probability distribution of the corresponding target object's category in the detection results output by the second model;

[0019] Calculate the sum of cross-entropy for all target objects, and determine the class dissimilarity based on the sum of cross-entropy.

[0020] In one optional implementation, the first detection result includes a first bounding box detection result and a first category prediction result, the bounding box consistency includes bounding box position consistency and bounding box size consistency, and the calculation of bounding box consistency and category consistency based on the first detection result includes:

[0021] Based on the bounding box coordinates in the first bounding box detection result, calculate the bounding box position consistency;

[0022] Based on the bounding box dimensions in the first bounding box detection result, calculate the bounding box dimension consistency.

[0023] Class consistency is calculated based on the class prediction probability vector of each augmented sample in the first class prediction result.

[0024] In one optional implementation, calculating the bounding box position consistency based on the bounding box coordinates in the first bounding box detection result includes:

[0025] Extract the center coordinates of the bounding box of each expanded sample after inverse transformation, and calculate the average value of the center coordinates of all expanded samples;

[0026] Calculate the Euclidean distance between the center coordinates of the bounding boxes of each expanded sample after inverse transformation and the average value of the center coordinates, and calculate the positional consistency of the bounding boxes based on the Euclidean distance.

[0027] In one optional implementation, calculating the bounding box size consistency based on the bounding box size in the first bounding box detection result includes:

[0028] Extract the bounding box dimensions of each augmented sample after inverse transformation, and calculate the average bounding box dimensions of all augmented samples;

[0029] Calculate the Euclidean distance between the bounding box size of each augmented sample and the average of the bounding box sizes, and calculate the bounding box size consistency based on the Euclidean distance.

[0030] Secondly, the present invention provides an optimization apparatus for a target detection model, the apparatus comprising:

[0031] An image acquisition module is used to acquire a target image and process the target image based on a first model and a second model respectively to obtain the detection results output by the first model and the detection results output by the second model. The detection results include at least the bounding box coordinates of the target object in the target image and the category of the target object in the target image. The first model is deployed on the target vehicle and the second model is deployed in the cloud.

[0032] The image augmentation module is used to calculate the overall difference between the detection results output by the first model and the detection results output by the second model. If the overall difference is greater than a preset difference threshold, the target image is augmented to obtain an augmented sample.

[0033] The first detection module is used to process the expanded sample based on the second model to obtain a first detection result;

[0034] The consistency determination module is used to calculate bounding box consistency and category consistency based on the first detection result, and fuse the bounding box consistency and the category consistency to obtain overall consistency;

[0035] The model optimization module is used to add the detection results of the second model to the training dataset and optimize the first model based on the training dataset if the overall consistency is greater than a preset consistency threshold.

[0036] Thirdly, the present invention provides a computer device, comprising: a memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, and the processor executing the computer instructions to perform the optimization method of the target detection model of the first aspect or any corresponding embodiment described above.

[0037] Fourthly, the present invention provides a computer-readable storage medium storing computer instructions for causing a computer to execute the optimization method of the target detection model described in the first aspect or any corresponding embodiment thereof. Attached Figure Description

[0038] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0039] Figure 1 This is a flowchart illustrating the optimization method for a target detection model according to an embodiment of the present invention;

[0040] Figure 2 This is a structural block diagram of an optimization device for a target detection model according to an embodiment of the present invention;

[0041] Figure 3 This is a schematic diagram of the hardware structure of a computer device according to an embodiment of the present invention. Detailed Implementation

[0042] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0043] According to an embodiment of the present invention, an optimization method for a target detection model is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0044] This embodiment provides an optimization method for an object detection model, used in a cloud-based model optimization module. Figure 1 This is a flowchart of an optimization method for a target detection model according to an embodiment of the present invention, such as... Figure 1 As shown, the process includes the following steps:

[0045] Step S101: Obtain the target image, and process the target image based on the first model and the second model respectively to obtain the detection result output by the first model and the detection result output by the second model.

[0046] The detection results include at least the bounding box coordinates of the target object in the target image and the category of the target object in the target image. The first model is deployed on the target vehicle, and the second model is deployed in the cloud. This embodiment of the invention applies to the model optimization module deployed in the cloud, which is used to optimize the first model deployed on the target vehicle.

[0047] The target images are real-time road scene images captured by the target vehicle during its operation using onboard cameras, LiDAR, and other devices. These images include various target objects such as pedestrians, vehicles, and obstacles. The first model is a lightweight target detection model deployed on the target vehicle to meet the real-time perception requirements of autonomous driving scenarios. It can quickly process the captured scene images and output detection results. The second model is deployed in the cloud, possessing higher detection accuracy and ensuring the reliability of detection results through a more complex network structure and ample computing resources.

[0048] The acquired target images are input into the first model and the second model respectively. The two models are applied to the target images in parallel and output detection results respectively. The detection results include at least the bounding box coordinates of the target object in the target image and the category of the target object. The bounding box coordinates are used to locate the position range of the target object in the target image, and the category information is used to identify the type of the target object. The types of target objects include pedestrians, small cars, large vehicles (such as trucks), traffic lights, etc.

[0049] Step S102: Calculate the overall difference between the detection results output by the first model and the detection results output by the second model. If the overall difference is greater than a preset difference threshold, then the target image is augmented to obtain augmented samples.

[0050] The detection results from the first and second models are aggregated in the cloud-based model optimization module, and the overall difference between them is calculated. The overall difference is calculated based on core information from the detection results, using a multi-dimensional difference metric to comprehensively evaluate the differences between the two models. For bounding box coordinates, an index related to the degree of overlap between the predicted bounding boxes of the two models for the same target object is calculated, quantifying the difference in location and range predictions. For target object categories, an index reflecting the consistency of category judgments is calculated. The above two types of difference indices are weighted and fused using preset weighting coefficients, which can be adjusted according to the needs of the actual application scenario.

[0051] The preset difference threshold is a judgment threshold set in advance based on factors such as model performance requirements and application scenario accuracy standards. It is used to screen samples with optimization value. When the overall difference is greater than the threshold, it indicates that there is a significant difference between the first model and the second model in the detection results of the target image. The target image belongs to the sample related to the performance weakness of the first model. Therefore, it can be augmented to generate augmented samples, providing sufficient data support for subsequent reliability verification. If the overall difference is less than or equal to the preset difference threshold, it indicates that the detection results of the two models are highly consistent. The target image contributes little to the optimization of the first model, and the output of the second model can be directly added to the training dataset.

[0052] The purpose of augmenting the selected target images is to generate multiple augmented samples of different forms through diverse image transformations, thereby comprehensively verifying the stability of the detection results of the second model.

[0053] In some optional implementations, the target image is augmented using geometric transformations and photometric transformations. Geometric transformations include rotation, scaling, and translation, while photometric transformations include brightness adjustment and contrast adjustment. Through augmentation, a new image is generated. One expanded sample: ,in , Indicates the first Types of transformation operations.

[0054] Step S103: Process the expanded sample based on the second model to obtain the first detection result.

[0055] The generated expanded samples are input into the second model, which performs target detection processing on the expanded samples and outputs the first detection result, which includes the bounding box coordinates and category of the target object in each expanded sample.

[0056] In some alternative implementations, the second model infers from the augmented samples to obtain a set of detection results. ,in Includes bounding box and category probability .

[0057] Step S104: Calculate bounding box consistency and category consistency based on the first detection result, and fuse the bounding box consistency and category consistency to obtain overall consistency.

[0058] Bounding box consistency is used to evaluate the stability of the second model's predictions of target object bounding boxes across different augmented samples. It is calculated based on the bounding box coordinates in the first detection results. A unified coordinate system is applied to the bounding box coordinates corresponding to each augmented sample to ensure that all bounding box data are compared based on the same benchmark. The difference between the bounding box-related features of each augmented sample and the average of the bounding box-related features of all augmented samples is calculated, and normalization is used to eliminate the influence of factors such as image size. Finally, the bounding box consistency is calculated based on the degree of difference; a higher value indicates that the second model's predictions of bounding boxes are more stable.

[0059] Category consistency is used to evaluate the stability of the second model's predictions of target object categories across different augmented samples. It is calculated based on the category information from the first detection results. Category prediction-related data for each augmented sample is extracted, and a category prediction feature vector is constructed. The similarity between the category prediction feature vector of each augmented sample and the average of the category prediction feature vectors of all augmented samples is calculated. Finally, by statistically processing all similarity results, the category consistency is obtained. A higher value for this index indicates that the second model's category predictions are more stable.

[0060] A weighted fusion method can be used to integrate bounding box consistency and category consistency to obtain overall consistency. During the fusion process, corresponding weight coefficients can be set according to the different requirements for bounding box prediction accuracy and category prediction accuracy in actual application scenarios, with the sum of the weight coefficients being 1. Through weighted summation, the two consistency indices are transformed into a single overall consistency index, comprehensively reflecting the reliability of the second model's detection results for the target image.

[0061] Step S105: If the overall consistency is greater than the preset consistency threshold, the detection results of the second model are added to the training dataset, and the first model is optimized based on the training dataset.

[0062] The preset consistency threshold is a pre-defined reliability criterion used to evaluate the credibility of the second model's detection results. When the overall consistency is greater than the preset consistency threshold, it indicates that the second model exhibits stable predictive performance in the detection of different augmented samples, and its detection results for the original target image have high reliability, and can be directly added to the preset training dataset as valid labeled data.

[0063] A pre-set training dataset is used to store various high-value labeled data, which is continuously accumulated and updated as the optimization process progresses. Based on the updated pre-set training dataset, the first model is optimized using incremental or full training methods. This allows the first model to specifically learn target features related to its own detection weaknesses, gradually improving its detection performance in significantly different scenarios. Through continuous iteration of this optimization process, the detection accuracy of the first model is steadily improved, better meeting the perception needs of autonomous driving scenarios.

[0064] In some optional implementations, if the overall consistency is less than or equal to a preset consistency threshold, the detection results of the second model are sent to the manual annotation process to ensure annotation accuracy.

[0065] The target detection model optimization method provided in this embodiment includes acquiring a target image and processing the target image based on a first model and a second model respectively to obtain the detection results output by the first model and the second model; calculating the overall difference between the detection results output by the first model and the second model, and if the overall difference is greater than a preset difference threshold, expanding the target image to obtain expanded samples; processing the expanded samples based on the second model to obtain a first detection result; calculating bounding box consistency and category consistency based on the first detection result, and fusing the bounding box consistency and category consistency to obtain overall consistency; if the overall consistency is greater than a preset consistency threshold, adding the detection results of the second model to the training dataset, and optimizing the first model based on the training dataset. This method mines information-rich samples by calculating the overall difference between the detection results output by the first model and the second model. When the overall difference between the detection results output by the first model and the second model is large, it expands the samples with significant differences, reducing the number of samples requiring manual annotation, thereby reducing annotation costs. Meanwhile, by verifying the consistency of the expanded samples, we can ensure that the annotation results included in the training dataset are reliable, effectively avoid the accumulation of erroneous annotations, and continuously optimize the first model on the vehicle side, thereby improving the detection accuracy of the first model.

[0066] This embodiment provides an optimization method for an object detection model, which includes the following steps:

[0067] Step S201: Obtain the target image, and process the target image based on the first model and the second model respectively to obtain the detection result output by the first model and the detection result output by the second model.

[0068] Please see details Figure 1 Step S101 of the illustrated embodiment will not be described again here.

[0069] Step S202: Calculate the overall difference between the detection results output by the first model and the detection results output by the second model. If the overall difference is greater than a preset difference threshold, then the target image is augmented to obtain augmented samples.

[0070] Specifically, step S202 includes:

[0071] Step S2021: Calculate the bounding box difference based on the bounding box coordinates of the target object in the detection results output by the first model and the bounding box coordinates of the target object in the detection results output by the second model.

[0072] Bounding box difference is used to quantify the degree of difference between two models in predicting the location and extent of a target object. Specifically, it can be quantified through the overlap index of the bounding boxes.

[0073] Further, step S2021 includes: calculating the intersection-union ratio (IUR) of the bounding box coordinates of the target objects in the detection results output by the first model and the bounding box coordinates of the target objects in the detection results output by the second model; calculating the sum of IURs for all target objects, and determining the bounding box difference based on the sum of IURs.

[0074] For each target object jointly detected by the first and second models in the same frame of the target image, the bounding box coordinates of the target object output by the first model and the bounding box coordinates of the same target object output by the second model are extracted respectively. The intersection-union ratio (IUR) between the two bounding boxes of each pair of corresponding target objects is calculated. The IUR is the ratio of the area of ​​the overlapping region of the two bounding boxes to the area of ​​the merged region of the two bounding boxes, used to visually reflect the degree of overlap of the bounding boxes. The IURs corresponding to all jointly detected target objects are summed to obtain the total IUR. The bounding box difference is determined based on the total IUR, which can be derived by the ratio of the total IUR to the total number of target objects, ultimately obtaining a quantitative index that can characterize the difference in bounding box predictions between the two models.

[0075] The bounding box difference can be determined using the following formula:

[0076]

[0077] in, Indicates the difference in bounding boxes. Indicates the number of target objects. This indicates that the detection results output by the second model are for the first... The bounding box coordinates of the target object This indicates that the detection results output by the first model are for the first... The bounding box coordinates of the target object This indicates the calculation of the intersection-union ratio.

[0078] Step S2022: Calculate the category difference based on the category of the target object in the detection results output by the first model and the category of the target object in the detection results output by the second model.

[0079] Category dissimilarity is used to measure the consistency between two models in judging the type of the target object. Specifically, it can be accurately quantified by the difference in the probability distribution related to category prediction.

[0080] Further, step S2022 includes: calculating the cross-entropy between the category prediction probability distribution of the target object in the detection results output by the first model and the category prediction probability distribution of the corresponding target object in the detection results output by the second model; calculating the sum of the cross-entropy corresponding to all target objects, and determining the category difference degree based on the sum of the cross-entropy.

[0081] For each target object jointly detected by the first and second models in the same frame of the target image, the category prediction probability distribution of the target object output by the first model and the category prediction probability distribution of the same target object output by the second model are extracted. The cross-entropy function is used to calculate the cross-entropy between the two prediction probability distributions for each pair of corresponding target objects. The larger the cross-entropy value, the greater the difference between the two probability distributions, i.e., the more obvious the discrepancy in the category judgments of the two models. The cross-entropy values ​​corresponding to all jointly detected target objects are summed to obtain the total cross-entropy. The category difference is determined based on the total cross-entropy, specifically calculated as the ratio of the total cross-entropy to the total number of target objects.

[0082] Category dissimilarity can be determined using the following formula:

[0083]

[0084] in, Indicates the degree of class difference. Indicates the number of target objects. This indicates that the detection results output by the first model are for the first... Predicted probability distribution of the category of each target object This indicates that the detection results output by the second model are for the first... Predicted probability distribution of the category of each target object This represents the cross-entropy function.

[0085] Step S2023: The bounding box difference and the category difference are fused to obtain the overall difference.

[0086] Pre-set the weight coefficients corresponding to the bounding box differences Weight coefficients corresponding to category differences ,in and All are values ​​greater than 0, and satisfy the following conditions: The weighting coefficient can be adjusted according to the actual application scenario.

[0087] The overall variability can be determined using the following formula:

[0088]

[0089] in, Indicates the overall degree of difference. Indicates the degree of difference in bounding boxes. Indicates the degree of class difference.

[0090] Step S203: Process the expanded sample based on the second model to obtain the first detection result.

[0091] Please see details Figure 1 Step S103 of the illustrated embodiment will not be described again here.

[0092] Step S204: Calculate bounding box consistency and category consistency based on the first detection result, and fuse the bounding box consistency and category consistency to obtain overall consistency.

[0093] The first detection result includes the first bounding box detection result and the first category prediction result. Bounding box consistency includes bounding box position consistency and bounding box size consistency. Specifically, step S204 includes:

[0094] Step S2041: Calculate the consistency of bounding box position based on the bounding box coordinates in the first bounding box detection result.

[0095] Further, step S2041 includes: extracting the center coordinates of the bounding boxes of each expanded sample after inverse transformation, and calculating the average value of the center coordinates of all expanded samples; calculating the Euclidean distance between the center coordinates of the bounding boxes of each expanded sample after inverse transformation and the average value of the center coordinates, and calculating the consistency of the bounding box position based on the Euclidean distance.

[0096] The second model performs inference on the expanded samples to obtain the set of detection results. ,in Includes bounding box and category probability For each bounding box Applying inverse transform Obtain the bounding box in the original image coordinate system. .

[0097] The consistency of bounding box positions can be determined using the following formula:

[0098]

[0099]

[0100] in, This indicates consistency in the bounding box position. Indicates the number of augmented samples. Indicates the first The center coordinates of the bounding box after inverse transformation of the augmented sample This represents the average center coordinates of all augmented samples, and diag represents the normalization of the diagonal length of the original image.

[0101] Step S2042: Calculate the bounding box size consistency based on the bounding box size in the first bounding box detection result.

[0102] Further, step S2042 includes: extracting the bounding box size of each expanded sample after inverse transformation, and calculating the average value of the bounding box size of all expanded samples; calculating the Euclidean distance between the bounding box size of each expanded sample and the average value of the bounding box size, and calculating the consistency of the bounding box size based on the Euclidean distance.

[0103] The consistency of bounding box dimensions can be determined according to the following formula:

[0104]

[0105]

[0106] in, This indicates that the bounding box dimensions are consistent. Indicates the number of augmented samples. Indicates the first The bounding box dimensions (width and height) of the expanded sample after inverse transformation. This represents the average size of the bounding boxes of all expanded samples. To minimize the value, avoid dividing by zero.

[0107] Step S2043: Calculate class consistency based on the class prediction probability vector of each expanded sample in the first class prediction result.

[0108] Category consistency can be determined using the following formula:

[0109]

[0110]

[0111] in, Indicates category consistency. Indicates the first The class prediction probability vector of each expanded sample. Indicates the number of augmented samples. This represents the average probability vector of all augmented samples.

[0112] Pre-set the weighting coefficients corresponding to the consistency of bounding box positions. Weighting coefficients corresponding to the consistency of bounding box dimensions The weighting coefficient corresponding to category consistency , , as well as All are values ​​greater than 0, and satisfy the following conditions: The weighting coefficient can be adjusted according to the actual application scenario.

[0113] Overall Disparity It can be determined using the following formula:

[0114]

[0115] In step S205, if the overall consistency is greater than the preset consistency threshold, the detection results of the second model are added to the training dataset, and the first model is optimized based on the training dataset.

[0116] Please see details Figure 1 Step S105 of the illustrated embodiment will not be described again here.

[0117] The target detection model optimization method provided in this embodiment mines information-rich samples by calculating the overall difference between the detection results output by the first model and the second model. When the overall difference between the detection results output by the first model and the second model is large, the samples with significant differences are expanded, reducing the number of samples requiring manual annotation and thus lowering annotation costs. Simultaneously, by verifying the consistency of the expanded samples, the reliability of the annotation results included in the training dataset is ensured, effectively avoiding the accumulation of erroneous annotations, thereby continuously optimizing the first model on the vehicle side and improving its detection accuracy.

[0118] This embodiment also provides an optimization device for a target detection model, which is used to implement the above embodiments and preferred embodiments; details already described will not be repeated. As used below, the term "module" can be a combination of software and / or hardware that performs a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0119] This embodiment provides an optimization device for a target detection model, such as... Figure 2 As shown, it includes:

[0120] An image acquisition module is used to acquire a target image and process the target image based on a first model and a second model respectively to obtain the detection results output by the first model and the detection results output by the second model. The detection results include at least the bounding box coordinates of the target object in the target image and the category of the target object in the target image. The first model is deployed on the target vehicle and the second model is deployed in the cloud.

[0121] The image augmentation module is used to calculate the overall difference between the detection results output by the first model and the detection results output by the second model. If the overall difference is greater than a preset difference threshold, the target image is augmented to obtain an augmented sample.

[0122] The first detection module is used to process the expanded sample based on the second model to obtain a first detection result;

[0123] The consistency determination module is used to calculate bounding box consistency and category consistency based on the first detection result, and fuse the bounding box consistency and the category consistency to obtain overall consistency;

[0124] The model optimization module is used to add the detection results of the second model to the training dataset and optimize the first model based on the training dataset if the overall consistency is greater than a preset consistency threshold.

[0125] In some alternative implementations, the image augmentation module includes:

[0126] The bounding box difference calculation unit is used to calculate the bounding box difference based on the bounding box coordinates of the target object in the detection results output by the first model and the bounding box coordinates of the target object in the detection results output by the second model.

[0127] The category difference calculation unit is used to calculate the category difference based on the category of the target object in the detection result output by the first model and the category of the target object in the detection result output by the second model.

[0128] The overall difference calculation unit is used to fuse the bounding box difference and the category difference to obtain the overall difference.

[0129] In some optional implementations, the bounding box difference calculation unit includes:

[0130] The intersection-union ratio (IUGR) calculation subunit is used to calculate the IUGR of the bounding box coordinates of the target object in the detection results output by the first model and the bounding box coordinates of the target object in the detection results output by the second model.

[0131] The bounding box difference calculation subunit is used to calculate the sum of the intersection-union ratios (IURs) for all target objects and determine the bounding box difference based on the sum of IURs.

[0132] In some optional implementations, the category difference calculation unit includes:

[0133] The cross-entropy calculation subunit is used to calculate the cross-entropy between the category prediction probability distribution of the target object in the detection result output by the first model and the category prediction probability distribution of the corresponding target object in the detection result output by the second model.

[0134] The category difference calculation subunit is used to calculate the sum of cross-entropy for all target objects and determine the category difference based on the sum of cross-entropy.

[0135] In some optional implementations, the first detection result includes a first bounding box detection result and a first category prediction result; bounding box consistency includes bounding box position consistency and bounding box size consistency; and the consistency determination module includes:

[0136] The first consistency calculation unit is used to calculate the consistency of the bounding box position based on the bounding box coordinates in the first bounding box detection result;

[0137] The second consistency calculation unit is used to calculate the consistency of the bounding box size based on the bounding box size in the first bounding box detection result;

[0138] The third consistency calculation unit is used to calculate the class consistency based on the class prediction probability vector of each expanded sample in the first class prediction result.

[0139] In some optional implementations, the first consistency calculation unit includes:

[0140] The first extraction subunit is used to extract the center coordinates of the bounding box of each expanded sample after inverse transformation, and to calculate the average value of the center coordinates of all expanded samples.

[0141] The first calculation subunit is used to calculate the Euclidean distance between the center coordinates of the bounding boxes of each expanded sample after inverse transformation and the average value of the center coordinates, and to calculate the positional consistency of the bounding boxes based on the Euclidean distance.

[0142] In some optional implementations, the second consistency calculation unit includes:

[0143] The second extraction subunit is used to extract the bounding box size of each expanded sample after inverse transformation and to calculate the average value of the bounding box size of all expanded samples.

[0144] The second calculation subunit is used to calculate the Euclidean distance between the bounding box size of each augmented sample and the average value of the bounding box size, and to calculate the consistency of the bounding box size based on the Euclidean distance.

[0145] Further functional descriptions of the above modules and units are the same as those in the corresponding embodiments described above, and will not be repeated here.

[0146] In this embodiment, the optimization device for the target detection model is presented in the form of a functional unit. Here, a unit refers to an ASIC (Application Specific Integrated Circuit), a processor and memory that execute one or more software or fixed programs, and / or other devices that can provide the above functions.

[0147] This invention also provides a computer device having the above-described features. Figure 2 The target detection model shown is an optimization device.

[0148] Please see Figure 3 , Figure 3 This is a schematic diagram of the structure of a computer device provided in an optional embodiment of the present invention, such as... Figure 3 As shown, the computer device includes one or more processors 10, memory 20, and interfaces for connecting the components, including high-speed interfaces and low-speed interfaces. The components communicate with each other via different buses and can be mounted on a common motherboard or otherwise installed as needed. The processors can process instructions executed within the computer device, including instructions stored in or on memory to display graphical information of a GUI on external input / output devices (such as display devices coupled to the interfaces). In some alternative implementations, multiple processors and / or multiple buses can be used with multiple memories and multiple memory modules, if desired. Similarly, multiple computer devices can be connected, each providing some of the necessary operations (e.g., as a server array, a group of blade servers, or a multiprocessor system). Figure 3 Take a processor 10 as an example.

[0149] Processor 10 may be a central processing unit, a network processor, or a combination thereof. Processor 10 may further include a hardware chip. The hardware chip may be an application-specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof. The programmable logic device may be a complex programmable logic device (CAMP), a field-programmable gate array (FPGA), a general-purpose array logic (GDA), or any combination thereof.

[0150] The memory 20 stores instructions executable by at least one processor 10 to cause the at least one processor 10 to perform the method shown in the above embodiments.

[0151] The memory 20 may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data created based on the use of the computer device. Furthermore, the memory 20 may include high-speed random access memory and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some alternative embodiments, the memory 20 may optionally include memory remotely located relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0152] The memory 20 may include volatile memory, such as random access memory; the memory may also include non-volatile memory, such as flash memory, hard disk or solid-state drive; the memory 20 may also include a combination of the above types of memory.

[0153] The computer device also includes a communication interface 30 for communicating with other devices or communication networks.

[0154] This invention also provides a computer-readable storage medium. The methods described above according to embodiments of the invention can be implemented in hardware or firmware, or implemented as computer code that can be recorded on a storage medium, or implemented as computer code downloaded via a network and originally stored on a remote storage medium or a non-transitory machine-readable storage medium and then stored on a local storage medium. Thus, the methods described herein can be processed by software stored on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. The storage medium can be a magnetic disk, optical disk, read-only memory, random access memory, flash memory, hard disk, or solid-state drive, etc.; further, the storage medium can also include combinations of the above types of memory. It is understood that computers, processors, microprocessor controllers, or programmable hardware include storage components capable of storing or receiving software or computer code, which, when accessed and executed by the computer, processor, or hardware, implements the methods shown in the above embodiments.

[0155] Although embodiments of the invention have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of the invention, and all such modifications and variations fall within the scope defined by the invention.

Claims

1. An optimization method for a target detection model, characterized in that, The method includes: A target image is acquired and processed based on a first model and a second model respectively to obtain the detection results output by the first model and the second model. The detection results include at least the bounding box coordinates of the target object in the target image and the category of the target object in the target image. The first model is deployed on the target vehicle and the second model is deployed in the cloud. Calculate the overall difference between the detection results output by the first model and the detection results output by the second model. If the overall difference is greater than a preset difference threshold, then the target image is augmented to obtain an augmented sample. The expanded sample is processed based on the second model to obtain the first detection result; Based on the first detection result, the bounding box consistency and category consistency are calculated, and the bounding box consistency and category consistency are fused to obtain the overall consistency; If the overall consistency is greater than a preset consistency threshold, the detection results of the second model are added to the training dataset, and the first model is optimized based on the training dataset.

2. The optimization method for the target detection model according to claim 1, characterized in that, The calculation of the overall difference between the detection results output by the first model and the detection results output by the second model includes: Based on the bounding box coordinates of the target object in the detection results output by the first model and the bounding box coordinates of the target object in the detection results output by the second model, the bounding box difference is calculated. Based on the target object category in the detection results output by the first model and the target object category in the detection results output by the second model, calculate the category difference. The overall difference is obtained by fusing the bounding box difference and the category difference.

3. The optimization method for the target detection model according to claim 2, characterized in that, The step of calculating the bounding box difference based on the bounding box coordinates of the target object in the detection results output by the first model and the bounding box coordinates of the target object in the detection results output by the second model includes: Calculate the intersection-union ratio (IoU) of the bounding box coordinates of the target object in the detection results output by the first model and the bounding box coordinates of the target object in the detection results output by the second model; Calculate the sum of the intersection-union ratios (IURs) for all target objects, and determine the bounding box difference based on the sum of the IURs.

4. The optimization method for the target detection model according to claim 2, characterized in that, The calculation of category difference based on the target object categories in the detection results output by the first model and the target object categories in the detection results output by the second model includes: Calculate the cross-entropy between the predicted probability distribution of the target object's category in the detection results output by the first model and the predicted probability distribution of the corresponding target object's category in the detection results output by the second model; Calculate the sum of cross-entropy for all target objects, and determine the class dissimilarity based on the sum of cross-entropy.

5. The optimization method for the target detection model according to claim 1, characterized in that, The first detection result includes a first bounding box detection result and a first category prediction result. The bounding box consistency includes bounding box position consistency and bounding box size consistency. The calculation of bounding box consistency and category consistency based on the first detection result includes: Based on the bounding box coordinates in the first bounding box detection result, calculate the bounding box position consistency; Based on the bounding box dimensions in the first bounding box detection result, calculate the bounding box dimension consistency. Class consistency is calculated based on the class prediction probability vector of each augmented sample in the first class prediction result.

6. The optimization method for the target detection model according to claim 5, characterized in that, The step of calculating the bounding box position consistency based on the bounding box coordinates in the first bounding box detection result includes: Extract the center coordinates of the bounding box of each expanded sample after inverse transformation, and calculate the average value of the center coordinates of all expanded samples; Calculate the Euclidean distance between the center coordinates of the bounding boxes of each expanded sample after inverse transformation and the average value of the center coordinates, and calculate the positional consistency of the bounding boxes based on the Euclidean distance.

7. The optimization method for the target detection model according to claim 5, characterized in that, The step of calculating the bounding box size consistency based on the bounding box size in the first bounding box detection result includes: Extract the bounding box dimensions of each augmented sample after inverse transformation, and calculate the average bounding box dimensions of all augmented samples; Calculate the Euclidean distance between the bounding box size of each augmented sample and the average of the bounding box sizes, and calculate the bounding box size consistency based on the Euclidean distance.

8. An optimization device for a target detection model, characterized in that, The device includes: An image acquisition module is used to acquire a target image and process the target image based on a first model and a second model respectively to obtain the detection results output by the first model and the detection results output by the second model. The detection results include at least the bounding box coordinates of the target object in the target image and the category of the target object in the target image. The first model is deployed on the target vehicle and the second model is deployed in the cloud. The image augmentation module is used to calculate the overall difference between the detection results output by the first model and the detection results output by the second model. If the overall difference is greater than a preset difference threshold, the target image is augmented to obtain an augmented sample. The first detection module is used to process the expanded sample based on the second model to obtain a first detection result; The consistency determination module is used to calculate bounding box consistency and category consistency based on the first detection result, and fuse the bounding box consistency and the category consistency to obtain overall consistency; The model optimization module is used to add the detection results of the second model to the training dataset and optimize the first model based on the training dataset if the overall consistency is greater than a preset consistency threshold.

9. A computer device, characterized in that, include: A memory and a processor are communicatively connected, the memory stores computer instructions, and the processor executes the computer instructions to perform the optimization method of the target detection model according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions for causing a computer to perform the optimization method of the target detection model according to any one of claims 1 to 7.

Images