A method, apparatus, system, and storage medium for image processing.

Abstract

This application discloses a method, apparatus, system, and storage medium for image processing, applicable to the automotive field. The method includes: in response to a received target image, determining multiple bounding boxes corresponding to the target image; determining a conflict coefficient of the target image based on the multiple bounding boxes, and using the conflict coefficient to determine the target image as an image for training a target object recognition model. The conflict coefficient represents the difficulty of identifying a target object from the target image. Thus, by using the conflict coefficient, high-value images are selected and low-value images are filtered out, using high-value images as images for training the recognition model, thereby avoiding the problem of high manual costs associated with training vehicle recognition models with low-value images when labeling images.

Description

Technical Field

[0001] This application relates to the field of vehicle recognition, and in particular to a method, apparatus, system and storage medium for image processing. Background Technology

[0002] With the development of automotive technology, vehicle recognition models have become an important technical means for detecting traffic congestion and optimizing traffic, as they can identify vehicles in complex scenes.

[0003] Current technologies often involve pre-collecting massive amounts of image data before training vehicle recognition models. However, this massive image data includes both high-value and low-value images. High-value images are those whose vehicle-related information values ​​exceed a preset threshold. Low-value images are those whose vehicle-related information values ​​fall below the preset threshold. For example, an image that does not contain vehicle information is considered a low-value image. These low-value images are useless for training vehicle recognition models and increase the manual cost of image labeling. Summary of the Invention

[0004] In view of this, this application provides a method, apparatus, system and storage medium for image processing, which aims to reduce the human cost of image labeling by selecting high-value images from massive images as images for training vehicle recognition models.

[0005] Firstly, this application provides a method for image processing, the method comprising:

[0006] In response to the received target image, determine multiple bounding boxes corresponding to the target image;

[0007] Calculate the boundary box deviation of the multiple boundary boxes based on the multiple boundary boxes;

[0008] The collision coefficient of the target image is determined based on the boundary box deviations of the multiple bounding boxes; the collision coefficient is used to indicate the difficulty of recognizing the target object in the target image.

[0009] In response to the conflict coefficient being greater than a preset conflict threshold, the target image is used as the image for training the target object recognition model.

[0010] Optionally, the method further includes:

[0011] Obtain the predicted values ​​within the bounding boxes of the plurality of bounding boxes;

[0012] Calculate the in-boundary prediction bias of the multiple bounding boxes based on the in-boundary prediction values ​​of the multiple bounding boxes;

[0013] Determining the collision coefficient of the target image based on the multiple bounding box deviations includes:

[0014] The collision coefficient of the target image is determined based on the bounding box deviation and the in-bounding box prediction deviation of the plurality of bounding boxes.

[0015] Optionally, the predicted values ​​within the bounding box are the output results of the softmax layer;

[0016] The step of calculating the in-boundary prediction bias of the multiple bounding boxes based on their in-boundary prediction values ​​includes...

[0017] The softmax layer outputs of the multiple bounding boxes are divided into the softmax layer outputs of multiple targets;

[0018] The softmax layer outputs results for the same target are used to calculate the prediction bias within the bounding box of each of the multiple targets based on cross-entropy.

[0019] The prediction bias within the bounding boxes of the multiple targets is averaged to obtain the prediction bias within the bounding boxes of the multiple bounding boxes.

[0020] Optionally, determining the collision coefficient of the target image based on the bounding box deviation and the in-bounding box prediction deviation of the plurality of bounding boxes includes:

[0021] The conflict coefficient of the target image is calculated by substituting the bounding box deviation and the prediction deviation within the bounding box into the preset conflict coefficient calculation formula.

[0022] The preset conflict coefficient calculation formula is positively correlated with the bounding box deviation and positively correlated with the prediction deviation within the bounding box.

[0023] Optionally, the plurality of bounding boxes includes a plurality of first bounding boxes and a plurality of second bounding boxes;

[0024] The determination of multiple bounding boxes corresponding to the target image includes:

[0025] The target image is input into a first preset model to obtain the plurality of first bounding boxes; the target image is input into a second preset model to obtain the plurality of second bounding boxes;

[0026] Among them, the first preset model and the second preset model are models based on different principles;

[0027] The step of calculating the boundary box deviation of the plurality of boundary boxes includes:

[0028] Based on the preset bounding box deviation calculation formula, the bounding box deviation of the multiple bounding boxes is calculated using the multiple first bounding boxes and the multiple second bounding boxes;

[0029] The formula for calculating the bounding box deviation is positively correlated with the degree of difference among the multiple bounding boxes.

[0030] Optionally, the preset boundary box deviation calculation formula is the GIoU calculation formula.

[0031] Secondly, this application provides an image processing apparatus, the apparatus comprising:

[0032] The first response unit is used to determine multiple bounding boxes corresponding to the received target image in response to the target image.

[0033] The first calculation unit is used to calculate the boundary box deviation of the plurality of boundary boxes based on the plurality of boundary boxes;

[0034] The determining unit is used to determine the conflict coefficient of the target image based on the boundary box deviations of the plurality of bounding boxes; the conflict coefficient is used to indicate the difficulty of recognizing the target object in the target image;

[0035] The second response unit is used to use the target image as an image for training the target object recognition model in response to the conflict coefficient being greater than a preset conflict threshold.

[0036] Optionally, the device further includes:

[0037] The second calculation unit is used to calculate the in-boundary prediction bias of the plurality of bounding boxes; the second calculation unit is specifically used to: obtain the in-boundary prediction values ​​of the plurality of bounding boxes; and calculate the in-boundary prediction bias of the plurality of bounding boxes based on the in-boundary prediction values ​​of the plurality of bounding boxes.

[0038] The determining unit is specifically used to determine the conflict coefficient of the target image based on the boundary box deviation and the in-boundary prediction deviation of the plurality of boundary boxes.

[0039] Thirdly, this application provides a vehicle system including an image processing apparatus as described in the second aspect.

[0040] Fourthly, this application provides a computer storage medium storing code, wherein when the code is executed, a device executing the code implements the method described in any of the first aspects above.

[0041] This application discloses a method, apparatus, system, and storage medium for image processing. When executing the method, firstly, in response to a received target image, bounding boxes corresponding to the target image are determined. Based on multiple bounding boxes, the bounding box deviations are determined, and the conflict coefficient of the target image is calculated based on the bounding box deviations. The target image is then determined as an image for training a vehicle recognition model based on the conflict coefficient. The conflict coefficient represents the difficulty of recognizing objects in the target image. Thus, by using the conflict coefficient, high-value images are selected and low-value images are filtered out. High-value images are used as images for training the recognition model, thereby avoiding the problem of high manual costs associated with training the vehicle recognition model with low-value images. Attached Figure Description

[0042] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and the originals and elements are not necessarily drawn to scale.

[0043] Figure 1 A flowchart of an image processing method provided in this application embodiment;

[0044] Figure 2 A schematic diagram provided for an embodiment of this application;

[0045] Figure 3 A schematic diagram illustrating the acquisition of multiple first bounding boxes and multiple second bounding boxes provided in an embodiment of this application;

[0046] Figure 4 A schematic diagram of bounding box calculation provided for an embodiment of this application;

[0047] Figure 5 This is a schematic diagram of an image processing apparatus provided in an embodiment of this application. Detailed Implementation

[0048] The term "comprising" and its variations as used herein are open-ended inclusions, meaning "including but not limited to". The term "based on" means "at least partially based on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". Definitions of other terms will be given in the description below.

[0049] It should be noted that the concepts of "first" and "second" mentioned in this disclosure are used only to distinguish different devices, modules or units, and are not used to limit the order of functions performed by these devices, modules or units or their interdependencies.

[0050] It should be noted that the terms "a" and "a plurality of" used in this disclosure are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".

[0051] As mentioned earlier, training a vehicle recognition model first requires collecting massive amounts of image data, which is then input into the model to train it to recognize vehicles. However, the collected massive image data may include low-value images, i.e., images whose vehicle-related information is below a preset threshold. Inputting such low-value images into the vehicle recognition model does not improve the model's training effect, but it does increase the manual cost of subsequent image labeling.

[0052] Based on this, this application provides an image processing method that aims to obtain different bounding boxes corresponding to a target image, calculate the conflict coefficient of the target image based on the bounding box deviation, and select high-value images and filter out low-value images based on the conflict coefficient. High-value images are used as target images for training the recognition model to avoid the problem of high manual costs during image labeling caused by low-value images entering the training of the vehicle recognition model.

[0053] The image processing method provided in the embodiments of this application will now be described in detail with reference to the accompanying drawings.

[0054] See Figure 1 This is a flowchart illustrating an image processing method provided in an embodiment of this application. This method can be applied to a vehicle detection system, and its execution entity is the detection server within the detection system. The method includes the following steps:

[0055] S101: The detection server receives massive amounts of image data.

[0056] The detection system's server receives massive amounts of image data. This massive amount of images may include high-value and low-value images. High-value images are those whose target-object related information values ​​are higher than a preset relevant threshold. Low-value images are those whose target-object related information values ​​are lower than a preset relevant threshold.

[0057] Understandably, low-value images are used to indicate that object detection in an image is relatively easy, such as images with simple layouts or few or no objects. For example, assuming the object is a "vehicle," low-value images would be images with few or no vehicles. See also Figure 2 This is a schematic diagram provided for an embodiment of this application. Wherein, as shown... Figure 2 As shown in (a), this is a schematic diagram of a low-value image provided in an embodiment of this application, which does not contain vehicle information.

[0058] High-value images indicate that target detection in the image is more difficult. There are many reasons for this difficulty, such as the presence of a large number of objects within the image. For example, suppose the target object is a "vehicle". Figure 2 As shown in (b), this is a schematic diagram of a high-value image provided in an embodiment of this application, which contains a large number of vehicles.

[0059] In this embodiment, low-value images among a massive number of images have extremely poor training effects on the vehicle recognition model, but increase the manual cost of image labeling. Therefore, when training the vehicle recognition model, it is necessary to filter out low-value images. For example, in the vehicle recognition model, low-value images are filtered out... Figure 2 (a) Type of image. Only high-value images are selected. For example, in a vehicle recognition model, select... Figure 2 (b) type images are used as samples for model training.

[0060] S102: Determine multiple bounding boxes corresponding to the target image.

[0061] The detection server receives a massive number of images, including the target image. In this embodiment, the target image can be any image from the massive number of images.

[0062] To select high-value images and filter out low-value images, one possible implementation is to process the target image to obtain multiple bounding boxes corresponding to the target image.

[0063] In the embodiments of this application, multiple bounding boxes can be obtained in various ways, such as by using different recognition models to obtain multiple bounding boxes corresponding to the target object in the target image and the target prediction value within the bounding boxes.

[0064] In one possible implementation, the multiple bounding boxes include multiple first bounding boxes and multiple second bounding boxes. The bounding boxes represent the positions of objects in the target image and can be rectangular boxes or other shapes. No limitations are placed on the bounding boxes here. The following explanation will only use rectangular boxes as an example.

[0065] In this embodiment of the application, the first bounding box and the second bounding box are bounding boxes corresponding to the target image obtained by different methods.

[0066] In one possible implementation, a first bounding box can be obtained by inputting the target image into a first preset model, and a second bounding box can be obtained by inputting the target image into a second preset model. The first and second preset models have different implementation principles, but both input the target image and output two preset models of bounding boxes.

[0067] In one possible implementation, the first preset model is a first-order object recognition model, such as the YOLO series of algorithms. The second preset model is obtained from a second-order object recognition model, such as the Faster R-CNN model.

[0068] In one possible implementation, the first preset model is obtained by a second-order target recognition model, and the second preset model is obtained by a first-order target recognition model.

[0069] It is worth noting that the first and second bounding boxes can also be obtained in other ways in the embodiments of this application, and those skilled in the art can adjust them as needed.

[0070] To improve the efficiency and accuracy of filtering massive amounts of image data, the Faster R-CNN model can be selected for second-order object recognition, while the YOLOV7 model can be selected for first-order object recognition.

[0071] YoloV7 uses an architecture based on E-ELAN (Extended Efficient Layer Aggregation Network) and cascaded model scaling, employing a trainable BOF method, including planned reparameterization of convolutions. Cascaded model scaling considers factors such as resolution (i.e., the size of the input image), width (i.e., the number of channels), and depth (i.e., the number of network layers).

[0072] Train multiple models using different training data but with the same settings. Then average their weights to obtain the final model. Take the average of the model weights at different epochs to obtain the planned reparameterized convolution.

[0073] Therefore, compared to other models, such as YOLOv4 and YOLOv7, it has higher processing speed and accuracy.

[0074] The specific implementation process for obtaining the bounding box in YOLOv7 is as follows:

[0075] S1021: Set the number of categories.

[0076] For large images, such as 640*640 or 1280*1280, first preset the number of grids to divide the image into, and the number of bounding boxes predicted for each grid. For example, divide the image into 7*7 grids, and predict one bounding box for each grid.

[0077] S1022: Scale the image to 448*448, input it into a CNN convolutional network for processing, and output bounding boxes.

[0078] A CNN (Convolutional Neural Network) consists of two fully connected processes: convolution and pooling. It processes bounding boxes to obtain 7x7x2 values ​​and then outputs these bounding boxes.

[0079] S1023: Set threshold to filter borders.

[0080] Calculate the score for all vehicles in all bounding boxes, including bounding box confidence and bounding box grid position. Obtain the highest score for each bounding box and record its index. Filter by a threshold and create a mask. Output the threshold-filtered score, bounding box, and rating using the mask value.

[0081] S1024: Perform nonmaximum suppression.

[0082] Select the bounding box with the highest score and add it to the output list, then remove it from the bounding box list. Calculate the Interchange of Union (IOU) between the bounding box with the highest score and other candidate boxes, and remove bounding boxes with an IOU greater than a set threshold IOU. Repeat this process until the bounding box list is empty.

[0083] S1025: Display the bounding box of the final output list in the target image.

[0084] The Faster R-CNN network framework can be divided into two parts according to its model function: feature extraction and decision-making. The feature extraction part generates high-quality region proposal candidate boxes, which are then judged and refined using a classification function and a bounding box regression function to initially locate the target.

[0085] The target image is first subjected to feature extraction. Proposed feature blocks of the same size are extracted and input into the decision part. Then, the classification function is used to calculate the category of the proposed feature blocks, and the bounding box regression function is used to accurately detect the position of the bounding box.

[0086] Example illustration: See Figure 3 This diagram illustrates an embodiment of the present application for obtaining multiple first bounding boxes and multiple second bounding boxes. The white bounding boxes represent the second bounding boxes, obtained using Faster R-CNN, while the gray bounding boxes represent the first bounding boxes, obtained using YOLOv7.

[0087] In this embodiment, the color, thickness, and text style of the frame are independent of the model and are all settings related to image processing programs. Those skilled in the art can adjust them as needed.

[0088] It is worth noting that the first and second bounding boxes are only illustrative; a third, fourth, and other bounding boxes can also be obtained from the target image. This application does not limit the number of types of bounding boxes that can be obtained.

[0089] In addition, while acquiring multiple bounding boxes, this embodiment of the application can also simultaneously acquire the predicted values ​​within the bounding boxes of multiple bounding boxes, so as to use the predicted values ​​within the bounding boxes to determine the predicted value deviation within the bounding boxes, and combine the bounding box deviation to obtain the conflict coefficient of the target image.

[0090] In one possible implementation, the predicted value within the bounding box is the output of the softmax layer. That is, the output layer is combined with the softmax activation function to output a probability value between 0 and 1.

[0091] Furthermore, the predicted bias within the bounding box can also be the sigmoid output, i.e., the probability value output by the output layer and the sigmoid activation function. Here, the predicted bias within the bounding box is not limited to the softmax layer output.

[0092] The softmax layer outputs of multiple bounding boxes are divided into softmax layer outputs of multiple targets. For the softmax layer outputs of the same target, cross-entropy is used to calculate the in-boundary prediction bias for each target. The in-boundary prediction biases of the multiple targets are then averaged to obtain the in-boundary prediction bias of the multiple bounding boxes.

[0093] S103: Calculate the bounding box deviations of multiple bounding boxes.

[0094] The detection server calculates the bounding box deviations of the multiple bounding boxes based on the acquired bounding boxes.

[0095] In one possible implementation, the multiple bounding boxes include multiple first bounding boxes obtained from a first preset model and multiple second bounding boxes obtained from a second preset model. The multiple first bounding boxes and multiple second bounding boxes are input into a preset bounding box deviation calculation formula, and the bounding box deviations of the multiple bounding boxes can be calculated.

[0096] The formula for calculating bounding box deviation is positively correlated with the degree of difference among multiple bounding boxes. This means that for a target image, different methods of labeling the target object will result in different bounding boxes. For low-value images, such as those without target objects, there are no bounding boxes, and the bounding boxes obtained using different methods are indistinguishable, meaning the degree of difference is low and the bounding box deviation is almost zero. For high-value images, such as those containing multiple image types, the obtained bounding boxes show greater differences, meaning the degree of difference is high and the bounding box deviation is large. Therefore, the difference in bounding boxes can be used to determine whether a target image is high-value.

[0097] In one possible implementation, the bounding box deviation is calculated using the formula GIOU.

[0098] Assuming the first bounding box is A and the second bounding box is B, as follows: Figure 4 The diagram shown illustrates a bounding box calculation method according to an embodiment of this application. First, the smallest bounding box C, including A and B, is obtained. Then:

[0099]

[0100]

[0101] Where A∩B represents the intersection area of ​​the first and second bounding boxes. C\(A∪B) is the area of ​​C minus the area of ​​A∪B. A∪B is the union area of ​​the first and second bounding boxes.

[0102] The boundary box deviation of multiple boundary boxes can be obtained by calculating the average of the GIOU values ​​of multiple boundary boxes.

[0103] In one possible implementation, DIoU can be chosen to represent the conflict coefficient. Compared to GIoU, it has better convergence speed and accuracy, and the obtained conflict coefficients are more precise.

[0104]

[0105] Where ρ represents b and b gt The Euclidean distance between them. b represents the center point of the first predicted bounding box. gt This represents the center point of the second prediction box. ρ 2 The squared distance *c* between the two center points represents the length of the diagonal of the smallest bounding rectangle of the two rectangles. If the two boxes perfectly overlap, IoU = 1, DIoU = 1 - 0 = 1. If the two boxes are far apart, DIoU = 0 - 1 = -1. Therefore, the value range of DIoU is [-1, 1].

[0106] The boundary box deviation of multiple boundary boxes can be obtained by calculating the average of the DIoU values ​​of multiple boundary boxes.

[0107] In the embodiments of this application, other formulas for calculating the boundary box deviation may also be used, and those skilled in the art can adjust them as needed.

[0108] S104: Determine the collision coefficient of the target image based on the bounding box deviations of multiple bounding boxes.

[0109] The conflict coefficient is used to represent the difficulty of object detection in a target image. The conflict coefficient is positively correlated with the difficulty; the greater the difficulty, the larger the conflict coefficient. High-value images contain more relevant information, therefore, object detection is more difficult, and the conflict coefficient is also larger.

[0110] The embodiments of this application can determine the collision coefficient of a target image based on the boundary box deviations of multiple bounding boxes.

[0111] One possible implementation uses the bounding box deviation as the conflict coefficient of the target image. This is because high-value images have more labeled bounding boxes and larger bounding box deviations, while low-value images have almost no labeled bounding boxes and smaller bounding box deviations. Therefore, the bounding box deviation can be directly used as the conflict coefficient of the target image.

[0112] In another possible implementation, the conflict coefficient of the target image is determined using bounding box deviation and in-bounding box prediction deviation. Specifically, the bounding box deviation and in-bounding box prediction deviation can be substituted into a preset conflict coefficient calculation formula to calculate the conflict coefficient of the target image. The preset conflict coefficient calculation formula is positively correlated with both the bounding box deviation and the in-bounding box prediction deviation.

[0113] Assuming the prediction bias within the bounding box is P, the bounding box bias is W, and the collision coefficient of the target image is C, the preset collision coefficient formula can be:

[0114] C = aP + bW (4)

[0115] Wherein, a and b are parameters that can be adjusted by those skilled in the art as needed.

[0116] Furthermore, the preset conflict coefficient formula can be directly proportional to the square of the prediction deviation within the bounding box, and also directly proportional to the square of the bounding box deviation. Other positive correlations are also possible, and those skilled in the art can adjust them as needed.

[0117] The prediction bias within the bounding box is obtained in the following way:

[0118] The softmax layer output of multiple bounding boxes is divided into the softmax layer output of multiple targets;

[0119] The softmax layer outputs of the same target are used to calculate the prediction bias within the bounding box of each target among multiple targets based on cross-entropy.

[0120] The prediction bias within the bounding boxes of multiple targets is averaged to obtain the prediction bias within the bounding boxes of multiple bounding boxes.

[0121] Compared to calculating the conflict coefficient by only considering the bounding box deviation, the selected high-value images are more accurate.

[0122] S105: Determine if the conflict coefficient is greater than the preset conflict threshold. If yes, proceed to S106. Otherwise, proceed to S107.

[0123] The detection server determines whether a target image is high-value based on the obtained conflict coefficient. If the conflict coefficient is greater than a preset conflict threshold, the target image is determined to be high-value; otherwise, the target image is considered low-value.

[0124] For example, if the preset conflict threshold is set to 0.45, and the obtained conflict coefficient is 0.75, the target image is considered a high-value image because the conflict coefficient is greater than the preset conflict threshold. Conversely, if the obtained conflict coefficient is 0.15, the target image is considered a low-value image because the conflict coefficient is less than the preset conflict threshold.

[0125] S106: Use the target image as the image for training the vehicle recognition model.

[0126] For images with a conflict coefficient greater than a preset conflict threshold, these images are used as training samples in the object recognition model (e.g., a vehicle recognition model). Figure 2 (b) represents high-value images placed in the training sample database.

[0127] S107: Filter out target images.

[0128] When the conflict coefficient is not greater than the preset conflict threshold, the target image is deleted from the training sample database.

[0129] S108: Repeat steps S101 to S107 to obtain the training sample set.

[0130] The detection server repeatedly executes steps S101 to S106 until all images in the pre-collected massive image set have been processed, thus obtaining the training sample set. At this point, the training sample set only contains high-value images.

[0131] The image processing method disclosed in this application first determines multiple bounding boxes corresponding to the received target image. Based on these bounding boxes, the bounding box deviations are determined. Based on the bounding box deviations, the conflict coefficient of the target image is determined, and the target image is then used to train a vehicle recognition model. The conflict coefficient represents the difficulty of recognizing objects in the target image. Thus, by using the conflict coefficient, high-value images are selected and low-value images are filtered out. The high-value images are then used to train the recognition model, thereby avoiding the problem of high labor costs during image labeling caused by training the target object recognition model with low-value images.

[0132] Furthermore, embodiments of this application also provide an image processing apparatus. See also Figure 5 This is a schematic diagram of an image processing apparatus 500 provided in an embodiment of this application. The apparatus includes:

[0133] The first response unit 501 is used to determine multiple bounding boxes corresponding to the target image in response to the received target image.

[0134] The first calculation unit 502 is used to calculate the boundary box deviation of multiple boundary boxes based on multiple boundary boxes;

[0135] The determining unit 503 is used to determine the collision coefficient of the target image based on the boundary box deviations of multiple bounding boxes; the collision coefficient is used to indicate the difficulty of recognizing the target object in the target image.

[0136] The second response unit 504 is used to use the target image as the image for training the target object recognition model in response to a conflict coefficient greater than a preset conflict threshold.

[0137] Optionally, device 500 also includes:

[0138] The second calculation unit is used to calculate the in-boundary prediction bias of multiple bounding boxes; specifically, the second calculation unit is used to: obtain the in-boundary prediction values ​​of multiple bounding boxes; and calculate the in-boundary prediction bias of multiple bounding boxes based on the in-boundary prediction values ​​of multiple bounding boxes.

[0139] The determination unit 503 is specifically used to determine the collision coefficient of the target image based on the bounding box deviation and the prediction deviation within the bounding box of multiple bounding boxes.

[0140] The predicted values ​​within the bounding box are the output of the softmax layer.

[0141] Based on the predicted values ​​within the bounding boxes of multiple bounding boxes, calculate the predicted bias within the bounding boxes of multiple bounding boxes, including:

[0142] The softmax layer output of multiple bounding boxes is divided into the softmax layer output of multiple targets;

[0143] The softmax layer outputs results for the same target are used to calculate the prediction bias for each target among multiple targets based on cross-entropy.

[0144] The prediction biases of multiple targets are averaged to obtain the in-boundary prediction biases of multiple bounding boxes.

[0145] Optionally, the determining unit 503 is also used for:

[0146] Substitute the bounding box deviation and the prediction deviation within the bounding box into the preset conflict coefficient calculation formula to calculate the conflict coefficient of the target image.

[0147] Among them, the preset conflict coefficient calculation formula is positively correlated with the bounding box deviation and positively correlated with the prediction deviation within the bounding box.

[0148] Optionally, the multiple bounding boxes include multiple first bounding boxes and multiple second bounding boxes;

[0149] The first response unit 501 is also used for:

[0150] Input the target image into the first preset model to obtain multiple first bounding boxes; input the target image into the second preset model to obtain multiple second bounding boxes;

[0151] Among them, the first preset model and the second preset model are models based on different principles;

[0152] The first computing unit 502 is specifically used for:

[0153] Based on the preset bounding box deviation calculation formula, the bounding box deviation of multiple bounding boxes is calculated using multiple first bounding boxes and multiple second bounding boxes;

[0154] The formula for calculating bounding box deviation is positively correlated with the degree of difference among multiple bounding boxes.

[0155] Optionally, the default formula for calculating the bounding box deviation is the GIoU calculation formula.

[0156] The specific structural implementation is as described in the above image processing method embodiment, and will not be repeated here.

[0157] The image processing apparatus disclosed in this application includes a first response unit 501 that, in response to a received target image, determines a first bounding box set and a second bounding box corresponding to the target image. A first calculation unit 502 calculates the bounding box deviations of the multiple bounding boxes. A determination unit 503 determines the conflict coefficient of the target image based on the bounding box deviations. A second response unit 504 determines the target image as an image for training a vehicle recognition model based on the conflict coefficient. The conflict coefficient represents the difficulty of recognizing objects in the target image. Thus, by using the conflict coefficient, high-value images are selected and low-value images are filtered out, using high-value images as training images for the recognition model, thereby avoiding the problem of high labor costs during image labeling caused by training vehicle recognition models with low-value images.

[0158] This application also provides a vehicle system including the image processing apparatus described above. This application further provides corresponding devices and computer-readable storage media for implementing the solutions provided in this application.

[0159] The device includes a memory and a processor. The memory stores instructions or code, and the processor executes the instructions or code to cause the device to perform an image processing method according to any embodiment of this application.

[0160] In practical applications, the computer-readable storage medium can be any combination of one or more computer-readable media. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. For example, a computer-readable storage medium can be, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this embodiment, the computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0161] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, capable of sending, propagating, or transmitting programs for use by or in connection with an instruction execution system, apparatus, or device.

[0162] Program code contained on a computer-readable medium may be transmitted using any suitable medium, including but not limited to wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0163] Computer program code for performing the operations of this invention can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as "C" or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0164] It should also be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0165] The above description is merely one specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method of picture processing, the method comprising: The method comprises: in response to the received target picture, determining a plurality of bounding boxes corresponding to the target picture; according to the plurality of bounding boxes, calculating bounding box deviations of the plurality of bounding boxes; according to the bounding box deviations, determining a conflict coefficient of the target picture; the conflict coefficient is used to indicate the identification difficulty of the target object in the target picture; in response to the conflict coefficient being greater than a preset conflict threshold, taking the target picture as a picture for training a target object identification model; wherein the determination of the plurality of bounding boxes corresponding to the target picture comprises: processing the target picture by using a first preset model to obtain a plurality of first bounding boxes, and processing the target picture by using a second preset model to obtain a plurality of second bounding boxes; wherein the first preset model is one of a first-order target identification model and a second-order target identification model, the second preset model is one of a first-order target identification model and a second-order target identification model, and the first preset model is different from the second preset model; the calculation of the bounding box deviations of the plurality of bounding boxes according to the plurality of bounding boxes comprises: determining GIOU values of the plurality of first bounding boxes and the plurality of second bounding boxes; and determining the bounding box deviations according to the GIOU values.

2. The method of claim 1, wherein, The method further comprises: obtaining bounding box internal prediction values of the plurality of bounding boxes; calculating bounding box internal prediction deviations of the plurality of bounding boxes according to the bounding box internal prediction values; the determination of the conflict coefficient of the target picture according to the plurality of bounding box deviations comprises: determining the conflict coefficient of the target picture according to the bounding box deviations and the bounding box internal prediction deviations of the plurality of bounding boxes.

3. The method of claim 2, wherein, The bounding box internal prediction value is a softmax layer output result; the calculation of the bounding box internal prediction deviations of the plurality of bounding boxes according to the bounding box internal prediction values of the plurality of bounding boxes comprises: dividing the softmax layer output results of the plurality of bounding boxes into softmax layer output results of a plurality of targets; obtaining the bounding box internal prediction deviations of each target in the plurality of targets based on cross-entropy calculation of the softmax layer output results of the same target; averaging the bounding box internal prediction deviations of the plurality of targets to obtain the bounding box internal prediction deviations of the plurality of bounding boxes.

4. The method of claim 2, wherein, the determination of the conflict coefficient of the target picture according to the bounding box deviations and the bounding box internal prediction deviations of the plurality of bounding boxes comprises: substituting the bounding box deviations and the bounding box internal prediction deviations into a preset conflict coefficient calculation formula to calculate the conflict coefficient of the target picture; wherein the preset conflict coefficient calculation formula has a positive correlation with the bounding box deviations and a positive correlation with the bounding box internal prediction deviations.

5. An apparatus for picture processing, the apparatus comprising: The device comprises: a first response unit configured to determine a plurality of bounding boxes corresponding to a target picture in response to the received target picture; a first calculation unit configured to calculate bounding box deviations of the plurality of bounding boxes according to the plurality of bounding boxes; determining, by a determining unit, a conflict coefficient of the target picture according to the bounding box deviation; the conflict coefficient is used to indicate the difficulty of recognizing the target object in the target picture; a second response unit is configured to, in response to the conflict coefficient being greater than a preset conflict threshold, take the target picture as a picture for training a target object recognition model; wherein the determining the multiple bounding boxes corresponding to the target picture comprises: processing the target picture by using a first preset model to obtain multiple first bounding boxes, and processing the target picture by using a second preset model to obtain multiple second bounding boxes; wherein the first preset model is one of a first-order target recognition model and a second-order target recognition model, the second preset model is one of the first-order target recognition model and the second-order target recognition model, and the first preset model is different from the second preset model; the calculating the bounding box deviation of the multiple bounding boxes according to the multiple bounding boxes comprises: determining a GIOU value of the multiple first bounding boxes and the multiple second bounding boxes, and determining the bounding box deviation according to the GIOU value.

6. The apparatus of claim 5, wherein, The device further comprises: a second calculating unit is configured to calculate a bounding box intra-prediction deviation of the multiple bounding boxes; the second calculating unit is specifically configured to: obtain a bounding box intra-prediction value of the multiple bounding boxes; and calculate the bounding box intra-prediction deviation of the multiple bounding boxes according to the bounding box intra-prediction value of the multiple bounding boxes; the determining unit is specifically configured to determine the conflict coefficient of the target picture according to the bounding box deviation and the bounding box intra-prediction deviation of the multiple bounding boxes.

7. A vehicle system comprising the device for processing pictures according to any one of claims 5 or 6.

8. A computer-readable storage medium, characterized in that, The computer readable storage medium stores codes, when the codes are executed, the steps of the method according to any one of claims 1-4 are executed.

Images