A bounding box verification method, device, equipment and medium

Abstract

Embodiments of the present application provide a bounding box verification method and device, equipment and medium, which belong to the field of computer, and include: obtaining a plurality of first bounding boxes and a plurality of second bounding boxes for an object in the same image; wherein the first bounding box is a manually annotated box, and the second bounding box is a box predicted by a model; according to the intersection area between each of the plurality of first bounding boxes and the plurality of second bounding boxes, the plurality of first bounding boxes and the plurality of second bounding boxes are matched, and the second bounding box that fails to match is determined; based on the second bounding box that fails to match, a first target box is determined and fed back; wherein the first target box is a box predicted by the model and should be annotated but is not annotated. The bounding box comparison and verification based on the model prediction is realized, which greatly reduces the time cost and labor cost of checking after manual annotation, thereby greatly improving the overall efficiency.

Description

Technical Field

[0001] This application relates to the field of computers, and specifically to a method, apparatus, device, and medium for verifying bounding boxes. Background Technology

[0002] In existing technologies, most of the data used for model training is labeled manually. Therefore, quality inspectors need to check the results of the manual labeling in order to determine the accuracy of the labelers, which greatly increases labor costs and the overall efficiency is not high. Summary of the Invention

[0003] In view of the above problems, embodiments of this application are proposed to provide a method, apparatus, device and medium for annotation frame verification that overcomes or at least partially solves the above problems.

[0004] To address the aforementioned problems, this application discloses a method for validating bounding boxes, the method comprising:

[0005] Obtain multiple first bounding boxes and multiple second bounding boxes for objects in the same image; wherein, the first bounding boxes are manually annotated boxes, and the second bounding boxes are predicted boxes by the model;

[0006] Based on the intersection area between multiple first annotation boxes and multiple second annotation boxes, the multiple first annotation boxes and multiple second annotation boxes are matched, and the second annotation boxes that fail to match are identified.

[0007] Based on the second labeled box that failed to match, the first target box is determined and fed back; wherein, the first target box is the box that the model predicted and that should have been labeled manually but was not.

[0008] Optionally, after matching multiple first bounding boxes and multiple second bounding boxes, the method further includes:

[0009] Based on the multiple first labeled boxes after matching, the second target box is determined and fed back; the second target box is the box that was manually labeled but not predicted by the model.

[0010] Optionally, before matching the multiple first bounding boxes and the multiple second bounding boxes, the method further includes:

[0011] Based on the area of ​​each first annotation box, the multiple first annotation boxes are divided into multiple non-ignored boxes and multiple ignored boxes;

[0012] Based on the multiple first bounding boxes obtained after matching, the second target bounding boxes are determined and fed back, including:

[0013] Identify the first annotation box whose area is greater than a preset threshold after matching, and determine whether the first annotation box is an ignored box;

[0014] If the first annotation box is ignored, then the first annotation box is identified as the second target box and feedback is given.

[0015] Optionally, after dividing the multiple first annotation boxes into multiple non-ignore boxes and multiple ignore boxes, the method further includes:

[0016] Pair multiple non-ignored boxes with multiple ignored boxes based on the intersection area between each non-ignored box and each ignored box.

[0017] After identifying multiple first annotation boxes with areas greater than a preset threshold and determining whether the first annotation boxes are ignored boxes, the method further includes:

[0018] If the first annotation box is a non-ignore box, then determine whether the non-ignore box is successfully matched;

[0019] If the non-ignore box fails to match, the first labeled box is identified as the second target box and feedback is provided.

[0020] Optionally, after determining whether the first annotation box is a non-ignore box and whether the non-ignore box has been successfully paired, the method further includes:

[0021] If the non-ignore box is successfully matched, then determine whether the ignore box corresponding to the non-ignore box has successfully matched the second label box;

[0022] If the ignore box corresponding to the non-ignore box fails to match the second label box, the first label box is identified as the second target box and feedback is given.

[0023] Optionally, based on the area of ​​each first annotation box, the multiple first annotation boxes can be divided into multiple non-ignored boxes and multiple ignored boxes, including:

[0024] Determine whether the area of ​​the first annotation box is greater than a preset pixel value. If it is, it is determined to be a non-ignore box; otherwise, it is determined to be a non-ignore box.

[0025] Optionally, based on the second bounding box that failed to match, the first target bounding box is determined and returned, including:

[0026] Determine whether the area of ​​the second bounding box that failed to match is greater than a preset threshold. If it is, then the second bounding box that failed to match is identified as the first target box, and feedback is given.

[0027] This application also discloses a device for checking bounding boxes, which includes:

[0028] The data acquisition module is used to acquire multiple first bounding boxes and multiple second bounding boxes for objects in the same image; wherein, the first bounding boxes are manually labeled boxes, and the second bounding boxes are boxes predicted by the model;

[0029] The data matching module is used to match multiple first annotation boxes and multiple second annotation boxes based on the intersection area between each of the multiple first annotation boxes and multiple second annotation boxes, and to determine the second annotation boxes that fail to match.

[0030] The first target module is used to determine and return the first target box based on the second labeled box that failed to match; wherein, the first target box is the box that the model predicted and that should have been labeled manually but was not.

[0031] This application also discloses an electronic device, including a processor, a memory, and a computer program stored in the memory and capable of running on the processor. When the computer program is executed by the processor, it implements the above-described method for checking bounding boxes.

[0032] This application also discloses a non-volatile readable storage medium on which a computer program is stored. When the computer program is executed by a processor, it implements the above-described method for checking bounding boxes.

[0033] The embodiments of this application have the following advantages:

[0034] In this embodiment, multiple first and second bounding boxes for objects in the same image are first obtained, where the first bounding boxes are manually annotated and the second bounding boxes are predicted by the model. Then, based on the intersection area between the multiple first bounding boxes and the multiple second bounding boxes, the multiple first bounding boxes and the multiple second bounding boxes are matched, and the second bounding boxes that fail to match are identified. Finally, based on the second bounding boxes that fail to match the first bounding boxes, the first target box is identified and fed back. The first target box is the box that the model predicted and that should have been manually annotated but was not. This realizes the comparison and verification of bounding boxes based on model prediction, which greatly reduces the time and manpower costs of checking after manual annotation, thereby greatly improving the overall efficiency. Attached Figure Description

[0035] Figure 1 This is a flowchart illustrating the steps of an embodiment of the annotation box verification method of this application;

[0036] Figure 2 This is a flowchart illustrating the steps of another embodiment of the annotation box verification method in this application;

[0037] Figure 3 This is an overall flowchart of an embodiment of the annotation box verification method of this application;

[0038] Figure 4 This is a structural block diagram of an embodiment of a label box verification device according to this application. Detailed Implementation

[0039] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0040] Reference Figure 1 The diagram illustrates a step flowchart of an embodiment of a method for verifying bounding boxes according to this application. The method may include the following steps:

[0041] Step 101: Obtain multiple first bounding boxes and multiple second bounding boxes for objects in the same image; wherein, the first bounding boxes are manually labeled boxes, and the second bounding boxes are predicted boxes by the model.

[0042] When it is necessary to verify manually labeled data, model prediction can be performed on objects in the same image after the manual labeling is completed, thereby obtaining multiple first labeled boxes by manual labeling and multiple second labeled boxes predicted by the model, for subsequent comparison and verification.

[0043] Step 102: Match the multiple first annotation boxes and multiple second annotation boxes according to the intersection area between each of the multiple first annotation boxes and multiple second annotation boxes, and determine the second annotation boxes that fail to match.

[0044] After obtaining multiple first and second bounding boxes for objects in the same image, the multiple first and second bounding boxes can be matched based on the intersection area between each of the multiple first and second bounding boxes, thereby determining the second bounding boxes that failed to match.

[0045] Step 103: Based on the second labeled box that failed to match, determine and return the first target box; wherein, the first target box is the box that the model predicted and that should have been labeled manually but was not.

[0046] After identifying the second bounding boxes that failed to match, each of these boxes can be analyzed to determine the first target box and provide feedback. This helps pinpoint the objects that were missed by manual annotation, thereby optimizing the manual annotation process.

[0047] In some embodiments of this application, step 103 may include the following sub-steps:

[0048] Sub-step 11: Determine whether the area of ​​the second bounding box that failed to match is greater than a preset threshold. If it is, then the second bounding box that failed to match is identified as the first target box, and feedback is given.

[0049] The first target box is determined by judging whether the area of ​​the second unmatched annotation box is greater than a preset threshold. When the area of ​​a second unmatched annotation box is greater than the preset threshold, it can be identified as the first target box and feedback is given to further check whether the first target box may be a mislabeled box in the model. When the area of ​​a second unmatched annotation box is less than or equal to the preset threshold, it can be identified as not the first target box and no feedback is needed.

[0050] In this embodiment, multiple first and second bounding boxes for objects in the same image are first obtained, where the first bounding boxes are manually annotated and the second bounding boxes are predicted by the model. Then, based on the intersection area between the multiple first bounding boxes and the multiple second bounding boxes, the multiple first bounding boxes and the multiple second bounding boxes are matched, and the second bounding boxes that fail to match are identified. Finally, based on the second bounding boxes that fail to match the first bounding boxes, the first target box is identified and fed back. The first target box is the box that the model predicted and that should have been manually annotated but was not. This realizes the comparison and verification of bounding boxes based on model prediction, which greatly reduces the time and manpower costs of checking after manual annotation, thereby greatly improving the overall efficiency.

[0051] Reference Figure 2 The diagram illustrates a flowchart of another embodiment of the annotation box verification method of this application, which may include the following steps:

[0052] Step 201: Obtain multiple first bounding boxes and multiple second bounding boxes for objects in the same image; wherein, the first bounding boxes are manually labeled boxes, and the second bounding boxes are boxes predicted by the model.

[0053] When it is necessary to verify manually labeled data, model prediction can be performed on objects in the same image after the manual labeling is completed, thereby obtaining multiple first labeled boxes by manual labeling and multiple second labeled boxes predicted by the model, for subsequent comparison and verification.

[0054] Step 202: Match the multiple first annotation boxes and multiple second annotation boxes according to the intersection area between each of the multiple first annotation boxes and the multiple second annotation boxes, and determine the second annotation boxes that fail to match.

[0055] After obtaining multiple first and second bounding boxes for objects in the same image, the multiple first and second bounding boxes can be matched based on the intersection area between each of the multiple first and second bounding boxes, thereby determining the second bounding boxes that failed to match.

[0056] Step 203: Based on the second labeled box that failed to match, determine and return the first target box; wherein, the first target box is the box that the model predicted and that should have been labeled manually but was not.

[0057] After identifying the second bounding boxes that failed to match, each of these boxes can be analyzed to determine the first target box and provide feedback. This helps pinpoint the objects that were missed by manual annotation, thereby optimizing the manual annotation process.

[0058] Step 204: Based on the multiple first bounding boxes after matching, determine and feed back the second target box; wherein, the second target box is the box that was manually labeled but not predicted by the model.

[0059] After the matching is completed, the multiple first bounding boxes can be analyzed to determine the second target box and provide feedback. This allows for further verification of whether the second target box was incorrectly labeled manually or was omitted by the model, thereby optimizing the manual labeling process or the model prediction process.

[0060] In some embodiments of this application, prior to step 202, the method may further include the following steps:

[0061] Based on the area of ​​each first annotation box, the multiple first annotation boxes are divided into multiple non-ignored boxes and multiple ignored boxes.

[0062] After obtaining multiple first annotation boxes, the multiple first annotation boxes can be divided into multiple non-ignored boxes and multiple ignored boxes according to the area of ​​each first annotation box, thereby distinguishing which first annotation boxes can be ignored.

[0063] In some embodiments of this application, step 204 may include the following sub-steps:

[0064] Sub-step 21: Determine the first annotation box whose area is greater than the preset threshold among the multiple first annotation boxes after matching, and determine whether the first annotation box is an ignored box.

[0065] By identifying first-bound boxes with an area greater than a preset threshold after matching, first-bound boxes with an area that is too small can be excluded, thereby more accurately locking the range of the second target box among the multiple first-bound boxes after matching.

[0066] Sub-step 22: If the first annotation box is an ignored box, then the first annotation box is identified as the second target box and feedback is given.

[0067] When the first bounding box is an ignored box, it can be determined that the object selected by the first bounding box is too small to be used for model training, so the object does not need to be labeled. Therefore, the first bounding box is identified as the second target box and feedback is given.

[0068] In some examples of this application, after the step of dividing the multiple first annotation boxes into multiple non-ignored boxes and multiple ignored boxes, the method may further include the following steps:

[0069] Pair multiple non-ignored boxes with multiple ignored boxes based on the intersection area between each non-ignored box and each ignored box.

[0070] Since there may be multiple dense objects in an image, there may be duplicate annotations during the manual annotation process. Therefore, the non-ignored boxes and the ignored boxes can be paired according to the intersection area between each other to determine which non-ignored boxes are paired with which ignored boxes.

[0071] In some examples of this application, after sub-step 22, step 204 may also include the following sub-steps:

[0072] Sub-step 23: If the first annotation box is a non-ignore box, determine whether the non-ignore box is successfully paired.

[0073] When the first annotation box is a non-ignore box, it is also necessary to determine whether the non-ignore box is successfully paired, so as to determine whether the non-ignore box is caused by duplicate annotation.

[0074] Sub-step 24: If the non-ignore box pairing fails, then the first annotation box is identified as the second target box.

[0075] If the non-ignore box fails to match, it can be determined that the non-ignore box does not have a matching ignore box, that is, the non-ignore box is not caused by repeated annotation, so the non-ignore box can be identified as the second target box, that is, the first annotation box is the second target box.

[0076] In some specific embodiments, after sub-step 23, step 204 may further include the following sub-steps:

[0077] Sub-step 25: If the non-ignore box is successfully matched, determine whether the ignore box corresponding to the non-ignore box has successfully matched the second annotation box.

[0078] If the non-ignore box fails to match, it can be determined that there is a matching ignore box for the non-ignore box, that is, the non-ignore box is caused by duplicate annotation. Therefore, it is necessary to further determine whether the ignore box corresponding to the non-ignore box has successfully matched the second annotation box.

[0079] Sub-step 26: If the ignore box corresponding to the non-ignore box does not successfully match the second label box, then the first label box is identified as the second target box.

[0080] If the ignored box corresponding to the non-ignored box does not successfully match the second labeled box, it means that the model did not predict the box and it was only manually labeled. Therefore, the first labeled box can be identified as the second target box.

[0081] In some embodiments of this application, the step of dividing multiple first annotation boxes into multiple non-ignored boxes and multiple ignored boxes based on the area of ​​each first annotation box may include the following sub-steps:

[0082] Determine whether the area of ​​the first annotation box is greater than a preset pixel value. If it is, it is determined to be a non-ignore box; otherwise, it is determined to be a non-ignore box.

[0083] By determining whether the area of ​​the first bounding box is greater than a preset pixel value, ignore boxes with an area less than or equal to the preset pixel value and non-ignore boxes with an area greater than the preset pixel value are identified. This helps determine which objects selected by the first bounding box cannot be used for model training due to their small size, and which objects selected by the first bounding box can be used for model training.

[0084] In this embodiment, multiple first and second bounding boxes for objects in the same image are first obtained, where the first bounding boxes are manually annotated and the second bounding boxes are predicted by the model. Then, based on the intersection area between the multiple first bounding boxes and the multiple second bounding boxes, the multiple first bounding boxes and the multiple second bounding boxes are matched, and the second bounding boxes that fail to match are identified. Finally, based on the second bounding boxes that fail to match the first bounding boxes, a first target box is identified and fed back, where the first target box is a box predicted by the model but not manually annotated. Based on the multiple first bounding boxes after matching, a second target box is identified and fed back, where the second target box is a box manually annotated but not predicted by the model. This achieves the comparison and verification of bounding boxes based on model prediction, greatly reducing the time and manpower costs of checking after manual annotation, thereby greatly improving the overall efficiency.

[0085] The following describes the steps and flow involved in the above-described method embodiments:

[0086] like Figure 3 As shown, the overall flow of the annotation box verification method embodiment involved in this application is as follows:

[0087] Upload data:

[0088] The model is used to predict the images to be labeled. Then, the images predicted by the model and the images to be labeled are uploaded to the verification platform. Preferably, JSON data exchange format can be selected for uploading.

[0089] Create a task:

[0090] After uploading the images, you also need to create a task and configure the images to be verified and compared for that task.

[0091] Labeling and quality inspection:

[0092] The images to be annotated are manually annotated by the annotators and then inspected by the quality inspectors before entering the comparison process.

[0093] Prediction Comparison:

[0094] During the comparison process, the program automatically collects error information, including manually over-labeled boxes, manually missed boxes, and the corresponding annotators. It also displays the status of each image, including pending processing, returned, modified, and accepted. By summarizing the error information, the program can better identify annotators with more errors and more accurately identify the data that the model lacks, thereby better supplementing effective training data.

[0095] Pending acceptance:

[0096] By reviewing the comparison results, the annotation results can be directly returned to the corresponding annotator.

[0097] Sampling inspection:

[0098] Quality inspection is conducted through random sampling to further confirm whether manually over-labeled boxes are boxes that were missed by the model, and whether manually missed boxes are boxes that were incorrectly labeled by the model, so as to optimize the model in the future.

[0099] In some embodiments of this application, error information can also be statistically analyzed:

[0100] Let A be the number of correctly labeled boxes by human, B be the number of incorrectly labeled boxes by human, and C be the number of manually labeled boxes that can be ignored (i.e., the number of ignored boxes is C). Then, determine the number of boxes predicted by the model, D = A + BC.

[0101] The accuracy rate of manual annotation is calculated as A / (A+B), the manual omission rate is calculated as A / D, and the proportion of ignored boxes is calculated as C / (A+B+C).

[0102] The comparison logic in this application is explained below:

[0103] 1. First, compare the manually labeled boxes with the original ones:

[0104] The manually labeled boxes are divided into non-ignored boxes and ignored boxes. Each non-ignored box is compared with each ignored box. The specific comparison method is as follows: calculate the ioof value between the two boxes, that is, the value obtained by dividing the intersection area of ​​the two boxes by the area of ​​the non-ignored box. If the ioof value is greater than or equal to the preset ioof threshold, then the non-ignored box is matched with the ignored box, indicating that the two boxes overlap. The non-ignored box is then marked with the specific value of the matched ignored box.

[0105] 2. Next, compare the bounding boxes predicted by the model with the bounding boxes annotated by the manual labelers:

[0106] If the manually labeled bounding box is a non-ignored bounding box, the IoU value between the two boxes is calculated, which is the value obtained by dividing the intersection area of ​​the two boxes by the total area of ​​the two boxes. Since there may be a situation where a box predicted by the model matches multiple manually labeled boxes, if the IoU value is greater than or equal to the preset IoU threshold, it is added to the candidate queue of non-ignored bounding boxes that the predicted box has matched. After the comparison is completed, the non-ignored bounding box with the largest corresponding IoU value is found, and it is determined that the non-ignored bounding box matches the predicted box.

[0107] If the manually labeled box is an ignored box, the ioof value between the two boxes is calculated, which is the value obtained by dividing the intersection area of ​​the two boxes by the area of ​​the box predicted by the model. If the ioof value is greater than the preset ioof threshold, it is considered that the box predicted by the model matches the manually labeled box. When matching with ignored boxes, only the first matched ignored box needs to be considered, and there is no need to find the ignored box with the largest corresponding ioof value.

[0108] If a model predicts a bounding box that is matched with both ignored and non-ignored bounding boxes, the ignored bounding box has the highest priority. That is, only the Iof value needs to be calculated, and the largest IoU value does not need to be considered. The Iof value will directly cover the IoU value, thus giving priority to matching the ignored bounding box.

[0109] All matching model-predicted boxes and manually labeled boxes will be marked as matched, which will be used to calculate boxes that were missed or over-labeled manually in subsequent calculations.

[0110] 3. Calculate the boxes for manually omitted labels and manually added labels:

[0111] Check if all model-predicted bounding boxes are marked as matched. If they are, it means the box was both manually labeled and predicted by the model, and can be used directly for model training without further verification. If they are not marked as matched, determine the area of ​​the model-predicted box. If the area of ​​the model-predicted box is less than the set comparable area threshold, it means the area is too small to be used for model training, so it is not considered a manually missed label. If the area of ​​the model-predicted box is greater than the set comparable area threshold, it is considered a manually missed label, and further verification is needed to determine if it was a model mislabeling.

[0112] Then, determine the area of ​​all manually labeled boxes. If the area of ​​a manually labeled box is less than the set threshold for comparable areas, it is considered too small and not considered a case of multiple labeling. If the area of ​​a manually labeled box is greater than the set threshold for comparable areas, determine whether the manually labeled box is an ignored box. If it is an ignored box, it means that the manual labeling has included an object that should not have been labeled, so it is considered a case of multiple labeling. If the manually labeled box is not an ignored box, check whether it matches a certain ignored box. If it does not match, it means that it is not due to duplicate labeling, so it is considered a case of multiple labeling. If it matches, check whether the corresponding ignored box matches the box predicted by the model. If it does not match, it means that the model did not predict the box and only the manual labeling was done, so it can be considered a case of multiple labeling. If it matches, it is not considered a case of multiple labeling.

[0113] in:

[0114] The preset Iof threshold and preset IoU threshold can be set according to the algorithm suggestions. For example, the preset Iof threshold can be 0.5, while the maximum preset IoU threshold can be 1, which means they are completely identical. The minimum preset IoU threshold can be 0.5, which means at least half of them match. The pixel threshold for distinguishing between non-ignored boxes and ignored boxes can be 1024.

[0115] It should be noted that, for the sake of simplicity, the method embodiments are all described as a series of actions. However, those skilled in the art should understand that the embodiments of this application are not limited to the described order of actions, because according to the embodiments of this application, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also understand that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of this application.

[0116] Reference Figure 4 The diagram shows a structural block diagram of an embodiment of a bounding box verification device according to this application. The device may include the following modules:

[0117] The data acquisition module 401 is used to acquire multiple first bounding boxes and multiple second bounding boxes for objects in the same image; wherein the first bounding boxes are manually labeled boxes and the second bounding boxes are predicted boxes by the model.

[0118] The data matching module 402 is used to match multiple first annotation boxes and multiple second annotation boxes based on the intersection area between each of the multiple first annotation boxes and multiple second annotation boxes, and to determine the second annotation boxes that fail to match.

[0119] The first target module 403 is used to determine and return the first target box based on the second labeled box that failed to match; wherein the first target box is a box that was predicted by the model and should have been labeled manually but was not.

[0120] In some embodiments of this application, the device may further include the following modules:

[0121] The second target module is used to determine and feed back the second target box based on multiple first labeled boxes after matching; wherein the second target box is the box that was manually labeled but not predicted by the model.

[0122] In some examples of this application, the device may further include the following modules:

[0123] The box grouping module is used to divide multiple first annotation boxes into multiple non-ignored boxes and multiple ignored boxes based on the area of ​​each first annotation box.

[0124] In some examples of this application, the second target module may include the following sub-modules:

[0125] The first judgment submodule is used to identify first annotation boxes with an area greater than a preset threshold among multiple first annotation boxes, and to determine whether the first annotation box is an ignored box.

[0126] The first output submodule is used to identify the first annotation box as the second target box and provide feedback when the first annotation box is an ignored box.

[0127] In some specific embodiments, the device may further include the following modules:

[0128] The grouping and pairing module is used to pair multiple non-ignored boxes with multiple ignored boxes based on the intersection area between each non-ignored box and each ignored box.

[0129] In some specific embodiments, the second target module may further include the following sub-modules:

[0130] The second judgment submodule is used to determine whether the non-ignore box is successfully matched when the first annotation box is a non-ignore box.

[0131] The second output submodule is used to identify the first labeled box as the second target box and provide feedback if the pairing of the non-ignore boxes fails.

[0132] In some specific examples, the second target module may also include the following sub-modules:

[0133] The third judgment submodule is used to determine whether the ignore box corresponding to the non-ignore box has successfully matched the second annotation box if the non-ignore box is successfully matched.

[0134] The third output submodule is used to identify the first target box as the second target box and provide feedback if the ignore box corresponding to the non-ignore box fails to match the second label box.

[0135] In some specific embodiments, the box-shaped grouping module may include the following sub-modules:

[0136] The first annotation box grouping submodule is used to determine whether the area of ​​the first annotation box is greater than a preset pixel value. If it is greater, it is determined to be a non-ignore box; otherwise, it is determined to be a non-ignore box.

[0137] In some embodiments of this application, the first target module 403 may include the following sub-modules:

[0138] The second annotation box judgment submodule determines whether the area of ​​the second annotation box that failed to match is greater than a preset threshold. If it is greater, the second annotation box that failed to match is identified as the first target box, and feedback is given.

[0139] In this embodiment, multiple first and second bounding boxes for objects in the same image are first obtained, where the first bounding boxes are manually annotated and the second bounding boxes are predicted by the model. Then, based on the intersection area between the multiple first bounding boxes and the multiple second bounding boxes, the multiple first bounding boxes and the multiple second bounding boxes are matched, and the second bounding boxes that fail to match are identified. Finally, based on the second bounding boxes that fail to match the first bounding boxes, a first target box is identified and fed back, where the first target box is a box predicted by the model but not manually annotated. Based on the multiple first bounding boxes after matching, a second target box is identified and fed back, where the second target box is a box manually annotated but not predicted by the model. This achieves the comparison and verification of bounding boxes based on model prediction, greatly reducing the time and manpower costs of checking after manual annotation, thereby greatly improving the overall efficiency.

[0140] As the device embodiment is basically similar to the method embodiment, the description is relatively simple, and relevant parts can be found in the description of the method embodiment.

[0141] This application also provides an electronic device that may include a processor, a memory, and a computer program stored in the memory and capable of running on the processor. When the computer program is executed by the processor, it implements the method for checking bounding boxes as described above.

[0142] This application also provides a non-volatile readable storage medium on which a computer program is stored. When the computer program is executed by a processor, it implements the method for checking bounding boxes as described above.

[0143] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0144] Those skilled in the art will understand that embodiments of this application can be provided as methods, apparatus, or computer program products. Therefore, embodiments of this application can take the form of entirely hardware embodiments, entirely software embodiments, or embodiments combining software and hardware aspects. Furthermore, embodiments of this application can take the form of computer program products implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0145] This application describes embodiments with reference to flowchart illustrations and / or block diagrams of methods, terminal devices (systems), and computer program products according to embodiments of this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing terminal device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal device, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0146] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing terminal device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0147] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal equipment, causing a series of operational steps to be performed on the computer or other programmable terminal equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable terminal equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0148] Although preferred embodiments of the present application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of the embodiments of the present application.

[0149] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or terminal device. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or terminal device that includes said element.

[0150] The above provides a detailed description of the method, apparatus, device, and medium for verifying annotation boxes provided in this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A method for validating annotation boxes, characterized in that, The method includes: Obtain multiple first bounding boxes and multiple second bounding boxes for objects in the same image; wherein, the first bounding boxes are manually annotated boxes, and the second bounding boxes are predicted boxes by the model; Based on the intersection area between the plurality of first annotation boxes and the plurality of second annotation boxes, the plurality of first annotation boxes and the plurality of second annotation boxes are matched, and the second annotation boxes that fail to match are determined. Based on the second labeled box that failed to match, a first target box is determined and fed back; wherein, the first target box is a box that was predicted by the model and should have been labeled manually but was not. The method further includes, after matching the plurality of first annotation boxes and the plurality of second annotation boxes: Based on the multiple first bounding boxes after matching, a second target box is determined and fed back; wherein, the second target box is a box that was manually labeled but not predicted by the model; The method further includes, prior to matching the plurality of first annotation boxes and the plurality of second annotation boxes: Based on the area of ​​each first annotation box, the plurality of first annotation boxes are divided into a plurality of non-ignored boxes and a plurality of ignored boxes; The process of determining and feeding back the second target box based on the multiple first bounding boxes after matching includes: Based on the intersection area between the plurality of non-ignored boxes and the plurality of ignored boxes, the plurality of non-ignored boxes are paired with the plurality of ignored boxes; If it is determined that the area of ​​the first annotation box is less than a preset threshold, then the first annotation box does not belong to the second target box; If the area of ​​the first annotation box is determined to be greater than a preset threshold, the first annotation box is paired with the ignore box; If the first annotation box fails to match the ignore box, the first annotation box is identified as the second target box and feedback is given.

2. The method according to claim 1, characterized in that, After the first annotation box and the ignore box are successfully paired, the method further includes: Pair the ignore box with the second annotation box; If the ignored box is successfully paired with the second annotation box, then it is determined that the first annotation box does not belong to the second target box; If the ignored box fails to match the second annotation box, the first annotation box is identified as the second target box and feedback is provided.

3. The method according to any one of claims 1 to 2, characterized in that, The step of dividing the plurality of first annotation boxes into a plurality of non-ignored boxes and a plurality of ignored boxes based on the area of ​​each first annotation box includes: Determine whether the area of ​​the first annotation box is greater than a preset pixel value. If it is greater, it is determined to be a non-ignore box; otherwise, it is determined to be an ignore box.

4. The method according to any one of claims 1 to 2, characterized in that, The process of determining and returning the first target box based on the second bounding box that failed to match includes: Determine whether the area of ​​the second bounding box that failed to match is greater than a preset threshold. If it is, then the second bounding box that failed to match is identified as the first target box, and feedback is given.

5. A device for verifying annotation boxes, characterized in that, The device includes: The data acquisition module is used to acquire multiple first bounding boxes and multiple second bounding boxes for objects in the same image; wherein, the first bounding boxes are manually labeled boxes, and the second bounding boxes are boxes predicted by the model; The data matching module is used to match the plurality of first annotation boxes and the plurality of second annotation boxes based on the intersection area between each of the plurality of first annotation boxes and the plurality of second annotation boxes, and to determine the second annotation boxes that fail to match. The first target module is used to determine and return a first target box based on the second labeled box that failed to match; wherein, the first target box is a box that was predicted by the model and should have been labeled manually but was not. The device further includes: The second target module is used to determine and return the second target box based on multiple first labeled boxes after matching; wherein, the second target box is the box that was manually labeled but not predicted by the model; The box grouping module is used to divide multiple first annotation boxes into multiple non-ignored boxes and multiple ignored boxes based on the area of ​​each first annotation box; The second target module includes: The grouping and pairing module is used to pair multiple non-ignored boxes with multiple ignored boxes based on the intersection area between each non-ignored box and each ignored box. The second judgment submodule is used to determine if the area of ​​the first annotation box is less than a preset threshold, then the first annotation box does not belong to the second target box; if the area of ​​the first annotation box is greater than the preset threshold, then the first annotation box is paired with the ignore box. The second output submodule is used to identify the first labeled box as the second target box and provide feedback if the pairing of the non-ignore boxes fails.

6. An electronic device, characterized in that, It includes a processor, a storage device, and a computer program stored on the storage device and capable of running on the processor, wherein the computer program, when executed by the processor, implements the method as described in any one of claims 1 to 4.

7. A non-volatile readable storage medium, characterized in that, A computer program is stored on the non-volatile readable storage medium, which, when executed by a processor, implements the method as described in any one of claims 1 to 4.

Images