A method, device and storage medium for processing image prediction frame overlap
By performing IOU calculation and filtering on the detection bounding boxes of high-resolution images, the problem of overlapping detection boxes under the sliding overlap cropping method is solved, improving detection efficiency and accuracy, and making it suitable for industrial inspection.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- BEIJING INST OF TECH
- Filing Date
- 2023-09-18
- Publication Date
- 2026-06-30
Smart Images
Figure CN117372674B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of deep learning technology, and in particular to a method, apparatus, device, and storage medium for processing overlapping image prediction boxes. Background Technology
[0002] Intelligent image recognition and detection technology combines computer vision, image processing, and deep learning. Currently, when using deep learning object detection models to perform detection tasks on high-resolution images, limitations in hardware computing power and memory prevent the direct input of the original image into the model for training and inference. Using a sliding overlapping cropping box to crop the original image to a fixed size, using the cropped sub-image as the model's input, and then stitching the model's output back into the original image to retrieve the target information, is the mainstream method for high-resolution image detection.
[0003] Image processing using cropping methods may result in some targets appearing in multiple cropped images due to the variable size of targets in the image. Therefore, when the model infers and predicts targets in high-resolution images, multiple overlapping detection boxes may appear, each containing some information about the target to be detected. Summary of the Invention
[0004] Therefore, it is necessary to provide a method, apparatus, device, and storage medium for processing overlapping image prediction boxes to address the aforementioned technical problems.
[0005] A method for handling overlapping image prediction bounding boxes includes the following steps:
[0006] Obtain the target detection box, and determine the detection box type based on the target detection box;
[0007] Perform IOU calculations on each pair of target detection boxes of the same type to obtain the first IOU calculation result;
[0008] The target detection box is marked according to the first IOU calculation result, and the target marking result is obtained;
[0009] IOU calculation is performed on each pair of the remaining target detection boxes, including the target label results, to obtain the second IOU calculation result;
[0010] Based on the second IOU calculation result, the processing result of the predicted box overlap is obtained.
[0011] In one embodiment, obtaining the target detection box further includes:
[0012] The image to be tested is cropped by sliding, resulting in a cropped image;
[0013] The cropped image is input into the object detection model to obtain the object detection box, the category of the object detection box, and the confidence score of the object detection box.
[0014] In one embodiment, the target detection box is marked based on the first IOU calculation result, and the target marking result is obtained as follows:
[0015] The IOU calculation result that is greater than the preset first threshold is taken as the first preferred result, and the detection box of the same type corresponding to the first preferred result is marked to obtain the marked detection box;
[0016] The maximum bounding rectangle of the marker detection box is drawn to obtain the target marker detection result.
[0017] In one embodiment, marking the target detection box based on the first IOU calculation result to obtain the target marking result further includes:
[0018] The confidence score of the detected target labeling result is calculated according to the following formula:
[0019]
[0020] Where S represents the confidence score of the detected target labeling result, s i represents the confidence score of the i-th bounding box within the maximum bounding rectangle, and N represents the total number of bounding boxes within the maximum bounding rectangle.
[0021] In one embodiment, the processing result of predictive bounding box overlap based on the second IOU calculation result includes:
[0022] The IOU calculation result that is greater than the preset second threshold is taken as the second preferred result, and the remaining target detection boxes corresponding to the second preferred result are marked to obtain the marked remaining target detection boxes;
[0023] Based on the remaining target detection boxes marked, the processing result of the predicted box overlap is obtained.
[0024] In one embodiment, the processing result of obtaining the predicted box overlap based on the marked remaining target detection boxes includes:
[0025] Calculate the importance score of the remaining target detection boxes, and retain the remaining target detection boxes if the importance score is greater than a preset third threshold.
[0026] In response to the importance score being less than a preset third threshold, the remaining target detection boxes are deleted, resulting in a processing result of overlapping prediction boxes.
[0027] In one embodiment, calculating the importance score of the marked remaining target detection boxes includes:
[0028] The importance score is calculated using the following formula:
[0029] G = w i ×S ′
[0030] Where G represents the importance score, w i S represents the importance coefficient of the remaining detection boxes in the i-th category. ′ This represents the confidence score of the remaining detection boxes.
[0031] An image prediction bounding box overlap processing apparatus, used to implement the image prediction bounding box overlap processing method described above, includes:
[0032] The category determination module is used to obtain target detection boxes and determine the detection box type based on the target detection boxes; the same type calculation module is used to perform IOU calculation between each pair of target detection boxes of the same type to obtain the first IOU calculation result.
[0033] A similar labeling module is used to label the target detection box based on the first IOU calculation result to obtain the target labeling result;
[0034] The heterogeneous calculation module is used to perform IOU calculations between each pair of the remaining target detection boxes, including the target labeling results, to obtain the second IOU calculation result.
[0035] The overlap processing module is used to obtain the processing result of the predicted box overlap based on the calculation result of the second IOU.
[0036] An apparatus includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the image prediction box overlap processing method described in the above embodiments.
[0037] A storage medium storing a computer program that, when executed by a processor, implements the steps of the image prediction box overlap processing method described in the above embodiments.
[0038] Compared with the prior art, the advantages and beneficial effects of the present invention are as follows: The present invention can provide a method, apparatus, device and storage medium for processing overlapping image prediction boxes. For high-resolution weld seam images, the target detection model is trained and inferred using a sliding overlap cropping method. The large number of overlapping target detection boxes generated after inference are merged and filtered to solve the problem of overlapping target detection boxes. The target detection boxes are clear and easy to understand, which is more suitable for industrial inspection. This method reduces manpower consumption and improves the efficiency and accuracy of inspection. Attached Figure Description
[0039] Figure 1 This is a flowchart illustrating a method for handling overlapping image prediction boxes in one embodiment.
[0040] Figure 2 This is a schematic diagram of the original high-resolution image in one embodiment;
[0041] Figure 3 This is a schematic diagram of a target detection box in one embodiment;
[0042] Figure 4 This is a schematic diagram illustrating the calculation process of the target marker detection result in one embodiment;
[0043] Figure 5 This is a schematic diagram of the target marker detection results in one embodiment;
[0044] Figure 6 This is a schematic diagram illustrating the processing results for predicting overlap in one embodiment;
[0045] Figure 7 This is a schematic diagram illustrating the processing of the prediction box overlap result in one embodiment;
[0046] Figure 8 This is a schematic diagram of the structure of an image prediction box overlap processing device in one embodiment;
[0047] Figure 9 This is a schematic diagram of the internal structure of an electronic device in one embodiment. Detailed Implementation
[0048] Before describing the specific embodiments of the present invention, the overall concept of the present invention will be explained as follows:
[0049] This invention primarily focuses on the development of image detection processes. Currently, when using deep learning object detection models for high-resolution image detection tasks, limitations in hardware computing power and memory prevent the direct input of the original image into the model for training and inference. This invention utilizes a sliding overlapping cropping box to crop the original image to a fixed size, using the cropped sub-image as the model's input, and then stitching the model's output back into the original image to retrieve the target information. This cropping method is the mainstream approach for high-resolution image detection.
[0050] Image processing using cropping methods may result in some targets appearing in multiple cropped images due to the variable size of targets in the image. Therefore, when the model infers and predicts targets in high-resolution images, multiple overlapping detection boxes may appear, each containing some information about the target to be detected.
[0051] In reality, most object detection models output multiple highly overlapping bounding boxes for each object. These are then processed using the Non-Maximum Suppression (NMS) algorithm to calculate the Intersection over Union (IOU) and sorted by score, removing many overlapping and duplicate boxes while retaining only the most suitable ones. However, in high-resolution image object detection using the sliding overlap cropping method, while some object bounding boxes may overlap, none may completely contain all objects, or the bounding box containing all objects may not be the highest-scoring one. This is because cropping may cause the model to learn only a portion of the object during training, not the entirety. If the NMS algorithm is used to address this issue, it cannot guarantee that the retained bounding boxes will contain all objects.
[0052] For the task of detecting defects (false defects, weld seams) in weld seam images, different targets to be detected will not overlap highly, that is, the IOU of two detection boxes of different categories will not be large. However, the NMS algorithm only filters overlapping boxes for the same category of detection targets and cannot handle the overlapping of different categories of detection targets.
[0053] The inventors, through analysis, discovered that the main reason for the aforementioned problems is that for high-resolution weld seam images, using a sliding overlap cropping method for training and inference of the target detection model results in a large number of overlapping detection boxes. Merging and filtering these boxes can avoid the problems mentioned above. Therefore, this invention proposes a method, apparatus, device, and storage medium for processing overlapping prediction boxes in images. First, image target detection boxes are acquired and categorized into similar and dissimilar detection boxes based on their different types. Interchange of Union (IOU) calculations are performed between each pair of similar detection boxes within each type to obtain a first IOU result. Based on the first IOU result, the target label result with the largest bounding rectangle is obtained. Then, IOU calculations are performed on all remaining detection boxes to obtain a second IOU result. Based on the second IOU result, the processing result for overlapping prediction boxes is obtained.
[0054] After introducing the overall concept of the present invention, in order to make the objectives, technical solutions and advantages of the present invention clearer, the present invention will be further described in detail below through specific embodiments and in conjunction with the accompanying drawings.
[0055] It should be noted that, unless otherwise defined, the technical or scientific terms used in one or more embodiments of this specification should have the ordinary meaning understood by one of ordinary skill in the art to which this invention pertains. The terms "first," "second," and similar terms used in one or more embodiments of this specification do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Terms such as "comprising" or "including" mean that the element or object preceding the word covers the element or object listed following the word and its equivalents, without excluding other elements or objects. Terms such as "connected" or "linked" are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. Terms such as "upper," "lower," "left," and "right" are used only to indicate relative positional relationships; when the absolute position of the described object changes, the relative positional relationship may also change accordingly.
[0056] For ease of understanding, the terms used in the embodiments of this invention are explained below:
[0057] NMS: Non-Maximum Suppression, suppresses elements that are not local maxima, and can be understood as a local maximum search. This "local" refers to a neighborhood, which has two variable parameters: the dimension of the neighborhood and its size.
[0058] IOU: Intersection over Union, is a concept used in object detection. IoU calculates the overlap ratio between the predicted bounding box and the ground truth bounding box, that is, the ratio of their intersection to their union.
[0059] In one embodiment, such as Figure 1 As shown, a method for handling overlapping image prediction boxes is provided, including the following steps:
[0060] Step S101: Obtain the target detection box and determine the detection box type based on the target detection box.
[0061] Specifically, the target detection bounding box of the image to be tested is obtained, and different categories of the detection bounding box are obtained based on the target detection bounding box. There are multiple defect categories for the target detection bounding box, such as: weld, concavity, incomplete penetration, porosity, and lack of fusion. Different detection bounding boxes may select the same defect.
[0062] Based on this, the target detection bounding box is obtained, which previously also included:
[0063] The image to be tested is cropped by sliding, resulting in a cropped image;
[0064] The cropped image is input into the object detection model to obtain the object detection box, the category of the object detection box, and the confidence score of the object detection box.
[0065] Specifically, a high-resolution image with a resolution of 8376*1794 is acquired as the image to be tested. The high-resolution image is as follows: Figure 2 As shown, a sliding overlap cropping process is performed on the original high-resolution image, with a crop size of 436*436 and an overlap ratio of 0.25. The cropped image is then input into an object detection model for inference. The inference results are then positionally shifted according to the cropping strategy to obtain the model's output of all object detection boxes, the categories of the object detection boxes, and the confidence scores of the object detection boxes. The results obtained after inference are as follows: Figure 3 As shown.
[0066] In this embodiment, the overlapping of target detection boxes is not conducive to the verification by quality inspection personnel and affects the weld rating. The image to be tested is cropped and the target is detected. Then, the image is restored according to the cropping strategy to obtain target detection boxes with category and confidence scores. This allows for clearer handling of overlapping target detection boxes and improves the efficiency of target detection box overlap processing.
[0067] Step S102: Perform IOU calculation on each pair of target detection boxes of the same detection box type to obtain the first IOU calculation result.
[0068] Specifically, IOU calculation is performed on each pair of multiple target detection boxes of the same detection type to obtain multiple first IOU calculation results. Each target detection box corresponds to only one detection target. Multiple target detection boxes of the same detection type may be multiple detection targets. Different detection targets can be obtained based on IOU calculation.
[0069] Step S103: Mark the target detection box according to the first IOU calculation result to obtain the target marking result.
[0070] Specifically, by marking the detection boxes of the same type based on the IOU calculation results, different marking results can be obtained for different detection targets. For example, the detection box of the second concave defect with the detection box type of concave defect can be marked. The same detection target uses the same marking method, such as the same color marking or the same label.
[0071] Based on this, the target detection box is marked according to the first IOU calculation result, and the target marking result is obtained as follows:
[0072] The IOU calculation result that is greater than the preset first threshold is taken as the first preferred result, and the detection box of the same type corresponding to the first preferred result is marked to obtain the marked detection box;
[0073] The maximum bounding rectangle of the marker detection box is drawn to obtain the target marker detection result.
[0074] Specifically, a pair of target detection boxes with an IOU greater than a first threshold is taken as the first preferred result. These first preferred results are then labeled to obtain labeled detection boxes. The pair of target detection boxes that are considered the first preferred results correspond to the same detection target. However, the IOU of detection boxes for the same detection target is not necessarily greater than the first threshold. For example... Figure 4 As shown, for stomata, the first threshold is set to 0.5, and for others, the first threshold is set to 0.2. Although the IOU of ① and ③ is less than the first threshold, the IOU of ① and ② is greater than the first threshold, that is, ① and ② are the same detection target. Also, the IOU of ② and ③ is greater than the first threshold, that is, ② and ③ are the same detection target. Therefore, ① and ③ are also the same detection target.
[0075] In actual industrial production, each weld image to be tested generally contains only one weld, and the detection of welds generally does not use the sliding overlap cropping method. Therefore, the weld category can be ignored in the post-processing method of this invention.
[0076] In the image to be tested, pores may be concentrated in a certain area, i.e. dense pores. Because of their small area, the detection boxes are easy to overlap. In order to better distinguish each pore, the IOU threshold of the pores should be slightly larger. Other defects have larger areas and generally do not appear in a concentrated manner, so the IOU threshold of other defects should be smaller.
[0077] For all bounding boxes belonging to the same detection target, the largest bounding rectangle is taken as the final detection box for that target, thus obtaining the target labeling result. Specifically, the processed result of the target detection box is as follows: Figure 5 As shown.
[0078] Based on this, the target detection box is marked according to the first IOU calculation result to obtain the target marking result, which also includes:
[0079] The confidence score of the detected target labeling result is calculated according to the following formula:
[0080]
[0081] Where S represents the confidence score of the detected target labeling result, s i represents the confidence score of the i-th bounding box within the maximum bounding rectangle, and N represents the total number of bounding boxes within the maximum bounding rectangle.
[0082] Specifically, based on the confidence scores of all the bounding boxes contained in the target detection result, an average confidence score is calculated and used as the confidence score of the target detection result.
[0083] In this embodiment, calculating the average confidence score of all bounding boxes within the detected target marking result can improve the processing accuracy and precision of the detection.
[0084] Step S104: Perform IOU calculations on each pair of the remaining target detection boxes, including the target detection label results, to obtain the second IOU calculation result.
[0085] Specifically, IOU calculations are performed on each pair of the remaining target detection boxes, including the target labeling results, to obtain multiple second IOU calculation results, with each target detection box corresponding to only one target.
[0086] Step S105: Based on the second IOU calculation result, obtain the processing result of the prediction box overlap.
[0087] Specifically, after processing based on the second IOU calculation result, the obtained prediction overlap processing result is as follows: Figure 6 As shown.
[0088] Based on this, and according to the second IOU calculation result, the processing result of the predicted box overlap is as follows:
[0089] The IOU calculation result that is greater than the preset second threshold is taken as the second preferred result, and the remaining target detection boxes corresponding to the second preferred result are marked to obtain the marked remaining target detection boxes;
[0090] Based on the remaining target detection boxes marked, the processing result of the predicted box overlap is obtained.
[0091] Specifically, the remaining target detection boxes whose IOU calculation results are greater than a preset second threshold are marked to obtain marked remaining target detection boxes. Based on the marked remaining target detection boxes, the processing result of the predicted box overlap is obtained. For pores and other defects, the second threshold is set to 0.5. For non-pore defects, the second threshold is set to 0.2.
[0092] Based on this, the processing result of overlapping predicted boxes obtained according to the marked remaining target detection boxes includes:
[0093] Calculate the importance score of the remaining target detection boxes, and retain the remaining target detection boxes if the importance score is greater than a preset third threshold.
[0094] In response to the importance score being less than a preset third threshold, the remaining target detection boxes are deleted, resulting in a processing result of overlapping prediction boxes.
[0095] Specifically, the importance score of each remaining target detection box is calculated, and a preset third threshold for the importance score is set. Remaining target detection boxes with an importance score greater than the preset third threshold are retained, while remaining target detection boxes with a lower importance score are deleted, resulting in the final processing result of overlapping prediction boxes.
[0096] Based on this, the importance score of the marked remaining target detection boxes is calculated as follows:
[0097] The importance score is calculated using the following formula:
[0098] G = w i ×S ′
[0099] Where G represents the importance score, w i S represents the importance coefficient of the remaining detection boxes in the i-th category. ′ This represents the confidence score of the remaining detection boxes.
[0100] Specifically, an importance coefficient is assigned to each detection box type. The importance coefficient is selected based on the detection box type to which each remaining target detection box belongs. An importance score is calculated for each remaining target detection box, and the result of predictive box overlap is obtained based on the importance score. The importance coefficients for porosity, concavity, lack of fusion, and incomplete penetration are set to 0.4, 0.7, 0.8, and 0.9, respectively. Figure 7In the right region, the four targets—concave, incomplete penetration, lack of fusion, and porosity—all overlap. The second threshold for porosity, concave, incomplete penetration, and lack of fusion is calculated; all are less than 0.5, so the porosity is retained. The IOU for concave, incomplete fusion, and incomplete penetration is greater than 0.2. The importance scores for these three are calculated: incomplete penetration > lack of fusion > concave. The target detection box with the higher importance score is retained (i.e., incomplete penetration), while the target detection boxes with the lower importance score are deleted (i.e., lack of fusion and concave). The same logic applies to the left region, yielding the final result.
[0101] In this embodiment, because the amount of target data varies for different detection box types, and the feature complexity of defect types also varies, the model's ability to identify defect types for each detection box also varies. Furthermore, considering that hazardous defects are more noteworthy than general defects in practical industrial applications, an importance coefficient w is required. i To balance the scores of different defect types in the detection bounding box, the worse the defect recognition ability, the higher the defect severity, and the more important the defect w. i The larger the value, the stronger the defect identification capability, the smaller the defect hazard, and the greater the defect importance coefficient w. i The smaller.
[0102] This invention provides a method, apparatus, device, and storage medium for processing overlapping image prediction boxes. For high-resolution weld seam images, a sliding overlap cropping method is used for training and inference of the target detection model. The large number of overlapping target detection boxes generated after inference are merged and filtered to solve the problem of overlapping target detection boxes. The target detection boxes are clear and easy to understand, making them more suitable for industrial inspection. This method reduces manpower consumption and improves the efficiency and accuracy of inspection.
[0103] It should be noted that the method of this embodiment can be executed by a single device, such as a computer or server. The method of this embodiment can also be applied to a distributed scenario, where multiple devices cooperate to complete the task. In such a distributed scenario, one of these devices may execute only one or more steps of the method of this embodiment, and the multiple devices will interact with each other to complete the method described.
[0104] It should be noted that the above description describes some embodiments of the present invention. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps described in the claims may be performed in a different order than that shown in the above embodiments and still achieve the desired results. Furthermore, the processes depicted in the drawings do not necessarily require a specific or sequential order to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.
[0105] Based on the same inventive concept, corresponding to any of the above embodiments, the present invention also provides an image prediction box overlap processing device.
[0106] refer to Figure 8 The image prediction bounding box overlap processing device includes:
[0107] The category determination module 801 is used to obtain the target detection box and determine the detection box type based on the target detection box;
[0108] The same type calculation module 802 is used to perform IOU calculation between pairs of detection boxes of the same type to obtain the first IOU calculation result;
[0109] The same type of marking module 803 is used to mark the detection box according to the first IOU calculation result to obtain the detection target marking result;
[0110] The heterogeneous calculation module 804 is used to perform IOU calculations between each pair of the remaining detection boxes, including the detection target label results, to obtain the second IOU calculation result;
[0111] The overlap processing module 805 is used to obtain the processing result of the predicted box overlap based on the calculation result of the second IOU.
[0112] For ease of description, the above apparatus is described in terms of its functions, divided into various modules. Of course, in implementing this invention, the functions of each module can be implemented in one or more software and / or hardware components.
[0113] The apparatus of the above embodiments is used to implement a corresponding image prediction box overlap processing method in any of the foregoing embodiments, and has the beneficial effects of the corresponding method embodiments, which will not be repeated here.
[0114] Based on the same inventive concept, corresponding to the methods of any of the above embodiments, the present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the image prediction box overlap processing method described in any of the above embodiments.
[0115] Figure 9 This embodiment illustrates a more specific hardware structure of an electronic device, which may include a processor 1010, a memory 1020, an input / output interface 1030, a communication interface 1040, and a bus 1050. The processor 1010, memory 1020, input / output interface 1030, and communication interface 1040 are interconnected internally via the bus 1050.
[0116] The processor 1010 can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this specification.
[0117] The memory 1020 can be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage device, dynamic storage device, etc. The memory 1020 can store the operating system and other applications. When the technical solutions provided in the embodiments of this specification are implemented by software or firmware, the relevant program code is stored in the memory 1020 and is called and executed by the processor 1010.
[0118] The input / output interface 1030 is used to connect input / output modules to realize information input and output. Input / output modules can be configured as components within the device (not shown in the figure) or externally connected to the device to provide corresponding functions. Input devices may include keyboards, mice, touchscreens, microphones, various sensors, etc., while output devices may include displays, speakers, vibrators, indicator lights, etc.
[0119] The communication interface 1040 is used to connect a communication module (not shown in the figure) to enable communication between this device and other devices. The communication module can communicate via wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).
[0120] Bus 1050 includes a pathway for transmitting information between various components of the device, such as processor 1010, memory 1020, input / output interface 1030, and communication interface 1040.
[0121] It should be noted that although the above-described device only shows the processor 1010, memory 1020, input / output interface 1030, communication interface 1040, and bus 1050, in specific implementations, the device may also include other components necessary for normal operation. Furthermore, those skilled in the art will understand that the above-described device may only include the components necessary for implementing the embodiments of this specification, and not necessarily all the components shown in the figures.
[0122] The electronic device described above is used to implement a corresponding image prediction box overlap processing method in any of the foregoing embodiments, and has the beneficial effects of the corresponding method embodiments, which will not be repeated here.
[0123] Based on the same inventive concept, corresponding to the methods of any of the above embodiments, the present invention also provides a non-transitory computer-readable storage medium storing computer instructions for causing the computer to execute an image prediction box overlap processing method as described in any of the above embodiments.
[0124] The computer-readable medium of this embodiment includes permanent and non-permanent, removable and non-removable media, and information storage can be implemented by any method or technology. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transfer medium that can be used to store information accessible by a computing device.
[0125] The computer instructions stored in the storage medium of the above embodiments are used to cause the computer to execute an image prediction box overlap processing method as described in any of the above embodiments, and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.
[0126] Those skilled in the art should understand that the discussion of any of the above embodiments is merely exemplary and is not intended to imply that the scope of the invention (including the claims) is limited to these examples; within the framework of the invention, the technical features of the above embodiments or different embodiments can also be combined, the steps can be implemented in any order, and there are many other variations of different aspects of the embodiments of the invention as described above, which are not provided in the details for the sake of brevity.
[0127] Any process or method description in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing a particular logical function or process, and the scope of the preferred embodiments of the invention includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as will be understood by those skilled in the art to which embodiments of the invention pertain.
[0128] Additionally, to simplify the description and discussion, and to avoid obscuring the embodiments of the invention, the well-known power / ground connections to integrated circuit (IC) chips and other components may or may not be shown in the provided drawings. Furthermore, the apparatus may be shown in block diagram form to avoid obscuring the embodiments of the invention, and this also takes into account the fact that the details of implementation of these block diagram apparatuses are highly dependent on the platform on which the embodiments of the invention will be implemented (i.e., these details should be fully understood by those skilled in the art). While specific details (e.g., circuits) have been set forth to describe exemplary embodiments of the invention, it will be apparent to those skilled in the art that the embodiments of the invention may be implemented without these specific details or with variations thereof. Therefore, these descriptions should be considered illustrative rather than restrictive.
[0129] Although the invention has been described in conjunction with specific embodiments thereof, many substitutions, modifications, and variations of these embodiments will be apparent to those skilled in the art from the foregoing description. For example, other memory architectures (e.g., dynamic RAM (DRAM)) may be used with the embodiments discussed.
[0130] The embodiments of this invention are intended to cover all such substitutions, modifications, and variations falling within the broad scope of the appended claims. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the embodiments of this invention should be included within the protection scope of this invention.
Claims
1. A method for processing overlapping image prediction boxes, characterized in that, include: Obtain the target detection box, and determine the detection box type based on the target detection box; Perform IOU calculations on each pair of target detection boxes of the same type to obtain the first IOU calculation result; The target detection box is marked according to the first IOU calculation result to obtain the target detection marking result; wherein, the step of marking the target detection box according to the first IOU calculation result to obtain the target detection marking result includes: The IOU calculation result that is greater than the preset first threshold is taken as the first preferred result, and the detection box of the same type corresponding to the first preferred result is marked to obtain the marked detection box; The maximum bounding rectangle of the marker detection box is drawn to obtain the target marker detection result; IOU calculation is performed on each pair of the remaining target detection boxes, including the target label results, to obtain the second IOU calculation result; Based on the second IOU calculation result, the processing result of the predicted box overlap is obtained; The process of obtaining the predicted bounding box overlap result based on the second IOU calculation result includes: The IOU calculation result that is greater than the preset second threshold is taken as the second preferred result, and the remaining target detection boxes corresponding to the second preferred result are marked to obtain the marked remaining target detection boxes; Based on the marked remaining target detection boxes, the processing result of the predicted box overlap is obtained; The process of obtaining the predicted bounding box overlap result based on the marked remaining target detection boxes includes: Calculate the importance score of the remaining target detection boxes, and retain the remaining target detection boxes if the importance score is greater than a preset third threshold. In response to the importance score being less than a preset third threshold, the remaining target detection boxes are deleted to obtain the processing result of overlapping prediction boxes; The calculation of the importance score of the marked remaining target detection boxes includes: The importance score is calculated using the following formula: ， in, Indicates importance score, Indicates the first Importance coefficients of remaining detection boxes for each category. This represents the confidence score of the remaining detection boxes.
2. The method for processing overlapping image prediction boxes according to claim 1, characterized in that, The process of obtaining the target detection box also includes: The image to be tested is cropped by sliding, resulting in a cropped image; The cropped image is input into the object detection model to obtain the object detection box, the category of the object detection box, and the confidence score of the object detection box.
3. The method for processing overlapping image prediction boxes according to claim 1, characterized in that, The step of marking the target detection box based on the first IOU calculation result to obtain the target marking result further includes: The confidence score of the detected target labeling results is calculated according to the following formula: ， in, This represents the confidence score of the detected target labeling results. Represents the th element within the largest bounding rectangle. The confidence score of each labeled detection box This represents the total number of marked detection boxes within the largest bounding rectangle.
4. A processing apparatus for image prediction box overlap, characterized in that, A method for processing overlapping image prediction boxes as described in claims 1-3 includes: The category determination module is used to obtain the target detection box and determine the detection box type based on the target detection box; The same type of calculation module is used to perform IOU calculation between pairs of target detection boxes of the same type to obtain the first IOU calculation result; A similar labeling module is used to label target detection boxes based on the first IOU calculation result to obtain target labeling results; wherein, labeling target detection boxes based on the first IOU calculation result to obtain target labeling results includes: The IOU calculation result that is greater than the preset first threshold is taken as the first preferred result, and the detection box of the same type corresponding to the first preferred result is marked to obtain the marked detection box; The maximum bounding rectangle of the marker detection box is drawn to obtain the target marker detection result; The heterogeneous calculation module is used to perform IOU calculations between each pair of the remaining target detection boxes, including the target labeling results, to obtain the second IOU calculation result. The overlap processing module is used to obtain the processing result of the predicted box overlap based on the calculation result of the second IOU; The process of obtaining the predicted bounding box overlap result based on the second IOU calculation result includes: The IOU calculation result that is greater than the preset second threshold is taken as the second preferred result, and the remaining target detection boxes corresponding to the second preferred result are marked to obtain the marked remaining target detection boxes; Based on the marked remaining target detection boxes, the processing result of the predicted box overlap is obtained; The process of obtaining the predicted bounding box overlap result based on the marked remaining target detection boxes includes: Calculate the importance score of the remaining target detection boxes, and retain the remaining target detection boxes if the importance score is greater than a preset third threshold. In response to the importance score being less than a preset third threshold, the remaining target detection boxes are deleted to obtain the processing result of overlapping prediction boxes; The calculation of the importance score of the marked remaining target detection boxes includes: The importance score is calculated using the following formula: ， in, Indicates importance score, Indicates the first Importance coefficients of remaining detection boxes for each category. This represents the confidence score of the remaining detection boxes.
5. An apparatus comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 3.
6. A storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 3.