Weakly supervised object detection method based on misclassification correction, and related device
The weakly supervised object detection method with misclassification correction utilizes convolutional neural networks and classifiers to identify and correct misclassifications, solving the problem of inaccurate category judgment in weakly supervised detection, improving the classification performance of the detection model and reducing the cost of manual annotation.
Patent Information
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- HARBIN INSTITUTE OF TECHNOLOGY (SHENZHEN) (INSTITUTE OF SCIENCE AND TECHNOLOGY INNOVATION HARBIN INSTITUTE OF TECHNOLOGY SHENZHEN)
- Filing Date
- 2025-06-27
- Publication Date
- 2026-07-30
AI Technical Summary
Existing weakly supervised visual object detection methods cannot accurately determine the category after detecting the target region, resulting in limited classification performance, and the high cost of manual annotation limits the use of large-scale data.
Regional features of target candidate locations are extracted using a convolutional neural network. Classifiers are used to initially assign category labels. A label assignment module driven by misclassification correction is designed to identify and correct misclassification cases and reassign labels to improve the classification performance of the detection model.
It significantly improves the classification performance of weakly supervised object detection methods, reduces reliance on costly manual annotation, and enhances the accuracy of the detection model.
Smart Images
Figure CN2025104189_30072026_PF_FP_ABST
Abstract
Description
Weakly Supervised Target Detection Method and Related Equipment Based on Misclassification Correction Technical Field
[0001] This invention belongs to the field of computer vision technology, specifically relating to a weakly supervised target detection method and related equipment based on misclassification correction. Background Technology
[0002] Driven by national policies, artificial intelligence (AI) technology is rapidly penetrating various industries, showing great promise, particularly in sectors such as industrial manufacturing, smart transportation, healthcare, and education. As a crucial component of AI, computer vision, especially advanced learning and cognitive technologies such as visual object detection, has become a key driver of industrial intelligent upgrading.
[0003] While deep learning-based visual object detection models have made significant progress in accuracy, their heavy reliance on large-scale manually labeled data remains a major bottleneck. This labeling process is time-consuming, labor-intensive, and costly, directly limiting the efficient use of large-scale data and severely restricting the development potential of related technologies.
[0004] To reduce reliance on costly manual annotation, researchers have proposed weakly supervised visual object detection techniques. However, these methods often overlook a crucial issue in their design: misclassification. Even if the detector can accurately locate the target region, it cannot accurately determine its category, thus affecting the improvement of classification performance. Summary of the Invention
[0005] To address the aforementioned problems, this invention provides a weakly supervised object detection method, system, electronic device, and storage medium based on misclassification correction, aiming to significantly improve the classification performance of the weakly supervised object detection method by correcting erroneous category labels generated during the training phase.
[0006] According to a first aspect of the present disclosure, a weakly supervised target detection method based on misclassification correction is provided, the method comprising the following steps:
[0007] Use convolutional neural networks to extract regional features of candidate target locations;
[0008] The extracted regional features are assigned category labels by a classifier;
[0009] Based on the confidence differences between categories, a label allocation module driven by misclassification correction is designed to identify and correct misclassification cases, and reassign labels for object detection model training.
[0010] In some embodiments, the convolutional neural network is based on the VGGNet neural network, and a spatial pyramid pooling layer is added to the output of the VGGNet to generate convolutional features corresponding to multi-scale combined target candidate regions; and two fully connected layers are added to the output of the spatial pyramid pooling layer of the VGGNet to generate feature vectors required by the classifier.
[0011] In some embodiments, the extracted region features are assigned category labels by a classifier, and the training process includes the following steps:
[0012] The region feature vectors are input into the two fully connected layers of the classifier to obtain the first matrix and the second matrix.
[0013] The first matrix and the second matrix are respectively input into the category dimension Softmax layer and the region dimension Softmax layer to obtain the third matrix representing the confidence score of each candidate region belonging to each category and the fourth matrix representing the normalization contribution of each candidate region to the image containing each category.
[0014] Perform element-wise multiplication on the third and fourth matrices to generate candidate region score matrices;
[0015] The candidate region score matrix is summed along the candidate region dimension to obtain the confidence score of the image containing each category;
[0016] The confidence scores and true classes of the image are input into a multi-class cross-entropy loss function.
[0017] In some embodiments, during the training of the misclassification correction-driven label allocation module:
[0018] The iteration continues with the following steps until the maximum number of iterations is reached:
[0019] Input the region feature vector into the Softmax layer branch, which contains fully connected layers and class dimensions. The candidate region score matrix is obtained. ;
[0020] From the candidate region score of the previous branch In the middle, select each positive class Candidate region with the highest confidence level The corresponding confidence level is Select all negative classes Candidate region with the highest confidence level The corresponding confidence level is ;
[0021] like , regarded as category pair A potential misclassification occurred;
[0022] If the training progress is less than three-tenths complete, when the category is... When a potential misclassification occurs, it is recorded in the misclassification memory. Update ,in Representative category pair Number of occurrences; constructing a positive class memory This is used to record the number of times each category appears as a positive class in previous iterations;
[0023] If the training process reaches three-tenths complete, calculate the frequency of potential misclassification for each class pair: ;Will The elements in the array are sorted in descending order to obtain the sorted category-to-frequency vector. ,in and the corresponding ranking matrix ,in Represents category pairs Rank the occurrence frequency of elements; calculate the difference between neighboring elements to obtain the frequency difference vector. ;turn up The largest element in the array and its ranking. ;
[0024] If the training process exceeds three-tenths, when the potential misclassification is... When it occurs, if the conditions are met Then the candidate region with the highest negative class score will be obtained. Mark as a positive example; otherwise, the positive example is the region that receives the highest positive class score. ;
[0025] The intersection-union ratio (IUU) of other candidate regions with the positive examples is calculated. Candidate regions with an IUU greater than 0.5 are also selected as positive examples, regions with an IUU less than 0.1 are ignored, and the remaining regions are set as negative examples as supervision information. ;
[0026] Will and The input is fed into the multi-class cross-entropy loss function;
[0027] Calculation after iteration The average value is used as the final prediction result for the confidence level of each candidate region.
[0028] According to a second aspect of the present disclosure, a weakly supervised target detection system based on misclassification correction is provided, the system comprising:
[0029] The region feature acquisition unit is used to extract region features of candidate target locations using a convolutional neural network.
[0030] The category label acquisition unit is used to assign category labels to the extracted region features through a classifier.
[0031] The label assignment module design unit is used to design a misclassification correction-driven label assignment module based on the confidence difference between categories, so as to identify and correct misclassification cases, and reassign labels for object detection model training.
[0032] In some embodiments, the convolutional neural network in the region feature acquisition unit is based on the VGGNet neural network, and a spatial pyramid pooling layer is added at the output of the VGGNet to generate convolutional features corresponding to the multi-scale combined target candidate regions; and two fully connected layers are added at the output of the spatial pyramid pooling layer of the VGGNet to generate the feature vectors required by the classifier.
[0033] In some embodiments, the category label acquisition unit assigns category labels to the extracted region features using a classifier, and the training process includes the following steps:
[0034] The region feature vectors are input into the two fully connected layers of the classifier to obtain the first matrix and the second matrix.
[0035] The first matrix and the second matrix are respectively input into the category dimension Softmax layer and the region dimension Softmax layer to obtain the third matrix representing the confidence score of each candidate region belonging to each category and the fourth matrix representing the normalization contribution of each candidate region to the image containing each category.
[0036] Perform element-wise multiplication on the third and fourth matrices to generate candidate region score matrices;
[0037] The candidate region score matrix is summed along the candidate region dimension to obtain the confidence score of the image containing each category;
[0038] The confidence scores and true classes of the image are input into a multi-class cross-entropy loss function.
[0039] In some embodiments, during the training of the misclassification correction-driven label allocation module in the label allocation module design unit:
[0040] The iteration continues with the following steps until the maximum number of iterations is reached:
[0041] Input the region feature vector into the Softmax layer branch, which contains fully connected layers and class dimensions. The candidate region score matrix is obtained. ;
[0042] From the candidate region score of the previous branch In the middle, select each positive class Candidate region with the highest confidence level The corresponding confidence level is Select all negative classes Candidate region with the highest confidence level The corresponding confidence level is ;
[0043] like , regarded as category pair A potential misclassification occurred;
[0044] If the training progress is less than three-tenths complete, when the category is... When a potential misclassification occurs, it is recorded in the misclassification memory. Update ,in Representative category pair Number of occurrences; constructing a positive class memory This is used to record the number of times each category appears as a positive class in previous iterations;
[0045] If the training process reaches three-tenths complete, calculate the frequency of potential misclassification for each class pair: ;Will The elements in the array are sorted in descending order to obtain the sorted category-to-frequency vector. ,in and the corresponding ranking matrix ,in Represents category pairs Rank the occurrence frequency of elements; calculate the difference between neighboring elements to obtain the frequency difference vector. ;turn up The largest element in the array and its ranking. ;
[0046] If the training process exceeds three-tenths, when the potential misclassification is... When it occurs, if the conditions are met Then the candidate region with the highest negative class score will be obtained. Mark as a positive example; otherwise, the positive example is the region that receives the highest positive class score. ;
[0047] The intersection-union ratio (IUU) of other candidate regions with the positive examples is calculated. Candidate regions with an IUU greater than 0.5 are also selected as positive examples, regions with an IUU less than 0.1 are ignored, and the remaining regions are set as negative examples as supervision information. ;
[0048] Will and The input is fed into the multi-class cross-entropy loss function;
[0049] Calculation after iteration The average value is used as the final prediction result for the confidence level of each candidate region.
[0050] According to a third aspect of the present disclosure, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the weakly supervised target detection method based on misclassification correction described above.
[0051] According to a fourth aspect of the present disclosure, a non-transitory computer-readable storage medium is provided, wherein computer instructions are stored on the storage medium, and when executed by a processor, the instructions implement the steps of the weakly supervised target detection method based on misclassification correction described above.
[0052] This disclosure provides a weakly supervised object detection method, system, electronic device, and storage medium based on misclassification correction. A convolutional neural network is used to extract region features corresponding to candidate target locations. A classifier categorizes the extracted region features, initially assigning a category label to each candidate region. A label allocation module driven by misclassification correction identifies and corrects misclassifications, reassigning labels for model training. This allows for more accurate learning of features across categories, providing a practical solution to the misclassification problem and possessing significant application value. This invention significantly improves the classification performance of weakly supervised object detection methods by correcting erroneous category labels generated during the training phase.
[0053] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description
[0054] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.
[0055] Figure 1 is a schematic flowchart of the weakly supervised target detection method based on misclassification correction in an embodiment of the present invention;
[0056] Figure 2 is an overall block diagram of the weakly supervised target detection method based on misclassification correction in an embodiment of the present invention;
[0057] Figure 3 is a schematic diagram of the structure of the weakly supervised target detection system based on misclassification correction in an embodiment of the present invention;
[0058] Figure 4 is a schematic diagram of an electronic device according to an embodiment of the present invention. Detailed Implementation
[0059] The present invention will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the scope of the invention. Furthermore, it should be noted that, for ease of description, only the parts relevant to the present invention are shown in the drawings, not the entire structure.
[0060] Before discussing the exemplary embodiments in more detail, it should be noted that some exemplary embodiments are described as processes or methods depicted as flowcharts. Although the flowcharts describe the steps as sequential processes, many of these steps can be performed in parallel, concurrently, or simultaneously. Furthermore, the order of the steps can be rearranged. The process can be terminated when its operation is complete, but may also have additional steps not included in the figures. The process can correspond to a method, function, procedure, subroutine, subroutine, etc.
[0061] This invention provides the following embodiments for a weakly supervised target detection method, system, electronic device, and storage medium based on misclassification correction:
[0062] As shown in Figure 1, the weakly supervised target detection method based on misclassification correction includes the following steps:
[0063] S1. Use a convolutional neural network to extract regional features of the target candidate location;
[0064] S2. Assign category labels to the extracted regional features using a classifier; specifically, classify the extracted regional features using a classifier, and initially assign category labels to each candidate region.
[0065] S3. Based on the confidence differences between categories, a misclassification correction-driven label allocation module is designed to identify and correct misclassifications, and then reassign labels for object detection model training. Specifically, based on the confidence differences between categories, a misclassification correction-driven label allocation module is designed to identify and correct misclassifications, and then reassign labels for model training, thereby learning the features of each category more accurately.
[0066] Overall, as shown in Figure 2, the convolutional neural network is based on the VGGNet neural network, with a spatial pyramid pooling layer added to the output of VGGNet to generate convolutional features corresponding to multi-scale combined target candidate regions; and two fully connected layers added to the output of the spatial pyramid pooling layer of VGGNet to generate the feature vectors required by the classifier.
[0067] In one specific embodiment, the convolutional neural network in S1 is modified from the original version of VGGNet (Simonyan, Karen, and Andrew Zisserman. "Very deep convolutional networks for large-scale image recognition." ArXiv. 2014.).
[0068] Specifically, the specific method for building a convolutional neural network is as follows:
[0069] 1) Add a spatial pyramid pooling layer to the output of VGGNet to generate convolutional features corresponding to the target candidate regions provided by the multi-scale combination grouping method;
[0070] 2) Add two fully connected layers at the output of the spatial pyramid pooling layer.
[0071] The establishment of the convolutional neural network enables the generation of feature vectors for the target candidate region, providing input for the classifier in S2.
[0072] In S2, a classifier assigns category labels to the extracted region features. The training process includes the following steps:
[0073] The region feature vectors are input into the two fully connected layers of the classifier to obtain the first matrix and the second matrix.
[0074] The first matrix and the second matrix are respectively input into the category dimension Softmax layer and the region dimension Softmax layer to obtain the third matrix representing the confidence score of each candidate region belonging to each category and the fourth matrix representing the normalization contribution of each candidate region to the image containing each category.
[0075] Perform element-wise multiplication on the third and fourth matrices to generate candidate region score matrices;
[0076] The candidate region score matrix is summed along the candidate region dimension to obtain the confidence score of the image containing each category;
[0077] The confidence scores and true classes of the image are input into a multi-class cross-entropy loss function.
[0078] In one specific embodiment, the classifier is modified from the original version of WSDDN (Bilen, Hakan, and Andrea Vedaldi. "Weakly supervised deep detection networks." Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2016.).
[0079] The specific method for building the classifier is as follows:
[0080] S21. Input the feature vectors generated in S1 into the two fully connected layers respectively to generate two matrices. and ;
[0081] S22. Input the two matrices generated in S21 into the Softmax layer on the category dimension and the Softmax layer on the region dimension, respectively, to obtain two matrices. and , representing the confidence score of each candidate region belonging to each category, and the normalized contribution of each candidate region to the image containing each category, respectively;
[0082] S23. Perform element-wise multiplication on the two matrices generated in S22 to generate the candidate region score matrix. ;
[0083] S24. Add the candidate region score matrices from S23 along the candidate region dimensions to obtain the confidence score of the image containing each category;
[0084] S25. Input the image class confidence and class ground truth generated in S24 into the multi-class cross-entropy loss function;
[0085] If it is during the training phase, proceed to S21 to S25; if it is during the application phase, skip S21 to S25.
[0086] This completes the establishment of the classifier, which has been trained to classify images and provide supervision information for the next branch.
[0087] The misclassification correction-driven label assignment module described in S3 includes three structurally identical branches.
[0088] The construction method of the misclassification correction-driven label allocation module is as follows:
[0089] S31. Input the feature vector generated in step 1) into the branch with a fully connected layer and a Softmax layer in the class dimension. In the process, a candidate region score matrix is generated. ;
[0090] S32, Score of candidate regions from the previous branch In the middle, select each positive class Candidate region with the highest confidence level The corresponding confidence level is Select the candidate region with the highest confidence among all negative classes. The corresponding confidence level is If the difference between the two satisfies , regarded as category pair A potential misclassification occurred;
[0091] Step S33: If the training progress is less than three-tenths, when the class pairs When a potential misclassification occurs, it is recorded in the misclassification memory. middle: ,in Representative class pairs Number of occurrences; constructing a positive class memory This is used to record the number of times each class appeared as the positive class in previous iterations. If the training process reaches 30%, the frequency of potential misclassification for each class pair is calculated: Then The elements in the array are sorted in descending order to obtain the sorted category-to-frequency vector. ,in and the corresponding ranking matrix ,in Represents category pairs Rank the occurrence frequency of elements; calculate the difference between neighboring elements to obtain the frequency difference vector. ;turn up The largest element in the array and its ranking. If the training process exceeds three-tenths, when the potential misclassification is... When it occurs, if the conditions are met Then the candidate region with the highest negative class score will be obtained. Mark as a positive example; otherwise, the positive example is the region that receives the highest positive class score. ;
[0092] S34. Calculate the intersection-union ratio (IUU) of other candidate regions with the positive examples. Candidate regions with an IUU greater than 0.5 are also selected as positive examples, regions with an IUU less than 0.1 are ignored, and the remaining regions are set as negative examples as supervision information. ;
[0093] S35, will and The input is fed into the multi-class cross-entropy loss function;
[0094] S36. If it is in the training phase, repeat S31 to S35 3 times; if it is in the application phase, repeat only step S31 3 times.
[0095] S37, Calculation The average value is used as the final prediction result for the confidence level of each candidate region.
[0096] This completes the creation of the misclassification correction-driven label allocation module.
[0097] Another embodiment illustrates a weakly supervised target detection system based on misclassification correction, as shown in Figure 3. The system 300 includes:
[0098] The region feature acquisition unit 310 is used to extract region features of the target candidate location using a convolutional neural network.
[0099] The category label acquisition unit 320 is used to assign category labels to the extracted region features through a classifier.
[0100] The label assignment module design unit 330 is used to design a misclassification correction-driven label assignment module based on the confidence difference between categories, so as to identify and correct misclassification cases, and reassign labels for object detection model training.
[0101] The convolutional neural network in the region feature acquisition unit 310 is based on the VGGNet neural network, and a spatial pyramid pooling layer is added to the output of VGGNet to generate convolutional features corresponding to the multi-scale combined target candidate regions; and two fully connected layers are added to the output of the spatial pyramid pooling layer of VGGNet to generate the feature vectors required by the classifier.
[0102] In the category label acquisition unit 320, the extracted region features are assigned category labels by a classifier. The training process includes the following steps:
[0103] The region feature vectors are input into the two fully connected layers of the classifier to obtain the first matrix and the second matrix.
[0104] The first matrix and the second matrix are respectively input into the category dimension Softmax layer and the region dimension Softmax layer to obtain the third matrix representing the confidence score of each candidate region belonging to each category and the fourth matrix representing the normalization contribution of each candidate region to the image containing each category.
[0105] Perform element-wise multiplication on the third and fourth matrices to generate candidate region score matrices;
[0106] The candidate region score matrix is summed along the candidate region dimension to obtain the confidence score of the image containing each category;
[0107] The confidence scores and true classes of the image are input into a multi-class cross-entropy loss function.
[0108] In the label allocation module design unit 330, during the training of the misclassification correction-driven label allocation module:
[0109] The iteration continues with the following steps until the maximum number of iterations is reached:
[0110] Input the region feature vector into the Softmax layer branch, which contains fully connected layers and class dimensions. The candidate region score matrix is obtained. ;
[0111] From the candidate region score of the previous branch In the middle, select each positive class Candidate region with the highest confidence level The corresponding confidence level is Select all negative classes Candidate region with the highest confidence level The corresponding confidence level is ;
[0112] like , regarded as category pair A potential misclassification occurred;
[0113] If the training progress is less than three-tenths complete, when the category is... When a potential misclassification occurs, it is recorded in the misclassification memory. Update ,in Representative category pair Number of occurrences; constructing a positive class memory This is used to record the number of times each category appears as a positive class in previous iterations;
[0114] If the training process reaches three-tenths complete, calculate the frequency of potential misclassification for each class pair: ;Will The elements in the array are sorted in descending order to obtain the sorted category-to-frequency vector. ,in and the corresponding ranking matrix ,in Represents category pairs Rank the occurrence frequency of elements; calculate the difference between neighboring elements to obtain the frequency difference vector. ;turn up The largest element in the array and its ranking. ;
[0115] If the training process exceeds three-tenths, when the potential misclassification is... When it occurs, if the conditions are met Then the candidate region with the highest negative class score will be obtained. Mark as a positive example; otherwise, the positive example is the region that receives the highest positive class score. ;
[0116] The intersection-union ratio (IUU) of other candidate regions with the positive examples is calculated. Candidate regions with an IUU greater than 0.5 are also selected as positive examples, regions with an IUU less than 0.1 are ignored, and the remaining regions are set as negative examples as supervision information. ;
[0117] After the iteration is completed, and The input is fed into the multi-class cross-entropy loss function;
[0118] Calculation after iteration The average value is used as the final prediction result for the confidence level of each candidate region.
[0119] In addition to the above module, unit 300 may also include other components; however, since these components are not relevant to the content of the embodiments of this disclosure, their illustrations and descriptions are omitted here.
[0120] Other specific working processes of the weakly supervised target detection system 300 based on misclassification correction are described in the above-described embodiment of the weakly supervised target detection method based on misclassification correction, and will not be repeated here.
[0121] Another embodiment illustrates that the system of the present invention can also be implemented using the architecture of the computing device shown in FIG4. FIG4 shows the architecture of the computing device. As shown in FIG4, the computer system 410, system bus 430, one or more CPUs 440, input / output 420, memory 450, etc. The memory 450 can store various data or files used by the computer for processing and / or communication, as well as program instructions executed by the CPU, including the weakly supervised target detection method based on misclassification correction in the embodiments. The architecture shown in FIG4 is only exemplary, and one or more components in FIG4 can be adjusted according to actual needs when implementing different devices. The memory 450, as a computer-readable storage medium, can be used to store software programs, computer-executable programs, and modules, such as the program instructions / modules corresponding to the weakly supervised target detection method based on misclassification correction in the embodiments of the present invention (e.g., the region feature acquisition unit 310, the category label acquisition unit 320, and the label allocation module design unit 330 in the weakly supervised target detection system 300 based on misclassification correction). One or more CPUs 440 execute various functional applications and data processing of the system of the present invention by running software programs, instructions, and modules stored in memory 450, thereby realizing the above-mentioned weakly supervised target detection method based on misclassification correction. The method includes the following steps:
[0122] Use convolutional neural networks to extract regional features of candidate target locations;
[0123] The extracted regional features are assigned category labels by a classifier;
[0124] Based on the confidence differences between categories, a label allocation module driven by misclassification correction is designed to identify and correct misclassification cases, and reassign labels for object detection model training.
[0125] Of course, the processor of the server provided in the embodiments of the present invention is not limited to performing the method operations described above, but can also perform related operations in the weakly supervised target detection method based on misclassification correction provided in any embodiment of the present invention.
[0126] The memory 450 may primarily include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data created based on terminal usage. Furthermore, the memory 450 may include high-speed random access memory and non-volatile memory, such as at least one disk storage device, flash memory device, or other non-volatile solid-state storage device. In some instances, the memory 450 may further include memory remotely configured relative to one or more CPUs 440, these remote memories being connected to the device via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.
[0127] Input / output 420 can be used to receive input digital or character information, and to generate key signal inputs related to user settings and function control of the device. Input / output 420 may also include a display device such as a display screen.
[0128] This invention also provides a non-transitory computer-readable storage medium storing a computer program thereon. When executed by a processor, this computer program implements the weakly supervised target detection method based on misclassification correction described in the above embodiments. The computer-readable storage medium of this invention can be any combination of one or more computer-readable media. A computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. For example, a computer-readable storage medium can be, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this document, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.
[0129] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, capable of sending, propagating, or transmitting programs for use by or in connection with an instruction execution system, apparatus, or device.
[0130] The program code contained on the storage medium can be transmitted using any suitable medium, including but not limited to wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.
[0131] Furthermore, other specific working processes of a non-temporary computer-readable storage medium are described in the above-described embodiment of the weakly supervised target detection method based on misclassification correction, and will not be repeated here.
[0132] Based on the technical solutions provided in the above embodiments, a weakly supervised object detection method, system, electronic device, and storage medium based on misclassification correction are presented. A convolutional neural network is used to extract region features corresponding to candidate target locations; a classifier classifies the extracted region features, initially assigning a category label to each candidate region; a label allocation module driven by misclassification correction identifies and corrects misclassifications, reassigning labels for model training, thereby learning the features of each category more accurately. This provides a practical solution to the misclassification problem and has significant application value. The method of this invention significantly improves the classification performance of weakly supervised object detection methods by correcting erroneous category labels generated during the training phase.
[0133] In this document, the terms “comprising,” “including,” or any other variations thereof are intended to cover non-exclusive inclusion, such that a step or method that comprises a list of elements includes not only those elements but also other elements not expressly listed or inherent to such a step or method.
[0134] The above description, in conjunction with specific preferred embodiments, provides a further detailed explanation of the present invention. It should not be construed that the specific implementation of the present invention is limited to these descriptions. For those skilled in the art, various simple deductions or substitutions can be made without departing from the concept of the present invention, and all such modifications and substitutions should be considered within the scope of protection of the present invention.
Claims
1. A weakly supervised object detection method based on misclassification correction, characterized in that, The method includes the following steps: Use convolutional neural networks to extract regional features of candidate target locations; The extracted regional features are assigned category labels by a classifier; Based on the confidence differences between categories, a label allocation module driven by misclassification correction is designed to identify and correct misclassification cases, and reassign labels for object detection model training.
2. The weakly supervised object detection method based on misclassification correction according to claim 1, characterized in that, The convolutional neural network is based on the VGGNet neural network, and a spatial pyramid pooling layer is added to the output of the VGGNet to generate convolutional features corresponding to multi-scale combined target candidate regions. In addition, two fully connected layers are added to the output of the spatial pyramid pooling layer of VGGNet to generate the feature vectors required by the classifier.
3. The weakly supervised object detection method based on misclassification correction of claim 1, wherein, The extracted region features are assigned category labels by a classifier. The training process includes the following steps: The region feature vectors are input into the two fully connected layers of the classifier to obtain the first matrix and the second matrix. The first matrix and the second matrix are respectively input into the category dimension Softmax layer and the region dimension Softmax layer to obtain the third matrix representing the confidence score of each candidate region belonging to each category and the fourth matrix representing the normalization contribution of each candidate region to the image containing each category. Perform element-wise multiplication on the third and fourth matrices to generate candidate region score matrices; The candidate region score matrix is summed along the candidate region dimension to obtain the confidence score of the image containing each category; The confidence scores and true classes of the image are input into a multi-class cross-entropy loss function.
4. The weakly supervised target detection method based on misclassification correction according to claim 1, characterized in that, During the training of the misclassification correction-driven label assignment module: The iteration continues with the following steps until the maximum number of iterations is reached: inputting the regional feature vector into a branch containing a fully connected layer and a softmax layer of class dimension , to obtain a candidate region score matrix ； from the candidate region scores of the previous branch In this case, each positive class is selected top candidate regions The corresponding confidence is ; select all negatives Candidate region with the highest confidence level The corresponding confidence level is ； like , regarded as category pair A potential misclassification occurred; If the training progress is less than three-tenths complete, when the category is... When a potential misclassification occurs, it is recorded in the misclassification memory. Update ,in Representative category pair Number of occurrences; constructing a positive class memory This is used to record the number of times each category appears as a positive class in previous iterations; If the training process reaches three-tenths complete, calculate the frequency of potential misclassification for each class pair: ;Will The elements in the array are sorted in descending order to obtain the sorted category-to-frequency vector. ,in and the corresponding ranking matrix ,in Represents category pairs Ranking of frequency of occurrence; Calculate the difference between neighboring elements to obtain the frequency difference vector. ;turn up The largest element in the array and its ranking. ； If the training process exceeds three-tenths, when the potential misclassification is... When it occurs, if the conditions are met Then the candidate region with the highest negative class score will be obtained. Mark as a positive example; otherwise, the positive example is the region that receives the highest positive class score. ； The intersection-union ratio (IUU) of other candidate regions with the positive examples is calculated. Candidate regions with an IUU greater than 0.5 are also selected as positive examples, regions with an IUU less than 0.1 are ignored, and the remaining regions are set as negative examples as supervision information. ； Will and The input is fed into the multi-class cross-entropy loss function; Calculation after iteration The average value is used as the final prediction result for the confidence level of each candidate region.
5. A weakly supervised target detection system based on misclassification correction, characterized in that, The system includes: The region feature acquisition unit is used to extract region features of candidate target locations using a convolutional neural network. The category label acquisition unit is used to assign category labels to the extracted region features through a classifier. The label assignment module design unit is used to design a misclassification correction-driven label assignment module based on the confidence difference between categories, so as to identify and correct misclassification cases, and reassign labels for object detection model training.
6. The weakly supervised target detection system based on misclassification correction according to claim 5, characterized in that, The convolutional neural network in the region feature acquisition unit is based on the VGGNet neural network, and a spatial pyramid pooling layer is added to the output of VGGNet to generate convolutional features corresponding to multi-scale combined target candidate regions. In addition, two fully connected layers are added to the output of the spatial pyramid pooling layer of VGGNet to generate the feature vectors required by the classifier.
7. The weakly supervised target detection system based on misclassification correction as described in claim 5, characterized in that, The category label acquisition unit assigns category labels to the extracted region features using a classifier. The training process includes the following steps: The region feature vectors are input into the two fully connected layers of the classifier to obtain the first matrix and the second matrix. The first matrix and the second matrix are respectively input into the category dimension Softmax layer and the region dimension Softmax layer to obtain the third matrix representing the confidence score of each candidate region belonging to each category and the fourth matrix representing the normalization contribution of each candidate region to the image containing each category. Perform element-wise multiplication on the third and fourth matrices to generate candidate region score matrices; The candidate region score matrix is summed along the candidate region dimension to obtain the confidence score of the image containing each category; The confidence scores and true classes of the image are input into a multi-class cross-entropy loss function.
8. The weakly supervised target detection system based on misclassification correction as described in claim 5, characterized in that, In the label allocation module design unit, during the training of the misclassification correction-driven label allocation module: The iteration continues with the following steps until the maximum number of iterations is reached: Input the region feature vector into the Softmax layer branch, which contains fully connected layers and class dimensions. The candidate region score matrix is obtained. ； From the candidate region score of the previous branch In the middle, select each positive class Candidate region with the highest confidence level The corresponding confidence level is Select all negative classes Candidate region with the highest confidence level The corresponding confidence level is ； like , regarded as category pair A potential misclassification occurred; If the training progress is less than three-tenths complete, when the category is... When a potential misclassification occurs, it is recorded in the misclassification memory. Update ,in Representative category pair Number of occurrences; constructing a positive class memory This is used to record the number of times each category appears as a positive class in previous iterations; If the training process reaches three-tenths complete, calculate the frequency of potential misclassification for each class pair: ;Will The elements in the array are sorted in descending order to obtain the sorted category-to-frequency vector. ,in and the corresponding ranking matrix ,in Represents category pairs Ranking of frequency of occurrence; Calculate the difference between neighboring elements to obtain the frequency difference vector. ;turn up The largest element in the array and its ranking. ； If the training process exceeds three-tenths, when the potential misclassification is... When it occurs, if the conditions are met Then the candidate region with the highest negative class score will be obtained. Mark as a positive example; otherwise, the positive example is the region that receives the highest positive class score. ； The intersection-union ratio (IUU) of other candidate regions with the positive examples is calculated. Candidate regions with an IUU greater than 0.5 are also selected as positive examples, regions with an IUU less than 0.1 are ignored, and the remaining regions are set as negative examples as supervision information. ； Will and The input is fed into the multi-class cross-entropy loss function; Calculation after iteration The average value is used as the final prediction result for the confidence level of each candidate region.
9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the weakly supervised target detection method based on misclassification correction as described in any one of claims 1 to 4.
10. A non-transitory computer-readable storage medium, wherein computer instructions are stored on the storage medium, characterized in that, When the instructions are executed by the processor, they implement the steps of the weakly supervised target detection method based on misclassification correction as described in any one of claims 1 to 4.