Weak supervision object detection method based on global information learning

Abstract

The invention discloses a weak supervision object detection method based on global information learning, and belongs to the technical field of object detection in machine vision. The method aims at solving the problem that an existing weak supervision object detection model cannot pay attention to global information of an input image and is not sensitive to background noise, so that a detection result contains redundant information. Comprising the steps of obtaining a reconstructed image through singular value decomposition of a sample image; a candidate frame feature extraction module is adopted to obtain sample image candidate frame features and reconstructed image candidate frame features; obtaining a multi-instance learning score of each candidate box by adopting a multi-instance learning network; in the training process, an online example classification extraction network is adopted to carry out category scoring on candidate frames based on sample image candidate frame features and reconstructed image candidate frame features; the multi-example learning score is used as supervision information to optimize model parameters of the weak supervision object detector; in actual detection, the to-be-detected image scores each candidate frame through the multi-instance learning network to determine a target frame. The method is used for object detection.

Description

Technical Field

[0001] The invention relates to a weakly supervised object detection method based on global information learning, and belongs to the technical field of object detection in machine vision. Background Art

[0002] Object detection is a key research topic in the field of computer vision. Its purpose is to identify the target object of interest from an image or video and accurately locate its position in the image. Position information is usually represented by a bounding box, which is a rectangular box that can enclose an object and is defined by the coordinates of the upper left and lower right corners (for two-dimensional images). Category information refers to the category to which the object belongs. For example, in a traffic scene, the categories may include cars, pedestrians, traffic lights, etc. Object detection technology has been developed to quickly and accurately identify and locate objects in images and videos, and is widely used in modern society, such as automatically detecting abnormal behaviors or suspicious targets in videos in the security field; realizing automatic commodity identification and inventory management in smart cities; and identifying vehicles, pedestrians, and traffic signs on the road in the field of autonomous driving.

[0003] Existing object detection methods can be divided into fully supervised object detection (FSOD) and weakly supervised object detection (WSOD) based on the model training data method. However, the existing FSOD method requires a large amount of accurately labeled data to obtain high detection accuracy, and the current research is conducted on fixed databases (PASCAL VOC, Microsoft COCO, etc., see Table 1 for details). In many practical application scenarios, it is very difficult to obtain high-quality fully supervised labeled data. For example, in a traffic scene dataset, for each image containing vehicles, pedestrians, and traffic signs, it is necessary to manually carefully outline the bounding box of each object and mark it as belonging to the category of vehicle, pedestrian, or traffic sign. This kind of labeling work is not only time-consuming and laborious, but also requires professional knowledge and experience. More importantly, manual labeling relies heavily on the subjective judgment of the labeler. Different labelers may have different standards for the bounding box delineation or category determination of the same object, which will cause the object detection model to learn noise during the training process. In some complex scenes, manual labeling is difficult to achieve high accuracy. For example, when processing images with occluded objects, it is difficult for annotators to accurately determine the boundaries of the occluded parts and the complete shape of the object.

[0004] In order to alleviate the above problems, weakly supervised object detection (WSOD) was proposed. WSOD attempts to achieve effective object detection with only annotation information (such as only knowing whether there is an image-level label of a certain type of object in the image). This method alleviates the pressure of annotation costs to a certain extent, and is especially suitable for scenarios where it is difficult to obtain accurately annotated data. By searching with keywords (such as "pedestrians", "vehicles", etc.) in search engines, training samples can be easily obtained without additional manual annotation.

[0005] Table 1 Summary of commonly used databases for object detection

[0006]

[0007] However, the existing weakly supervised object detection models have difficulty focusing on the global information of the training data (input image) during the training process and are insensitive to background noise, resulting in the detection results containing redundant information. Figure 1 As shown in Figure 1, we expect the weakly supervised object detection model to detect the entire TV without including the table (the table is considered noise when detecting the TV class).

[0008] Therefore, a weakly supervised object detection technique is needed that can focus on the global information of the image and reduce background noise learning. Summary of the invention

[0009] Aiming at the problem that the existing weakly supervised object detection model cannot pay attention to the global information of the input image and is insensitive to background noise, resulting in the detection result containing redundant information, the present invention provides a weakly supervised object detection method based on global information learning.

[0010] A weakly supervised object detection method based on global information learning of the present invention comprises:

[0011] Acquire a sample image, wherein the sample image is configured with a category label; extract a candidate frame of a target object in the sample image;

[0012] The sample image is split into R, G, and B channels, and then the split image is subjected to singular value decomposition and reconstructed to obtain a reconstructed image;

[0013] A weakly supervised object detector is used to detect the sample image and the reconstructed image based on the candidate box, wherein the weakly supervised object detector includes a candidate box feature extraction module, a multi-instance learning network and an online example classification and refinement network;

[0014] The candidate frame feature extraction module extracts features from the sample image and the reconstructed image respectively, and obtains the sample image candidate frame features and the reconstructed image candidate frame features by combining the candidate frames;

[0015] The multi-instance learning network scores the candidate boxes based on the candidate box features of the sample image and obtains the multi-instance learning score of each candidate box;

[0016] During the training process, the online example classification and refinement network scores the candidate boxes based on the features of the sample image candidate boxes and the features of the reconstructed image candidate boxes. The multi-example learning scores of each candidate box are used as supervision information, and the model parameters of the weakly supervised object detector are optimized through back propagation in combination with the category labels configured in the sample images.

[0017] In actual detection, the image to be detected uses a multi-instance learning network to score each candidate box and determine the target box.

[0018] According to the weakly supervised object detection method based on global information learning of the present invention, the method of acquiring the sample image includes:

[0019] For general object detection tasks, use search engines to obtain sample images through keyword searches;

[0020] For general object detection tasks, sample images are obtained through public object detection datasets;

[0021] For specific object detection tasks, sample images are obtained through dedicated datasets.

[0022] According to the weakly supervised object detection method based on global information learning of the present invention, the sample image includes an acquired original image and an expanded image after data expansion is performed on the original image.

[0023] According to the weakly supervised object detection method based on global information learning of the present invention, the method for obtaining the reconstructed image is:

[0024] The sample image is split into R, G, and B channels to obtain a three-channel image; singular value decomposition is performed on each channel image to obtain a left singular value matrix, a singular value matrix with singular values ​​arranged from large to small, and a right singular value matrix; the first q singular values ​​in the singular value matrix with singular values ​​arranged from large to small are selected to truncate the three matrices and reconstruct them to obtain a reconstructed image; where q is a selected constant.

[0025] According to the weakly supervised object detection method based on global information learning of the present invention, the candidate box feature extraction module includes a VGG16 network and a RoI pooling layer pre-trained on the ImageNet dataset,

[0026] The VGG16 network extracts features from the sample image and the reconstructed image respectively to obtain sample image features and reconstructed image features; the RoI pooling layer maps the candidate boxes to the sample image features and the reconstructed image features to obtain a fixed-size feature representation, and further extracts features from the feature representation corresponding to each candidate box through two serially connected fully connected layers to obtain sample image candidate box features and reconstructed image candidate box features.

[0027] According to the weakly supervised object detection method based on global information learning of the present invention, the multi-instance learning network includes a classification branch and a detection branch. The classification branch uses a fully connected layer and a category Softmax layer to score the candidate box based on the candidate box features of the sample image to obtain a category score; the detection branch uses a fully connected layer and a candidate box Softmax layer to score the candidate box based on the candidate box features of the sample image to obtain a position score; the category score and the position score are dot-multiplied to obtain a multi-instance learning score for each candidate box.

[0028] According to the weakly supervised object detection method based on global information learning of the present invention, the online example classification and refinement network includes a sample image detection branch and a reconstructed image detection branch. The sample image detection branch uses a fully connected layer and a category Softmax layer to score the candidate box based on the sample image candidate box features to obtain the category score of the sample image; the reconstructed image detection branch uses a fully connected layer and a category Softmax layer to score the candidate box based on the reconstructed image candidate box features to obtain the category score of the reconstructed image.

[0029] According to the weakly supervised object detection method based on global information learning of the present invention, in actual detection, the multi-instance learning network scores the candidate boxes based on the candidate box features of the image to be detected, sorts them according to the category scores, takes out the candidate boxes with the highest scores in turn, calculates the intersection over union (IOU) ratio with the adjacent next-scoring candidate boxes, and suppresses the adjacent next-scoring candidate boxes that exceed the set threshold until the target box is determined.

[0030] According to the weakly supervised object detection method based on global information learning of the present invention, the set threshold is 0.5.

[0031] According to the weakly supervised object detection method based on global information learning of the present invention, the position information of the target box is used as a true value or a pseudo-true value to train a fully supervised object detector.

[0032] The beneficial effects of the invention are as follows: the invention is a weakly supervised object detection method based on global information learning when the model lacks global information perception of the image. It enables the weakly supervised object detection model to focus on the global information of the image during training, reduces background noise learning, overcomes the redundant detection existing in weak target object detection technology, and promotes the application of WSOD in object detection in real scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 This is an example of a weakly supervised object detector whose detection results contain redundant information.

[0034] Figure 2 This is the existing weakly supervised object detection flow chart based on multi-instance learning;

[0035] Figure 3 It is a comparison diagram of the original image and the reconstructed image;

[0036] Figure 4 is a flow chart of the method of the present invention;

[0037] Figure 5 It is a schematic diagram of obtaining a reconstructed image by performing singular value decomposition on a sample image;

[0038] Figure 6 It is a schematic diagram of the principle of weakly supervised object detector;

[0039] Figure 7 It is a diagram of the detection results in the verification experiment. DETAILED DESCRIPTION

[0040] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0041] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments may be combined with each other.

[0042] The present invention will be further described below in conjunction with the accompanying drawings and specific embodiments, but they are not intended to limit the present invention.

[0043] Specific implementation method 1. Combination Figures 2 to 6 As shown, the present invention provides a weakly supervised object detection method based on global information learning, comprising:

[0044] Acquire a sample image, wherein the sample image is configured with a category label; extract a candidate frame of a target object in the sample image;

[0045] The sample image is split into R, G, and B channels, and then the split image is subjected to singular value decomposition and reconstructed to obtain a reconstructed image;

[0046] A weakly supervised object detector is used to detect the sample image and the reconstructed image based on the candidate box, wherein the weakly supervised object detector includes a candidate box feature extraction module, a multi-instance learning network and an online example classification and refinement network;

[0047] The candidate frame feature extraction module extracts features from the sample image and the reconstructed image respectively, and obtains the sample image candidate frame features and the reconstructed image candidate frame features by combining the candidate frames;

[0048] The multi-instance learning network scores the candidate boxes based on the candidate box features of the sample image and obtains the multi-instance learning score of each candidate box;

[0049] During the training process, the online example classification and refinement network scores the candidate boxes based on the features of the sample image candidate boxes and the features of the reconstructed image candidate boxes. The multi-example learning scores of each candidate box are used as supervision information, and the model parameters of the weakly supervised object detector are optimized through back propagation in combination with the category labels configured in the sample images.

[0050] In actual detection, the image to be detected uses a multi-instance learning network to score each candidate box and determine the target box.

[0051] This implementation method uses images (video frames) in real scenes as detection objects, and the specific categories can be determined according to actual problems. With the development of Internet technology, image and video acquisition equipment has become popular. According to statistics, the images and videos on YouTube are now growing at a rate of 58 images per second and 3.6 videos per second. It is only necessary to crawl images on the search engine in the form of keywords according to the detection category to establish a database that conforms to the actual problem, solving the problem that there are few object categories in the fixed database and the object categories do not match the actual detection categories. At the same time, since there is no need to mark the location information, a large amount of manpower and material resources are not required to mark the database, and the deviation introduced by the subjectivity of manual marking is also avoided.

[0052] After establishing the training database, we can use the existing weakly supervised object detection technology to train a weakly supervised object detector. The existing weakly supervised object detection technology regards object detection under weakly supervised information as a multiple instance learning (MIL) problem. The existing weakly supervised object detection flow chart based on MIL is as follows: Figure 2As shown in the figure, the input image (training data) is first extracted by a pre-trained convolutional neural network, such as VGG16, to obtain the input image features. At the same time, the input image also needs to pass through a candidate box generation method such as Select Search to obtain about 2,000 candidate boxes of the input image; then the input image features and the candidate boxes of the input image are simultaneously input into a feature pooling layer such as the ROIPooling layer to obtain the mapping features of the candidate boxes on the image, i.e., the candidate box features; finally, the features of these candidate boxes are input into two branches, one for multi-instance learning and the other for online instance classifier refinement (OICR). However, using a pre-trained convolutional neural network to extract features cannot effectively learn the global information of the input image. In other words, the model cannot perceive the relationship between the pixels of the input image. In addition, the model is insensitive to the background noise information in the input image, resulting in the detection results containing redundant information.

[0053] The input image feature extraction module in the weakly supervised object detector is studied, and it is found that the existing feature extractors mainly rely on the local convolution operation of the ocean neural network to extract features when processing object detection tasks, which makes them have certain limitations in capturing the global information of the image. Specifically, these feature extractors (such as VGG16) tend to focus on significant local features, such as the edges and textures of objects, but it is difficult to understand the overall semantic context information. In addition, the background information in the image is also regarded as part of the target area by the feature extractor for feature extraction. This insensitivity to background information further limits the detection performance of the model. In view of the above problems and analysis, this embodiment proposes a weakly supervised object detection method based on global information learning. Specifically, in order to enable the weakly supervised object detector to learn the global information of the input image in the image feature extraction stage, a global information learning (GIL) module is proposed. For the input image, three matrices are obtained by singular value decomposition, including a left singular value matrix, a singular value matrix with singular values ​​arranged from large to small, and a right singular value matrix. Among them, the singular values ​​reflect the importance of different features in the input image. Larger singular values ​​correspond to important information in the input image, and smaller singular values ​​correspond to noise or minor features of the input image. Then, the singular value matrices are truncated using the first q singular values, and the input image is reconstructed. Each pixel in the reconstructed image naturally contains the features of other pixels, which means that the candidate box feature extraction module in this embodiment can capture the global information of the input image. In addition, the image reconstructed with the first q singular values ​​retains the important information of the input image, reducing the burden of weakly supervised object detectors learning background noise and redundant information. Figure 3(a) The original input image and the image reconstructed by singular value decomposition Figure 3 The comparison diagram of (b) is as follows Figure 3 As shown. Therefore, based on the multi-instance learning paradigm, the reconstructed image can encourage the weakly supervised object detector to learn the important features of the input image from a global perspective while ignoring background noise and redundant information. This has promoted the weakly supervised object detection technology from the laboratory to practical applications and promoted the development of weakly supervised object detection technology.

[0054] Therefore, this embodiment solves the problem that the existing weakly supervised object detection method is difficult to focus on the global information of the training image and is insensitive to background noise, and solves the problem that the weakly supervised object detection results have redundant information.

[0055] Combination Figure 4 As shown, this implementation method first prepares training samples according to actual needs. Then global information learning is performed. The input training image is decomposed using singular value decomposition to obtain a left singular value matrix, a singular value matrix with singular values ​​arranged from large to small, and a right singular value matrix. Then the first q singular values ​​are intercepted and reconstructed to obtain a reconstructed input image. The reconstructed image includes the global information of the image. Secondly, the original input image and the reconstructed image are input into the weakly supervised trainer together, and the multiple instance learning (MIL) method is used to carry out the training of the weakly supervised object detector to obtain the position information of each object in the training sample. Finally, this position information is used as the true value (pseudo-true value) to train a fully supervised object detector, and the fully supervised object detector will give a more accurate detection result.

[0056] Prepare training samples and obtain samples according to the actual needs of object detection:

[0057] As an example, the sample image is obtained by:

[0058] For general object detection tasks, such as pedestrian detection and vehicle inspection, a search engine can be used to search for pedestrians, vehicles, etc. by keywords to obtain sample images;

[0059] For general object detection tasks, sample images are obtained through public object detection datasets such as PASCAL VOC and MS COCO.

[0060] For specific object detection tasks, such as face detection, sample images are obtained through dedicated datasets such as WIDER FACE and FDDB.

[0061] In order to maintain generality, the PASCAL VOC 2007 dataset can be selected as sample images in this embodiment. The "trainval" part of the dataset is used as training data for the weakly supervised object detector, and the "test" part of the dataset is used as test data for the weakly supervised object detector. In the training phase, this embodiment only uses the category label information, and does not use the location information of any object.

[0062] The sample image includes the acquired original image and the expanded image after data expansion of the original image. In order to further enhance the performance and robustness of the model, this embodiment adopts some data expansion strategies, namely data enhancement and multi-scale processing. Data enhancement refers to flipping all training samples left and right, and adding the flipped images to the training set, thereby increasing the sample size and improving the generalization ability of the model. Multi-scale processing: In order to adapt to the characteristics of multi-scale changes of objects in real scenes, the aspect ratio of the picture is kept unchanged, and one of the five scales {480, 576, 688, 864, 1200} is randomly selected as the shortest side of the training sample. In order to avoid insufficient GPU memory due to excessively high resolution, the longest side of the training sample is limited to no more than 2000. Through the preparation and enhancement strategies of the above-mentioned training samples, it is possible to improve its adaptability to diversified scenes while ensuring the efficiency of model training, and provide better support for the practical application of weakly supervised target detection tasks.

[0063] Global information learning. Existing weakly supervised object detectors rely on local convolution operations of convolutional neural networks to extract features, which makes these object detectors have certain limitations in capturing global information of the image. In addition, the background information in the image is also regarded as part of the target area by the feature extractor for feature extraction. This insensitivity to background information further limits the detection performance of the model. Specifically, the objects detected by the weakly supervised object detector contain redundant background information.

[0064] Further, combined with Figure 5 As shown, in order to make the weakly supervised object detector focus on global information during the training process and reduce the interference of redundant information, this embodiment learns global information. First, the training data (input image) is split into three channels of R, G, and B, and the image is reconstructed.

[0065] The method to obtain the reconstructed image is:

[0066] The sample image is split into R, G, and B channels to obtain a three-channel image, such as Figure 5Step 1 in ; perform singular value decomposition on each channel image to obtain a left singular value matrix, a singular value matrix arranged from large to small, and a right singular value matrix, select the first q singular values ​​in the singular value matrix arranged from large to small to truncate the three matrices, and reconstruct them to obtain a reconstructed image, such as Figure 5 As shown in steps 2 and 3 in ; where q is a selected constant. Each pixel in the reconstructed image naturally contains the features of other pixels, which means that the global information of the input image can be captured. In addition, the image reconstructed with the first q singular values ​​retains the important information of the input image, reducing the burden of weakly supervised object detectors to learn background noise and redundant information.

[0067] Training weakly supervised detectors. This embodiment uses the Multiple Instance Learning (MIL) method to implement the training of weakly supervised object detectors. Due to the lack of clear location information as a supervisory signal, traditional weakly supervised object detectors tend to converge to a local optimal solution, resulting in a low recognition rate for object detection. To solve this problem, this embodiment introduces several optimization networks in parallel with the multiple instance learning detection network during the model training process, namely, the online example classification and refinement network, such as Figure 6 As shown in the figure, the recognition rate of the detector is improved. For each input sample, the Selective Search algorithm is used to extract about 2,000 candidate regions. These candidate regions cover the locations that may contain the target object. At the same time, for each input sample, global information learning is required to obtain the reconstructed image.

[0068] Combination Figure 6 As shown, the candidate box feature extraction module includes a VGG16 network and a RoI pooling layer pre-trained on the ImageNet dataset.

[0069] The VGG16 network extracts features from the sample image and the reconstructed image respectively to obtain sample image features and reconstructed image features; then the features of the original image and the reconstructed image are input into the RoI pooling layer (Region of Interest Pooling) together with the candidate box. The RoI pooling layer maps the candidate box to the sample image features and the reconstructed image features to obtain a fixed-size feature representation. The feature representation corresponding to each candidate box is further extracted through two series-connected fully connected layers to obtain the sample image candidate box features and the reconstructed image candidate box features, with a dimension of 4096.

[0070] In this embodiment, the multi-instance learning network includes a classification branch and a detection branch. The classification branch uses a fully connected layer and a category Softmax layer to score the candidate box based on the sample image candidate box features to obtain a category score; the detection branch uses a fully connected layer and a candidate box Softmax layer to score the candidate box based on the sample image candidate box features to obtain a position score; the category score and the position score are dot-multiplied to obtain a multi-instance learning score for each candidate box.

[0071] In the multi-instance learning network, the input is the candidate box features in the original image. Each candidate box feature is sent to two parallel fully connected layers (classification and detection) and a Softmax layer. The two parallel classification and detection branches are used to determine the category of each candidate area and score the location information of each candidate area. Finally, the scores of the classification branch and the detection branch are multiplied to obtain the score of this candidate area.

[0072] The online example classification and refinement network includes a sample image detection branch and a reconstructed image detection branch. The sample image detection branch uses a fully connected layer and a category Softmax layer to score the candidate box based on the sample image candidate box features to obtain the category score of the sample image; the reconstructed image detection branch uses a fully connected layer and a category Softmax layer to score the candidate box based on the reconstructed image candidate box features to obtain the category score of the reconstructed image.

[0073] In the online example classification and refinement network, the candidate box features of the original image and the reconstructed image candidate box features use the final scores of the candidate areas in the multi-example learning network as supervision signals, and further optimize the model parameters through back propagation, thereby improving the recognition rate of the weakly supervised object detector. In order to balance the training time and recognition rate, this embodiment sets the number of online example classification and refinement networks to 3 to achieve a higher recognition rate while controlling the training time within a reasonable range. The three online example classification and refinement networks are connected in series, and the output of the kth online example classification and refinement network is used to supervise the k+1th online example classification and refinement network.

[0074] Furthermore, in actual detection, the multi-instance learning network scores the candidate boxes based on the candidate box features of the image to be detected, sorts them according to the category scores, takes out the candidate boxes with the highest scores in turn, calculates the IOU with the adjacent next-scoring candidate boxes, and suppresses the adjacent next-scoring candidate boxes that exceed the set threshold until the target box is determined.

[0075] Since multiple high-scoring candidate boxes may overlap significantly, in order to remove redundant boxes, OICR usually applies a non-maximum suppression strategy:

[0076] For each category, sort by score from high to low. Take out the candidate boxes with the highest scores one by one, and suppress other boxes whose IOU with them exceeds the set threshold; continue to process the remaining candidate boxes until all boxes are processed and the target box is determined.

[0077] As an example, the threshold is set to 0.5.

[0078] In this implementation, the position information of the target box is used as a true value or a pseudo-true value to train a fully supervised object detector.

[0079] After the weakly supervised detector is processed, each object in the training sample obtains relatively accurate location information. This location information is used as the ground truth and can be used to train a fully supervised object detector. The fully supervised object detector can be any existing mainstream detector, such as Fast R-CNN, Faster R-CNN, YOLO, SSD, etc.

[0080] Verification experiment:

[0081] Fast R-CNN is selected as the fully supervised object detector. The specific settings of the training are as follows: total number of iterations: 70,000 times; learning rate strategy: the learning rate of the first 40,000 iterations is set to 0.01 to quickly optimize the model parameters. The learning rate of the next 30,000 iterations is reduced to 0.001 to ensure that the model gradually converges and avoids overfitting. Through the above training process, based on the position information generated by the weakly supervised detector, the trained fully supervised detector can further improve the accuracy of target detection and provide higher quality detection results for practical applications.

[0082] The weakly supervised object detection network trained by the above steps can realize the object detection function without the need for location information annotation. The network can enable the model to learn the global information of the image and reduce the interference of redundant information and noise. The experimental results show that the positioning accuracy of the method of the present invention is accurate and the detection efficiency is high. Table 2 shows the experimental results and comparison data.

[0083] Table 2 Experimental results comparison data

[0084]

[0085] Among them, mAP is the mean Average Precision, which is used to evaluate the detection accuracy in the test samples and reflects the detection effect of the model on different target categories. Corloc is the correct location rate (Correct Location), which is used to evaluate the positioning effect of the training samples during the training process and indicates whether the candidate area correctly covers the target object. From the comparison data, it can be seen that the framework of "weakly supervised object detector + global information learning + fully supervised detector" proposed by the method of the present invention has a huge improvement in the detection results compared with the "weakly supervised detector". The "global information learning" of the method of the present invention is compared with the detection results without using "global information learning". Figure 7 It is an experimental result diagram, in which the green detection box is the detection result of the "weakly supervised object detection method based on global feature learning" in the present invention, and the red detection box is the detection result of the "weakly supervised object detector + fully supervised detector". The advantage of the method of the present invention can be seen from the figure: the detection result of the present invention contains less redundant information.

[0086] Although the present invention is described herein with reference to specific embodiments, it should be understood that these embodiments are merely examples of the principles and applications of the present invention. It should therefore be understood that many modifications may be made to the exemplary embodiments and that other arrangements may be devised without departing from the spirit and scope of the present invention as defined by the appended claims. It should be understood that the various dependent claims and features described herein may be combined in a manner different from that described in the original claims. It should also be understood that the features described in conjunction with a single embodiment may be used in other described embodiments.

Claims

1. A weakly supervised object detection method based on global information learning, characterized in that: include, Acquire a sample image, wherein the sample image is configured with a category label; extract a candidate frame of a target object in the sample image; The sample image is split into R, G, and B channels, and then the split image is subjected to singular value decomposition and reconstructed to obtain a reconstructed image; A weakly supervised object detector is used to detect the sample image and the reconstructed image based on the candidate box, wherein the weakly supervised object detector includes a candidate box feature extraction module, a multi-instance learning network and an online example classification and refinement network; The candidate frame feature extraction module extracts features from the sample image and the reconstructed image respectively, and obtains the sample image candidate frame features and the reconstructed image candidate frame features by combining the candidate frames; The multi-instance learning network scores the candidate boxes based on the candidate box features of the sample image and obtains the multi-instance learning score of each candidate box; During the training process, the online example classification and refinement network scores the candidate boxes based on the features of the sample image candidate boxes and the features of the reconstructed image candidate boxes. The multi-example learning scores of each candidate box are used as supervision information, and the model parameters of the weakly supervised object detector are optimized through back propagation in combination with the category labels configured in the sample images. In actual detection, the image to be detected uses a multi-instance learning network to score each candidate box and determine the target box.

2. The weakly supervised object detection method based on global information learning according to claim 1 is characterized in that: The sample images are obtained by: For general object detection tasks, use search engines to obtain sample images through keyword searches; For general object detection tasks, sample images are obtained through public object detection datasets; For specific object detection tasks, sample images are obtained through dedicated datasets.

3. The weakly supervised object detection method based on global information learning according to claim 2 is characterized in that: The sample image includes an acquired original image and an expanded image obtained by performing data expansion on the original image.

4. The weakly supervised object detection method based on global information learning according to claim 1, characterized in that: The method to obtain the reconstructed image is: The sample image is split into R, G, and B channels to obtain a three-channel image; singular value decomposition is performed on each channel image to obtain a left singular value matrix, a singular value matrix with singular values ​​arranged from large to small, and a right singular value matrix; the first q singular values ​​in the singular value matrix with singular values ​​arranged from large to small are selected to truncate the three matrices and reconstruct them to obtain a reconstructed image; where q is a selected constant.

5. The weakly supervised object detection method based on global information learning according to claim 1, characterized in that: The candidate box feature extraction module includes a VGG16 network and a RoI pooling layer pre-trained on the ImageNet dataset. The VGG16 network extracts features from the sample image and the reconstructed image respectively to obtain sample image features and reconstructed image features; the RoI pooling layer maps the candidate boxes to the sample image features and the reconstructed image features to obtain a fixed-size feature representation, and further extracts features from the feature representation corresponding to each candidate box through two serially connected fully connected layers to obtain sample image candidate box features and reconstructed image candidate box features.

6. The weakly supervised object detection method based on global information learning according to claim 1, characterized in that: The multi-instance learning network includes a classification branch and a detection branch. The classification branch uses a fully connected layer and a category Softmax layer to score the candidate boxes based on the features of the candidate boxes in the sample image to obtain the category score. The detection branch uses a fully connected layer and a candidate box Softmax layer to score the candidate box based on the sample image candidate box features to obtain a position score; the category score and the position score are dot-multiplied to obtain a multi-instance learning score for each candidate box.

7. The weakly supervised object detection method based on global information learning according to claim 1, characterized in that: The online example classification and refinement network includes a sample image detection branch and a reconstructed image detection branch. The sample image detection branch uses a fully connected layer and a category Softmax layer to score the candidate box based on the sample image candidate box features to obtain the category score of the sample image; the reconstructed image detection branch uses a fully connected layer and a category Softmax layer to score the candidate box based on the reconstructed image candidate box features to obtain the category score of the reconstructed image.

8. The weakly supervised object detection method based on global information learning according to claim 1, characterized in that: In actual detection, the multi-instance learning network scores the candidate boxes based on the candidate box features of the image to be detected, sorts them according to the category scores, takes out the candidate boxes with the highest scores in turn, calculates the intersection over union (IOU) with the adjacent next-scoring candidate boxes, and suppresses the adjacent next-scoring candidate boxes that exceed the set threshold until the target box is determined.

9. The weakly supervised object detection method based on global information learning according to claim 8, characterized in that: The threshold is set to 0.

5.

10. The weakly supervised object detection method based on global information learning according to claim 1, characterized in that: The position information of the target box is used as a true value or a pseudo-true value to train a fully supervised object detector.

Images