Multi-head weakly supervised object detection method based on cross attention mechanism

Abstract

The present invention provides a multi-head weakly supervised target detection method based on a cross-attention mechanism, comprising: acquiring an image, processing the image, and dividing the processed image into a training set and a test set; defining a class prototype feature, and constructing a weakly supervised target detection network WCAN model based on a cross-attention mechanism containing the class prototype feature; using a training data set to train the WCAN model; and based on the trained WCAN model, performing target detection on the test set image. Under the condition of only having a class label, the method can more comprehensively and fully perceive the position and class information, and realize more accurate and efficient target detection.

Description

Technical Field

[0001] The present invention relates to the field of computer vision, and in particular to a multi-head weakly supervised target detection method based on a cross-attention mechanism. Background Art

[0002] Object detection is to find all the objects of interest in an image or video and obtain the category information and location information of this object. Object detection is the basis of visual recognition and analysis, and is one of the core issues in the field of computer vision. In recent years, with the development of deep learning, the performance of object detection models has been greatly improved. However, most of these studies are proposed for fully supervised object detection. The learning of fully supervised models requires large-scale data with accurate object box annotations. At present, accurate annotations are mostly completed manually by algorithm staff, and the annotation work consumes manpower, material resources and financial resources, especially when there are multiple objects in an image, the objects need to be manually annotated one by one. In order to solve the annotation problem, relevant researchers proposed weakly supervised object detection. This method only needs to calibrate the category information of the objects contained in this image and provide image-level annotation information. This annotation method greatly reduces the annotation workload and increases the annotation speed by more than 10 times. Therefore, how to improve the performance of weakly supervised object detection has become a current research hotspot.

[0003] The task of weakly supervised object detection is to find and locate all objects through algorithms under the condition that there are object categories in a given image. The Multiple Instance Learning (MIL) algorithm is the most commonly used method for weakly supervised object detection. It treats the entire image as a package and generates a large number of candidate boxes through the Selective Search (SS) algorithm or the Multiscale Combinatorial Grouping (MCG) method. These candidate boxes are taken as objects and a classifier is trained to assign category labels to these candidate boxes through the multi-instance learning algorithm. On this basis, the methods used in the prior art iteratively perform candidate box selection and appearance model estimation, and gradually detect the object. However, this iterative method can easily cause the model to fall into a local optimum. To address the above problems, a concise end-to-end weakly supervised object detection framework WSDDN is proposed, which independently performs candidate box region classification and candidate region localization, and then integrates the outputs of the two branches into image-level supervised learning to implicitly learn the object detector. However, the two branches in this method are learned independently. The localization branch does not consider the overall semantic information and may output an incorrect candidate region localization score. The classifier in the classification branch only focuses on the local discriminative parts of the target and cannot obtain the optimal candidate region score. The two independent branches cannot fully perceive the category and location information, which does not meet the requirements of the goal. To solve this problem, some existing methods improve the performance of weakly supervised face detection by utilizing contextual information, introducing additional segmentation modules, or training online object optimization modules. Although these methods have improved the existing methods, their base classifiers are all based on the WSDDN framework. This basic model hinders the further improvement of the performance of existing methods.

[0004] Therefore, there is an urgent need for a method that can prevent falling into local optimality and achieve more accurate target detection. Summary of the invention

[0005] The present invention provides a multi-head weakly supervised target detection method based on a cross-attention mechanism to break through the existing technical bottleneck and solve the defects in the existing technical problems.

[0006] In order to achieve the above object, the present invention adopts the following technical scheme.

[0007] A multi-head weakly supervised object detection method based on a cross-attention mechanism, comprising:

[0008] S1 acquires images, processes the images, and divides the processed images into a training set and a test set;

[0009] S2 defines the prototype features and constructs a weakly supervised target detection network WCAN model based on the cross attention mechanism that includes the prototype features;

[0010] S3 uses the training data set to train the WCAN model;

[0011] S4 performs object detection on the test set images based on the trained WCAN model.

[0012] Preferably, processing the image includes: for each image, marking the category label contained in the image, and using a selective search method or a multi-scale combined grouping method to obtain a candidate region in each image in the data set.

[0013] Preferably, defining the class prototype feature includes: defining the class prototype feature Each class prototype features E i Corresponding to a specific semantic category, C is the number of categories, D is the dimension of the class prototype feature, Represents a real number.

[0014] Preferably, the WCAN model includes a candidate region feature extraction network, a class prototype feature, and a multi-head weakly supervised target detection network;

[0015] The multi-head weakly supervised object detection network includes a parallel category-aware cross-attention module and a location-aware cross-attention module.

[0016] Preferably, the WCAN model also includes a multi-head weakly supervised position localization network, which includes a position-aware cross-attention module.

[0017] Preferably, the WCAN model is trained using a training data set, including:

[0018] S31 initializes the WCAN model;

[0019] S32 inputs the image in the training set into the candidate region feature extraction network of the WCAN model, and extracts the feature map through the backbone network;

[0020] S33 simultaneously inputs the extracted image feature map and the corresponding candidate region into the region pooling layer to extract the candidate region feature F p ; Candidate region feature F p The prototype feature E is simultaneously input into the class-aware cross-attention module and the positioning-aware cross-attention module in the multi-head weakly supervised target detection network to obtain the multi-head candidate region category similarity score and the multi-head candidate region target similarity score respectively. The multi-head candidate region category similarity score and the multi-head candidate region target similarity score of the same region are fused to obtain the candidate region score, and the multi-head candidate region scores are aggregated to obtain the image score Si ;

[0021] S34 uses the cross entropy loss function to calculate the score S for each image i The error between the image label and the image label is calculated, and multiple error values ​​are summed up. The parameters of the backbone network, class prototype features, and multi-head weakly supervised target detection network model in the WCAN model are updated using the stochastic gradient descent algorithm.

[0022] S35 repeats steps S32-S34 until the result of the cross entropy loss function no longer decreases, and a trained WCAN model is obtained.

[0023] Preferably, between S33 and S34, the following steps are also included:

[0024] The image feature map extracted by the backbone network is transformed in the spatial dimension to obtain the position feature, and the linear mapping layer is used to align the position feature dimension to the prototype feature dimension;

[0025] The position feature F pi The prototype feature E is input into the position-aware cross-attention module in the multi-head weakly supervised position localization network to obtain the multi-head position score, and the multi-head position scores are aggregated to obtain the multi-head image score.

[0026] The cross entropy loss function is used to calculate the score for each image. The error between the image label and the image label is calculated, and multiple error values ​​are summed up. The stochastic gradient descent algorithm is used to update the parameters of the backbone network, class prototype features, and multi-head weakly supervised target detection network model in the WCAN model.

[0027] Preferably, the candidate region feature F p The prototype feature E is simultaneously input into the class-aware cross-attention module and the positioning-aware cross-attention module in the multi-head weakly supervised target detection network to obtain the multi-head candidate region category similarity score and the multi-head candidate region target similarity score, which specifically include:

[0028] For the class-aware cross-attention module in the multi-head weakly supervised object detection network: using the linear mapping layer FC ck Map the candidate region features, using layer normalization layer and linear mapping layer FC cq Map the class prototype features, and then split the mapped class prototype features and the mapped candidate region features in the feature dimension; then use the dot-product operation to calculate the similarity score between the i-th class prototype feature slice and the i-th candidate region feature slice, and then perform softmax on the similarity score in the category dimension. c Normalized to get the similarity score P of the i-th candidate region category i,c, a total of candidate region category similarity scores equal to the number of segmentations can be obtained;

[0029] For the positioning-aware cross-attention module in the multi-head weakly supervised target detection network: using the linear mapping layer FC dk Map the input candidate region features, using layer normalization layer and linear mapping layer FC dq Map the class prototype features, then slice the mapped class prototype features and the mapped candidate region features to obtain class prototype feature slices and candidate region feature slices; then use the dot-product operation to calculate the similarity score between the i-th class prototype feature slice and the i-th candidate region feature slice, and then perform softmax on the similarity score in the candidate region dimension d Normalized to get the target similarity score P of the i-th candidate region i,d , a total of the candidate region target similarity scores, which is the same as the number of slices, can be obtained.

[0030] Preferably, the position feature F pi The prototype feature E is input into the position-aware cross-attention module in the multi-head weakly supervised position localization network to obtain the multi-head position score, which includes:

[0031] Using linear mapping layer FC pi_k Map the position features, use layer normalization layer, linear mapping layer FC pi_q Map the class prototype; then split the mapped class prototype features and the mapped position features in the feature dimension; then use the dot-product operation to calculate the similarity score of the i-th class prototype feature slice and the i-th position feature slice, and then perform softmax on the similarity score in the position dimension l Normalized to get the i-th position score PI i , a total of the same number of position scores as the number of cuts can be obtained.

[0032] Preferably, S4 includes:

[0033] S41 uses the backbone network to extract image feature maps in the test set;

[0034] S42 uses a regional pooling layer to extract candidate region features of the image;

[0035] S43 inputs the candidate region features and the class prototype features into the class-aware cross-attention module and the positioning-aware cross-attention module in the multi-head weakly supervised target detection network to obtain the multi-head candidate region score;

[0036] S44 averages the scores of all the multiple candidate regions to obtain the final candidate region score;

[0037] S45 uses the NMS algorithm to remove overlapping and low-scoring candidate regions to obtain the final target detection result.

[0038] It can be seen from the technical solution provided by the multi-head weakly supervised target detection method based on the cross-attention mechanism of the present invention that the present invention takes into account that there are only category labels under weak supervision conditions. In order to make the model comprehensively consider the classification and positioning tasks, the class prototype features are defined, and cross-attention learning is performed simultaneously with the candidate area features from the category perception and positioning dimensions, thereby encouraging the model to learn knowledge that is sensitive to both classification and positioning, which is more in line with the task requirements of target detection; a multi-detection head is proposed to realize weakly supervised target detection, guiding the network to pay attention to different target areas at the same time, thereby preventing it from falling into the local optimum, obtaining a more complete detection result, and alleviating the problem that the weakly supervised target detection model only pays attention to the most conspicuous parts of the local area; it is proposed to optimize the prototype features on the fine-grained image features to prevent the introduction of background features and learn more accurate class prototype features. Therefore, under the condition of only category labels, the model of the present invention can perceive the position and category information more comprehensively and fully, and the target detection is more accurate and efficient.

[0039] Additional aspects and advantages of the present invention will be given in part in the following description, which will become obvious from the following description, or may be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0041] Figure 1 This is a flow chart of the multi-head weakly supervised target detection method based on the cross attention mechanism of this embodiment;

[0042] Figure 2 This is a schematic diagram of the structure of a multi-head weakly supervised detection network model based on a cross-attention mechanism;

[0043] Figure 3 This is a diagram of the training process;

[0044] Figure 4 It is a test process diagram;

[0045] Figure 5 Schematic diagram of class-aware cross-attention module processing in multi-head weakly supervised object detection network;

[0046] Figure 6Schematic diagram of the cross-attention module processing for positioning perception in a multi-head weakly supervised target detection network;

[0047] Figure 7 Schematic diagram of the position-aware cross-attention module processing in a multi-head weakly supervised position localization network;

[0048] Figure 8 A schematic diagram of the test results. DETAILED DESCRIPTION

[0049] The embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and cannot be interpreted as limiting the present invention.

[0050] It will be understood by those skilled in the art that, unless expressly stated, the singular forms "one", "said", and "the" used herein may also include plural forms. It should be further understood that the term "comprising" used in the specification of the present invention refers to the presence of the features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof. It should be understood that when we refer to an element as being "connected" or "coupled" to another element, it may be directly connected or coupled to the other element, or there may be intermediate elements. In addition, the "connection" or "coupling" used herein may include wireless connection or coupling. The term "and / or" used herein includes any unit and all combinations of one or more associated listed items.

[0051] It will be understood by those skilled in the art that, unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as those generally understood by those skilled in the art in the art to which the present invention belongs. It should also be understood that terms such as those defined in common dictionaries should be understood to have meanings consistent with the meanings in the context of the prior art, and will not be interpreted with idealized or overly formal meanings unless defined as herein.

[0052] To facilitate understanding of the embodiments of the present invention, further explanation will be given below by taking specific embodiments as examples in conjunction with the accompanying drawings, which does not constitute a limitation on the embodiments of the present invention.

[0053] Example

[0054] Figure 1 This is a flow chart of the multi-head weakly supervised target detection method based on the cross attention mechanism of this embodiment, referring to Figure 1 , the method comprising:

[0055] S1 obtains images, processes them, and divides the processed images into training sets and test sets. Dataset images can be captured from websites, or public target detection datasets such as PASCAL VOC or MS-COCO can be used. For each image obtained, annotate the category label Y contained in the image = {Y1, Y2, ..., Y C},Y c ∈{0,1}, use the selective search method (SS, Selective Search) or the multiscale combinatorial grouping method (MCG, Multiscale Combinatorial Grouping) to obtain the candidate region in each image in the dataset. Where Y is the total number of candidate regions (also called candidate boxes) extracted from each image, and each candidate region is represented by a 4-dimensional coordinate vector, indicating the position of the candidate region in the image. Then the image dataset with labeled and divided candidate regions is divided into a training set and a test set. Divide the dataset into a training set and a test set according to a certain ratio, such as 8:2 or 7:3. The public target detection dataset has been divided into training sets and test sets.

[0056] S2 defines class prototype features and constructs a weakly supervised object detection network (WCAN) model based on a cross-attention mechanism that includes class prototype features.

[0057] Defining Class Prototype Characteristics Each class prototype features E i Corresponding to a specific semantic category, C is the number of categories, D is the dimension of the class prototype feature, Represents a real number.

[0058] Figure 2 This is a schematic diagram of the structure of a multi-head weakly supervised detection network model based on a cross-attention mechanism. Figure 2 The WCAN model includes a candidate region feature extraction network, a class prototype feature, and a multi-head weakly supervised target detection network; the multi-head weakly supervised target detection network includes a parallel category-aware cross-attention module and a location-aware cross-attention module. It should be noted that the WCAN model also includes a multi-head weakly supervised position localization network, which includes a location-aware cross-attention module.

[0059] Since the category-aware cross-attention module, the location-aware cross-attention module, and the position-aware cross-attention module constructed in this embodiment can output multi-head prediction values ​​at the same time, the constructed network model is a multi-head weakly supervised target detection network model based on the cross-attention mechanism.

[0060] S3 uses the training data set to train the WCAN model. Figure 3 A diagram of the training process.

[0061] Initialize the WCAN model:

[0062] For the backbone network in the candidate region feature extraction network, the existing network model parameters pre-trained on the ImageNet dataset are used for initialization, and other network model parameters are initialized with random parameters. The class prototype feature E is also initialized with random parameters.

[0063] After initialization, a training image is sampled from the training set, and the category label Y = {Y1, Y2, ..., Y C},Y c ∈{0,1} and candidate regions

[0064] The images in the training set are input into the candidate region feature extraction network of the WCAN model, and the image feature map is extracted through the backbone network. Among them, H, W and Ch represent the width, height and feature dimensions of the image feature map respectively.

[0065] The extracted image feature map F and the corresponding candidate region P are simultaneously input into the regional pooling layer (RP, ROIPooling) to extract the candidate region feature F p It should be noted that the candidate region feature map must also be transformed in the spatial dimension, and then pass through two nonlinear mapping layers FC1 and FC2 to obtain the candidate region feature D1 is the dimension of the candidate region feature. Each nonlinear mapping layer consists of a fully connected layer FC, a nonlinear activation function ReLU and a Dropout layer.

[0066] Candidate region features F p The prototype feature E is simultaneously input into the class-aware cross-attention module and the positioning-aware cross-attention module in the multi-head weakly supervised target detection network to obtain the multi-head candidate region category similarity score P i,c , i = {1,...,H1} and the similarity score P of multiple candidate regions i,d , i={1,...,H1}.

[0067] The schematic diagram of class-aware cross-attention module processing in multi-head weakly supervised target detection network is as follows Figure 5 As shown: Linear mapping layer FC is used ck For the candidate region feature F p Mapping is performed to obtain the mapped candidate region feature Fck . Use layer normalization layer, linear mapping layer FC cq Map the class prototype features to obtain the mapped class prototype features E cq Then, the mapped class prototype features and the mapped candidate region features are segmented in the feature dimension. For example, the D-dimensional features are segmented into H1 parts to obtain H1 feature slices. The feature dimension of each slice is D / H1, and H1 class prototype feature slices and H1 candidate region feature slices are obtained. Specifically, F ck Cut into H1 parts in the feature dimension to obtain the slice candidate region features E cq Cut into H1 parts in the feature dimension to get the prototype feature slice Next, the dot-product operation is used to calculate the similarity score between the i-th class prototype feature slice and the i-th candidate region feature slice. and class prototype feature slices Perform vector inner product and output the similarity matrix of class prototype features and candidate region features ⊙ is the dot-product operation. The same operation is performed on other slice pairs, and a total of H1 similarity matrices are output. Then perform softmax on the similarity score in the category dimension c Normalized to get the similarity score P of the i-th candidate region category i,c , a total of candidate region category similarity scores P equal to the number of segmentations can be obtained i,c ,i={1,...,H1}, output candidate region category score matrix Represents the candidate region P n With class prototype E m The category similarity score of .

[0068] The schematic diagram of the cross-attention module processing of positioning perception in the multi-head weakly supervised target detection network is as follows Figure 6 As shown: Linear mapping layer FC is used dk For the input candidate region feature F p Mapping is performed to obtain the mapped candidate region feature F dk . Use layer normalization layer, linear mapping layer FC dq Map the class prototype features to obtain the mapped class prototype features E dq Then, the mapped class prototype features and the mapped candidate region features are sliced ​​to obtain the class prototype feature slices and candidate region feature slices Specifically, H1 slices are performed to obtain H1 class prototype feature slices and H1 candidate region feature slices; then the dot-product operation is used to calculate the similarity score between the i-th class prototype feature slice and the i-th candidate region feature slice. Specifically, for each candidate region feature slice and class prototype feature slices Perform vector inner product and output the similarity matrix of class prototype features and candidate region features ⊙ is the dot-product operation. The same operation is performed on other slice pairs, and a total of H1 similarity matrices are output. Then perform softmax on the similarity score in the candidate region dimension d Normalized to get the target similarity score P of the i-th candidate region i,d , a total of candidate region target similarity scores P, which are the same as the number of slices, can be obtained i,d ,i={1,...,H1}, output candidate region target similarity matrix Represents the candidate region P n With class prototype E m The target similarity score.

[0069] Specifically, the class-aware cross-attention module in the multi-head weakly supervised target detection network outputs H1 candidate region category similarity scores, and the positioning-aware cross-attention module outputs H1 candidate region target similarity scores. Preferably, H1=4.

[0070] By measuring the similarity between class prototype features and candidate region features from different dimensions through the class-aware cross-attention module and the positioning-aware cross-attention module, more accurate candidate region scores can be output.

[0071] The candidate region score is obtained by fusing the category similarity scores of multiple candidate regions and the target similarity scores of multiple candidate regions in the same region. Specifically, the category similarity score of the i-th candidate region and the target similarity score of the i-th candidate region are fused by element-wise product to obtain the i-th candidate region score. It is an element-by-element multiplication operation. The outputs of the two modules are combined to obtain the multi-head candidate region score P i ,i={1,...,H1}.

[0072] Aggregate the scores of multiple candidate regions to get the image score S i , the sum pooling layer is used to aggregate the candidate region scores to obtain the image score. Specifically, for each category c, the cumulative sum of the candidate region scores is calculated to obtain the score / prediction value of the image under category c Therefore, the image score is obtained Aggregating the scores of multiple candidate regions can obtain the multi-head image score S i ,i={1,...,H1}.

[0073] S34 uses the cross entropy loss function to calculate the score S for each image i The error between the image label And sum up multiple error values Use the stochastic gradient descent algorithm to update the parameters of the backbone network, prototype features, and multi-head weakly supervised target detection network model in the WCAN model;

[0074] S35 repeats steps S32-S34 until the result of the cross entropy loss function no longer decreases, and a trained WCAN model is obtained.

[0075] The above steps realize weakly supervised target detection based on the cross-attention mechanism. The constructed class-aware cross-attention module and the positioning-aware cross-attention module constrain the similarity between the candidate region and the class prototype from different dimensions, and the accuracy is higher. The proposed class prototype feature increases the interaction between the two modules, which can learn both class discriminant knowledge and candidate region target knowledge. With the help of comprehensive class prototype features, the network model can output accurate candidate region scores, thereby correctly selecting the target candidate region. At the same time, the two modules constructed in this embodiment can simultaneously output multi-head candidate region scores, which is a multi-head weakly supervised target detection network, which can prevent the network from falling into a local minimum.

[0076] Furthermore, this embodiment also proposes a multi-head weakly supervised position localization network, which is mainly composed of a position-aware cross-attention module. This module can further optimize the prototype features, optimize the prototype features with the most similar position features, and suppress the interference of background position features in non-target areas. The specific content is:

[0077] Between S33 and S34, the following may also be included:

[0078] Image feature map after backbone network feature extraction The shape is transformed in the spatial dimension to obtain the position feature, and the linear mapping layer FC is used align2 Align the position feature dimension to the class prototype feature dimension.

[0079] The position feature F pi The prototype feature E is input into the position-aware cross-attention module in the multi-head weakly supervised position localization network to obtain the multi-head position score PI i ,i={1,...,H2}.

[0080] The schematic diagram of the position-aware cross-attention module processing in the multi-head weakly supervised position localization network is as follows Figure 7 As shown, specifically, a linear mapping layer FC is used pi_k Map the position features to obtain the mapped position features F pi_k . Use layer normalization layer, linear mapping layer FC pi_q Map the class prototype to obtain the mapped class prototype feature E pi_q Then, the mapped class prototype features and the mapped position features are segmented in the feature dimension. For example, the D-dimensional features are segmented into H2 parts to obtain H2 feature slices. The feature dimension of each slice is D / H2, and H2 class prototype feature slices can be obtained. and H2 position feature slices Next, the dot-product operation is used to calculate the similarity score between the i-th class prototype feature slice and the i-th position feature slice. Specifically, the output Slice the prototype feature of the i-th class and the feature slice at the i-th position Perform vector inner product and output the similarity matrix of class prototype features and candidate region features ⊙ is the dot-product operation. The same operation is performed on other slice pairs to obtain H2 similarity matrices. Then perform softmax on the similarity score in the position dimension l Normalized to get the i-th position score PI i , a total of the same number of position scores PI as the number of cuts can be obtained i ,i={1,...,H2}, output position score matrix Indicates position PI n With class prototype E m The matching score.

[0081] Aggregate the multi-head position scores separately to get the multi-head image scores The position scores are aggregated using the maximum pooling layer. Specifically, for each category c, the maximum position score is selected to obtain the predicted value of the image under category c. Therefore, the image score is obtained Aggregate the multi-head position scores separately to get the multi-head image scores

[0082] The maximum position score is calculated in the position dimension to obtain the image score. This is equivalent to optimizing the class prototype feature with the most similar position feature, which can suppress the interference of background position features in non-target areas.

[0083] The cross entropy loss function is used to calculate the score for each image. The error between the image label And sum up multiple error values, and use the stochastic gradient descent algorithm to update the parameters of the backbone network, class prototype features and multi-head weakly supervised target detection network model in the WCAN model. It should be noted here that during the execution of the steps, the cross entropy loss function here must also meet the conditions in step S35.

[0084] S4 performs object detection on the test set images based on the trained WCAN model. Figure 4 Schematic diagram of the test process of this embodiment.

[0085] S41 uses the backbone network to extract image feature maps in the test set.

[0086] S42 uses a region pooling layer to extract candidate region features of the image.

[0087] S43 inputs the candidate region features and class prototype features into the class-aware cross-attention module and the positioning-aware cross-attention module in the multi-head weakly supervised target detection network to obtain the multi-head candidate region scores.

[0088] S44 averages the scores of all the multiple candidate regions to obtain the final candidate region score;

[0089] S45 uses the non-maximum suppression (NMS) algorithm to remove overlapping and low-scoring candidate regions to obtain the final target detection result. The method of this embodiment is verified by the PASCAL VOC 2007 dataset. Specifically, the model is trained using the training set and validation set in PASCAL VOC 2007, and the model is tested using the test set in PASCAL VOC 2007. On the PASCAL VOC 2007 test set, the mAP (mean Average Precision) index of the method of this embodiment is 0.401. Compared with the WSDDN method, a significant improvement has been achieved. Figure 8 As shown, this is a schematic diagram of the detection results obtained by the method of this embodiment.

[0090] Since all network model parameters have converged in the testing phase, the multi-head weakly supervised position localization network can be omitted (in the training phase, its main role is to assist in optimizing the network parameters of the backbone network and prototype-like features and suppress the interference of position features in non-target areas) to improve the efficiency of target detection.

[0091] Those skilled in the art should understand that Figure 1The number of various network elements shown for simplicity may be smaller than the number in an actual network, but such omission is undoubtedly based on the premise that it will not affect the clear and sufficient disclosure of the embodiments of the invention.

[0092] It can be known from the description of the above implementation methods that those skilled in the art can clearly understand that the present invention can be implemented by means of software plus a necessary general hardware platform. Based on such an understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which can be stored in a storage medium such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in the various embodiments of the present invention or certain parts of the embodiments.

[0093] The above is only a preferred specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed by the present invention should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A multi-head weakly supervised target detection method based on a cross-attention mechanism, characterized in that: include: S1 acquires images, processes the images, and divides the processed images into a training set and a test set; The process of processing the image includes: for each image, marking the category label contained in the image, and using a selective search method or a multi-scale combined grouping method to obtain a candidate region in each image in the data set; S2 defines a class prototype feature, and constructs a weakly supervised target detection network WCAN model based on a cross attention mechanism including the class prototype feature; the WCAN model includes a candidate region feature extraction network, a class prototype feature, and a multi-head weakly supervised target detection network; The multi-head weakly supervised object detection network includes a parallel category-aware cross-attention module and a location-aware cross-attention module; S3 uses the training data set to train the WCAN model; specifically, it includes: S31 initializes the WCAN model; S32 inputs the image in the training set into the candidate region feature extraction network of the WCAN model, and extracts the feature map through the backbone network; S33 simultaneously inputs the extracted image feature map and the corresponding candidate region into the region pooling layer to extract the candidate region feature F p ; Candidate region feature F p The prototype feature E is simultaneously input into the class-aware cross-attention module and the positioning-aware cross-attention module in the multi-head weakly supervised target detection network to obtain the multi-head candidate region category similarity score and the multi-head candidate region target similarity score respectively. The multi-head candidate region category similarity score and the multi-head candidate region target similarity score of the same region are fused to obtain the candidate region score, and the multi-head candidate region scores are aggregated to obtain the image score S i ; The image feature map extracted by the backbone network is transformed in the spatial dimension to obtain the position feature, and the linear mapping layer is used to align the position feature dimension to the prototype feature dimension; The position feature F pi The prototype feature E is input into the position-aware cross-attention module in the multi-head weakly supervised position localization network to obtain the multi-head position score, and the multi-head position scores are aggregated to obtain the multi-head image score. The cross entropy loss function is used to calculate the score for each image. The error between the image label and the image label is calculated, and multiple error values ​​are summed up. The parameters of the backbone network, class prototype features, and multi-head weakly supervised target detection network model in the WCAN model are updated using the stochastic gradient descent algorithm. S34 uses the cross entropy loss function to calculate the score S for each image i The error between the image label and the image label is calculated, and multiple error values ​​are summed up. The parameters of the backbone network, class prototype features, and multi-head weakly supervised target detection network model in the WCAN model are updated using the stochastic gradient descent algorithm. S35 repeats steps S32-S34 until the result of the cross entropy loss function no longer decreases, and a trained WCAN model is obtained; S4 performs object detection on the test set images based on the trained WCAN model.

2. The method according to claim 1, characterized in that Defining class prototype features includes: defining class prototype features Each class prototype features E i Corresponding to a specific semantic category, C is the number of categories, D is the dimension of the class prototype feature, Represents a real number.

3. The method according to claim 1, characterized in that The WCAN model also includes a multi-head weakly supervised position localization network, which includes a position-aware cross-attention module.

4. The method according to claim 1, characterized in that: The candidate region feature F p The prototype feature E is simultaneously input into the class-aware cross-attention module and the positioning-aware cross-attention module in the multi-head weakly supervised target detection network to obtain the multi-head candidate region category similarity score and the multi-head candidate region target similarity score, which specifically include: For the class-aware cross-attention module in the multi-head weakly supervised object detection network: using the linear mapping layer FC ck Map the candidate region features, using layer normalization layer and linear mapping layer FC cq Map the class prototype features, and then split the mapped class prototype features and the mapped candidate region features in the feature dimension; then use the dot-product operation to calculate the similarity score between the i-th class prototype feature slice and the i-th candidate region feature slice, and then perform softmax on the similarity score in the category dimension. c Normalized to get the similarity score P of the i-th candidate region category i,c , a total of candidate region category similarity scores equal to the number of segmentations can be obtained; For the positioning-aware cross-attention module in the multi-head weakly supervised target detection network: using the linear mapping layer FC dk Map the input candidate region features, using layer normalization layer and linear mapping layer FC dq Map the class prototype features, then slice the mapped class prototype features and the mapped candidate region features to obtain class prototype feature slices and candidate region feature slices; then use the dot-product operation to calculate the similarity score between the i-th class prototype feature slice and the i-th candidate region feature slice, and then perform softmax on the similarity score in the candidate region dimension d Normalized to get the target similarity score P of the i-th candidate region i,d , a total of the candidate region target similarity scores, which is the same as the number of slices, can be obtained.

5. The method according to claim 1, characterized in that The position feature F pi The prototype feature E is input into the position-aware cross-attention module in the multi-head weakly supervised position localization network to obtain the multi-head position score, which includes: Using linear mapping layer FC pi_k Map the position features, use layer normalization layer, linear mapping layer FC pi_q Map the class prototype; then split the mapped class prototype features and the mapped position features in the feature dimension; then use the dot-product operation to calculate the similarity score of the i-th class prototype feature slice and the i-th position feature slice, and then perform softmax on the similarity score in the position dimension l Normalized to get the i-th position score PI i , a total of the same number of position scores as the number of cuts can be obtained.

6. The method according to claim 1, characterized in that The S4 includes: S41 uses the backbone network to extract image feature maps in the test set; S42 uses a regional pooling layer to extract candidate region features of the image; S43 inputs the candidate region features and the class prototype features into the class-aware cross-attention module and the positioning-aware cross-attention module in the multi-head weakly supervised target detection network to obtain the multi-head candidate region score; S44 averages the scores of all the multiple candidate regions to obtain the final candidate region score; S45 uses the NMS algorithm to remove overlapping and low-scoring candidate regions to obtain the final target detection result.

Images