Weakly supervised object localization method and apparatus

Abstract

The application provides a kind of weakly supervised target positioning method and device, belong to target positioning field, method includes: S1: import original image dataset, and original image dataset is divided into image training set and image test set;S2: each original image in image training set is rotated at multiple angles, obtain multiple rotated images of each original image;S3: each original image in image training set and each rotated image of each original image are extracted in multiple channels Feature, obtain the feature spectrum group of each original image in image training set and the feature spectrum group of each rotated image;S4: according to the preset layer feature spectrum in the feature spectrum group of each original image in image training set and the preset layer feature spectrum in the feature spectrum group of each rotated image, the calculation of rotation prediction total loss function is carried out, and the rotation prediction total loss function is obtained.The application can improve the accuracy of weakly supervised positioning by mining the complementarity and consistency of fine-grained targets.

Description

TECHNICAL FIELD

[0001] The present application mainly relates to the technical field of target positioning, and particularly relates to a weakly supervised target positioning method and device. BACKGROUND

[0002] Weakly supervised target positioning aims to locate the position of target objects using only image-level labels. Compared with fully supervised target positioning methods, weakly supervised target positioning methods do not require expensive bounding box annotations. Therefore, weakly supervised target positioning methods reduce the large amount of manpower and time required for fine-grained labeling of training samples. Weakly supervised target co-localization is a special case of weakly supervised target positioning, which aims to find common objects in a dataset containing one superclass. These fine-grained datasets contain two characteristics: 1) the co-occurrence problem of specific fine-grained objects and their surrounding environment, which can cause ambiguity in object localization; 2) the complex and diverse surrounding environment of fine-grained objects, which reduces the localization accuracy. These characteristics pose great challenges to weakly supervised target co-localization.

[0003] In order to alleviate these problems, existing methods usually use multi-instance learning (MIL) methods and self-supervised learning (SSL) methods for object co-localization. MIL-based methods use alternating optimization between selecting positive instances and learning object detectors to achieve object co-localization, and these methods identify and localize target objects from complex and diverse environments by using rich object information in positive instances. However, these methods require high computational cost due to the generation of a series of instance bags. SSL-based methods use image transformation as a pretext task, and then train the pretext task to learn a representation with good semantic or structural meaning to localize objects. In order to alleviate the problem of coexistence of fine-grained objects and their surrounding environment, these methods use different image augmentation strategies to learn meaningful feature representations. Although existing target co-localization methods have achieved impressive performance, they ignore the common part information shared by fine-grained objects and the influence of complementary part pairs of fine-grained objects on the co-localization of fine-grained targets. This results in their inability to comprehensively and accurately describe object patterns, thereby affecting the final co-localization performance. SUMMARY

[0004] The technical problem to be solved by the present application is to provide a weakly supervised target positioning method and device to solve the problems of the prior art.

[0005] The technical solution of the present application to solve the above technical problems is as follows: a weakly supervised target positioning method, comprising the following steps:

[0006] S1: importing an original image dataset and dividing the original image dataset into an image training set and an image test set;

[0007] S2: rotate each original image in the image training set by multiple angles to obtain multiple rotated images of each original image;

[0008] S3: perform multi-channel feature extraction on each original image in the image training set and each rotated image of each original image to obtain a feature spectrum group of each original image in the image training set and a feature spectrum group of each rotated image;

[0009] S4: calculate a rotation prediction total loss function according to the preset layer feature spectrum in the feature spectrum group of each original image in the image training set and the preset layer feature spectrum in the feature spectrum group of each rotated image, to obtain the rotation prediction total loss function;

[0010] S5: construct a pseudo-label generation network, and perform pseudo-label generation network update analysis on the pseudo-label generation network according to the rotation prediction total loss function, the feature spectrum group of each original image in the image training set and the feature spectrum group of each rotated image, to obtain a comparison total loss function and a target object positioning map and a pseudo-label of each original image in the image training set;

[0011] S6: construct a cooperative positioning network, and perform cooperative positioning network update analysis on the cooperative positioning network according to the comparison total loss function and the target object positioning map and the pseudo-label of each original image in the image training set, to obtain a target cooperative positioning network;

[0012] S7: test all original images in the image test set through the target cooperative positioning network to obtain a target positioning result.

[0013] Based on the above-mentioned weakly supervised target positioning method, the application further provides a weakly supervised target positioning device.

[0014] Another technical solution for solving the above technical problems is as follows: a weakly supervised target positioning device, comprising:

[0015] The division module imports an original image data set and divides the original image data set into an image training set and an image test set;

[0016] The rotation module is configured to rotate each original image in the image training set by multiple angles to obtain multiple rotated images of each original image;

[0017] The feature extraction module is configured to perform multi-channel feature extraction on each original image in the image training set and each rotated image of each original image to obtain a feature spectrum group of each original image in the image training set and a feature spectrum group of each rotated image;

[0018] a loss calculation module configured to calculate a total rotation prediction loss function according to the preset layer feature spectrum in the feature spectrum group of each original image in the image training set and the preset layer feature spectrum in the feature spectrum group of each rotated image, and obtain the total rotation prediction loss function;

[0019] a first network updating module configured to construct a pseudo-label generation network, analyze pseudo-label generation network updating of the pseudo-label generation network according to the total rotation prediction loss function, the feature spectrum group of each original image in the image training set and the feature spectrum group of each rotated image, and obtain a total comparison loss function and a target object positioning map and a pseudo-label of each original image in the image training set;

[0020] a second network updating module configured to construct a cooperative positioning network, analyze cooperative positioning network updating of the cooperative positioning network according to the total comparison loss function and the target object positioning map and the pseudo-label of each original image in the image training set, and obtain a target cooperative positioning network;

[0021] a target positioning result obtaining module configured to test all original images in the image test set through the target cooperative positioning network, and obtain a target positioning result.

[0022] Based on the above-mentioned weakly supervised target positioning method, the application further provides a weakly supervised target positioning system.

[0023] Another technical solution for solving the above technical problems is as follows: a weakly supervised target positioning system, comprising a memory, a processor and a computer program stored in the memory and executable on the processor, when the processor executes the computer program, the weakly supervised target positioning method as described above is realized.

[0024] Based on the above-mentioned weakly supervised target positioning method, the application further provides a computer readable storage medium.

[0025] Another technical solution for solving the above technical problems is as follows: a computer readable storage medium, the computer readable storage medium stores a computer program, when the computer program is executed by a processor, the weakly supervised target positioning method as described above is realized.

[0026] The beneficial effects of the present application are: by dividing the original image data set into an image training set and an image test set, rotating the original image at multiple angles to obtain a rotated image, extracting the multi-channel features of the original image and the rotated image to obtain a feature spectrum group of the original image and a feature spectrum group of the rotated image, calculating the rotation prediction total loss function according to the rotation prediction total loss function of the preset layer feature spectrum in the feature spectrum group of the original image and the feature spectrum group of the rotated image, updating the pseudo label generation network of the pseudo label generation network according to the rotation prediction total loss function, the feature spectrum group of the original image and the feature spectrum group of the rotated image to obtain a comparison total loss function and a target object positioning map and a pseudo label, updating the cooperative positioning network of the cooperative positioning network according to the comparison total loss function and the target object positioning map and the pseudo label to obtain a target cooperative positioning network, and testing the original image through the target cooperative positioning network to obtain a target positioning result, the complementarity and consistency of the fine-grained target can be mined, and the precision of weak supervision positioning is improved. BRIEF DESCRIPTION OF DRAWINGS

[0027] Figure 1 A flowchart of a weakly supervised target positioning method provided by an embodiment of the present application is shown.

[0028] Figure 2 A model diagram of a pseudo label generation network provided by an embodiment of the present application is shown.

[0029] Figure 3 A model diagram of a shared weight multiplication feature fusion provided by an embodiment of the present application is shown.

[0030] Figure 4 A model diagram of a cooperative positioning network provided by an embodiment of the present application is shown.

[0031] Figure 5 A module block diagram of a weakly supervised target positioning device provided by an embodiment of the present application is shown. DETAILED DESCRIPTION

[0032] The principles and characteristics of the present application are described below in conjunction with the accompanying drawings, and the examples are only used to explain the present application and are not used to limit the scope of the present application.

[0033] Figure 1 A flowchart of a weakly supervised target positioning method provided by an embodiment of the present application is shown.

[0034] As shown in Figure 1 and 2 , a weakly supervised target positioning method comprises the following steps:

[0035] S1: Import an original image data set, and divide the original image data set into an image training set and an image test set;

[0036] S2: Rotating each original image in the image training set at multiple angles to obtain multiple rotated images of each original image;

[0037] S3: performing multi-channel feature extraction on each original image in the image training set and each rotated image of each original image to obtain a feature spectrum group of each original image in the image training set and a feature spectrum group of each rotated image;

[0038] S4: calculating a rotation prediction total loss function according to the preset layer feature spectrum in the feature spectrum group of each original image in the image training set and the preset layer feature spectrum in the feature spectrum group of each rotated image to obtain the rotation prediction total loss function;

[0039] S5: constructing a pseudo-label generation network, performing pseudo-label generation network update analysis on the pseudo-label generation network according to the rotation prediction total loss function, the feature spectrum group of each original image in the image training set and the feature spectrum group of each rotated image to obtain a comparison total loss function and a target object positioning map and a pseudo-label of each original image in the image training set;

[0040] S6: constructing a cooperative positioning network, performing cooperative positioning network update analysis on the cooperative positioning network according to the comparison total loss function and the target object positioning map and the pseudo-label of each original image in the image training set to obtain a target cooperative positioning network;

[0041] S7: testing all original images in the image test set through the target cooperative positioning network to obtain a target positioning result.

[0042] It should be understood that the multi-channel feature extraction on each original image in the image training set and each rotated image of each original image is performed through a residual network to obtain the feature spectrum group of each original image in the image training set and the feature spectrum group of each rotated image.

[0043] It should be understood that, as shown in Figure 2 , the original image (i.e. each original image in the image training set) is randomly rotated at 0 degrees, 90 degrees, 180 degrees and 270 degrees to obtain a set of images at four different angles as input images (i.e. multiple rotated images), and the residual network (ResNet50) is used as a feature extractor, where y rota is the true angle label of the original image I rotation, Rota(I, φ) represents the operator of the original image I rotation φ, and the image rotation process is:

[0044] h(I|y rota )=Rota(I,90×(y rota -1));

[0045] The original image I is rotated by 0 degrees, 90 degrees, 180 degrees or 270 degrees by using R=4 discrete geometric transformations G, which can be represented as:

[0046]

[0047] It should be understood that the fine-grained weakly supervised object co-localization network (i.e., the object pseudo-label generation network and the object co-localization network) is tested by using the test sample set (i.e., all original images in the image test set).

[0048] It should be understood that the fine-grained object co-localization network (i.e., the object pseudo-label generation network and the object co-localization network) is tested based on the training set weight by using the test sample set (i.e., all original images in the image test set).

[0049] In the above embodiment, by dividing the original image dataset into an image training set and an image test set, rotating the original image at multiple angles to obtain rotated images, extracting the multi-channel features of the original image and the rotated images to obtain the feature spectrum group of the original image and the feature spectrum group of the rotated images, calculating the rotation prediction total loss function according to the preset layer feature spectrum in the feature spectrum group of the original image and the preset layer feature spectrum in the feature spectrum group of the rotated images, updating the pseudo-label generation network of the pseudo-label generation network according to the rotation prediction total loss function, the feature spectrum group of the original image and the feature spectrum group of the rotated images to obtain the comparison total loss function and the target object localization map and the pseudo-label, updating the co-localization network of the co-localization network according to the comparison total loss function and the target object localization map and the pseudo-label to obtain the target co-localization network, and testing the original image by using the target co-localization network to obtain the target localization result, the complementarity and consistency of the fine-grained target can be mined, and the accuracy of the weakly supervised localization is improved.

[0050] Optionally, as an embodiment of the present application, the process of S4 comprises:

[0051] Importing a real angle label corresponding to each original image in the image training set;

[0052] Calculating a loss function according to the preset layer feature spectrum in the feature spectrum group of each original image in the image training set, the preset layer feature spectrum in the feature spectrum group of each rotated image and all real angle labels to obtain a rotation prediction total loss function.

[0053] Specifically, the step of calculating the loss function according to the preset layer feature spectrum in the feature spectrum group of each original image in the image training set, the preset layer feature spectrum in the feature spectrum group of each rotated image, and all real angle labels to obtain the total rotation prediction loss function is specifically: based on the first formula, the total rotation prediction loss function is calculated to obtain the total rotation prediction loss function, and the first formula is:

[0054]

[0055] wherein, is the total rotation prediction loss function, I is the total number of original images in the image training set or the total number of first preset rotated images or the total number of second preset rotated images or the total number of third preset rotated images, y i-rota is the real angle label corresponding to the i-th original image, is the preset layer feature spectrum of the i-th original image, is the preset layer feature spectrum of the i-th first preset rotated image, is the preset layer feature spectrum of the i-th second preset rotated image, is the preset layer feature spectrum of the i-th third preset rotated image.

[0056] Specifically, the first preset rotated image is one of the multiple rotated images corresponding to the original image, and the first preset rotated image is obtained by rotating the original image by a first preset angle, the second preset rotated image is one of the multiple rotated images corresponding to the original image, and the second preset rotated image is obtained by rotating the original image by a second preset angle, and the third preset rotated image is one of the multiple rotated images corresponding to the original image, and the third preset rotated image is obtained by rotating the original image by a third preset angle.

[0057] It should be understood that the preset layer feature spectrum is the feature spectrum of the last layer.

[0058] It should be understood that the first preset angle can be 90 degrees, the second preset angle can be 180 degrees, and the third preset angle can be 270 degrees.

[0059] It should be understood that , wherein Z represents the angle of the image rotated predicted by the feature spectrum Z (i.e. the preset layer feature spectrum of the original image or the preset layer feature spectrum of the first preset rotated image or the preset layer feature spectrum of the second preset rotated image or the preset layer feature spectrum of the third preset rotated image), and the rotation prediction loss function (i.e. the total rotation prediction loss function) is:

[0060]

[0061] In the above embodiment, the rotation prediction total loss function is calculated according to the preset layer feature spectrum in the feature spectrum group of the original image and the preset layer feature spectrum in the feature spectrum group of the rotated image, the discriminative representation of the object part region is learned, and the rotation invariance of the part positioning network is improved.

[0062] Optionally, as an embodiment of the present application, the process of S5 comprises:

[0063] S51: constructing a pseudo-label generation network;

[0064] S52: sequentially performing upsampling analysis, element multiplication fusion, first global average pooling processing and attention erasing analysis according to the pseudo-label generation network, the feature spectrum group of each original image in the image training set and the feature spectrum group of each rotated image, to correspondingly obtain the upsampling feature spectrum group of each original image in the image training set, the original positioning spectrum, the upsampling feature vector and the erased positioning spectrum, and the upsampling feature spectrum group of each rotated image, the original positioning spectrum, the upsampling feature vector and the erased positioning spectrum;

[0065] S53: sequentially performing second global average pooling and first normalization processing on the original positioning spectrum and the erased positioning spectrum of each original image in the image training set and the original positioning spectrum and the erased positioning spectrum of each rotated image, to obtain the original positioning spectrum normalization vector and the erased positioning spectrum normalization vector of each original image in the image training set, and the original positioning spectrum normalization vector and the erased positioning spectrum normalization vector of each rotated image;

[0066] S54: importing the ground truth class label corresponding to each original image in the image training set, and calculating a loss function according to the upsampling feature vector, the original positioning spectrum normalization vector and the erased positioning spectrum normalization vector of all original images in the image training set, the upsampling feature vector, the original positioning spectrum normalization vector and the erased positioning spectrum normalization vector of each rotated image, and all ground truth class labels, to obtain a feature spectrum classification total loss function and a contrast total loss function;

[0067] S55: performing auxiliary total loss function analysis according to the upsampling feature vector, the original positioning spectrum and the erased positioning spectrum of each original image in the image training set, the upsampling feature vector, the original positioning spectrum and the erased positioning spectrum of each rotated image, and the ground truth class label corresponding to all original images, to obtain an auxiliary total loss function, the target object positioning map of each original image in the image training set and the target object positioning map of each rotated image;

[0068] S56: Perform loss function analysis on the target object positioning maps of all original images in the image training set, the target object positioning maps of the plurality of rotated images, and the ground truth class labels corresponding to all original images, and calculate a loss function according to the results of the loss function analysis, the feature spectrum classification total loss function, the auxiliary total loss function, the rotation prediction total loss function, and the contrast total loss function, and perform parameter updating on the pseudo label generation network according to the calculated results, and perform test analysis on each original image in the image training set according to the results of the parameter updating, to obtain the pseudo label of each original image in the image training set.

[0069] Specifically, the upsampling analysis in S52 is specifically: performing upsampling analysis on the pseudo label generation network according to the feature spectrum groups of each original image and the feature spectrum groups of each rotated image in the image training set, to obtain the upsampling feature spectrum groups of each original image and the upsampling feature spectrum groups of each rotated image in the image training set; the element multiplication fusion in S52 is specifically: performing element multiplication fusion on all feature spectra in the upsampling feature spectrum groups of each original image and all feature spectra in the upsampling feature spectrum groups of each rotated image in the image training set, to correspondingly obtain the original positioning spectrum of each original image and the original positioning spectrum of each rotated image in the image training set; the first global average pooling processing in S52 is specifically: performing global average pooling processing on the preset layer feature spectra in the upsampling feature spectrum groups of each original image and the preset layer feature spectra in the upsampling feature spectrum groups of each rotated image in the image training set, to obtain the upsampling feature vectors of each original image and the upsampling feature vectors of each rotated image in the image training set; the attention erasing analysis in S52 is specifically: performing attention erasing analysis on the pseudo label generation network according to the feature spectrum groups and the upsampling feature spectrum groups of each original image and the feature spectrum groups and the upsampling feature spectrum groups of each rotated image in the image training set, to obtain the erased positioning spectrum of each original image and the erased positioning spectrum of each rotated image in the image training set.

[0070] It should be understood that the second global average pooling in S53 is specifically: performing global average pooling on the original positioning spectrum and the erased positioning spectrum of each original image in the image training set and the original positioning spectrum and the erased positioning spectrum of each rotated image, to obtain the original positioning spectrum feature vector and the erased positioning spectrum feature vector of each original image in the image training set and the original positioning spectrum feature vector and the erased positioning spectrum feature vector of each rotated image; the first normalization processing in S53 is specifically: performing normalization processing on the original positioning spectrum feature vector and the erased positioning spectrum feature vector of each original image in the image training set and the original positioning spectrum feature vector and the erased positioning spectrum feature vector of each rotated image, to obtain the original positioning spectrum normalized vector and the erased positioning spectrum normalized vector of each original image in the image training set and the original positioning spectrum normalized vector and the erased positioning spectrum normalized vector of each rotated image.

[0071] Specifically, the calculation of the feature spectrum classification total loss function in S54 is specifically: based on the second formula, the calculation of the feature spectrum classification total loss function is performed to obtain the feature spectrum classification total loss function, and the second formula is:

[0072]

[0073] wherein, is the feature spectrum classification total loss function, y i-GT is the ground truth class label corresponding to the i-th original image, I is the total number of original images in the image training set or the total number of first preset rotated images or the total number of second preset rotated images or the total number of third preset rotated images, is the up-sampling feature vector of the i-th original image, is the up-sampling feature vector of the i-th first preset rotated image, is the up-sampling feature vector of the i-th second preset rotated image, is the up-sampling feature vector of the i-th third preset rotated image.

[0074] The calculation of the comparison total loss function in S54 is specifically: based on the third formula, the calculation of the comparison total loss function is performed to obtain the comparison total loss function, and the third formula is:

[0075]

[0076] wherein,

[0077] Specifically, is the comparison total loss function, is the original image comparison loss function, is a first rotated image contrast loss function, is a second rotated image contrast loss function, is a third rotated image contrast loss function, is an original localization spectrum normalized vector of the i-th original image, is an erased localization spectrum normalized vector of the j-th original image, is an original localization spectrum normalized vector of the i-th first preset rotated image, is an erased localization spectrum normalized vector of the j-th first preset rotated image, is an original localization spectrum normalized vector of the i-th second preset rotated image, is an erased localization spectrum normalized vector of the j-th second preset rotated image, is an original localization spectrum normalized vector of the i-th third preset rotated image, is an erased localization spectrum normalized vector of the j-th third preset rotated image.

[0078] Specifically, the analysis of the localization spectrum classification total loss function in S56 is specifically: analyzing the localization spectrum classification total loss function according to the target object localization map of all original images in the image training set, the target object localization map of a plurality of rotated images, and the ground truth class label corresponding to all original images, to obtain the localization spectrum classification total loss function; the loss function calculation according to the result after loss function analysis, the feature spectrum classification total loss function, the auxiliary total loss function, the rotation prediction total loss function and the contrast total loss function is specifically: based on the fourth formula, the pseudo label generation loss function is calculated to obtain the pseudo label generation loss function, the fourth formula is:

[0079]

[0080] wherein, is a pseudo label generation loss function, is a feature spectrum classification total loss function, is an auxiliary total loss function, is a localization spectrum classification total loss function, is a rotation prediction total loss function, is a contrast total loss function.

[0081] It should be understood that the pseudo label generation network is updated according to the calculated result to obtain a target pseudo label generation network.

[0082] It should be understood that the calculated result is the pseudo label generation loss function, and the result after parameter updating is the target pseudo label generation network.

[0083] It should be understood that the three feature spectra of X4, Y4 and Z4 (i.e. all feature spectra in the up-sampling feature spectrum group of each original image in the image training set and all feature spectra in the up-sampling feature spectrum group of each rotated image) are fused by multiplication feature fusion to obtain a fused positioning spectrum (i.e. the original positioning spectrum of each original image in the image training set and the original positioning spectrum of each rotated image).

[0084] It should be understood that the first normalization process in S53 can be max-min normalization. Specifically, by contrastive learning, it is ensured that the two positioning spectra have complementary and consistent partial semantic information; using global average pooling, the fused positioning spectra F1 (i.e. the original positioning spectrum of each original image in the image training set and the original positioning spectrum of each rotated image) and F2 (i.e. the erased positioning spectrum of each original image in the image training set and the erased positioning spectrum of each rotated image) are compressed into corresponding feature vectors (the original positioning spectrum feature vector and the erased positioning spectrum feature vector of each original image in the image training set and the original positioning spectrum feature vector and the erased positioning spectrum feature vector of each rotated image); by max-min normalization operation, the two feature vectors are normalized to [0, 1]; define a cross-class object complementary partial contrast loss:

[0085]

[0086] where p = q indicates that the two positioning spectra are output by multiplication feature fusion of the same shared weight, p ≠ q indicates that the two positioning spectra are output by multiplication feature fusion of different shared weights; i and j represent the index of the batch image.

[0087] It should be understood that in order to emphasize the consistency and complementarity of the object part, the cross-class target complementarity component contrast learning is used to pull the same component features closer and push the different component features apart.

[0088] Specifically, let the prediction probability of Z4 be y Z4 (i.e. the up-sampling feature vector of the original image or the up-sampling feature vector of the first preset rotated image or the up-sampling feature vector of the second preset rotated image or the up-sampling feature vector of the third preset rotated image), y GT is the ground truth class label (i.e. the ground truth class label corresponding to the original image), then the classification loss function (i.e. the feature spectrum classification total loss function) is:

[0089]

[0090] The probability score predicted by Z4 is calculated by cross-entropy loss respectively.

[0091] It should be understood that the pseudo-label generation loss function includes a classification loss function (i.e., the feature spectrum classification total loss function), an auxiliary loss function (i.e., the auxiliary total loss function), a rotation prediction loss function (i.e., the rotation prediction total loss function), and a cross-category target complementarity component contrast loss function (i.e., the contrast total loss function).

[0092] It should be understood that the pseudo-label generation loss function is:

[0093]

[0094] In the above embodiment, the pseudo-label generation network update analysis of the pseudo-label generation network according to the rotation prediction total loss function, the feature spectrum group of the original image, and the feature spectrum group of the rotated image obtains the contrast total loss function and the target object positioning map of the original image and the pseudo-label, narrows the same component features and pushes away the different component features, and emphasizes the consistency and complementarity of the object part.

[0095] Optionally, as one embodiment of the present application, as shown in the figure, the pseudo-label generation network includes a first 1*1 convolutional layer group, a first 1*1 convolutional layer, a Sigmoid activation function layer, a second 1*1 convolutional layer group, and a first ReLU activation function layer. Figures 1 to 3

[0096] In the S52, the up-sampling analysis process according to the pseudo-label generation network, the feature spectrum group of each original image in the image training set, and the feature spectrum group of each rotated image includes:

[0097] performing third global average pooling processing according to the feature spectrum group of each original image in the image training set and the feature spectrum group of each rotated image;

[0098] performing first dimension conversion on the third global average pooling processing result by the first 1*1 convolutional layer group;

[0099] performing first element multiplication fusion on the result after the first dimension conversion;

[0100] performing second dimension conversion on the result after the first element multiplication fusion by the first 1*1 convolutional layer;

[0101] performing second normalization processing on the result after the second dimension conversion by the Sigmoid activation function layer;

[0102] performing second element multiplication fusion on the result after the second normalization processing;

[0103] performing third dimension conversion on the result after the second element multiplication fusion by the second 1*1 convolutional layer group; ​

[0104] performing third normalization processing on the result of the third dimension conversion through the first ReLU activation function layer;

[0105] performing up-sampling processing on the result of the third normalization processing to obtain the up-sampling feature spectrum group of each original image in the image training set and the up-sampling feature spectrum group of each rotated image.

[0106] Specifically, in S52, the up-sampling analysis process is performed according to the pseudo-label generation network, the feature spectrum groups of the original images in the image training set, and the feature spectrum groups of the rotated images. The process is as follows: global average pooling is performed on the feature spectrum groups of the original images in the image training set and the feature spectrum groups of the rotated images to obtain the pooled feature spectrum groups of the original images in the image training set and the pooled feature spectrum groups of the rotated images; the first 1x1 convolutional layer group is used to convert the dimensions of the pooled feature spectrum groups of the original images in the image training set and the pooled feature spectrum groups of the rotated images, thereby obtaining the converted feature spectrum groups of the original images in the image training set and the converted feature spectrum groups of the rotated images; element-wise multiplication fusion is performed on all feature spectra in the converted feature spectrum groups of the original images in the image training set and all feature spectra in the converted feature spectrum groups of the rotated images, thereby obtaining the first fusion feature spectrum groups of the original images in the image training set and the first fusion feature spectrum groups of the rotated images; the first 1x1 convolutional layer is used to convert the dimensions of the first fusion feature spectrum groups of the original images in the image training set and the first fusion feature spectrum groups of the rotated images, thereby obtaining the converted first fusion feature spectrum groups of the original images in the image training set and the converted first fusion feature spectrum groups of the rotated images; the Sigmoid activation function layer is used to normalize the converted first fusion feature spectrum groups of the original images in the image training set and the converted first fusion feature spectrum groups of the rotated images, thereby obtaining the normalized first fusion feature spectrum groups of the original images in the image training set and the normalized first fusion feature spectrum groups of the rotated images; element-wise multiplication fusion is performed on the normalized first fusion feature spectrum groups of the original images in the image training set and the normalized first fusion feature spectrum groups of the rotated images and the feature spectrum groups of the original images in the image training set and the feature spectrum groups of the rotated images, thereby obtaining the second fusion feature spectrum groups of the original images in the image training set and the second fusion feature spectrum groups of the rotated images; the second 1x1 convolutional layer group is used to convert the dimensions of the second fusion feature spectrum groups of the original images in the image training set and the second fusion feature spectrum groups of the rotated images, thereby obtaining the converted second fusion feature spectrum groups of the original images in the image training set and the converted second fusion feature spectrum groups of the rotated images; the first ReLU activation function layer is used to normalize the converted second fusion feature spectrum groups of the original images in the image training set and the converted second fusion feature spectrum groups of the rotated images, thereby obtaining the normalized second fusion feature spectrum groups of the original images in the image training set and the normalized second fusion feature spectrum groups of the rotated images;The normalized second fusion feature spectrum groups of each original image and each rotated image in the image training set are subjected to up-sampling processing, and corresponding up-sampling feature spectrum groups of each original image and each rotated image in the image training set are obtained.

[0107] It should be understood that the first 1x1 convolutional layer group includes three parallel 1x1 convolutional layers, and the second 1x1 convolutional layer group includes three parallel 1x1 convolutional layers.

[0108] Specifically, as shown in Figure 3 The multiplication feature fusion of shared weights is roughly filtered by channel attention; for the last three layers of feature spectrum of the residual network (ResNet50) and (i.e. the feature spectrum groups of each original image and each rotated image in the image training set); using global average pooling to realize and (i.e. the pooled feature spectrum groups of each original image and each rotated image in the image training set); then using three parallel 1x1 convolutional layers (i.e. the first 1x1 convolutional layer group) to convert the channel number of X1, Y1 and Z1 to the same dimension (i.e. the converted feature spectrum groups of each original image and each rotated image in the image training set); using element multiplication fusion to obtain the latent representation vector V=X2·Y2·Z2 (i.e. the first fusion feature spectrum groups of each original image and each rotated image in the image training set); in the opposite direction, the latent representation vector V (i.e. the first fusion feature spectrum groups of each original image and each rotated image in the image training set) is converted to the original dimension and (i.e. the normalized first fusion feature spectrum group of each original image in the image training set and the normalized first fusion feature spectrum group of each rotated image); multiply and fuse X and X3, Y and Y3, Z and Z3 (i.e. the normalized first fusion feature spectrum group of each original image in the image training set and the normalized first fusion feature spectrum group of each rotated image and the feature spectrum group of each original image in the image training set and the feature spectrum group of each rotated image) in the channel dimension respectively; then convert the three feature spectrum channels and dimensions into the same size through three parallel 1x1 convolution layers (i.e. the second 1x1 convolution layer group), ReLU activation function (i.e. the first ReLU activation function layer) and up-sampling operation (i.e. the up-sampled feature spectrum group of each original image in the image training set and the up-sampled feature spectrum group of each rotated image).

[0109] It should be understood that the shallow and deep features can be aggregated to produce clearer object boundaries.

[0110] In the above embodiment, according to the pseudo label generation network, the feature spectrum group of each original image in the image training set and the feature spectrum group of each rotated image, the up-sampling analysis can aggregate the shallow and deep features to produce clearer object boundaries.

[0111] Optionally, as an embodiment of the present application, the pseudo label generation network further comprises a second 1x1 convolution layer and a second ReLU activation function layer;

[0112] In the S52, the process of attention erasing analysis according to the pseudo label generation network, the feature spectrum group of each original image in the image training set and the feature spectrum group of each rotated image comprises:

[0113] The up-sampled feature spectrum group of each original image in the image training set and the up-sampled feature spectrum group of each rotated image are subjected to fourth dimension conversion through the second 1x1 convolution layer;

[0114] The fourth dimension conversion result is subjected to fourth normalization processing through the second ReLU activation function layer;

[0115] The fourth normalization processing result is subjected to attention spectrum extraction according to the preset category;

[0116] The extracted result is subjected to maximum and minimum normalization processing to obtain the normalized attention spectrum group of each original image in the image training set and the normalized attention spectrum group of each rotated image;

[0117] The normalized attention spectrum set of each original image in the image training set and the normalized attention spectrum set of each rotated image are sequentially subjected to pixel point calculation and set pixel point, so as to correspondingly obtain the attention erasing mask set of each original image in the image training set and the attention erasing mask set of each rotated image.

[0118] The feature spectrum set of each original image in the image training set and the feature spectrum set of each rotated image are respectively subjected to third element multiplication fusion with the attention erasing mask set of each original image in the image training set and the attention erasing mask set of the plurality of rotated images, and the result after the third element multiplication fusion is subjected to the upsampling analysis, element multiplication fusion and first global average pooling processing in S52, so as to obtain the post-erasing positioning spectrum of each original image in the image training set and the post-erasing positioning spectrum of each rotated image.

[0119] Specifically, in the S52, the process of the fourth dimension conversion, the fourth normalization processing, the attention spectrum extraction and the maximum and minimum normalization processing is specifically as follows: the dimension conversion is performed on the up-sampling feature spectrum set of each original image in the image training set and the up-sampling feature spectrum set of each rotated image through the second 1x1 convolution layer, so as to correspondingly obtain the converted up-sampling feature spectrum set of each original image in the image training set and the converted up-sampling feature spectrum set of each rotated image; the normalized processing is performed on the converted up-sampling feature spectrum set of each original image in the image training set and the converted up-sampling feature spectrum set of each rotated image through the second ReLU activation function layer, so as to correspondingly obtain the normalized up-sampling feature spectrum set of each original image in the image training set and the normalized up-sampling feature spectrum set of each rotated image; the attention spectrum extraction is performed on the normalized up-sampling feature spectrum set of each original image in the image training set and the normalized up-sampling feature spectrum set of each rotated image according to the preset category, so as to obtain the attention spectrum set of each original image in the image training set and the attention spectrum set of each rotated image; and the maximum and minimum normalization processing is performed on the attention spectrum set of each original image in the image training set and the attention spectrum set of each rotated image, so as to correspondingly obtain the normalized attention spectrum set of each original image in the image training set and the normalized attention spectrum set of each rotated image.

[0120] Specifically, the pixel point calculation and the set pixel point process of the normalized attention spectrum group of each original image and the normalized attention spectrum group of each rotated image in the image training set in turn are specifically: based on the fifth formula, the calculation of the attention erasing mask pixel point is performed, and a plurality of attention erasing mask pixel points of each attention spectrum in the normalized attention spectrum group of each original image and a plurality of attention erasing mask pixel points of each attention spectrum in the normalized attention spectrum group of each rotated image in the image training set are correspondingly obtained, and the fifth formula is:

[0121]

[0122] wherein D(x, y) is the (x, y)th attention erasing mask pixel point of each attention spectrum in the normalized attention spectrum group of the original image or the (x, y)th attention erasing mask pixel point of each attention spectrum in the normalized attention spectrum group of each rotated image, is the (x, y)th attention spectrum pixel point of each attention spectrum in the normalized attention spectrum group of the original image or the (x, y)th attention spectrum pixel point of each attention spectrum in the normalized attention spectrum group of each rotated image, and θ is a threshold value, θ ∈ [0, 1]; the plurality of attention erasing mask pixel points of each attention spectrum in the normalized attention spectrum group of each original image and the plurality of attention erasing mask pixel points of each attention spectrum in the normalized attention spectrum group of each rotated image in the image training set are respectively collected to obtain the attention erasing mask group of each original image and the attention erasing mask group of each rotated image in the image training set.

[0123] The feature spectrum group of each original image and the feature spectrum group of each rotated image in the image training set are respectively multiplied by the attention erasing mask group of each original image and the attention erasing mask group of each rotated image in the image training set for the third time, and the process of performing the upsampling analysis, the element multiplication fusion and the first global average pooling processing on the result of the third time element multiplication fusion is specifically: the feature spectrum group of each original image and the feature spectrum group of each rotated image in the image training set are respectively multiplied by the attention erasing mask group of each original image and the attention erasing mask group of each rotated image in the image training set for the third time, and the third fusion feature spectrum group of each original image and the third fusion feature spectrum group of each rotated image in the image training set are correspondingly obtained; the third fusion feature spectrum group of each original image and the third fusion feature spectrum group of each rotated image in the image training set are subjected to the upsampling analysis, the element multiplication fusion and the first global average pooling processing in S52, and the post-erasing positioning spectrum of each original image and the post-erasing positioning spectrum of each rotated image in the image training set are correspondingly obtained.

[0124] It should be understood that other discriminative regions of the fine-grained object are excavated.

[0125] Specifically, for X4, Y4 and Z4 (i.e. the up-sampled feature spectrum group of each original image in the image training set and the up-sampled feature spectrum group of each rotated image), a 1x1 convolution layer (i.e. the second 1x1 convolution layer) is applied to adjust the channel number K to the class number C, and then the feature spectrum is activated by ReLU (i.e. the second ReLU activation function layer) to suppress negative values; let represent the attention spectrum of the nth stage and the cth class (i.e. the attention spectrum group of each original image in the image training set and the attention spectrum group of each rotated image), and the attention spectrum (i.e. the attention spectrum group of each original image in the image training set and the attention spectrum group of each rotated image) is applied to the maximum and minimum normalization as follows:

[0126]

[0127] Set the threshold θ∈[0,1], get the attention erasure mask (i.e. the attention erasure mask group of each original image in the image training set and the attention erasure mask group of each rotated image), as follows:

[0128]

[0129] Multiply the attention erasure mask of each stage (i.e. the attention erasure mask group of each original image in the image training set and the attention erasure mask group of each rotated image) with the corresponding feature spectrum X, Y, Z of the residual network (ResNet50) (i.e. the attention erasure mask group of each original image in the image training set and the attention erasure mask group of each rotated image), respectively, to get the erased feature spectrum X', Y', Z' (i.e. the third fusion feature spectrum group of each original image in the image training set and the third fusion feature spectrum group of each rotated image); take the erased feature spectrum (i.e. the third fusion feature spectrum group of each original image in the image training set and the third fusion feature spectrum group of each rotated image) as input, repeat the process of up-sampling analysis, element multiplication fusion and first global average pooling in S52, to get the erased fusion positioning spectrum (i.e. the erased positioning spectrum of each original image in the image training set and the erased positioning spectrum of each rotated image).

[0130] It should be understood that the network is encouraged to learn other complementary component regions.

[0131] In the above embodiment, the attention erasing analysis is performed according to the pseudo label generation network, the feature spectrum group of each original image in the image training set, and the feature spectrum group of each rotated image, the other discriminative regions of the fine-grained object are mined, and the network is encouraged to learn the other complementary component regions.

[0132] Optionally, as an embodiment of the present application, the process of S55 comprises:

[0133] The original positioning spectrum and the erased positioning spectrum of each original image in the image training set and the original positioning spectrum and the erased positioning spectrum of each rotated image are sequentially subjected to fifth dimension conversion and fifth normalization processing by a third 1x1 convolutional layer and a third ReLU activation function layer, to obtain the original object positioning spectrum and the erased object positioning spectrum of each original image in the image training set and the original object positioning spectrum and the erased object positioning spectrum of each rotated image.

[0134] According to the original object positioning spectrum and the erased object positioning spectrum of each original image in the image training set and the original object positioning spectrum and the erased object positioning spectrum of each rotated image, pixel point calculation and set pixel points are sequentially performed to obtain the target object positioning map of each original image in the image training set and the target object positioning map of each rotated image.

[0135] The original object positioning spectrum of each original image in the image training set and the original object positioning spectrum of each rotated image are subjected to fourth global average pooling, and a loss function calculation is performed according to the results after the fourth global average pooling, the up-sampling feature vectors of all original images in the image training set and the up-sampling feature vectors of all rotated images, to obtain an auxiliary total loss function.

[0136] Specifically, the process of sequentially performing fifth dimension conversion and fifth normalization processing on the original positioning spectrum and the post-erasure positioning spectrum of each original image and the original positioning spectrum and the post-erasure positioning spectrum of each rotated image in the image training set by the third 1x1 convolutional layer and the third ReLU activation function layer is specifically as follows: performing dimension conversion on the original positioning spectrum and the post-erasure positioning spectrum of each original image and the original positioning spectrum and the post-erasure positioning spectrum of each rotated image in the image training set by the third 1x1 convolutional layer to obtain the converted original positioning spectrum and the converted post-erasure positioning spectrum of each original image and the converted original positioning spectrum and the converted post-erasure positioning spectrum of each rotated image; performing normalization processing on the converted original positioning spectrum and the converted post-erasure positioning spectrum of each original image and the converted original positioning spectrum and the converted post-erasure positioning spectrum of each rotated image in the image training set by the third ReLU activation function layer to obtain the original object positioning spectrum and the post-erasure object positioning spectrum of each original image and the original object positioning spectrum and the post-erasure object positioning spectrum of each rotated image.

[0137] It should be understood that the process of sequentially performing pixel point calculation and set pixel point calculation on the original object positioning spectrum and the post-erasure object positioning spectrum of each original image and the original object positioning spectrum and the post-erasure object positioning spectrum of each rotated image in the image training set is specifically as follows: based on the sixth formula, the object positioning image pixel point calculation is performed to obtain a plurality of object positioning image pixel points of each original image and a plurality of object positioning image pixel points of each rotated image, and the sixth formula is as follows:

[0138] F”(x',y')=max(F”1(x',y'),F”2(x',y')),

[0139] wherein F”(x',y') is the (x',y')th object positioning image pixel point of the original image or the (x',y')th object positioning image pixel point of each rotated image, F”1(x',y') is the (x',y')th object positioning spectrum pixel point in the original object positioning spectrum of the original image or the (x',y')th object positioning spectrum pixel point of each rotated image, and F”2(x',y') is the (x',y')th post-erasure object positioning spectrum pixel point in the post-erasure object positioning spectrum of the original image or the (x',y')th post-erasure object positioning spectrum pixel point in the post-erasure object positioning spectrum of each rotated image; the plurality of object positioning image pixel points of each original image and the plurality of object positioning image pixel points of each rotated image are respectively set to obtain the target object positioning map of each original image and the target object positioning map of each rotated image in the image training set.

[0140] Specifically, the fourth global average pooling is performed on the original object localization spectrum of each original image in the image training set and the original object localization spectrum of each rotated image, and the loss function calculation process is performed according to the results after the fourth global average pooling, the up-sampling feature vectors of all original images in the image training set and the up-sampling feature vectors of all rotated images. The process is as follows: the global average pooling is performed on the original object localization spectrum of each original image in the image training set and the original object localization spectrum of each rotated image, and the original object localization spectrum feature vectors of each original image in the image training set and the original object localization spectrum feature vectors of each rotated image are obtained. Based on the seventh formula, the calculation of the auxiliary total loss function is performed to obtain the auxiliary total loss function, and the seventh formula is:

[0141]

[0142] wherein, is the auxiliary total loss function, is the original object localization spectrum feature vector of the i-th original image, is the up-sampling feature vector of the i-th original image, and I is the total number of original images in the image training set or the total number of first preset rotated images or the total number of second preset rotated images or the total number of third preset rotated images, is the original object localization spectrum feature vector of the i-th first preset rotated image, is the up-sampling feature vector of the i-th first preset rotated image, is the original object localization spectrum feature vector of the i-th second preset rotated image, is the up-sampling feature vector of the i-th second preset rotated image, is the original object localization spectrum feature vector of the i-th third preset rotated image, is the up-sampling feature vector of the i-th third preset rotated image.

[0143] It should be understood that the two complementary feature spectra are fused using the element maximum operation.

[0144] Specifically, the channel number K is adjusted to the class number C using a 1x1 convolution layer (i.e., the third 1x1 convolution layer), and then the feature spectrum is activated through a ReLU activation function (i.e., the third ReLU activation function layer) to suppress negative values, so that the object localization spectrum (i.e., the original object localization spectrum and the erased object localization spectrum of each original image in the image training set and the original object localization spectrum and the erased object localization spectrum of each rotated image) can be obtained, as follows:

[0145]

[0146] The two object localization maps (i.e., the original object localization spectrum of each original image and the object localization spectrum after erasing in the image training set, and the original object localization spectrum of each rotated image and the object localization spectrum after erasing) are fused by taking the element maximum value to obtain the fused object localization map (i.e., the target object localization map of each original image and the target object localization map of each rotated image in the image training set), as follows:

[0147] F'(x, y) = max(F1'(x, y), F2'(x, y)),

[0148] where x and y are the indices of the positions in the feature matrix. The function max(a, b) returns the corresponding maximum value of the two matrices a and b; the feature vector f' of the localization spectrum (i.e., the original object localization spectrum feature vector of each original image and the original object localization spectrum feature vector of each rotated image in the image training set) can be obtained by global average pooling.

[0149] It should be understood that in order to obtain a more comprehensive and complete target region, the two complementary feature maps are fused by element maximum value fusion.

[0150] Specifically, the prediction probability of the object localization spectrum F1' is y F1 (i.e., the original object localization spectrum feature vector of the original image or the original object localization spectrum feature vector of the first preset rotated image or the original object localization spectrum feature vector of the second preset rotated image or the original object localization spectrum feature vector of the third preset rotated image), then the auxiliary loss function (i.e., the auxiliary total loss function) is:

[0151]

[0152] The cross-entropy between the feature spectrum Z4 (i.e., the up-sampling feature spectrum group of each original image and the up-sampling feature spectrum group of each rotated image in the image training set) and the object localization spectrum F1' (i.e., the original object localization spectrum of each original image and the original object localization spectrum of each rotated image in the image training set) is calculated by the cross-entropy loss.

[0153] In the above embodiment, the auxiliary total loss function, the target object localization map of the original image, and the target object localization map of the rotated image are obtained according to the auxiliary total loss function of the up-sampling feature vector of the original image, the original localization spectrum and the localization spectrum after erasing, the up-sampling feature vector of the rotated image, the original localization spectrum and the localization spectrum after erasing, and the ground truth class label. In order to obtain a more comprehensive and complete target region, the two complementary feature maps are fused by element maximum value fusion.

[0154] Optionally, as one embodiment of the present application, in the S56, the process of analyzing the total loss function of the localization spectrum classification according to the target object localization map of all original images in the image training set, the target object localization map of the plurality of rotated images, and the ground truth class label corresponding to all original images comprises:

[0155] The target object localization map of each original image in the image training set and the target object localization map of each rotated image are subjected to fifth global average pooling, and a loss function is calculated according to the results after the fifth global average pooling and all ground truth class labels, to obtain the total loss function of the localization spectrum classification.

[0156] Specifically, the target object localization map of each original image in the image training set and the target object localization map of each rotated image are subjected to global average pooling, to obtain the target object localization map feature vector of each original image in the image training set and the target object localization map feature vector of each rotated image; based on the eighth formula, the total loss function of the localization spectrum classification is calculated, to obtain the total loss function of the localization spectrum classification, and the eighth formula is:

[0157]

[0158] wherein, is the total loss function of the localization spectrum classification, y i-GT is the ground truth class label corresponding to the i-th original image, I is the total number of original images in the image training set or the total number of first preset rotated images or the total number of second preset rotated images or the total number of third preset rotated images, y 1 i-f' is the target object localization map feature vector of the i-th original image, y 2 i-f' is the target object localization map feature vector of the i-th first rotated image, y 3 i-f' is the target object localization map feature vector of the i-th second rotated image, y 4 i-f' is the target object localization map feature vector of the i-th third rotated image.

[0159] Specifically, the predicted probability of f' is y f′ (i.e., the target object localization map feature vector of the original image or the target object localization map feature vector of the first rotated image or the target object localization map feature vector of the second rotated image or the target object localization map feature vector of the third rotated image), y GT is the ground truth class label, then the classification loss function (i.e., the total loss function of the localization spectrum classification) is:

[0160]

[0161] The probability score predicted by f' is calculated by a cross-entropy loss.

[0162] In the above embodiment, the probability score is calculated by analyzing the total loss function of the localization spectral classification according to the target object localization map of all original images in the image training set, the target object localization map of multiple rotated images, and the ground truth class label corresponding to all original images, which can improve the accuracy of weakly supervised localization by mining the complementarity and consistency of fine-grained targets.

[0163] Optionally, as an embodiment of the present application, in the process of S56, the process of obtaining the pseudo label of each original image in the image training set by testing each original image in the image training set according to the result after updating the parameters includes:

[0164] Each original image in the image training set is tested by the result after updating the parameters to obtain the original activation spectrum of each original image in the image training set, and the original activation spectrum includes multiple original activation spectrum pixel points.

[0165] The average pixel point, variance, and correlation coefficient of all original activation spectrum pixel points of all original images are sequentially calculated to obtain the original activation spectrum average pixel point, the original activation spectrum pixel point variance, and the original activation spectrum pixel point correlation coefficient.

[0166] The original activation spectrum average pixel point, the original activation spectrum pixel point variance, the original activation spectrum pixel point correlation coefficient, and the original activation spectrum of each original image in the image training set are sequentially calculated and set to obtain the Gaussian activation spectrum of each original image in the image training set.

[0167] The original activation spectrum of each original image in the image training set is multiplied by the Gaussian activation spectrum of each original image in the image training set for the fourth time, and the result after the fourth time element multiplication fusion is segmented according to three preset segmentation thresholds, and the segmented result is taken as the pseudo label of each original image in the image training set.

[0168] Specifically, the process of sequentially calculating and setting the original activation spectrum average pixel point, the original activation spectrum pixel point variance, the original activation spectrum pixel point correlation coefficient, and the original activation spectrum of each original image in the image training set is: based on the ninth formula, the Gaussian activation spectrum pixel point is calculated to obtain multiple Gaussian activation spectrum pixel points of each original image in the image training set, and the ninth formula is:

[0169]

[0170] wherein f(x", y") is a Gaussian activation spectrum pixel point of the original activation spectrum pixel point (x", y"), p is a correlation coefficient of the original activation spectrum pixel point, x" is an x coordinate of the original activation spectrum pixel point, y" is a y coordinate of the original activation spectrum pixel point, μ x” is an x coordinate of the average pixel point of the original activation spectrum, is an x coordinate of the variance of the pixel point of the original activation spectrum, is a y coordinate of the variance of the pixel point of the original activation spectrum; a plurality of Gaussian activation spectrum pixel points of each original image in the image training set are obtained by performing the fourth element multiplication fusion on the original activation spectrum of each original image in the image training set and the Gaussian activation spectrum of each original image in the image training set, and performing segmentation on the fourth element multiplication fusion result according to three preset segmentation thresholds.

[0171] Specifically, the fourth element multiplication fusion of the original activation spectrum of each original image in the image training set and the Gaussian activation spectrum of each original image in the image training set is performed, and the fusion activation spectrum of each original image in the image training set is obtained; the fusion activation spectrum of each original image in the image training set is segmented according to three preset segmentation thresholds, and the high response region, the low response region and the conflict region of each original image in the image training set are obtained, and the high response region, the low response region and the conflict region of each original image in the image training set are taken as the pseudo label of each original image in the image training set.

[0172] Specifically, in the test stage, the initial class activation map (CAM) is refined using the Gaussian prior information, and then the pseudo label is generated by setting a threshold on the Gaussian enhanced feature spectrum; each point (x, y) on the initial class activation map is regarded as a sample, and the mean value (μ x ,μ y ) (i.e. the average pixel point of the original activation spectrum), the variance (i.e. the variance of the pixel point of the original activation spectrum) and the correlation coefficient p (i.e. the correlation coefficient of the pixel point of the original activation spectrum) of all samples are calculated:

[0173]

[0174] These parameters are used to generate a two-dimensional Gaussian distribution (i.e. the Gaussian activation spectrum pixel point), as follows:

[0175]

[0176] The initial class activation map and the Gaussian enhanced class activation map are combined by taking the element maximum value to obtain a complete prediction; the enhanced class activation map is further divided into three parts by two predefined thresholds, a high response region corresponding to the foreground, a low response region corresponding to the background, and a conflict region corresponding to a low confidence region, and the three-value spectrum is used as a pseudo label.

[0177] Specifically, the average value (μ x ,μ y ) of all samples (i.e., the average pixel point of the original activation map), the variance (i.e., the variance of the pixel points of the original activation map), and the correlation coefficient ρ between x and y (i.e., the correlation coefficient of the pixel points of the original activation map) are calculated.

[0178]

[0179]

[0180]

[0181] where w i is the weight of the two-dimensional Gaussian distribution, and n is the total number of sampling points.

[0182] It should be understood that although the element maximum value fusion can provide more comprehensive target regions, these regions cannot be directly used to extract accurate positioning bounding boxes in the class-independent collaborative positioning network, and pseudo labels need to be generated by Gaussian enhancing the initial class activation map.

[0183] In the above embodiment, the pseudo labels are obtained by testing and analyzing each original image in the image training set according to the result after updating the parameters, which can enhance the initial class activation map to generate the pseudo labels.

[0184] Optionally, as shown in FIGS. Figure 1 and 4 , the target object positioning map includes a plurality of target object positioning image points, and the pseudo label includes a plurality of pseudo label image points corresponding to the target object positioning image points.

[0185] The process of S6 includes:

[0186] Constructing a collaborative positioning network;

[0187] According to all target object positioning image points and all pseudo label image points of each original image in the image training set, a loss function is calculated, and the calculated result is summed to obtain a total segmentation loss function;

[0188] According to the contrast total loss function and the segmentation total loss function, loss function calculation is performed to obtain a cooperative positioning network loss function;

[0189] According to the cooperative positioning network loss function, parameter updating is performed on the cooperative positioning network to obtain a target cooperative positioning network.

[0190] Specifically, according to all target object positioning image pixels and all pseudo-label image pixels of each original image in the image training set, loss function calculation is performed, and the calculated results are summed to obtain a segmentation loss function of each original image in the image training set, based on the tenth formula, the calculation of the segmentation loss function is performed to obtain the segmentation loss function of each original image in the image training set, and the tenth formula is:

[0191]

[0192] Wherein, is the segmentation loss function of the i-th original image, H1W1 is the total number of target object positioning image pixels, g ab is a preset indicator function, m ab is the target object positioning image pixel (a, b), is the pseudo-label image pixel corresponding to the target object positioning image pixel (a, b); the segmentation loss functions of all original images are summed to obtain a segmentation total loss function.

[0193] Specifically, according to the contrast total loss function and the segmentation total loss function, loss function calculation is performed to obtain a cooperative positioning network loss function, based on the eleventh formula, the calculation of the cooperative positioning network loss function is performed to obtain the cooperative positioning network loss function, and the eleventh formula is:

[0194]

[0195] Wherein, is the cooperative positioning network loss function, is the contrast total loss function, is the segmentation total loss function.

[0196] Specifically, as shown in Figure 4 , a 1x1 convolution layer and a ReLU activation function are used to extract the mask spectrum of the object and By taking the element maximum value, the two mask spectra are fused to obtain a fused mask spectrum , that is, the target object positioning map;

[0197] According to the mask M' (that is, the target object positioning map) and the pseudo-label, the segmentation loss (that is, the segmentation loss function) is calculated; let m ij and respectively are the predicted probability (i.e. the target object localization pixel) and the corresponding pseudo label (i.e. the pseudo label pixel) at spatial coordinates (i,j); in the training process, only the deterministic regions (foreground region and background region) are considered, and the loss of the conflict region is ignored to prevent misleading the network; then the segmentation loss (i.e. the segmentation loss function) is:

[0198]

[0199] where g represents an indicator function. For the deterministic region, g ij is set to 1, while for the conflict region, g ij is set to 0.

[0200] It should be understood that ignoring the loss of the conflict region can prevent misleading the network and affect the performance of the network.

[0201] It should be understood that the class-agnostic co-localization network loss function (i.e. the co-localization network loss function) includes a cross-class target complementarity component contrast loss function (i.e. the contrast total loss function) and a segmentation loss (i.e. the segmentation loss function).

[0202] Specifically, the class-agnostic co-localization network loss function (i.e. the co-localization network loss function) is:

[0203]

[0204] By mining the complementarity and consistency components of fine-grained objects, the classification performance of the system is improved.

[0205] In the above embodiment, the target co-localization network is obtained by analyzing the co-localization network update of the co-localization network based on the contrast total loss function and the target object localization map and the pseudo label of the original image, and the loss of the conflict region is ignored, which can prevent misleading the network and affect the performance of the network.

[0206] Alternatively, as another embodiment of the present application, the present application establishes a classification network by using an existing convolutional neural network (EfficientNet-B7); uses the original image to train the network to predict the probability of the correct class of the input image and the probability of the correct class being included in the top 5; and sends the original image into the existing convolutional neural network (EfficientNet-B7) to obtain the TOP-1 classification accuracy and the TOP-5 classification accuracy of the original image. The beneficial effect is that by using the most advanced classification network, better classification accuracy can be obtained, and further better localization accuracy can be obtained.

[0207] Optionally, as another embodiment of the present application, the present application predicts the category probability score of the original image through the classification network; generates a more refined class activation map as a pseudo label through the pseudo label generation network; and trains the class-agnostic co-localization network through the generated pseudo label, so as to better locate the fine-grained object.

[0208] Optionally, as another embodiment of the present application, the present application follows the assumption that fine-grained objects with the same / different semantics should have similar / different feature representations in the feature space, and by mining the common part information shared by fine-grained objects and the influence of the complementary parts of fine-grained objects on the co-localization of fine-grained objects, the final classification performance of the model reaches the current most advanced level.

[0209] Optionally, as another embodiment of the present application, the present application produces a clearer object boundary by multiplying elements to aggregate shallow and deep features; in order to improve the rotation invariance of part localization and extract discriminative parts of fine-grained objects; in addition, attention erasing is used to encourage the network to learn other complementary part regions by erasing discriminative parts; in addition to exploring the features of complementary parts of fine-grained objects, the same part features are pulled closer and the different part features are pushed away to mine consistent shared parts of fine-grained objects; finally, by taking the element maximum value to fuse two complementary consistent feature maps, a comprehensive and accurate target region can be captured.

[0210] Optionally, as another embodiment of the present application, the present application applies a bounding box extractor to the object mask to obtain the final object localization; and combines the predicted bounding box extracted from the predicted mask and the S11 predicted classification label together to evaluate the fine-grained object co-localization performance.

[0211] Optionally, as another embodiment of the present application, the present application optimizes the fine-grained weakly supervised object co-localization using a training sample set; and tests the fine-grained weakly supervised object co-localization network using a test sample set, specifically:

[0212] By using the automatic differentiation technology provided in Pytorch, the fine-grained object co-localization network is optimized according to the pseudo label generation loss and the class-agnostic co-localization network loss using the stochastic gradient descent and back propagation algorithm; and the fine-grained object co-localization network is tested based on the training set weight using the test sample set.

[0213] The beneficial effect is that the network parameters are optimized by the stochastic gradient descent, the back propagation algorithm and the loss function; and then the test set is used to test based on the training set weight.

[0214] Optionally, as another embodiment of the present application, as Figure 4As shown, a class-agnostic co-localization network is established by a deep convolutional neural network; a residual network (ResNet50) is trained using original images and generated pseudo labels, and then a refined object mask is extracted through a shared weight multiplication feature fusion module, an attention erasing module, a cross-class target complementarity component contrast learning module and an element maximum fusion module, respectively; a bounding box extractor is applied to the object mask to obtain the final object localization;

[0215] A class-agnostic co-localization network is established by a deep convolutional neural network, including a shared weight multiplication feature fusion module, an attention erasing module, a cross-class target complementarity component contrast learning module and an element maximum fusion module;

[0216] The original image is used as the input image, and a residual network (ResNet50) is used as the feature extractor;

[0217] The shared weight multiplication feature fusion module operates S52 to S53;

[0218] The attention erasing module repeats operation S56;

[0219] The cross-class target complementarity component contrast learning module operates repeatedly S57-S59.

[0220] Optionally, as another embodiment of the present application, the proposed algorithm is tested on four published datasets CUB-200-2011, Stanford Cars, FGCV-Aircraft and Stanford Dogs. The CUB-200-2011 dataset is a fine-grained dataset containing a total of 11788 bird images of 200 classes, of which the training set contains 5994 images and the test set contains 5794 images. The Stanford Cars dataset is a fine-grained dataset containing a total of 16185 car images of 196 classes, of which the training set contains 8144 images and the test set contains 8041 images. The FGCV-Aircraft dataset is a fine-grained dataset containing a total of 10000 aircraft images of 100 classes, of which the training set contains 6667 images and the test set contains 3333 images. The Stanford Dogs dataset is a fine-grained dataset containing a total of 20580 dog images of 120 classes, of which the training set contains 12000 images and the test set contains 8580 images.

[0221] The method is compared with several weakly supervised object localization and weakly supervised co-localization methods, including SPA, CAMSC, PSOL and SPOL. It is worth noting that the method proposed in the application uses ResNet50 as a feature extractor, and uses an NVIDIA A6000 and an open source machine learning library Pytorch to implement the proposed method. The same parameter settings are used for the four data sets, the input image is adjusted to a fixed size of 256x256, and then randomly cropped to a fixed size of 224x224 for subsequent processing. The training batch size is set to 64, the learning rate for the pre-trained feature extraction layer is 0.002, and the learning rate for the newly added module is 0.02. For the pseudo-label generation network, the number of iterations is 32, and the initial learning rate gradually decreases as the number of iterations increases. In the Gaussian enhancement, we use the region greater than 0.2 as the foreground. For the pseudo-label generation after Gaussian enhancement, the two thresholds of foreground and background are set to 0.2 and 0.01 respectively. For the class-agnostic co-localization network, the number of iterations is 10, and the training batch size (batch-size) is set to 128.

[0222] Top-1 / Top-5 localization accuracy (Top-1 / Top-5 Loc) and ground truth known localization accuracy (GT-known Loc) are used as evaluation metrics. In these localization evaluation metrics, classification and localization performance are intertwined. The final localization accuracy is calculated as the fraction of images for which both tasks have correct predictions. 1) Top-1 Loc: Top-1 predicted class contains the correct label and the predicted bounding box has an IoU greater than 50% with the ground truth box; 2) Top-5 Loc: Top-5 predicted classes contain the correct label and the predicted bounding box has an IoU greater than 50% with the ground truth box; 3) GT-known Loc: regardless of classification, the predicted bounding box has an IoU greater than 50% with the ground truth box.

[0223] As shown in the following table, the method proposed in the application shows better performance compared with other state-of-the-art methods. Table one is the result on the CUB-200-2011 dataset, and table two is the result on the other three datasets. The results show that the proposed fine-grained weakly supervised object co-localization algorithm based on complementary part contrast learning can better find the discriminative regions of objects for fine-grained object co-localization. The proposed method performs better on the CUB-200-2011 and Stanford Cars datasets. The results show that the method has superiority compared with other methods.

[0224] Table one

[0225]

[0226]

[0227] Table II

[0228]

[0229] Figure 5 A module block diagram of a weakly supervised target positioning device provided for an embodiment of the present application.

[0230] Optionally, as another embodiment of the present application, as shown in Figure 5 A weakly supervised target positioning device comprises:

[0231] A division module is used to import an original image dataset and divide the original image dataset into an image training set and an image test set;

[0232] A rotation module is used to rotate each original image in the image training set at multiple angles to obtain multiple rotated images of each original image;

[0233] A feature extraction module is used to perform multi-channel feature extraction on each original image in the image training set and each rotated image of each original image to obtain a feature spectrum group of each original image in the image training set and a feature spectrum group of each rotated image;

[0234] A loss calculation module is used to calculate a rotation prediction total loss function according to a preset layer feature spectrum in the feature spectrum group of each original image in the image training set and a preset layer feature spectrum in the feature spectrum group of each rotated image to obtain the rotation prediction total loss function;

[0235] A first network update module is used to construct a pseudo-label generation network, analyze pseudo-label generation network update of the pseudo-label generation network according to the rotation prediction total loss function, the feature spectrum group of each original image in the image training set and the feature spectrum group of each rotated image to obtain a comparison total loss function and a target object positioning map and a pseudo-label of each original image in the image training set;

[0236] A second network update module is used to construct a collaborative positioning network, analyze collaborative positioning network update of the collaborative positioning network according to the comparison total loss function and the target object positioning map and the pseudo-label of each original image in the image training set to obtain a target collaborative positioning network;

[0237] A target positioning result obtaining module is used to test all original images in the image test set through the target collaborative positioning network to obtain a target positioning result.

[0238] Optionally, another embodiment of the present application provides a weakly supervised target positioning system, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, when the processor executes the computer program, the weakly supervised target positioning method as described above is implemented. The system can be a computer or the like.

[0239] Optionally, another embodiment of the present application provides a computer readable storage medium, the computer readable storage medium stores a computer program, when the computer program is executed by a processor, the weakly supervised target positioning method as described above is implemented.

[0240] It should be noted that, in this paper, the relationship terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between the entities or operations. Moreover, the terms "include", "contain" or any other variants thereof are intended to cover non-exclusive inclusion, so that the process, method, article or equipment including a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such process, method, article or equipment.

[0241] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the above-described device and unit can refer to the corresponding process in the foregoing method embodiment, which will not be described here.

[0242] In several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of units is only a logical function division, and actual implementation can have another division manner, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed.

[0243] The units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, that is, they can be located in one place, or they can be distributed on a plurality of network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiment of the present application.

[0244] In addition, the functional units in each embodiment of the present application can be integrated in one processing unit, or each unit can be physically present alone, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of software functional unit.

[0245] The integrated unit, if implemented in the form of a software function unit and sold or used as an independent product, can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application essentially or say the part of the prior art that contributes, or the whole or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various media that can store program codes.

[0246] The above description is only the preferred embodiment of the present application and is not intended to limit the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A weakly supervised target localization method, characterized in that, The method comprises the following steps: S1: importing an original image data set and dividing the original image data set into an image training set and an image test set; S2: rotating each original image in the image training set at multiple angles to obtain multiple rotated images of each original image; S3: performing multi-channel feature extraction on each original image in the image training set and each rotated image of each original image to obtain a feature spectrum group of each original image in the image training set and a feature spectrum group of each rotated image; S4: calculating a rotation prediction total loss function according to a preset layer feature spectrum in the feature spectrum group of each original image in the image training set and a preset layer feature spectrum in the feature spectrum group of each rotated image, to obtain the rotation prediction total loss function; S5: constructing a pseudo-label generation network, performing pseudo-label generation network update analysis on the pseudo-label generation network according to the rotation prediction total loss function, the feature spectrum group of each original image in the image training set and the feature spectrum group of each rotated image, to obtain a comparison total loss function and a target object positioning map and a pseudo-label of each original image in the image training set; S6: constructing a cooperative positioning network, performing cooperative positioning network update analysis on the cooperative positioning network according to the comparison total loss function and the target object positioning map and the pseudo-label of each original image in the image training set, to obtain a target cooperative positioning network; S7: testing all original images in the image test set through the target cooperative positioning network to obtain a target positioning result; The process of S5 comprises: S51: constructing a pseudo-label generation network; S52: sequentially performing upsampling analysis, element multiplication fusion, first global average pooling processing and attention erasing analysis on the pseudo-label generation network, the feature spectrum group of each original image in the image training set and the feature spectrum group of each rotated image, to correspondingly obtain an upsampling feature spectrum group of each original image in the image training set, an original positioning spectrum, an upsampling feature vector and an erased positioning spectrum, and an upsampling feature spectrum group of each rotated image, an original positioning spectrum, an upsampling feature vector and an erased positioning spectrum; S53: sequentially performing second global average pooling and first normalization processing on the original positioning spectrum and the erased positioning spectrum of each original image in the image training set and the original positioning spectrum and the erased positioning spectrum of each rotated image, to obtain an original positioning spectrum normalization vector and an erased positioning spectrum normalization vector of each original image in the image training set, and an original positioning spectrum normalization vector and an erased positioning spectrum normalization vector of each rotated image; S54: introducing ground truth class labels corresponding to each original image in the image training set, calculating a loss function according to the up-sampling feature vectors, the original positioning spectrum normalized vectors and the erased positioning spectrum normalized vectors of all original images in the image training set, the up-sampling feature vectors, the original positioning spectrum normalized vectors and the erased positioning spectrum normalized vectors of each rotated image, and all ground truth class labels, obtaining a feature spectrum classification total loss function and a contrast total loss function; S55: performing auxiliary total loss function analysis according to the up-sampling feature vectors, the original positioning spectrum and the erased positioning spectrum of each original image in the image training set, the up-sampling feature vectors, the original positioning spectrum and the erased positioning spectrum of each rotated image, and the ground truth class labels corresponding to all original images, obtaining an auxiliary total loss function, target object positioning maps of each original image in the image training set, and target object positioning maps of each rotated image; S56: performing positioning spectrum classification total loss function analysis according to the target object positioning maps of all original images in the image training set, the target object positioning maps of a plurality of rotated images, and the ground truth class labels corresponding to all original images, and calculating a loss function according to the results after loss function analysis, the feature spectrum classification total loss function, the auxiliary total loss function, the rotated prediction total loss function and the contrast total loss function, and performing parameter updating on the pseudo label generation network according to the results after parameter updating, and performing test analysis on each original image in the image training set according to the results after parameter updating, obtaining pseudo labels of each original image in the image training set; The target object positioning map includes a plurality of target object positioning image pixels, and the pseudo label includes a plurality of pseudo label image pixels corresponding to the target object positioning image pixels. The process of S6 includes: constructing a cooperative positioning network; performing loss function calculation according to all target object positioning image pixels and all pseudo label image pixels of each original image in the image training set, and performing summation operation on the results after calculation, obtaining a segmentation total loss function; performing loss function calculation according to the contrast total loss function and the segmentation total loss function, obtaining a cooperative positioning network loss function; performing parameter updating on the cooperative positioning network according to the cooperative positioning network loss function, obtaining a target cooperative positioning network.

2. The weakly supervised target positioning method of claim 1, wherein, The process of S4 includes: introducing real angle labels corresponding to each original image in the image training set; calculating a loss function according to the pre-set layer feature spectrum in the feature spectrum group of each original image in the image training set, the pre-set layer feature spectrum in the feature spectrum group of each rotated image, and all real angle labels, obtaining a rotated prediction total loss function.

3. The weakly supervised target positioning method of claim 1, wherein, The pseudo label generation network includes a first 1x1 convolutional layer group, a first 1x1 convolutional layer, a Sigmoid activation function layer, a second 1x1 convolutional layer group and a first ReLU activation function layer; The process of upsampling analysis in S52 includes: performing third global average pooling processing on the feature spectrum groups of each original image and each rotated image in the image training set; performing first dimension conversion on the third global average pooling processing result by the first 1x1 convolution layer group; performing first element multiplication fusion on the result after the first dimension conversion; performing second dimension conversion on the result after the first element multiplication fusion by the first 1x1 convolution layer; performing second normalization processing on the result after the second dimension conversion by the Sigmoid activation function layer; performing second element multiplication fusion on the result after the second normalization processing; performing third dimension conversion on the result after the second element multiplication fusion by the second 1x1 convolution layer group; performing third normalization processing on the result after the third dimension conversion by the first ReLU activation function layer; performing upsampling processing on the result after the third normalization processing to obtain the upsampling feature spectrum groups of each original image and each rotated image in the image training set.

4. The weakly supervised target positioning method of claim 3, wherein, The pseudo-label generation network further includes a second 1x1 convolution layer and a second ReLU activation function layer. The process of attention erasing analysis in S52 includes: performing fourth dimension conversion on the upsampling feature spectrum groups of each original image and each rotated image in the image training set by the second 1x1 convolution layer; performing fourth normalization processing on the result after the fourth dimension conversion by the second ReLU activation function layer; extracting attention spectrum from the result after the fourth normalization processing according to the preset category; performing maximum and minimum normalization processing on the extracted result to obtain the normalized attention spectrum groups of each original image and each rotated image in the image training set; performing pixel point calculation and set pixel point on the normalized attention spectrum groups of each original image and each rotated image in the image training set in sequence to obtain the attention erasing mask groups of each original image and each rotated image in the image training set; performing third element multiplication fusion on the feature spectrum groups of each original image and each rotated image in the image training set and the attention erasing mask groups of each original image and each rotated image in the image training set, and performing upsampling analysis, element multiplication fusion and first global average pooling processing on the result after the third element multiplication fusion to obtain the post-erasing localization spectrum of each original image and each rotated image in the image training set.

5. The weakly supervised target positioning method of claim 1, wherein, The process of S55 includes: The original object localization spectrum and the post-erasing object localization spectrum of each original image in the image training set and the original object localization spectrum and the post-erasing object localization spectrum of each rotated image are sequentially subjected to fifth dimension conversion and fifth normalization processing through a third 1*1 convolutional layer and a third ReLU activation function layer, to obtain the original object localization spectrum and the post-erasing object localization spectrum of each original image in the image training set and the original object localization spectrum and the post-erasing object localization spectrum of each rotated image; The target object localization map of each original image in the image training set and the target object localization map of each rotated image are obtained by sequentially performing pixel point calculation and set pixel point calculation according to the original object localization spectrum and the post-erasing object localization spectrum of each original image in the image training set and the original object localization spectrum and the post-erasing object localization spectrum of each rotated image. The original object localization spectrum of each original image in the image training set and the original object localization spectrum of each rotated image are subjected to fourth global average pooling, and a loss function is calculated according to the results of the fourth global average pooling, the up-sampling feature vectors of all original images in the image training set and the up-sampling feature vectors of all rotated images, to obtain an auxiliary total loss function.

6. The weakly supervised target positioning method of claim 5, wherein, In the S56, the process of analyzing the localization spectrum classification total loss function according to the target object localization map of each original image in the image training set, the target object localization map of each rotated image and the ground truth class label corresponding to all original images includes: The target object localization map of each original image in the image training set and the target object localization map of each rotated image are subjected to fifth global average pooling, and a loss function is calculated according to the results of the fifth global average pooling and all ground truth class labels, to obtain a localization spectrum classification total loss function.

7. The weakly supervised target positioning method of claim 1, wherein, In the S56, the process of testing and analyzing each original image in the image training set according to the results of the parameter update to obtain pseudo-labels of each original image in the image training set includes: Each original image in the image training set is tested by the results of the parameter update to obtain an original activation spectrum of each original image in the image training set, wherein the original activation spectrum includes a plurality of original activation spectrum pixel points; The average pixel point, the variance and the correlation coefficient of all original activation spectrum pixel points of all original images are sequentially calculated, to obtain the original activation spectrum average pixel point, the original activation spectrum pixel point variance and the original activation spectrum pixel point correlation coefficient; The Gaussian activation spectrum of each original image in the image training set is obtained by sequentially performing pixel point calculation and set pixel point calculation according to the original activation spectrum average pixel point, the original activation spectrum pixel point variance, the original activation spectrum pixel point correlation coefficient and the original activation spectrum of each original image in the image training set. The original activation spectrum of each original image in the image training set is fourthly element-wise multiplied with the Gaussian activation spectrum of each original image in the image training set, and the fourth element-wise multiplication fusion result is segmented according to three preset segmentation thresholds, and the segmented result is taken as the pseudo label of each original image in the image training set.

8. A weakly supervised object localization apparatus, characterized by, Comprise: The division module imports the original image data set, and divides the original image data set into an image training set and an image test set; The rotation module is configured to rotate each original image in the image training set by a plurality of angles to obtain a plurality of rotated images of each original image; The feature extraction module is configured to perform multi-channel feature extraction on each original image in the image training set and each rotated image of each original image to obtain a feature spectrum group of each original image in the image training set and a feature spectrum group of each rotated image; The loss calculation module is configured to calculate a rotation prediction total loss function according to the preset layer feature spectrum in the feature spectrum group of each original image in the image training set and the preset layer feature spectrum in the feature spectrum group of each rotated image to obtain the rotation prediction total loss function; The first network update module is configured to construct a pseudo label generation network, analyze pseudo label generation network update of the pseudo label generation network according to the rotation prediction total loss function, the feature spectrum group of each original image in the image training set and the feature spectrum group of each rotated image to obtain a comparison total loss function and a target object positioning map and a pseudo label of each original image in the image training set; The second network update module is configured to construct a cooperative positioning network, analyze cooperative positioning network update of the cooperative positioning network according to the comparison total loss function and the target object positioning map and the pseudo label of each original image in the image training set to obtain a target cooperative positioning network; The target positioning result obtaining module is configured to test all original images in the image test set through the target cooperative positioning network to obtain a target positioning result; The first network update module is specifically configured to: S51: Construct a pseudo label generation network; S52: According to the pseudo label generation network, the feature spectrum group of each original image in the image training set and the feature spectrum group of each rotated image, sequentially perform upsampling analysis, element-wise multiplication fusion, first global average pooling processing and attention erasure analysis, and correspondingly obtain the upsampling feature spectrum group, the original positioning spectrum, the upsampling feature vector and the erased positioning spectrum of each original image in the image training set, and the upsampling feature spectrum group, the original positioning spectrum, the upsampling feature vector and the erased positioning spectrum of each rotated image; S53: sequentially performing second global average pooling and first normalization processing on the original localization spectrum and the post-erasing localization spectrum of each original image and the original localization spectrum and the post-erasing localization spectrum of each rotated image in the image training set to obtain the original localization spectrum normalized vector and the post-erasing localization spectrum normalized vector of each original image and the original localization spectrum normalized vector and the post-erasing localization spectrum normalized vector of each rotated image in the image training set; S54: importing the ground truth class labels corresponding to each original image in the image training set, and calculating a loss function according to the up-sampling feature vector, the original localization spectrum normalized vector and the post-erasing localization spectrum normalized vector of all original images, the up-sampling feature vector, the original localization spectrum normalized vector and the post-erasing localization spectrum normalized vector of each rotated image, and all ground truth class labels to obtain a feature spectrum classification total loss function and a contrast total loss function; S55: performing auxiliary total loss function analysis according to the up-sampling feature vector, the original localization spectrum and the post-erasing localization spectrum of each original image, the up-sampling feature vector, the original localization spectrum and the post-erasing localization spectrum of each rotated image, and the ground truth class labels corresponding to all original images to obtain an auxiliary total loss function, the target object localization map of each original image in the image training set, and the target object localization map of each rotated image; S56: performing localization spectrum classification total loss function analysis according to the target object localization map of all original images, the target object localization map of a plurality of rotated images, and the ground truth class labels corresponding to all original images, and calculating a loss function according to the results after loss function analysis, the feature spectrum classification total loss function, the auxiliary total loss function, the rotated prediction total loss function and the contrast total loss function, and performing parameter updating on the pseudo label generation network according to the results after parameter updating, and performing test analysis on each original image in the image training set according to the results after parameter updating to obtain the pseudo label of each original image in the image training set; The target object localization map includes a plurality of target object localization image pixels, and the pseudo label includes a plurality of pseudo label image pixels corresponding to the target object localization image pixels; The second network updating module is specifically configured to: construct a cooperative localization network; perform loss function calculation according to all target object localization image pixels and all pseudo label image pixels of each original image in the image training set, and perform summation operation on the results after calculation to obtain a segmentation total loss function; perform loss function calculation according to the contrast total loss function and the segmentation total loss function to obtain a cooperative localization network loss function; perform parameter updating on the cooperative localization network according to the cooperative localization network loss function to obtain a target cooperative localization network.

Images