Method and apparatus for training image detection model

Abstract

Embodiments of the present specification provide a training method and device of an image detection model, wherein the training method of an image detection model comprises: inputting a first sample image set and a second sample image set into a teacher network to be trained for category matching to obtain a category matching probability; inputting the first sample image set and the second sample image set into a student network to be trained, to perform multi-element feature generation and feature fusion on each sample image according to a similarity matrix corresponding to the first sample image set and the second sample image set, to obtain a multi-element feature set and a fused feature set, and to calculate a training loss based on the category matching probability, the multi-element feature set and the fused feature set, and to adjust parameters of the student network to be trained and the teacher network to be trained based on the training loss, to realize cooperative training of the student network and the teacher network.

Description

TECHNICAL FIELD

[0001] The present document relates to the technical field of data processing, and in particular to a training method and device of an image detection model. BACKGROUND

[0002] In the field of machine learning, there are two traditional learning methods: supervised learning and unsupervised learning. Semi-supervised learning is a key problem in the field of pattern recognition and machine learning, and is a learning method combining supervised learning and unsupervised learning. It mainly considers how to use a small amount of labeled samples and a large amount of unlabeled samples for training and classification. It is mainly divided into semi-supervised classification, semi-supervised regression, semi-supervised clustering and semi-supervised dimensionality reduction algorithm. How to use semi-supervised learning to train a more diverse and comprehensive model is the focus of users. SUMMARY

[0003] One or more embodiments of the present specification provide a training method of an image detection model. The training method of the image detection model comprises: inputting a first sample image set and a second sample image set into a teacher network to be trained for category matching to obtain a category matching probability; the first sample image set is composed of first sample images labeled with image categories; the second sample image set is composed of second sample images without image category labeling. The first sample image set and the second sample image set are input into a student network to be trained to perform the following operations: according to a similarity matrix corresponding to the first sample image set and the second sample image set, performing multi-element feature generation and feature fusion on each sample image to obtain a multi-element feature set and a fused feature set; based on the category matching probability, the multi-element feature set and the fused feature set, calculating a training loss, and based on the training loss, adjusting parameters of the student network to be trained and the teacher network to be trained to construct an image detection model according to the trained teacher network.

[0004] One or more embodiments of the present specification provide an image detection processing method, comprising: obtaining at least two images to be detected. Input the at least two images to be detected into an image detection model for image detection to obtain a feature vector; the image detection model is constructed based on a trained teacher network. Based on the feature vector, a category matching probability of the at least two images to be detected is calculated. Wherein, the teacher network is trained in cooperation with a student network; the cooperation training comprises: based on a teacher network to be trained, performing category matching on a first sample image set and a second sample image set to obtain a sample category matching probability; based on a student network to be trained, adjusting parameters of the teacher network to be trained and the student network to be trained according to the sample category matching probability and a multi-element feature set and a fused feature set corresponding to the first sample image set and the second sample image set, to obtain a trained teacher network.

[0005] One or more embodiments of the present specification provide a training device of an image detection model, comprising: a category matching module configured to input a first sample image set and a second sample image set into a teacher network to be trained for category matching, to obtain a category matching probability; the first sample image set is composed of first sample images labeled with image categories; the second sample image set is composed of second sample images without image category labeling. The first sample image set and the second sample image set are input into a student network to be trained, to run the following modules: a feature fusion module configured to generate and fuse multi-element features of each sample image according to a similarity matrix corresponding to the first sample image set and the second sample image set, to obtain a multi-element feature set and a fused feature set; a parameter adjustment module configured to calculate a training loss based on the category matching probability, the multi-element feature set and the fused feature set, and to adjust parameters of the student network to be trained and the teacher network to be trained based on the training loss, to construct an image detection model according to the trained teacher network.

[0006] One or more embodiments of the present specification provide an image detection processing device, comprising: an image acquisition module configured to acquire at least two images to be detected. An image detection module is configured to input the at least two images to be detected into an image detection model for image detection to obtain a feature vector; the image detection model is constructed based on a trained teacher network. A probability calculation module is configured to calculate a category matching probability of the at least two images to be detected based on the feature vector. Wherein, the teacher network is trained in cooperation with a student network; the cooperation training includes: based on a teacher network to be trained, a first sample image set and a second sample image set are matched in category to obtain a sample category matching probability; based on a student network to be trained, the teacher network to be trained and the student network to be trained are adjusted in parameters according to the sample category matching probability and a multi-element feature set and a fused feature set corresponding to the first sample image set and the second sample image set, to obtain the trained teacher network.

[0007] One or more embodiments of the specification provide a training device of an image detection model, comprising: a processor; and a memory configured to store computer executable instructions that, when executed, cause the processor to: a training method of the image detection model, comprising: inputting a first sample image set and a second sample image set into a teacher network to be trained for category matching, to obtain a category matching probability; the first sample image set is composed of first sample images labeled with image categories; the second sample image set is composed of second sample images without image category labeling. Inputting the first sample image set and the second sample image set into a student network to be trained, to perform the following operations: according to a similarity matrix corresponding to the first sample image set and the second sample image set, performing multi-element feature generation and feature fusion on each sample image, to obtain a multi-element feature set and a fused feature set; based on the category matching probability, the multi-element feature set and the fused feature set, calculating a training loss, and based on the training loss, adjusting parameters of the student network to be trained and the teacher network to be trained, to construct an image detection model according to a teacher network obtained through training.

[0008] One or more embodiments of the specification provide an image detection processing device, comprising: a processor; and a memory configured to store computer executable instructions that, when executed, cause the processor to: obtain at least two images to be detected. Input the at least two images to be detected into an image detection model for image detection, to obtain a feature vector; the image detection model is constructed based on a trained teacher network. Calculate a category matching probability of the at least two images to be detected based on the feature vector. Wherein, the teacher network is trained in cooperation with a student network; the cooperation training comprises: based on a teacher network to be trained, performing category matching on a first sample image set and a second sample image set, to obtain a sample category matching probability; based on a student network to be trained, adjusting parameters of the teacher network to be trained and the student network to be trained according to the sample category matching probability, a multi-element feature set and a fused feature set corresponding to the first sample image set and the second sample image set, to obtain a trained teacher network.

[0009] The one or more embodiments of the specification provide a storage medium for storing computer executable instructions, which, when executed by a processor, implement the following process: a training method of an image detection model, comprising: inputting a first sample image set and a second sample image set into a teacher network to be trained for category matching, to obtain a category matching probability; the first sample image set is composed of first sample images labeled with image categories; the second sample image set is composed of second sample images without image category labeling. Input the first sample image set and the second sample image set into a student network to be trained to perform the following operations: according to the similarity matrix corresponding to the first sample image set and the second sample image set, perform multi-element feature generation and feature fusion on each sample image to obtain a multi-element feature set and a fusion feature set; based on the category matching probability, the multi-element feature set and the fusion feature set, calculate a training loss, and based on the training loss, adjust the parameters of the student network to be trained and the teacher network to be trained, to construct an image detection model according to the teacher network obtained by training.

[0010] The one or more embodiments of the specification provide another storage medium for storing computer executable instructions, which, when executed by a processor, implement the following process: obtaining at least two images to be detected. Input the at least two images to be detected into an image detection model for image detection to obtain a feature vector; the image detection model is constructed based on a trained teacher network. Calculate the category matching probability of the at least two images to be detected based on the feature vector. Wherein, the teacher network is trained in cooperation with a student network; the cooperation training comprises: based on the teacher network to be trained, performing category matching on a first sample image set and a second sample image set to obtain a sample category matching probability; based on the student network to be trained, adjusting the parameters of the teacher network to be trained and the student network to be trained according to the sample category matching probability and a multi-element feature set and a fusion feature set corresponding to the first sample image set and the second sample image set, to obtain the trained teacher network. BRIEF DESCRIPTION OF DRAWINGS

[0011] In order to more clearly illustrate the technical solutions in the one or more embodiments of the specification or the prior art, the drawings needed in the embodiment or prior art description will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments described in the specification, and those skilled in the art can also obtain other drawings according to these drawings without creative labor;

[0012] Figure 1 A schematic diagram of an image detection model training method implementation environment provided by the one or more embodiments of the specification;

[0013] Figure 2 A flowchart illustrating a training method for an image detection model provided in one or more embodiments of this specification;

[0014] Figure 3 A flowchart illustrating an image detection processing method provided in one or more embodiments of this specification;

[0015] Figure 4 A schematic diagram of an embodiment of a training device for an image detection model provided in one or more embodiments of this specification;

[0016] Figure 5 A schematic diagram of an embodiment of an image detection and processing apparatus provided in one or more embodiments of this specification;

[0017] Figure 6 A schematic diagram of the structure of a training device for an image detection model provided in one or more embodiments of this specification;

[0018] Figure 7 This is a schematic diagram of the structure of an image detection and processing device provided for one or more embodiments of this specification. Detailed Implementation

[0019] To enable those skilled in the art to better understand the technical solutions in one or more embodiments of this specification, the technical solutions in one or more embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this specification, and not all of the embodiments. Based on one or more embodiments of this specification, all other embodiments obtained by those skilled in the art without creative effort should fall within the protection scope of this document.

[0020] like Figure 1 As shown, in one or more embodiments of this specification, the implementation environment includes a new model framework, which includes two branches: a teacher network to be trained and a student network to be trained.

[0021] The student network to be trained consists of three parts: a feature extraction layer (Backbone), a pooling layer (Pooler), and a feature transformation layer (Conv+Pooler).

[0022] Among them, Backbone is used to extract the feature map of the input image; Pooler is used to extract the feature vector corresponding to the feature map; Conv+Pooler is used to extract the feature vector of the concatenated combined feature map;

[0023] The teacher network to be trained includes a feature extraction layer Backbone, a feature conversion layer Conv+Pooler, and a fully connected layer;

[0024] The Backbone is configured to extract a feature map of an input image, the Conv+Pooler is configured to extract a feature vector of a combined feature map after splicing, and the fully connected layer is configured to perform classification prediction on the combined feature.

[0025] In addition, the teacher network to be trained further includes a pooling layer Pooler configured to extract a feature vector corresponding to the feature map.

[0026] The teacher network to be trained and the student network to be trained share the structures of the Backbone and the Conv+Pooler, and the parameter values of the Backbone and the Conv+Pooler of the teacher network to be trained are updated by using an exponential moving average (EMA) method on the parameter values of the Backbone and the Conv+Pooler of the student network to be trained.

[0027] In this implementation environment, after obtaining a first sample image set for image category labeling and a second sample image set without image category labeling, the teacher network to be trained performs the following operations: extracting features of each sample image in the first sample image set and the second sample image set through the Backbone layer to obtain first sample image features and second sample image features, combining features of each sample image based on the first sample image features and the second sample image features to obtain combined features of each sample image, extracting features of the combined features through the Conv+Pooler to obtain combined feature vectors, inputting the combined feature vectors into the fully connected layer for classification prediction, and obtaining prediction probabilities of whether two sample images corresponding to each combined feature vector belong to the same image category.

[0028] The student network to be trained performs the following operations: feature extraction on each sample image in the first sample image set and the second sample image set through the Backbone layer to obtain first sample image features and second sample image features, feature vector extraction on each sample image based on the first sample image features and the second sample image features through the Pooler to obtain first sample feature vectors and second image feature vectors, similarity matrix calculation based on the first sample feature vectors and the second sample feature vectors, and multi-element feature generation on each sample image based on the similarity matrix to obtain a multi-element feature set, and fusion feature vector acquisition of the fusion features by combining each image feature in the first sample image features and the second sample image features and inputting the combined features into the Conv+Pooler for feature extraction, label processing on the multi-element features containing the second sample image features and the fusion features based on the teacher network to be trained, training loss calculation based on the multi-element feature set and the fusion feature set that are labeled by the category, and parameter adjustment on the teacher network to be trained and the student network to be trained based on the training loss.

[0029] After the teacher network and the student network are trained, an image detection model can be constructed based on part of the teacher network according to actual requirements.

[0030] One or more embodiments of the image detection model training method provided in the specification are as follows:

[0031] The image detection model training method provided in this embodiment predicts the probability of whether the second sample image without image category labeling belongs to the same image category as the first sample image with image category labeling through the teacher network to be trained, labels the multi-element features and the fusion features from two levels of multi-element features and fusion features based on the category matching probability obtained by the teacher network to be trained, calculates the training loss based on the labeled multi-element feature set and the fusion feature set, and adjusts the parameters of the teacher network to be trained and the student network to be trained. In this way, the prediction of the category matching probability of the second sample image without image category labeling and the first sample image with image category labeling through the teacher network to be trained realizes the utilization of unlabeled samples, applies the student network and the teacher network to a semi-supervised learning framework, improves the effectiveness of the training samples, and improves the generalization ability of the teacher network and the student network obtained by training.

[0032] Reference Figure 2 The image detection model training method provided in this embodiment specifically includes steps S202 to S204.

[0033] In step S202, the first sample image set and the second sample image set are input into the teacher network to be trained for category matching to obtain a category matching probability.

[0034] In this embodiment, the sample images in the first sample image set are sample images after image category labeling, and the sample images in the second sample image set are sample images without image category labeling. Optionally, the first sample image set is composed of first sample images with image category labeling; and the second sample image set is composed of second sample images without image category labeling. Optionally, the first sample image set is composed of a first quantity of first sample images with image category labeling; the first quantity is obtained based on the quantity of image categories and the quantity of sample images corresponding to each image category; the quantity of sample images corresponding to each image category is equal; and the second sample image set is composed of a second quantity of second sample images without image category labeling.

[0035] For example, the first sample image set is composed of P x K images, where P represents the quantity of image categories (i.e., image IDs), and K represents the quantity of images under each image ID; and the second sample image set is composed of any M images of unknown image categories.

[0036] In this embodiment, the teacher network and the student network are composed of a feature extraction layer, a pooling layer and / or a feature conversion layer; in addition, the teacher network can also include a full connection layer; optionally, the parameter value of the feature extraction layer in the teacher network is obtained by performing exponential moving average processing on the parameter value of the feature extraction layer in the student network; and the parameter value of the feature conversion layer in the teacher network is obtained by performing exponential moving average processing on the parameter value of the feature conversion layer in the student network.

[0037] In specific implementation, in order to improve the effectiveness of training, after obtaining the initial sample image set, sample image sampling is performed on the initial sample image set to obtain the first sample image set and the second sample image set; optionally, in an optional implementation provided by this embodiment, the first sample image set and the second sample image set are obtained by the following manner:

[0038] Obtain an initial sample image set;

[0039] Perform sample image sampling on the initial sample image set to obtain the first sample image set and the second sample image set.

[0040] Specifically, the first sample image set and the second sample image set are obtained by performing sample image sampling on the initial sample image set; wherein, first sample image sampling is performed in the initial sample image set according to a first quantity, and second sample image sampling is performed in the initial sample image set according to a second quantity.

[0041] Further, after obtaining the first sample image set and the second sample image set, in order to use the sample images without image category label as effective samples and realize the training of the teacher network and the student network, in the embodiment, the teacher network to be trained is matched and predicted with the sample images without image category label and the sample images with image category label, that is, category matching, so as to realize the effective use of the sample images without image category label.

[0042] In the specific implementation process, after obtaining the first sample image set and the second sample image set, the first sample image set and the second sample image set are input into the teacher network to be trained for category matching to obtain a category matching probability; in an optional implementation provided by the embodiment, the teacher network is matched in the following manner:

[0043] The first sample image set and the second sample image set are input into a feature extraction layer for feature extraction to obtain first sample image features and second sample image features;

[0044] The first sample image features and the second sample image features are combined to obtain a combined feature set, and the combined feature set is input into a feature conversion layer for feature conversion to obtain a combined feature vector set;

[0045] The combined features in the combined feature vector set containing the feature vector corresponding to the second sample image are input into a full connection layer for classification prediction to obtain a category matching probability of whether the combined features belong to the same category.

[0046] Specifically, the feature extraction layer in the teacher network to be trained extracts features of the first sample image set and the second sample image set to obtain first sample image features and second sample image features; the teacher network to be trained combines the first sample image features and the second sample image features to obtain a combined feature set; the feature conversion layer in the teacher network to be trained converts each combined feature in the combined feature set to obtain a combined feature vector set; and the full connection layer in the teacher network to be trained performs classification prediction on the feature vector corresponding to the second sample image in the combined feature vector set to obtain a category matching probability of whether the image corresponding to the combined feature containing the second sample image belongs to the same image category.

[0047] Further, in the embodiment, in the process of combining the first sample image features and the second sample image features, the first sample image features and the second sample image features can be combined according to the similarity matrix corresponding to the first sample image set and the second sample image set. Specifically, in an optional implementation provided by the embodiment, after the first sample image set and the second sample image set are input into the feature extraction layer to perform feature extraction and obtain the first sample image features and the second sample image features, the first sample image features and the second sample image features are combined to obtain a combined feature set, and before the combined feature set is input into the feature conversion layer to perform feature conversion and obtain a combined feature vector set, the following operation can also be performed:

[0048] The first sample image features and the second sample image features are input into a pooling layer to perform pooling processing, to obtain a first sample feature vector and a second sample feature vector.

[0049] Similarity calculation is performed based on the first sample feature vector and the second sample feature vector, to obtain a similarity matrix.

[0050] Correspondingly, the combining of the first sample image features and the second sample image features to obtain the combined feature set and the input of the combined feature set into the feature conversion layer to perform feature conversion and obtain the combined feature vector set include: combining the first sample image features and the second sample image features based on the similarity matrix obtained through similarity calculation, to obtain the combined feature set, and inputting the combined feature set into the feature conversion layer to perform feature conversion and obtain the combined feature vector set.

[0051] Optionally, the category matching probability is used for label processing of the multi-element feature set generated after multi-element feature generation of the student network to be trained and the fusion feature set obtained after feature fusion.

[0052] In step S204, the first sample image set and the second sample image set are input into the student network to be trained, to perform the following operations: according to the similarity matrix corresponding to the first sample image set and the second sample image set, multi-element feature generation and feature fusion are performed on each sample image, to obtain a multi-element feature set and a fusion feature set; based on the category matching probability, the multi-element feature set and the fusion feature set, a training loss is calculated, and based on the training loss, parameter adjustment is performed on the student network to be trained and the teacher network to be trained, to construct an image detection model according to the teacher network obtained through training.

[0053] The similarity matrix comprises similarities between each sample image in the first sample image set and each sample image in the second sample image set. Optionally, the similarity matrix is obtained according to feature vectors of each sample image in the first sample image set and each sample image in the second sample image set.

[0054] The multi-element feature of any sample image in the multi-element feature set refers to a feature group composed of the any sample image and at least one associated sample image. The multi-element feature of a plurality of any sample images can be a triple group of the any sample image. The fusion feature of any sample image refers to a fusion feature obtained after fusion of the any sample image and associated sample images. The multi-element feature of any sample image is without feature fusion. The fusion feature of any sample image is obtained by feature fusion of the any sample image and associated sample images.

[0055] The image detection model comprises a model constructed by extracting a part of the teacher network for model construction according to actual needs.

[0056] In the specific implementation process, after the first sample image set and the second sample image set are obtained, the first sample image set and the second sample image set are input into the student network to be trained. The student network to be trained performs similarity calculation based on the first sample image set and the second sample image set, obtains a similarity matrix, generates multi-element features of each sample image according to the similarity matrix to obtain a multi-element feature set, and performs feature fusion on each sample image according to the similarity matrix to obtain a fusion feature set. Optionally, the multi-element feature set is composed of multi-element features of each sample image. The fusion feature set is composed of fusion features of each sample image.

[0057] In order to improve the effectiveness of the obtained multi-element feature set and fusion feature set, in this embodiment, the similarity matrix is calculated according to the first sample image set and the second sample image set. In order to improve the accuracy of the calculated similarity matrix, in an optional implementation provided in this embodiment, the similarity matrix is calculated in the following manner:

[0058] The first sample image set and the second sample image set are input into a feature extraction layer for feature extraction to obtain first sample image features and second sample image features.

[0059] The first sample image features and the second sample image features are input into a pooling layer for pooling processing to obtain first sample feature vectors and second sample feature vectors.

[0060] Similarity calculation is performed based on the first sample feature vectors and the second sample feature vectors to obtain the similarity matrix.

[0061] Specifically, the first sample image set and the second sample image set are input into a feature extraction layer of the student network to be trained for feature extraction to obtain first sample image features and second sample image features, and then the first sample image features and the second sample image features are input into a pooling layer of the student network to be trained for pooling processing to obtain first sample feature vectors and second sample feature vectors. Finally, the student network to be trained performs similarity calculation based on the first sample feature vectors and the second sample feature vectors to obtain a similarity matrix composed of similarities between two sample images in the first sample image and the second sample image.

[0062] Further, after the similarity matrix is obtained, according to the similarity matrix, multi-feature sets are obtained by performing multi-feature generation on each sample image, and a fusion feature set is obtained by performing feature fusion on each sample image according to the similarity matrix; in an optional implementation provided by the embodiment, the process of performing multi-feature generation and feature fusion on each sample image according to the similarity matrix corresponding to the first sample image set and the second sample image set to obtain the multi-feature set and the fusion feature set is implemented in the following manner:

[0063] According to the similarity matrix, multi-features of each sample image in the first sample image set and the second sample image set are generated to obtain the multi-feature set.

[0064] According to the similarity matrix, the fusion feature set is obtained by performing feature fusion on each sample image.

[0065] In order to improve the efficiency of parameter adjustment and avoid the loss of parameter adjustment due to the selection of images of the same or the same image category for training, in the embodiment, the selection and determination of sample images associated with any sample image are performed from the aspect of "difficulty"; specifically, in an optional implementation provided by the embodiment, the generation process of multi-features of any sample image is described in detail taking a triple as an example:

[0066] In the first sample image set and the second sample image set, a first target sample image of an image category of the any sample image is selected, and the similarity of the first target sample image to the any sample image is less than other similarities.

[0067] A second target sample image of an image category other than the any sample image is selected, and the similarity of the second target sample image to the any sample image is greater than other similarities.

[0068] Multi-features of the any sample image are generated based on the any sample image, the first target sample image, and the second target sample image.

[0069] In an optional implementation of the embodiment, the fusion feature of any sample feature vector in the sample image is generated in the following manner:

[0070] The any sample image and the corresponding first target sample image are input into a feature fusion layer for feature fusion to obtain a first fusion feature of the any sample image; and,

[0071] The any sample image and the corresponding second target sample image are input into the feature fusion layer for feature fusion to obtain a second fusion feature of the any sample image, and the first fusion feature and the second fusion feature are taken as the fusion feature of the any sample image.

[0072] Specifically, in order to improve the effectiveness of the obtained multi-element feature and fusion feature on model training, for any sample image, a triplet is generated from the sample image, a sample image with the lowest similarity in the same image category, and a sample image with the highest similarity in a different image category, and feature fusion is performed on the sample image and the sample image with the lowest similarity in the same image category and on the sample image and the sample image with the lowest similarity in the different image category.

[0073] Optionally, any sample image and corresponding multi-element feature are one-to-one; any sample image and corresponding fusion feature are one-to-many.

[0074] In addition, according to the similarity matrix corresponding to the first sample image set and the second sample image set, the multi-element feature generation and feature fusion of each sample image to obtain the multi-element feature set and the fusion feature set can also be replaced by determining the first target sample image and the second target sample image corresponding to each sample image according to the similarity matrix corresponding to the first sample image set and the second sample image set and the category matching probability, and performing multi-element feature generation and feature fusion according to each sample image and the corresponding first target sample image and second target sample image to obtain the multi-element feature set and the fusion feature set, and forming a new implementation with other processing procedures provided in the embodiment; optionally, the first target sample image is a sample image with a lower similarity than other sample images in the same image category of the corresponding sample image; and the second target sample image is a sample image with a higher similarity than other sample images in a different image category of the corresponding sample image.

[0075] It should be noted that the multi-element feature is generated from the feature vector dimension, and the fusion feature is generated from the feature dimension; in an optional implementation of the embodiment, according to the similarity matrix corresponding to the first sample image set and the second sample image set, the multi-element feature generation and feature fusion of each sample image to obtain the multi-element feature set and the fusion feature set can also be implemented in the following manner:

[0076] performing multi-element feature generation on each of the first sample feature vector and the second sample feature vector based on the similarity matrix, to obtain the multi-element feature set;

[0077] performing feature fusion on each of the first sample image feature and the second sample image feature based on the similarity matrix, to obtain the fusion feature set.

[0078] In addition, the process of performing multi-element feature generation and feature fusion on each sample image based on the similarity matrix corresponding to the first sample image set and the second sample image set to obtain the multi-element feature set and the fusion feature set can also be replaced by a process of performing multi-element feature generation and feature fusion on each sample image based on the category matching probability and the similarity matrix corresponding to the first sample image set and the second sample image set to obtain the multi-element feature set and the fusion feature set, and combined with other processing procedures provided in the embodiment to form a new implementation manner.

[0079] It should be further noted that, since the second sample image is not labeled with an image category, the multi-element feature generation and feature fusion can also be performed only on the first sample image to obtain the multi-element feature set and the fusion feature set; that is, the process of performing multi-element feature generation and feature fusion on each sample image based on the similarity matrix corresponding to the first sample image set and the second sample image set to obtain the multi-element feature set and the fusion feature set can also be replaced by a process of performing multi-element feature generation and feature fusion on each first image sample in the first sample image set based on the similarity matrix corresponding to the first sample image set and the second sample image set to obtain the multi-element feature set and the fusion feature set, and combined with other processing procedures provided in the embodiment to form a new implementation manner.

[0080] In addition to the above-provided determination of the first target sample image and the second target sample image, multi-element feature generation and feature fusion based on the sample image and the corresponding first target sample image and second target sample image, the process of performing multi-element feature generation and feature fusion on each sample image based on the similarity matrix corresponding to the first sample image set and the second sample image set to obtain the multi-element feature set and the fusion feature set can also be performed in the following manner:

[0081] performing multi-element feature generation on the sample image other than the any sample image in the first sample image set and the second sample image set and the any sample image; and,

[0082] extracting a sample image other than the any sample image from the first sample image set and the second sample image set, and performing feature fusion between the sample image and the any sample image;

[0083] Optionally, the any sample image is another different sample image other than the any sample image.

[0084] In the specific implementation process, after obtaining the multi-element feature set and the fusion feature set, a training loss is calculated based on the category matching probability, the multi-element feature set and the fusion feature set, and the parameters of the student network to be trained and the teacher network to be trained are adjusted based on the training loss to obtain the trained student network and teacher network, and the image detection model is constructed based on part of the teacher network. Specifically, in the process of calculating the training loss based on the category matching probability, the multi-element feature set and the fusion feature set, that is, the multi-element features containing the second sample image in the multi-element feature set are marked based on the category matching probability, and the fusion features containing the second sample image in the fusion feature set are marked based on the category matching probability, and the training loss is calculated based on the marked multi-element feature set and fusion feature set.

[0085] In order to improve the effectiveness of the obtained training loss from the aspects of the multi-element feature set and the fusion feature set, in an optional implementation provided by the embodiment, the process of calculating the training loss based on the category matching probability, the multi-element feature set and the fusion feature set is implemented in the following manner:

[0086] Based on the category matching probability, the multi-element features containing the second sample image in the multi-element feature set are marked, and the fusion features containing the second sample image in the fusion feature set are marked to obtain the multi-element feature set and the fusion feature set containing classification marks;

[0087] The multi-element feature loss is calculated according to the multi-element feature set containing classification marks, and the binary classification loss is calculated according to the fusion feature set containing classification marks;

[0088] The training loss is calculated based on the multi-element feature loss and the binary classification loss.

[0089] Optionally, the multi-element features not containing the second sample image in the multi-element feature set and the fusion features not containing the second sample image in the fusion feature set are marked based on the image categories marked by each first sample image in the first sample image set;

[0090] The multi-element features containing the second sample image in the multi-element feature set and the fusion features containing the second sample image in the fusion feature set are marked based on the category matching probability.

[0091] Specifically, the multi-element features in the multi-element feature set and the fusion features in the fusion feature set are labeled based on the category matching probability, to obtain a multi-element feature set and a fusion feature set containing classification labels, the multi-element feature loss is calculated according to the multi-element feature set containing classification labels, the binary classification loss is calculated according to the fusion feature set containing classification labels, and the training loss is calculated based on the multi-element feature loss and the binary classification loss.

[0092] In addition, the calculation of the training loss based on the category matching probability, the multi-element feature set and the fusion feature set can be replaced by the calculation of the training loss based on the category matching probability, the image category of the first sample image, the multi-element feature set and the fusion feature set, or the labeling of the multi-element features in the multi-element feature set and the fusion features in the fusion feature set based on the category matching probability, to obtain a multi-element feature set and a fusion feature set containing classification labels, which can be replaced by the labeling of the multi-element features in the multi-element feature set and the fusion features in the fusion feature set based on the category matching probability and the image category of the first sample image.

[0093] It should be noted that the labels of the multi-element features and the fusion features are "0 (not belonging to the same image category)" and "1 (belonging to the same image category)". In the labeling process based on the category matching probability, if the category matching probability is higher than a preset threshold, it means that the corresponding two images belong to the same image category; if the category matching probability is less than or equal to the preset threshold, it means that the corresponding two images do not belong to the same image category.

[0094] After the training loss is calculated, the parameters of the student network to be trained and the teacher network to be trained are adjusted based on the training loss until the student network to be trained and the teacher network to be trained converge; wherein, during the parameter adjustment process, the Backbone and Conv+Pooler of the student network to be trained are adjusted, and then the exponential moving average method is used to adjust the Backbone and Conv+Pooler of the teacher network to be trained.

[0095] In addition, corresponding to the above-mentioned process of performing multi-feature generation and feature fusion on any sample image pair other than the any sample image extracted from the first sample image set and the second sample image set, in the process of calculating the training loss based on the category prediction probability, the multi-feature set and the fused feature set, first, the multi-feature, the image category and the fused feature are labeled based on the category prediction probability, the image category corresponding to the multi-feature and the image category corresponding to the fused feature, and then the training loss is calculated based on the multi-feature set and the fused feature set carrying the classification label. Optionally, in order to train from the "difficult" level, in the process of calculating the training loss based on the multi-feature set and the fused feature set carrying the classification label, a multi-feature subset composed of multi-features composed of sample images, first target sample images associated with the sample images and second target sample images associated with the sample images can be extracted from the multi-feature set, and a fused feature subset composed of fused features obtained by fusing the sample images and the associated first target sample images and / or the sample images and / or the second target sample images can be extracted from the fused feature set; the training loss is calculated based on the multi-feature subset and the fused feature subset.

[0096] It should be noted that in the process of calculating the training loss based on the multi-feature set and the fused feature set carrying the classification label, the training loss can be calculated for the multi-feature and the fused feature corresponding to each sample image, and then the parameters of the student network to be trained and the teacher network to be trained are adjusted; or the multi-feature loss can be calculated through the multi-feature set, and the binary classification loss can be calculated through the fused feature set, and the training loss can be calculated based on the multi-feature loss and the binary classification loss. The training loss can also be calculated based on other weights, which is not limited in the present embodiment.

[0097] In the specific implementation process, after the student network and the teacher network are trained, to realize image detection, an image detection model is constructed according to the trained teacher network; optionally, the image detection model can include part of the teacher network, or all of the teacher network, or a combination of part or all of the teacher network and part of the student network; for example, if image classification is to be performed, the image detection model can be constructed using the Backbone and the pooling layer in the teacher network; in an optional implementation provided by the present embodiment, the following method is used for image category detection:

[0098] Obtaining at least two images to be detected;

[0099] Inputting the at least two images to be detected into an image detection model constructed based on the feature extraction layer and the pooling layer in the trained teacher network to perform image detection and obtain a feature vector;

[0100] Calculating the category matching probability of the at least two images to be detected based on the feature vector.

[0101] Specifically, the image detection model constructed by the feature extraction layer and the pooling layer in the teacher network obtained after training is input with the to-be-detected image to perform image detection, and a feature vector is obtained, and then the class matching probability of the to-be-detected image is calculated based on the feature vector.

[0102] It should be noted that the student network and the teacher network obtained by training in the embodiment can be used to construct an image detection model according to actual needs, and can be applied to various scenes in addition to the class matching scene, such as a similarity sorting scene and a candidate screening scene, which are not limited in the embodiment.

[0103] It should be further noted that in the embodiment, step S202 and step S204 can be executed in the order of step S202 first and then step S204, or in the order of step S204 first and then step S202, or simultaneously, and the execution order of step S202 and step S204 is not limited in the embodiment.

[0104] One or more embodiments of the image detection processing method provided in the specification are as follows:

[0105] The related content of the image detection processing method provided in the embodiment is similar to the related content of the training method of the image detection model provided in the above embodiment. When reading the embodiment, the related content of the above embodiment or the related content of the above embodiment can be modified as appropriate. The embodiment will not be repeated here.

[0106] Referring to Figure 3 The image detection processing method provided in the embodiment specifically includes steps S302 to S306.

[0107] Step S302: obtaining at least two to-be-detected images.

[0108] Step S304: inputting the at least two to-be-detected images into an image detection model to perform image detection and obtain a feature vector.

[0109] Optionally, the image detection model is constructed based on the trained teacher network.

[0110] Step S306: calculating class matching probabilities of the at least two to-be-detected images based on the feature vector.

[0111] Optionally, the teacher network is trained in cooperation with the student network; the cooperation training comprises: performing category matching on the first sample image set and the second sample image set based on the teacher network to be trained, to obtain a sample category matching probability; and performing parameter adjustment on the teacher network to be trained and the student network to be trained based on the student network to be trained, according to the sample category matching probability and a multivariate feature set and a fusion feature set corresponding to the first sample image set and the second sample image set, to obtain a trained teacher network.

[0112] In this embodiment, whether the templates used by the documents are the same template can be identified by collecting multiple images of one or more documents. Optionally, steps S302 to S306 can also be replaced by: obtaining multiple document images obtained by image collection on at least two documents; inputting the multiple document images into an image detection model for image detection to obtain feature vectors; and calculating a category matching probability between each two of the multiple document images based on the feature vectors. Optionally, the multiple document images include at least two.

[0113] One or more embodiments of the training device of the image detection model provided in the specification are as follows:

[0114] In the above embodiments, a training method of an image detection model is provided, and a training device of the image detection model is also provided, which will be described below with reference to the accompanying drawings.

[0115] Reference is made to Figure 4 which shows a schematic diagram of an embodiment of the training device of the image detection model provided in this embodiment.

[0116] Since the device embodiment corresponds to the method embodiment, the description is relatively simple, and the related parts can be seen in the above-mentioned corresponding description of the method embodiment. The device embodiments described below are only illustrative.

[0117] This embodiment provides a training device of an image detection model, which comprises:

[0118] The category matching module 402 is configured to input the first sample image set and the second sample image set into the teacher network to be trained for category matching to obtain a category matching probability; the first sample image set is composed of first sample images that have been labeled with image categories; and the second sample image set is composed of second sample images that have not been labeled with image categories.

[0119] The first sample image set and the second sample image set are input into the student network to be trained to run the following modules:

[0120] The feature fusion module 404 is configured to perform multi-element feature generation and feature fusion on each sample image according to a similarity matrix corresponding to the first sample image set and the second sample image set, to obtain a multi-element feature set and a fused feature set.

[0121] The parameter adjustment module 406 is configured to calculate a training loss based on the category matching probability, the multi-element feature set and the fused feature set, and to perform parameter adjustment on the student network to be trained and the teacher network to be trained based on the training loss, to construct an image detection model according to the teacher network obtained through training.

[0122] One or more embodiments of the image detection processing apparatus provided in the specification are as follows.

[0123] In the above embodiments, an image detection processing method is provided, and a corresponding image detection processing apparatus is also provided, which will be described below with reference to the accompanying drawings.

[0124] Reference Figure 5 which shows a schematic diagram of an embodiment of an image detection processing apparatus provided in the present embodiment.

[0125] Since the apparatus embodiment corresponds to the method embodiment, the description is relatively simple, and the related parts can be referred to the corresponding description of the method embodiment provided above. The apparatus embodiment described below is only illustrative.

[0126] The present embodiment provides an image detection processing apparatus, which comprises:

[0127] The image acquisition module 502 is configured to acquire at least two images to be detected.

[0128] The image detection module 504 is configured to input the at least two images to be detected into an image detection model for image detection, to obtain a feature vector; the image detection model is constructed based on a trained teacher network.

[0129] The probability calculation module 506 is configured to calculate a category matching probability of the at least two images to be detected based on the feature vector.

[0130] The teacher network is trained in cooperation with a student network; the cooperation training comprises: performing category matching on a first sample image set and a second sample image set based on a teacher network to be trained, to obtain a sample category matching probability; performing parameter adjustment on the teacher network to be trained and the student network to be trained based on the sample category matching probability, a multi-element feature set and a fused feature set corresponding to the first sample image set and the second sample image set, to obtain a trained teacher network.

[0131] One or more implementations of a training device of an image detection model provided by the specification are as follows:

[0132] Based on the same technical concept as described above, one or more embodiments of the specification also provide a training device of an image detection model for executing the training method of the image detection model provided above, Figure 6 A structural schematic diagram of a training device of an image detection model provided by one or more embodiments of the specification.

[0133] The training device of an image detection model provided by the embodiment includes:

[0134] As shown in Figure 6 The training device of an image detection model can have great differences due to different configurations or performances, and can include one or more processors 601 and memories 602, and the memories 602 can store one or more storage applications or data. Among them, the memory 602 can be temporary storage or persistent storage. The application stored in the memory 602 can include one or more modules (not shown in the figure), and each module can include a series of computer executable instructions in the training device of the image detection model. Further, the processor 601 can be configured to communicate with the memory 602 and execute a series of computer executable instructions in the memory 602 on the training device of the image detection model. The training device of the image detection model can also include one or more power supplies 603, one or more wired or wireless network interfaces 604, one or more input / output interfaces 605, one or more keyboards 606, etc.

[0135] In one specific embodiment, the training device of the image detection model includes a memory and one or more programs, wherein one or more programs are stored in the memory, and one or more programs can include one or more modules, and each module can include a series of computer executable instructions in the training device of the image detection model, and the one or more processors are configured to execute the one or more programs include computer executable instructions for:

[0136] The first sample image set and the second sample image set are input into the teacher network to be trained to perform category matching to obtain a category matching probability; the first sample image set is composed of first sample images labeled with image categories; the second sample image set is composed of second sample images without image category labeling;

[0137] The first sample image set and the second sample image set are input into the student network to be trained to perform the following operations:

[0138] According to the similarity matrix corresponding to the first sample image set and the second sample image set, multi-element feature generation and feature fusion are performed on each sample image to obtain a multi-element feature set and a fused feature set;

[0139] Based on the category matching probability, the multi-element feature set and the fused feature set, a training loss is calculated, and the parameters of the student network to be trained and the teacher network to be trained are adjusted based on the training loss to construct an image detection model according to the teacher network obtained by training.

[0140] One or more embodiments of an image detection processing device provided by the present specification are as follows:

[0141] According to the same technical concept as described above, one or more embodiments of the present specification further provide an image detection processing device for executing the image detection processing method provided above, Figure 7 A structural schematic diagram of an image detection processing device provided by one or more embodiments of the present specification.

[0142] The image detection processing device provided in the present embodiment comprises:

[0143] As shown in Figure 7 The image detection processing device can have great differences due to different configurations or performances, and can include one or more processors 701 and memories 702. The memories 702 can store one or more storage applications or data. The memories 702 can be temporary storage or persistent storage. The applications stored in the memories 702 can include one or more modules (not shown in the figure), and each module can include a series of computer executable instructions in the image detection processing device. Furthermore, the processor 701 can be configured to communicate with the memory 702 and execute a series of computer executable instructions in the memory 702 on the image detection processing device. The image detection processing device can further include one or more power supplies 703, one or more wired or wireless network interfaces 704, one or more input / output interfaces 705, one or more keyboards 706, etc.

[0144] In one specific embodiment, the image detection processing device includes a memory and one or more programs, wherein one or more programs are stored in the memory, and one or more programs can include one or more modules, and each module can include a series of computer executable instructions in the image detection processing device, and the one or more programs configured to be executed by one or more processors include computer executable instructions for:

[0145] obtaining at least two images to be detected;

[0146] inputting the at least two images to be detected into an image detection model for image detection to obtain a feature vector; the image detection model is constructed based on a trained teacher network;

[0147] calculating a class matching probability of the at least two images to be detected based on the feature vector;

[0148] The teacher network is trained in cooperation with a student network; the cooperation training includes: performing class matching on a first sample image set and a second sample image set based on a teacher network to be trained to obtain a sample class matching probability; and performing parameter adjustment on the teacher network to be trained and the student network to be trained based on the sample class matching probability and a multivariate feature set and a fusion feature set corresponding to the first sample image set and the second sample image set according to the teacher network to be trained and the student network to be trained to obtain a trained teacher network.

[0149] One or more embodiments of the storage medium provided in the specification are as follows:

[0150] Based on the same technical concept, the training method of the image detection model described above, one or more embodiments of the specification also provide a storage medium.

[0151] The storage medium provided in the embodiment is used to store computer executable instructions, and the computer executable instructions are executed by a processor to implement the following processes:

[0152] inputting a first sample image set and a second sample image set into a teacher network to be trained for class matching to obtain a class matching probability; the first sample image set is composed of first sample images labeled with image classes; and the second sample image set is composed of second sample images without image class labeling;

[0153] inputting the first sample image set and the second sample image set into a student network to be trained to perform the following operations:

[0154] According to a similarity matrix corresponding to the first sample image set and the second sample image set, multi-element feature generation and feature fusion are performed on each sample image to obtain a multi-element feature set and a fused feature set;

[0155] Based on the category matching probability, the multi-element feature set and the fused feature set, a training loss is calculated, and the parameters of the student network to be trained and the teacher network to be trained are adjusted based on the training loss to construct an image detection model according to the teacher network obtained through training.

[0156] It should be noted that the embodiments of the storage medium in the present specification and the embodiments of the training method of the image detection model in the present specification are based on the same inventive concept, and therefore the specific implementation of this embodiment can be referred to the implementation of the corresponding method described above, and the repeated parts will not be described herein.

[0157] Another storage medium provided by the present specification has one or more embodiments as follows:

[0158] According to the image detection processing method described above, based on the same technical concept, the present specification one or more embodiments also provide a storage medium.

[0159] The storage medium provided by the present embodiment is used to store computer executable instructions, and the computer executable instructions realize the following processes when executed by a processor:

[0160] Obtain at least two images to be detected;

[0161] Input the at least two images to be detected into an image detection model for image detection to obtain a feature vector; the image detection model is constructed based on a trained teacher network;

[0162] Calculate the category matching probability of the at least two images to be detected based on the feature vector;

[0163] Wherein, the teacher network and the student network are trained in cooperation; the cooperative training includes: based on the teacher network to be trained, performing category matching on a first sample image set and a second sample image set to obtain a sample category matching probability; based on the student network to be trained, adjusting the parameters of the teacher network to be trained and the student network to be trained according to the sample category matching probability, a multi-element feature set and a fused feature set corresponding to the first sample image set and the second sample image set, to obtain a trained teacher network.

[0164] It should be noted that the embodiments of the storage medium in the present specification and the embodiments of the image detection processing method in the present specification are based on the same inventive concept, and therefore the specific implementation of this embodiment can be referred to the implementation of the corresponding method described above, and the repeated parts will not be described herein.

[0165] The various embodiments in the specification are described in progressive manner, and the same or similar parts among the various embodiments can be referred to each other, and each embodiment focuses on the difference from other embodiments, such as the device embodiment, the equipment embodiment and the storage medium embodiment, which are similar to the method embodiment, so the description is relatively simple, and the related content in the device embodiment, the equipment embodiment and the storage medium embodiment can be read by referring to the part of the description of the method embodiment.

[0166] The above describes specific embodiments of the specification. Other embodiments are within the scope of the appended claims. In some cases, the acts or steps recited in the claims can be performed in a different order than the order in which they are recited and still achieve desirable results. In addition, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve the desired results. In certain implementations, multitasking and parallel processing can be advantageous or necessary.

[0167] In the 1930s, it was clear to distinguish whether an improvement in a technology was in hardware (e.g., improvement in circuit structure of diodes, transistors, switches, etc.) or in software (e.g., improvement in method flow). However, as technology has evolved, many improvements in method flow today can be considered as direct improvements in hardware circuit structure. Designers almost always obtain the corresponding hardware circuit structure by programming the improved method flow into the hardware circuit. Therefore, it cannot be said that an improvement in a method flow cannot be implemented by a hardware entity module. For example, a programmable logic device (PLD) (e.g., a field programmable gate array (FPGA)) is an integrated circuit whose logic function is determined by user programming of the device. A digital system is "integrated" on a PLD by the designer programming it, rather than by asking a chip manufacturer to design and fabricate a custom integrated circuit chip. Moreover, instead of manually fabricating an integrated circuit chip, this programming is now mostly implemented by "logic compiler" software, which is similar to software compilers used in program development, and the original code to be compiled is written in a specific programming language, called a hardware description language (HDL), of which there are many, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, RHDL (Ruby Hardware Description Language), etc., the most commonly used being VHDL (Very-High-Speed Integrated Circuit Hardware Description Language) and Verilog. It should be clear to those skilled in the art that, by simply logically programming a method flow in one of the above hardware description languages and programming it into an integrated circuit, a hardware circuit implementing the logical method flow can be easily obtained.

[0168] The controller can be implemented in any suitable way, for example, the controller can take the form of a microprocessor or processor and a computer readable medium storing computer readable program code, e.g. software or firmware, executable by the (micro)processor, logic gates, switches, an application specific integrated circuit (ASIC), a programmable logic controller and an embedded microcontroller, examples of which include but are not limited to the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20 and Silicone Labs C8051F320, the memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art will also know that, in addition to being implemented in pure computer readable program code, the controller can equally well be implemented to perform the same functions using logic gates, switches, an application specific integrated circuit, a programmable logic controller and an embedded microcontroller, etc. by means of a logical programming of the method steps. The controller can thus be considered as a hardware component, and the means comprised therein for performing the various functions can be considered as structures within the hardware component. Alternatively, the means for performing the various functions can even be considered as both a software module implementing the method and a structure within the hardware component.

[0169] The systems, apparatuses, modules or units illustrated by the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, the computer can be a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0170] For the sake of description, the above apparatuses are described in various units with functions respectively. Of course, the functions of the units can be implemented in one or more software and / or hardware in implementing the embodiments of the present specification.

[0171] Those skilled in the art will understand that one or more embodiments of the present specification can be provided as a method, a system or a computer program product. Therefore, one or more embodiments of the present specification can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present specification can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROMs, optical storage devices, etc.) containing computer usable program code.

[0172] The specification is presented with reference to flow diagrams and / or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the specification. It will be understood that each block of the flow diagrams and / or block diagrams, and combinations of blocks in the flow diagrams and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general purpose computer, special purpose computer, embedded processing element or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flow diagrams and / or block diagrams block or blocks. Figure 1 The flow diagrams and / or block diagrams in the specification can present a method, apparatus or computer program product according to embodiments of the specification. Flow diagrams and / or block diagrams can also present a method, apparatus or computer program product to achieve functions specified in flow diagrams and / or block diagrams block or blocks. Figure 1 The flow diagrams and / or block diagrams in the specification can present a method, apparatus or computer program product according to embodiments of the specification. Flow diagrams and / or block diagrams can also present a method, apparatus or computer program product to achieve functions specified in flow diagrams and / or block diagrams block or blocks.

[0173] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instructions which implement the flow diagrams and / or block diagrams block or blocks. Figure 1 The flow diagrams and / or block diagrams in the specification can present a method, apparatus or computer program product according to embodiments of the specification. Flow diagrams and / or block diagrams can also present a method, apparatus or computer program product to achieve functions specified in flow diagrams and / or block diagrams block or blocks. Figure 1 The flow diagrams and / or block diagrams in the specification can present a method, apparatus or computer program product according to embodiments of the specification. Flow diagrams and / or block diagrams can also present a method, apparatus or computer program product to achieve functions specified in flow diagrams and / or block diagrams block or blocks.

[0174] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the flow diagrams and / or block diagrams block or blocks. Figure 1 The flow diagrams and / or block diagrams in the specification can present a method, apparatus or computer program product according to embodiments of the specification. Flow diagrams and / or block diagrams can also present a method, apparatus or computer program product to achieve functions specified in flow diagrams and / or block diagrams block or blocks. Figure 1 The flow diagrams and / or block diagrams in the specification can present a method, apparatus or computer program product according to embodiments of the specification. Flow diagrams and / or block diagrams can also present a method, apparatus or computer program product to achieve functions specified in flow diagrams and / or block diagrams block or blocks.

[0175] In a typical configuration, a computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0176] The memory can include non-persistent memory and / or storage mechanisms such as, for example, random access memory (RAM), non-volatile memory (NVM), and / or a persistent memory such as, for example, read-only memory (ROM) or flash memory. The memory is an example of computer-readable media.

[0177] Computer-readable media includes permanent and non-permanent, movable and non-movable media that can be implemented by any method or technology to store information. The information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassette, magnetic tape disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information accessible to a computing device. According to the definition herein, computer-readable media does not include transitory media such as modulated data signals and carriers.

[0178] It should also be noted that the terms "comprising", "containing", or any other variant thereof are intended to encompass non-exclusive inclusion, such that processes, methods, articles or devices that include a series of elements not only include those elements, but also include other elements not explicitly listed or inherent to such processes, methods, articles or devices. Without more limitations, the element defined by the statement "comprising a" does not exclude the presence of additional identical elements in the process, method, article or device that includes the element.

[0179] One or more embodiments of the present specification can be described in the general context of computer-executable instructions being executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, and the like that perform particular tasks or implement particular abstract data types. One or more embodiments of the present specification can also be practiced in a distributed computing environment, in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media, including storage devices.

[0180] Each embodiment in the present specification is described in a progressive manner, and the same or similar parts between each embodiment can be referred to each other, and each embodiment focuses on the difference from other embodiments. In particular, for system embodiments, since they are basically similar to method embodiments, the description is relatively simple, and the relevant parts can be referred to the part of the method embodiment.

[0181] The above merely provides the example of the present document and is not intended to limit the present document. For those skilled in the art, the present document can have various modifications and changes. Any modification, equivalent replacement, improvement, etc. within the spirit and principle of the present document shall be included in the scope of claims of the present document.

Claims

1. A method for training an image detection model, comprising: inputting a first set of sample images and a second set of sample images into a teacher network to be trained to perform category matching, to obtain a category matching probability; the first set of sample images is composed of first sample images that have been labeled with image categories; the second set of sample images is composed of second sample images that have not been labeled with image categories; inputting the first set of sample images and the second set of sample images into a student network to be trained to perform the following operations: performing multi-element feature generation and feature fusion on each sample image according to a similarity matrix corresponding to the first set of sample images and the second set of sample images, to obtain a set of multi-element features and a set of fused features; performing label processing on the multi-element features in the set of multi-element features and the fused features in the set of fused features based on the category matching probability, calculating a training loss based on the obtained set of multi-element features and the set of fused features containing classification labels, and adjusting parameters of the student network to be trained and the teacher network to be trained based on the training loss, to construct an image detection model from the obtained teacher network after training.

2. The method for training an image detection model according to claim 1, wherein the similarity matrix is calculated in the following manner: inputting the first set of sample images and the second set of sample images into a feature extraction layer to perform feature extraction, to obtain first sample image features and second sample image features; inputting the first sample image features and the second sample image features into a pooling layer to perform pooling processing, to obtain first sample feature vectors and second sample feature vectors; performing similarity calculation based on the first sample feature vectors and the second sample feature vectors, to obtain the similarity matrix.

3. The method for training an image detection model according to claim 1, wherein the performing multi-element feature generation and feature fusion on each sample image according to a similarity matrix corresponding to the first set of sample images and the second set of sample images, to obtain a set of multi-element features and a set of fused features, comprises: performing multi-element feature generation on each sample image in the first set of sample images and the second set of sample images based on the similarity matrix, to obtain the set of multi-element features; performing feature fusion on each sample image according to the similarity matrix, to obtain the set of fused features.

4. The method for training an image detection model according to claim 3, wherein the multi-element features of any sample image in the each sample image are generated in the following manner: In the first sample image set and the second sample image set, a first target sample image with a similarity less than other similarities to the any sample image under an image category of the any sample image is screened out; and filtering out a second target sample image that has a similarity greater than other similarities with the any sample image, other than the image category; generating multi-element features of the any sample image based on the any sample image, the first target sample image, and the second target sample image.

5. The method for training an image detection model according to claim 3, wherein the fused features of any sample image in the each sample image are generated in the following manner: inputting the any sample image and the corresponding first target sample image into a feature fusion layer to perform feature fusion, to obtain first fused features of the any sample image; and, The any sample image and the corresponding second target sample image are input into the feature fusion layer for feature fusion, to obtain second fusion features of the any sample image, and the first fusion features and the second fusion features are taken as fusion features of the any sample image.

6. The training method of the image detection model according to claim 1, wherein the training loss is calculated based on the obtained multi-feature set and the fusion feature set containing classification labels, and the calculation comprises: calculating a multi-feature loss based on the multi-feature set containing classification labels, and calculating a binary classification loss based on the fusion feature set containing classification labels; calculating the training loss based on the multi-feature loss and the binary classification loss.

7. The training method of the image detection model according to claim 6, wherein the multi-feature set does not contain multi-features of the second sample image, and the fusion feature set does not contain fusion features of the second sample image, and the label processing is based on the image categories of the labels of each first sample image in the first sample image set; the multi-feature set contains multi-features of the second sample image, and the fusion feature set contains fusion features of the second sample image, and the label processing is based on the category matching probability.

8. The training method of the image detection model according to claim 1, wherein the category matching comprises: inputting the first sample image set and the second sample image set into a feature extraction layer for feature extraction, to obtain first sample image features and second sample image features; combining the first sample image features and the second sample image features to obtain a combined feature set, and inputting the combined feature set into a feature conversion layer for feature conversion, to obtain a combined feature vector set; inputting a combined feature containing a feature vector corresponding to a second sample image in the combined feature vector set into a full connection layer for classification prediction, to obtain a category matching probability of whether the combined features belong to the same category.

9. The training method of the image detection model according to claim 8, wherein the parameter value of the feature extraction layer in the teacher network is obtained by performing exponential moving average processing on the parameter value of the feature extraction layer in the student network; the parameter value of the feature conversion layer in the teacher network is obtained by performing exponential moving average processing on the parameter value of the feature conversion layer in the student network.

10. The training method of the image detection model according to claim 1, wherein the first sample image set is composed of a first number of first sample images labeled with image categories; the first number is obtained based on the number of image categories and the number of sample images corresponding to each image category; and the number of sample images corresponding to each image category is equal; the second sample image set is composed of a second number of second sample images without image category labels.

11. The training method of the image detection model according to claim 1, further comprising: obtaining at least two images to be detected; inputting the at least two images to be detected into an image detection model constructed based on the feature extraction layer and the pooling layer in the trained teacher network for image detection, to obtain a feature vector. Calculate a class matching probability of the at least two images to be detected based on the feature vector.

12. The method of claim 2, wherein the generating multi-element features and the fusing features of each sample image based on the similarity matrix between the first sample image set and the second sample image set comprises: generating multi-element features of each sample feature vector in the first sample feature vector and the second sample feature vector based on the similarity matrix to obtain the multi-element feature set; and fusing each sample image feature in the first sample image feature and the second sample image feature based on the similarity matrix to obtain the fused feature set.

13. The method of claim 1, wherein the inputting the first sample image set and the second sample image set into the teacher network to be trained to obtain a class matching probability is performed before the class matching probability is obtained.

14. An image detection processing method, comprising: obtaining at least two images to be detected; inputting the at least two images to be detected into an image detection model to obtain a feature vector; the image detection model being constructed based on a trained teacher network; calculating a class matching probability of the at least two images to be detected based on the feature vector; wherein the teacher network is trained in cooperation with a student network; and the cooperation training comprises: obtaining a sample class matching probability by performing class matching on a first sample image set and a second sample image set based on a teacher network to be trained; performing label processing on multi-element features in a multi-element feature set and fused features in a fused feature set based on a student network to be trained according to the sample class matching probability; and adjusting parameters of the teacher network to be trained and the student network to be trained based on a training loss calculated based on the multi-element feature set and the fused feature set obtained by including classification labels, to obtain the trained teacher network.

15. An image detection model training device, comprising: a class matching module configured to input a first sample image set and a second sample image set into a teacher network to be trained to perform class matching and obtain a class matching probability; the first sample image set being composed of first sample images labeled with image classes; and the second sample image set being composed of second sample images not labeled with image classes; and the first sample image set and the second sample image set being input into a student network to be trained to run the following modules: a feature fusion module configured to generate multi-element features and fuse features of each sample image based on a similarity matrix corresponding to the first sample image set and the second sample image set to obtain a multi-element feature set and a fused feature set. ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ The parameter adjustment module is configured to mark the multi-element features in the multi-element feature set and the fusion features in the fusion feature set based on the category matching probability, calculate a training loss based on the obtained multi-element feature set and fusion feature set containing category labels, and adjust parameters of the student network to be trained and the teacher network to be trained based on the training loss, so as to construct an image detection model according to the trained teacher network.

16. An image detection processing apparatus, comprising: an image acquisition module configured to acquire at least two images to be detected; an image detection module configured to input the at least two images to be detected into an image detection model for image detection to obtain a feature vector; the image detection model is constructed based on a trained teacher network; a probability calculation module configured to calculate a category matching probability of the at least two images to be detected based on the feature vector; wherein the teacher network is trained in cooperation with a student network; the cooperation training comprises: performing category matching on a first sample image set and a second sample image set based on a teacher network to be trained to obtain a sample category matching probability; performing marking processing on multi-element features in a multi-element feature set and fusion features in a fusion feature set corresponding to the first sample image set and the second sample image set based on a student network to be trained according to the sample category matching probability, calculating a training loss based on the obtained multi-element feature set and fusion feature set containing category labels, and adjusting parameters of the teacher network to be trained and the student network to be trained based on the training loss to obtain a trained teacher network.

17. A training device of an image detection model, comprising: a processor; and a memory configured to store computer executable instructions which, when executed, cause the processor to: input a first sample image set and a second sample image set into a teacher network to be trained for category matching to obtain a category matching probability; the first sample image set is composed of first sample images labeled with image categories; the second sample image set is composed of second sample images without image category labels; input the first sample image set and the second sample image set into a student network to be trained to perform the following operations: generate multi-element features and perform feature fusion on each sample image according to a similarity matrix corresponding to the first sample image set and the second sample image set to obtain a multi-element feature set and a fusion feature set; mark multi-element features in the multi-element feature set and fusion features in the fusion feature set based on the category matching probability, calculate a training loss based on the obtained multi-element feature set and fusion feature set containing category labels, and adjust parameters of the student network to be trained and the teacher network to be trained based on the training loss to construct an image detection model according to a trained teacher network.

18. An image detection processing device, comprising: a processor; and a memory configured to store computer executable instructions which, when executed, cause the processor to: acquire at least two images to be detected; inputting the at least two images to be detected into an image detection model to perform image detection, and obtaining a feature vector; the image detection model is constructed based on a trained teacher network; calculating a category matching probability of the at least two images to be detected based on the feature vector; wherein the teacher network is trained in cooperation with a student network; the cooperation training comprises: performing category matching on a first sample image set and a second sample image set based on a teacher network to be trained, and obtaining a sample category matching probability; performing label processing on multi-element features in a multi-element feature set and fusion features in a fusion feature set corresponding to the first sample image set and the second sample image set according to the sample category matching probability based on a student network to be trained, and performing parameter adjustment on the teacher network to be trained and the student network to be trained based on a training loss calculated based on the obtained multi-element feature set and fusion feature set containing classification labels, so as to obtain a trained teacher network.

19. A storage medium for storing computer executable instructions, the computer executable instructions, when executed by a processor, implement the following processes: inputting a first sample image set and a second sample image set into a teacher network to be trained to perform category matching, and obtaining a category matching probability; the first sample image set is composed of first sample images labeled with image categories; the second sample image set is composed of second sample images without image category labels; inputting the first sample image set and the second sample image set into a student network to be trained to perform the following operations: generating multi-element features and performing feature fusion on each sample image according to a similarity matrix corresponding to the first sample image set and the second sample image set, and obtaining a multi-element feature set and a fusion feature set; performing label processing on multi-element features in the multi-element feature set and fusion features in the fusion feature set based on the category matching probability, calculating a training loss based on the obtained multi-element feature set and fusion feature set containing classification labels, and performing parameter adjustment on the student network to be trained and the teacher network to be trained based on the training loss, so as to construct an image detection model according to a teacher network obtained through training.

20. A storage medium for storing computer executable instructions, the computer executable instructions, when executed by a processor, implement the following processes: obtaining at least two images to be detected; inputting the at least two images to be detected into an image detection model to perform image detection, and obtaining a feature vector; the image detection model is constructed based on a trained teacher network; calculating a category matching probability of the at least two images to be detected based on the feature vector; The teacher network is trained in cooperation with the student network; the cooperation training comprises: performing category matching on the first sample image set and the second sample image set based on the teacher network to be trained, to obtain a sample category matching probability; performing label processing on multi-element features in a multi-element feature set and fusion features in a fusion feature set corresponding to the first sample image set and the second sample image set according to the sample category matching probability based on the student network to be trained; and performing parameter adjustment on the teacher network to be trained and the student network to be trained based on a training loss calculated based on the obtained multi-element feature set and fusion feature set containing classification labels, to obtain a trained teacher network.

Images