A deep metric learning method based on uniform penalty field loss function

Abstract

The present invention discloses a deep metric learning method based on a consistent penalty field loss function, which is mainly used to optimize sample embedding features in tasks such as image classification, image retrieval, face recognition, landmark recognition, zero-sample learning, and pedestrian re-identification in computer vision. The method provides an attraction field for samples of the same category, pushes the distance between samples of different categories, optimizes the quality of sample embedding features, and makes the decision boundaries of all categories consistent, thereby improving the accuracy of reasoning decisions. The present invention adopts a consistent penalty field loss function in combination with a deep learning model, and optimizes by calculating the cosine distance of the attraction field and the repulsion field, thereby enhancing the accuracy and robustness of the model in computer vision tasks. The invention can be widely used in computer vision tasks that require high-precision classification, and has high practical value and technical innovation.

Description

Technical Field

[0001] The present invention relates to the field of deep metric learning, and is particularly applicable to tasks such as classification and matching in computer vision. Specifically, the present invention provides a deep metric learning method based on a consistent penalty field loss function, which is used to optimize the sample embedding features of the representation learning model and improve the reasoning and decision-making accuracy of computer vision tasks. The method has a wide range of application value in tasks that require accurate classification, such as image retrieval, face recognition, and pedestrian re-identification. Background Art

[0002] Deep metric learning is a fundamental problem in computer vision and the core foundation of many important applications, such as product image retrieval, face recognition, landmark recognition, zero-shot learning, and person re-identification. The goal of deep metric learning is to learn a mapping from object samples to a low-dimensional dense vector space, so that objects of the same type are close in distance in the embedding space, while objects of different categories are far away from each other.

[0003] The similarity optimization of deep metric learning is achieved through the guidance of loss functions. At present, the loss functions used in deep metric learning can be divided into two categories: pair-based losses and proxy-based losses. The pair-based loss function optimizes the pairwise similarity between images in the embedding space by learning pairwise labels. The proxy-based loss function optimizes the similarity between image samples and proxies by learning class-level labels and using proxies as representatives of each category in the embedding space.

[0004] Among them, the proxy-based loss function performs better than the sample pair-based loss function in this field due to its fast convergence speed and low training complexity. The proxy-based loss function mainly uses the Softmax function, but the features learned by the original Softmax function do not distinguish different categories well enough. In order to enhance the recognition ability of unknown categories, people have proposed a margin-based loss function.

[0005] The decision boundary of each category of the Softmax-based loss function is a line. This decision boundary can only distinguish different categories, but it does not maximize the distance between the embeddings of image features of different categories. Therefore, when encountering samples of categories that have not been seen during training, the recognition effect will be poor. However, for the Arcface and other margin-based loss functions, the decision boundary is an angle buffer of size margin (m), and the similarity of each category differs by at least such a margin distance. The distance between features of samples of different categories will be larger, and samples of the same category will be more clustered. In this way, the distinction between categories will be greater, thus achieving better detection results.

[0006] However, due to the differences in samples of different categories in the data set, the decision boundaries of each category will be different regardless of whether it is based on Softmax or margin-based loss functions. In the inference stage, all categories need to use a unified threshold for inference decisions, so the differences in decision boundaries between categories will affect the accuracy of decision inference. Summary of the invention

[0007] In view of the above problems, the purpose of the present invention is to provide a representation learning method based on a consistent penalty field loss function, through which the decision boundaries of all categories can be made as consistent as possible, thereby improving the performance of the trained deep metric learning model. A representation learning method based on a consistent penalty field loss function comprises the following steps:

[0008] Step S10, data acquisition and preprocessing: obtain a data set, and divide the data set into a training set, a test set, and a validation set, and perform processing such as unifying the size and data enhancement on the training set images.

[0009] Step S20, using a consistent penalty field loss function to train and optimize the deep learning model: input the training set into the deep learning model to obtain the embedded features of the image, and then use the consistent penalty field loss function to obtain the loss value of the network model. The consistent penalty field loss function is implemented by executing steps S21 to S23.

[0010] Step S21, providing an attraction field for samples of the same category. The purpose is to shorten the distance between samples of the same category. The strength of the attraction field is proportional to the cosine distance between the proxy and the sample features. The smaller the similarity, the greater the attraction. The expression of the attraction field is as follows:

[0011]

[0012] in is the angle between the i-th sample feature and its proxy, N is the number of batches, is the weight of the attractiveness of the i-th sample.

[0013] Step S22, provide a penalty field for samples of different categories. The purpose is to push the distance between samples of different categories. The strength of the penalty field is inversely proportional to the cosine distance between the proxy and the sample features. The greater the similarity, the greater the thrust. The expression of the penalty field is as follows:

[0014]

[0015] where θ ij is the angle between the features of the i-th sample and the j-th proxy, is the penalty weight between the i-th sample and the j-th proxy, and τ is the threshold used to eliminate uninformative negative factors.

[0016] Step S23, balancing the attraction field and the penalty field. In deep metric learning, since samples of different categories are much more than samples of the same category, it is necessary to balance the attraction field and the penalty field. We balance the attraction field and the penalty field through the following expression and obtain the final consistent penalty field loss function value.

[0017]

[0018] Where λ is the weight hyperparameter between the attraction field and the penalty field, T p It is an evaluation value based on deep metric learning of cosine similarity between categories, and a constant is added to the numerator and denominator to increase the stability of the loss value.

[0019] Step S30, applying the deep learning model to make inference decisions in computer vision tasks: input the images in the test set into the optimal model, calculate the cosine similarity between each image embedding feature and all other image embedding features, and set a unified threshold. If the cosine similarity between the images is greater than the threshold, the two images are considered to be of the same category.

[0020] The present invention uses a consistent penalty field loss function to optimize the model during model training. The attraction field in the consistent penalty field loss function can aggregate the embedded features of images of the same category, while the penalty field can push away the embedded features of images of different categories. By providing a consistent field for all categories, the decision boundaries of different categories can be made as consistent as possible, thereby improving the model performance.

[0021] Through the above method, the present invention not only solves the reasoning error caused by the difference in decision boundaries of different categories in the prior art, but also significantly improves the performance of the model in actual classification tasks. Experimental results show that the deep metric learning model using the consistent penalty field loss function has higher accuracy in image classification and recognition tasks than traditional methods such as Softmax and Arcface, especially in unseen categories. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 Flowchart of the learning method based on the consistent penalty field loss function metric.

[0023] Figure 2 Schematic diagram of the uniform penalty field loss function.

[0024] Figure 3 Comparison diagram of decision boundaries for consistent penalty field loss functions. DETAILED DESCRIPTION

[0025] In order to make the objectives, technical solutions and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments.

[0026] This application uses Shopee product matching as an example. Shopee product matching is a competition in Kaggle, the goal of which is to determine whether two products are the same product through pictures. The public dataset provided by the competition has a total of 32,412 images in the training set, which are divided into 11,014 classes; there are about 21,000 images in the public test set, which we use as the validation set; there are about 49,000 images in the private test set, which we use as the test set.

[0027] The overall process of Shopee product matching is as follows Figure 1 As shown, the specific steps include:

[0028] The first step is to preprocess the training set, set the image size of the training set to 384×384, and perform operations such as horizontal flipping, scaling, and automatic data enhancement.

[0029] In the second step, this embodiment uses the deep learning model Resnet34 and deit_base_distilles_patch16_384 as the backbone network model. The pre-trained weights of the backbone network come from timm. The batch size is set to 20 and the number of iterations is set to 15 during training. The image is input into the network model, and after the last layer of convolution, the pooling layer-fully connected layer-batch normalization layer is used to obtain the final 768-dimensional high-dimensional embedding features.

[0030] In the third step, the image embedding features are used to calculate the loss value of the network model through the uniform penalty field loss function. The steps of the uniform penalty field loss function are as follows: Figure 2 As shown, specifically including:

[0031] S31 provides an attraction field for samples of the same category, with the goal of shortening the distance between samples of the same category. The strength of the attraction field is proportional to the cosine distance between the proxy and the sample features. The smaller the similarity, the greater the attraction. The expression of the attraction field is as follows:

[0032]

[0033] in is the angle between the i-th sample feature and its proxy, N is the number of batches, is the weight of the attractiveness of the i-th sample.

[0034] S32 provides a penalty field for samples of different categories, with the goal of pushing the distance between samples of different categories farther. The strength of the penalty field is also inversely proportional to the cosine distance between the proxy and the sample features. The greater the similarity, the greater the thrust. The expression of the penalty field is as follows:

[0035]

[0036] where θ ij is the angle between the features of the i-th sample and the j-th proxy, is the penalty weight between the i-th sample and the j-th proxy, and τ is the threshold for eliminating negative factors without information. In this embodiment, the τ value is set to 0.25.

[0037] S33, balancing the attraction field and the penalty field, balancing the attraction field and the penalty field through the following expression, and obtaining the final consistent penalty field loss function value.

[0038]

[0039] Where λ is the weight hyperparameter between the attraction field and the penalty field. In this embodiment, the value of λ is set to 2. p It is an evaluation value based on deep metric learning of cosine similarity between categories, and a constant is added to the numerator and denominator to increase the stability of the loss value.

[0040] The fourth step is to calculate the network model gradient through the loss function value and the back propagation algorithm, and use the AdamW algorithm to optimize the parameters of the network nodes to minimize the loss value of the network model and obtain the optimal model.

[0041] In the fifth step, the images in the test set are input into the trained model, the cosine similarity between each image embedding feature and all other image embedding features is calculated, and a unified threshold is set. If the cosine similarity between images is greater than the threshold, the two images are considered to be of the same category, thus obtaining matching products. The threshold set for the Resnet34 model is 0.68; the threshold set for the deit_base_distilles_patch16_384 model is 0.70.

[0042] The above embodiments are only preferred embodiments of the present invention in the field of product image matching, and their descriptions are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. The present invention can be widely used in computer vision tasks that require accurate classification, such as image retrieval, face recognition, pedestrian re-identification, landmark recognition, and zero-sample learning. It should be pointed out that for ordinary technicians in this field, several variations and improvements can be made without departing from the concept of the present application, which all belong to the scope of protection of the present application. Therefore, the scope of protection of the patent in this application shall be based on the attached claims.

Claims

1. A deep metric learning method based on a consistent penalty field loss function, which is applied to improve the accuracy of image classification and matching tasks in computer vision, characterized in that: The following steps are involved: S10, data acquisition and preprocessing: obtain the data set, and divide the data set into training set, test set and validation set, and unify the size of the training set images and perform data enhancement processing; S20, using a consistent penalty field loss function to train and optimize the deep learning model: input the training set into the deep learning model to obtain the embedded features of the image, and then use the consistent penalty field loss function to obtain the loss value of the network model, wherein the consistent penalty field loss function is implemented by executing steps S21 to S23; S21. Provide an attraction field for samples of the same category in order to shorten the distance between samples of the same category. The strength of the attraction field is proportional to the cosine distance between the proxy and the sample features. The smaller the similarity, the greater the attraction. The expression of the attraction field is as follows: in is the angle between the i-th sample feature and its proxy, N is the number of batches, is the weight of the attractiveness of the i-th sample; S22. Provide a penalty field for samples of different categories. The purpose is to push the distance between samples of different categories. The strength of the penalty field is inversely proportional to the cosine distance between the proxy and the sample features. The greater the similarity, the greater the thrust. The expression of the penalty field is as follows: where θ ij is the angle between the features of the i-th sample and the j-th proxy, is the penalty weight between the i-th sample and the j-th proxy, and τ is the threshold used to eliminate negative elements without information; S23. Balance the attraction field and the penalty field. In deep metric learning, since samples of different categories are much more than samples of the same category, it is necessary to balance the attraction field and the penalty field. The following expression is used to balance the attraction field and the penalty field, and the final consistent penalty field loss function value is obtained: Where λ is the weight hyperparameter between the attraction field and the penalty field, T p It is an evaluation value based on deep metric learning of cosine similarity between categories, and a constant is added to the numerator and denominator to increase the stability of the loss value; S30, applying deep learning models for reasoning and decision-making in computer vision tasks: input the images in the test set into the trained model, calculate the cosine similarity between each image embedding feature and all other image embedding features, and set a unified threshold. If the cosine similarity between images is greater than the threshold, the two images are considered to be of the same category, thereby obtaining image classification and matching results in computer vision tasks.

Images