Features between models are backward compatible with learning methods, electronic devices, and storage media

By applying perturbation to the old model's feature values ​​and constructing sample pairs for comparison learning, the problem of spatial aligning of new and old models is solved, the discrimination and compatibility of the new model is improved, and the image retrieval performance is improved.

CN118690201BActive Publication Date: 2024-11-12PENG CHENG LAB
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411155676.8
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-08-22
Publication Date
2024-11-12
Estimated Expiration
2044-08-22

AI Technical Summary

Technical Problem

In a content-based image retrieval system, the feature space of the new model is not aligned with the feature space of the old model, resulting in the embedding representation extracted by the new model that cannot be directly compared with the embedding vector of the old model, which in turn affects the search performance and mainstream features Backward compatibility learning methods require strict alignment, which impairs the discriminatory ability of the new model.

Method used

By applying perturbation to the eigenvalues ​​of the old model, new eigenvalues ​​are obtained, making it easier to distinguish the category feature centers that are difficult to distinguish in the feature space of the old model, and constructing positive and negative sample pairs. Through comparative learning, the new model is updated, so that the It has good discriminant performance while being compatible with the old model implementation.

Benefits of technology

The discriminant ability and retrieval performance of the new model are improved, and the compatibility between the new and old models is achieved, avoiding the damage to the discriminant ability of the new model.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118690201B_ABST
    Figure CN118690201B_ABST
Patent Text Reader

Abstract

The present application proposes a method for backward-compatible learning of features between models, which includes obtaining a first classification model, a second classification model, and a training data set; calculating the first eigenvalue and the second eigenvalue corresponding to each sub-data set based on the first classification model and the second classification model; performing perturbation calculation on the first eigenvalue to obtain the corresponding third eigenvalue; calculating the target eigenvector corresponding to each data sample based on the second classification model, and constructing a negative sample pair based on the target eigenvector and the first eigenvalue, and constructing a positive sample pair based on the target eigenvector and the third eigenvalue, and calculating the first loss function; classifying the training data set based on the second classification model, and calculating the second loss function of the second classification model based on the classification accuracy of the second classification model; and updating the model parameters of the second classification model based on the first loss function and the second loss function. This method can improve the discrimination ability of the new model.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a method for learning backward-compatible features between models, an electronic device, and a storage medium. Background Art

[0002] In content-based image retrieval systems, embedding models are usually used to extract features of the original image, and the features are used as the image embedding vector for similarity calculation in the retrieval process. However, when upgrading the embedding model of the retrieval system, the feature space of the new model is not aligned with the feature space of the old model, which results in the embedding representation of the query image extracted by the new model being unable to be directly compared with the old embedding vector already in the database corresponding to the old model, thereby completing the retrieval task.

[0003] In related technologies, mainstream feature backward-compatible learning methods usually require that the feature space of the new model be strictly aligned with the feature space of the old model. However, in the feature space of the old model, if the prototypes of two different categories are difficult to distinguish, constraining the new model to strictly align the feature space of the old model will damage the discrimination ability of the new model, thereby causing the retrieval performance of the new model to deteriorate. Summary of the invention

[0004] The embodiments of the present application provide a method for backward-compatible feature learning between models, an electronic device, and a storage medium, which can reduce the influence of the old model on the discrimination ability of the new model, thereby improving the discrimination ability of the new model.

[0005] To achieve the above-mentioned purpose, a first aspect of an embodiment of the present application proposes a method for learning backward-compatible features between models, the method comprising:

[0006] Obtain a first classification model, a second classification model, and a training data set; wherein the training data set includes a plurality of sub-data sets, and each of the sub-data sets includes at least one data sample;

[0007] Calculating each of the sub-datasets based on the first classification model to obtain a first eigenvalue corresponding to each of the sub-datasets, and calculating each of the sub-datasets based on the second classification model to obtain a second eigenvalue corresponding to each of the sub-datasets;

[0008] Performing perturbation calculation on the first eigenvalue corresponding to each of the sub-data sets to obtain a third eigenvalue corresponding to each of the sub-data sets;

[0009] Calculate the target feature vector corresponding to each of the data samples based on the second classification model, construct a negative sample pair according to the target feature vector corresponding to each of the data samples and the first feature value corresponding to the sub-dataset to which each of the data samples belongs, construct a positive sample pair according to the target feature vector corresponding to each of the data samples and the third feature value corresponding to the sub-dataset to which each of the data samples belongs, and calculate a first loss function according to the positive samples and the negative sample pairs;

[0010] Classifying the data samples in the training data set based on the second classification model to obtain the classification accuracy of the second classification model, and calculating the second loss function of the second classification model according to the classification accuracy of the second classification model;

[0011] A target loss function is constructed based on the first loss function and the second loss function, and the model parameters of the second classification model are updated according to the target loss function to obtain an updated second classification model.

[0012] In some embodiments, performing perturbation calculation on the first eigenvalue corresponding to each sub-data set to obtain a third eigenvalue corresponding to each sub-data set includes:

[0013] Based on the first eigenvalue, a first neighbor sub-dataset of each sub-dataset is calculated;

[0014] Calculating a first similarity between the first nearest neighbor sub-dataset and the sub-dataset;

[0015] Calculating a first disturbance amount based on the first similarity;

[0016] The first eigenvalue is subjected to a disturbance calculation based on the first disturbance amount to obtain a third eigenvalue corresponding to each of the sub-data sets.

[0017] In some embodiments, the method further comprises:

[0018] Based on the second eigenvalue, calculate and obtain a second nearest neighbor sub-dataset of each of the sub-datasets;

[0019] Calculating a second similarity between the second nearest neighbor sub-dataset and the sub-dataset;

[0020] Calculating a second disturbance amount based on the second similarity;

[0021] Based on the first disturbance amount and the second disturbance amount, a disturbance calculation is performed on the first eigenvalue to obtain a third eigenvalue corresponding to each of the sub-data sets.

[0022] In some embodiments, performing perturbation calculation on the first eigenvalue corresponding to each sub-data set to obtain a third eigenvalue corresponding to each sub-data set includes:

[0023] Constructing a disturbance objective function based on each of the sub-data sets, the first eigenvalue corresponding to each of the sub-data sets, and the second eigenvalue corresponding to each of the sub-data sets;

[0024] The disturbance objective function is solved to obtain a third disturbance, and the first eigenvalue is disturbed and calculated based on the third disturbance to obtain the third eigenvalue.

[0025] In some embodiments, the calculating each sub-dataset based on the first classification model to obtain a first feature value corresponding to each sub-dataset, and the calculating each sub-dataset based on the second classification model to obtain a second feature value corresponding to each sub-dataset, include:

[0026] A first feature vector of each of the data samples in each of the sub-datasets based on the first classification model, and a second feature vector of each of the data samples in each of the sub-datasets based on the second classification model;

[0027] A mean calculation is performed based on each of the first eigenvectors to obtain the first eigenvalue, and a mean calculation is performed based on each of the second eigenvectors to obtain the second eigenvalue.

[0028] In some embodiments, constructing a negative sample pair according to the target feature vector corresponding to each of the data samples and the first feature value corresponding to the sub-dataset to which each of the data samples belongs, and constructing a positive sample pair according to the target feature vector corresponding to each of the data samples and the third feature value corresponding to the sub-dataset to which each of the data samples belongs, includes:

[0029] Determine, in the training data set, a sub-data set other than the sub-data set to which each of the data samples belongs as a negative sample data set;

[0030] Constructing the negative sample pair according to the target feature vector and the first feature value corresponding to the negative sample data set;

[0031] Determine the sub-dataset corresponding to the sub-dataset to which each data sample belongs in the training data set as a positive sample data set;

[0032] The positive sample pair is constructed according to the target feature vector and the third feature value corresponding to the positive sample data set.

[0033] In some embodiments, classifying the data samples in the training data set based on the second classification model to obtain the classification accuracy of the second classification model, and calculating the second loss function of the second classification model according to the classification accuracy of the second classification model includes:

[0034] Classifying each of the data samples in the training data set based on the second classification model to obtain a classification result for each of the data samples;

[0035] Calculating the classification accuracy based on the classification result and the correct classification result preset in the training data set;

[0036] The second loss function is calculated based on the classification loss of the cross entropy and the classification accuracy.

[0037] In some embodiments, after constructing a target loss function based on the first loss function and the second loss function, and updating the model parameters of the second classification model according to the target loss function, and obtaining the updated second classification model, the method further includes:

[0038] Performing a perturbation calculation on the first eigenvalue corresponding to each of the sub-data sets based on the updated second classification model to obtain a fourth eigenvalue corresponding to each of the sub-data sets;

[0039] Calculate the target feature vector corresponding to each of the data samples based on the updated second classification model, and update the negative sample pair according to the target feature vector corresponding to each of the data samples and the first feature value corresponding to the sub-dataset to which each of the data samples belongs, and update the positive sample pair according to the target feature vector corresponding to each of the data samples and the fourth feature value corresponding to the sub-dataset to which each of the data samples belongs;

[0040] updating the first loss function according to the updated positive sample and the updated negative sample to obtain an updated first loss function, and classifying the data samples in the training data set based on the updated second classification model, and updating the second loss function of the second classification model according to the classification accuracy of the second classification model to obtain an updated second loss function;

[0041] Update the target loss function based on the updated first loss function and the updated second loss function to obtain an updated target loss function;

[0042] The model parameters of the updated second classification model are updated again based on the updated objective loss function to obtain a second classification model that is updated again.

[0043] To achieve the above-mentioned purpose, a second aspect of an embodiment of the present application proposes a feature backward compatibility learning device between models, including:

[0044] An acquisition unit, configured to acquire a first classification model, a second classification model, and a training data set; wherein the training data set includes a plurality of sub-data sets, and each of the sub-data sets includes at least one data sample;

[0045] a first calculation unit, configured to calculate each of the sub-datasets based on the first classification model to obtain a first eigenvalue corresponding to each of the sub-datasets, and to calculate each of the sub-datasets based on the second classification model to obtain a second eigenvalue corresponding to each of the sub-datasets;

[0046] a perturbation unit, configured to perform perturbation calculation on the first eigenvalue corresponding to each of the sub-data sets to obtain a third eigenvalue corresponding to each of the sub-data sets;

[0047] A second calculation unit is used to calculate the target feature vector corresponding to each of the data samples based on the second classification model, and construct a negative sample pair according to the target feature vector corresponding to each of the data samples and the first eigenvalue corresponding to the sub-dataset to which each of the data samples belongs, and construct a positive sample pair according to the target feature vector corresponding to each of the data samples and the third eigenvalue corresponding to the sub-dataset to which each of the data samples belongs, and calculate a first loss function according to the positive samples and the negative sample pairs;

[0048] a third computing unit, configured to classify the data samples in the training data set based on the second classification model to obtain a classification accuracy of the second classification model, and calculate a second loss function of the second classification model according to the classification accuracy of the second classification model;

[0049] An updating unit is used to construct a target loss function based on the first loss function and the second loss function, and to update the model parameters of the second classification model according to the target loss function to obtain an updated second classification model.

[0050] To achieve the above-mentioned purpose, the third aspect of an embodiment of the present application proposes an electronic device, which includes a memory and a processor, the memory stores a computer program, and when the processor executes the computer program, it implements the feature backward-compatible learning method between models as described in the first aspect.

[0051] To achieve the above-mentioned purpose, the fourth aspect of an embodiment of the present application proposes a storage medium, which is a computer-readable storage medium, and the storage medium stores a computer program. When the computer program is executed by a processor, it implements the feature backward-compatible learning method between models as described in the first aspect.

[0052] The present application proposes a method for backward-compatible learning of features between models, an electronic device, and a storage medium. First, a first classification model, a second classification model, and a training data set are obtained; wherein the training data set includes multiple sub-data sets, and each sub-data set includes at least one data sample; each sub-data set is calculated based on the first classification model to obtain a first eigenvalue corresponding to each sub-data set, and each sub-data set is calculated based on the second classification model to obtain a second eigenvalue corresponding to each sub-data set; a perturbation calculation is performed on the first eigenvalue corresponding to each sub-data set to obtain a third eigenvalue corresponding to each sub-data set; a target feature vector corresponding to each data sample is calculated based on the second classification model, and a target feature vector corresponding to each data sample is obtained according to the target feature vector of each data sample. A negative sample pair is constructed between the target feature vector corresponding to the present invention and the first eigenvalue corresponding to the sub-dataset to which each data sample belongs, a positive sample pair is constructed between the target feature vector corresponding to each data sample and the third eigenvalue corresponding to the sub-dataset to which each data sample belongs, and a first loss function is calculated based on the positive and negative sample pairs; the data samples in the training data set are classified based on the second classification model to obtain the classification accuracy of the second classification model, and a second loss function of the second classification model is calculated based on the classification accuracy of the second classification model; a target loss function is constructed based on the first loss function and the second loss function, and the model parameters of the second classification model are updated according to the target loss function to obtain an updated second classification model.

[0053] With respect to the problems in the background technology, the method for learning backward-compatible features between models provided in the embodiment of the present application first calculates the eigenvalues ​​of each sub-data set using the first classification model and the second classification model respectively to obtain the first eigenvalue and the second eigenvalue. In the prior art, the second eigenvalue is usually required to be strictly aligned with the first eigenvalue to achieve compatibility between models. This approach ignores the influence of the distribution of the feature space of the first classification model itself on the discrimination ability of the second classification model. The present solution relaxes the backward-compatibility constraint of the first classification model by applying a disturbance to the first eigenvalue to obtain the third eigenvalue. The third eigenvalue makes it easier to distinguish the category feature centers that were originally difficult to distinguish in the feature space of the first classification model. Next, Here, a positive sample pair is constructed based on the target feature vector corresponding to each data sample and the third eigenvalue corresponding to the sub-dataset to which each data sample belongs, so as to improve the discrimination ability of the second classification model for categories that were originally difficult to distinguish, and a negative sample pair is constructed between the target feature vector corresponding to each data sample and the first eigenvalue corresponding to the sub-dataset to which each data sample belongs, so as to ensure that the second classification model is still compatible with the first classification model. The first loss function is calculated according to the positive and negative sample pairs, and then the second loss function corresponding to the classification accuracy of the second classification model is calculated. By minimizing the target loss function composed of the first loss function and the second loss function, the second classification model can be made compatible with the old model while having good discrimination performance.

[0054] Other features and advantages of the present application will be described in the following description, and partly become apparent from the description, or understood by practicing the present application. The purpose and other advantages of the present application can be realized and obtained by the structures specifically pointed out in the description, claims and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Figure 1 A schematic diagram of a traditional offline backfill model update process in the prior art.

[0056] Figure 2 A schematic diagram of a backfill-free model update process based on backward-compatible learning in the prior art.

[0057] Figure 3 A flowchart of a method for backward-compatible feature learning between models provided in one embodiment of the present application.

[0058] Figure 4 An embodiment of the present application provides Figure 3 Flow chart of step 302 in FIG.

[0059] Figure 5 An embodiment of the present application provides Figure 3 Flow chart of step 303 in FIG.

[0060] Figure 6 Another flowchart of the method for backward-compatible feature learning between models provided in one embodiment of the present application.

[0061] Figure 7 An embodiment of the present application provides Figure 3 Another flow chart of step 303 in FIG.

[0062] Figure 8 An embodiment of the present application provides Figure 3 Flow chart of step 304 in FIG.

[0063] Fig. 9 An embodiment of the present application provides Figure 3 Flow chart of step 305 in FIG.

[0064] Fig.10 An embodiment of the present application provides Figure 3 Flowchart after step 306 in FIG.

[0065] Fig.11 A schematic diagram of a device for learning backward-compatible features between models provided in one embodiment of the present application.

[0066] Fig.12 A schematic diagram of the hardware structure of an electronic device provided in yet another embodiment of the present application. DETAILED DESCRIPTION

[0067] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0068] It should be noted that although the functional modules are divided in the device schematic and the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart.

[0069] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0070] Content-based image retrieval systems usually use embedding models to extract features of the original image. By extracting the features of the original image, it converts it into an embedding vector, which can represent the main content and features of the image. During the image retrieval process, the system will use this embedding vector to calculate the similarity between images, thereby achieving efficient image retrieval. In order to achieve this function, the system usually maintains a huge image embedding database. This database stores a large number of embedding vectors of original images, allowing the system to quickly access and retrieve. When a user submits a query image, the system first needs to calculate the embedding vector of the query image. Subsequently, the system will compare the query vector with the embedding vectors stored in the database to find the image set that is most similar to the query image in terms of features.

[0071] However, when upgrading an image retrieval system, since the new model is trained independently, the feature space it learns may not be fully aligned with the feature space of the old model, which means that even if two images are very similar visually, since their feature vectors are generated in different feature spaces, the embedding vector extracted by the new model may not be able to be directly compared for effective similarity with the embedding vector generated by the old model in the database.

[0072] In the related technology, there are currently two solutions to the problem of aligning the feature space of the new and old models. One is Figure 1 As shown in the figure, the basic idea of ​​offline backfill model update is to use the new model to recalculate the embedding vectors of all images in the database and replace the original embedding vectors with these newly calculated embedding vectors. Specifically, it includes: using the new model to process each image in the database, calculating their feature embedding vectors, and then storing the newly calculated embedding vectors back to the database to replace the original embedding vectors. In this way, the images in the database all have embedded representations that are aligned with the feature space of the new model. After the update is completed, the query embedding vector extracted by the new model can be directly compared with the embedding vector in the database, thereby achieving retrieval consistency. The advantage of offline backfill model update is that it can ensure that all images in the database use the same model for feature extraction, thereby avoiding the problem of inconsistent feature space. However, this method also has its limitations. For example, recalculating the embedding vectors of a large number of images may require a lot of computing resources and time. In addition, offline backfill is not applicable to application scenarios that require real-time retrieval, because before the backfill is completed, the entire retrieval system cannot use the new model and can only continue to use the old model.

[0073] Another is as Figure 2The backfill-free model based on backward-compatible learning shown in the figure aligns the feature space of the new model with the feature space of the old model, thereby achieving feature compatibility between the new and old models. The query embedding vector calculated by the new model can directly calculate the similarity with the existing image embedding, and the retrieval system can achieve backfill-free upgrade. However, mainstream feature backward-compatible learning methods usually require that the feature space of the new model be strictly aligned with the feature space of the old model. This approach ignores the impact of the distribution of the old model's feature space on the discriminative ability of the new model. In the feature space of the old model, if the prototypes of two different categories are close to each other and difficult to distinguish, constraining the new model to strictly align the feature space of the old model will damage the discriminative ability of the new model, resulting in a decrease in the retrieval performance of the new model.

[0074] In view of the above problems, the method for learning backward-compatible features between models provided in the embodiment of the present application first calculates the feature values ​​of each sub-data set using the first classification model and the second classification model respectively to obtain the first feature value and the second feature value. In the prior art, the second feature value is usually required to be strictly aligned with the first feature value to achieve compatibility between models. This approach ignores the influence of the distribution of the feature space of the first classification model itself on the discrimination ability of the second classification model. The present solution relaxes the backward-compatibility constraint of the first classification model by applying a disturbance to the first feature value to obtain the third feature value. The third feature value makes it easier to distinguish the category feature centers that were originally difficult to distinguish in the feature space of the first classification model. Next, A positive sample pair is constructed between the target feature vector corresponding to each data sample and the third eigenvalue corresponding to the sub-dataset to which each data sample belongs, so as to improve the discrimination ability of the second classification model for categories that were originally difficult to distinguish, and a negative sample pair is constructed between the target feature vector corresponding to each data sample and the first eigenvalue corresponding to the sub-dataset to which each data sample belongs, so as to ensure that the second classification model is still compatible with the first classification model. The first loss function is calculated according to the positive and negative sample pairs, and then the second loss function corresponding to the classification accuracy of the second classification model is calculated. By minimizing the target loss function composed of the first loss function and the second loss function, the second classification model can be made compatible with the old model while having good discrimination performance.

[0075] The following will further describe a method for learning backward-compatible features between models, an electronic device, and a storage medium provided by an embodiment of the present application. First, the method for learning backward-compatible features between models in an embodiment of the present application is specifically described.

[0076] Reference Figure 3 , which is an optional flowchart of the method for backward-compatible feature learning between models provided in an embodiment of the present application, Figure 3 The method in the embodiment may include but is not limited to steps 301 to 306. Figure 3The order of step 301 to step 306 is not specifically limited, and the order of steps can be adjusted or some steps can be reduced or added according to actual needs.

[0077] Step 301: Obtain a first classification model, a second classification model, and a training data set.

[0078] Step 301 is described in detail below.

[0079] The first classification model, the first classification model is the old model, which is the model used in the current system. It has been trained and used for image retrieval tasks. It has stable performance and has generated a large number of image embedding vectors stored in the database. The second classification model is a new model that needs to be replaced. It has a more advanced algorithm or architecture and can provide more accurate image feature extraction capabilities. However, the feature space obtained by independent training of the new model may not be completely consistent with the feature space of the old model. In order to achieve backward-compatible learning, a training dataset is required, which contains multiple sub-datasets. Each sub-dataset contains at least one data sample, which can be an image or a feature vector of an image. The purpose of the training dataset is to fine-tune the new model so that it learns feature representations similar to those of the old model.

[0080] Step 302: Calculate each sub-dataset based on the first classification model to obtain a first eigenvalue corresponding to each sub-dataset, and calculate each sub-dataset based on the second classification model to obtain a second eigenvalue corresponding to each sub-dataset.

[0081] Step 302 is described in detail below.

[0082] In some embodiments, based on the first classification model, a calculation operation is performed on each sub-dataset to obtain a first eigenvalue corresponding to each sub-dataset. At the same time, based on the second classification model, a calculation operation is performed on each sub-dataset to obtain a second eigenvalue corresponding to each sub-dataset. The first classification model (old model) and the second classification model (new model) are used to calculate the eigenvalues ​​of each sub-dataset. This step is to extract data features from different models, and the result is that each sub-dataset will correspond to a first and a second eigenvalue. There may be differences between these two eigenvalues, because the new and old models may not have completely consistent feature extraction results for the same sub-dataset due to different algorithms or architectures. Therefore, the execution of this step can provide two sets of eigenvalues ​​for comparison and adjustment for subsequent steps, which helps to ultimately achieve compatibility of the new model with the old model.

[0083] Reference Figure 4 In some embodiments, step 302 includes but is not limited to steps 401 to 402.

[0084] Step 401: A first feature vector of each data sample in each sub-data set is obtained based on a first classification model, and a second feature vector of each data sample in each sub-data set is obtained based on a second classification model.

[0085] Step 401 is described in detail below.

[0086] Specifically, for each sub-dataset, each data sample is input into the first classification model (old model) to obtain a feature vector, which is the representation of the data sample in the feature space of the old model. These feature vectors are called first feature vectors. Similarly, each data sample is input into the second classification model (new model) to obtain the corresponding second feature vector, which is the representation of the data sample in the feature space of the new model.

[0087] Through this step, the representation of each data sample in each sub-dataset in the new and old model feature spaces, namely the first eigenvector and the second eigenvector, can be obtained. These eigenvectors will be used in step 402 to calculate the overall eigenvalue of the sub-dataset.

[0088] Step 402: performing mean calculation based on each first eigenvector to obtain a first eigenvalue, and performing mean calculation based on each second eigenvector to obtain a second eigenvalue.

[0089] Step 402 is described in detail below.

[0090] Among them, the mean calculation is performed on all the first eigenvectors in each sub-dataset obtained in step 401. The purpose of this step is to obtain a single vector representing the characteristics of the entire sub-dataset, namely the first eigenvalue. This mean vector can be regarded as the center point of the sub-dataset in the old model feature space; then the mean calculation is performed on all the second eigenvectors in each sub-dataset obtained in step 401 to obtain the second eigenvalue. This mean vector represents the center point of the sub-dataset in the new model feature space.

[0091] In step 401 to step 402, for the sub-dataset c, its corresponding first eigenvalue and second eigenvalue are obtained by the mean of the embedding vectors of all data samples, which is specifically expressed as:

[0092]

[0093] in, represents the set of all data samples belonging to sub-dataset c, represents the first eigenvalue of the sub-dataset c corresponding to the first classification model, represents the second eigenvalue of the sub-dataset c corresponding to the second classification model, Indicates that the first classification model calculates the feature vector of the data sample x, Represents the feature vector of the data sample x calculated by the second classification model.

[0094] Step 303: Perform perturbation calculation on the first eigenvalue corresponding to each sub-data set to obtain the third eigenvalue corresponding to each sub-data set.

[0095] Step 303 is described in detail below.

[0096] Among them, in backward compatible learning, the new model needs to learn feature representations similar to those of the old model. However, due to the complexity of data distribution, the first eigenvalues ​​generated by the old model may be very close to each other in the feature space, resulting in low discrimination between them. By introducing perturbations, the distance between these eigenvalues ​​can be increased, thereby improving their distinguishability. It can be understood that the perturbation is not random, but adaptive, and the size and direction of the perturbation are determined according to the current state of the first eigenvalue. Adaptive perturbations can be based on a variety of strategies, such as gradient information, the distribution of cluster centers, or the distance from other eigenvalues. After applying adaptive perturbations to the first eigenvalues ​​of each sub-dataset, a new vector is obtained, called the third eigenvalue. These third eigenvalues ​​are perturbed, and their positions in the feature space of the old model are adjusted to increase their discrimination from the eigenvalues ​​of other sub-datasets. The third eigenvalues ​​are also called pseudo-old prototypes, which are the feature center points in the feature space of the old model and represent the features of the sub-datasets after perturbation. These pseudo-old prototypes serve as the goal of backward compatible learning to guide the new model to learn more discriminative feature representations. In the subsequent learning process, the new model will try to generate feature vectors that match these pseudo-old prototypes. By minimizing the difference between the second eigenvalues ​​generated by the new model and the pseudo-old prototypes, the new model can gradually learn feature representations that are similar to the old model and have high discrimination.

[0097] Through step 303, not only the distinguishability of the feature values ​​of the old model can be enhanced, but also a clearer learning goal is provided for the new model. This method helps to achieve a smooth transition between the old and new models and improve the retrieval performance and accuracy of the new model on the existing database.

[0098] Reference Figure 5 In some embodiments, step 303 includes but is not limited to steps 501 to 504.

[0099] Step 501: Based on the first eigenvalue, calculate and obtain the first nearest neighbor sub-dataset of each sub-dataset.

[0100] Step 501 is described in detail below.

[0101] Based on the first eigenvalue of each sub-dataset, the first nearest neighbor sub-dataset of the sub-dataset is calculated. Specifically, for each sub-dataset, the distance between its first eigenvalue and the first eigenvalues ​​of all other sub-datasets is calculated (such as Euclidean distance, cosine distance, etc.), and the K sub-datasets with the smallest distance are found, which are the first nearest neighbor sub-datasets of the sub-dataset. Among them, K is a predefined parameter that determines the number of neighbors to be considered. The value of K depends on the size of the data set, the dimension of the feature space, and the required discrimination. The first nearest neighbor sub-dataset reflects the K sub-datasets that are most similar or closest to the current sub-dataset in the old model feature space. By finding the first nearest neighbor sub-dataset, the position and surrounding environment of the current sub-dataset in the old model feature space can be understood, providing important reference information for subsequent perturbation calculations.

[0102] For example, the sub-dataset c corresponds to The first eigenvalue corresponding to the nearest neighbor sub-dataset is expressed as , the first eigenvalue set of all neighbor sub-datasets It is expressed as:

[0103]

[0104] Step 502: Calculate a first similarity between the first nearest neighbor sub-dataset and the sub-dataset.

[0105] Step 502 is described in detail below.

[0106] After determining the K nearest neighbor sub-datasets of each sub-dataset, it is necessary to calculate the similarity between these nearest neighbor sub-datasets and the atomic data set. The purpose of this step is to quantitatively evaluate the similarity between each sub-dataset and its nearest neighbor sub-datasets, providing more comprehensive and detailed reference information for subsequent perturbation calculations.

[0107] Specifically, for each sub-dataset, first obtain its first eigenvalue and the first eigenvalues ​​of its K nearest neighbor sub-datasets. Then, use a certain metric to calculate the similarity between the first eigenvalue of the atomic data set and the first eigenvalue of each nearest neighbor sub-dataset to obtain K similarity values. Commonly used similarity metrics include Euclidean distance, cosine similarity, etc.

[0108] For example, for sub-dataset c, its neighboring sub-datasets The similarity with sub-dataset c is expressed as , sub-dataset c and all neighboring sub-datasets Similarity set It is expressed as follows:

[0109]

[0110] Step 503: Calculate and obtain a first disturbance value based on the first similarity.

[0111] Step 503 is described in detail below.

[0112] Among them, the perturbation amount indicates the size and direction of the adjustment of the first eigenvalue to increase the distinguishability of the eigenvalue in the feature space. The core idea of ​​this step is that the higher the similarity, the closer the eigenvalues ​​of the sub-datasets are, so a larger perturbation amount is needed to separate them; conversely, the lower the similarity, the more distinguishable the eigenvalues ​​of the sub-datasets are, so only a smaller perturbation amount is needed to maintain their relative position relationship.

[0113] Specifically, a mapping function can be designed to map the first similarity to the first perturbation amount. The mapping function is expressed as follows:

[0114]

[0115] in, Represents the first nearest neighbor subdataset The first similarity with sub-dataset c, represents the first disturbance.

[0116] Step 503 converts the first similarity into a first disturbance value by designing a suitable mapping function, and quantifies the magnitude and direction of the disturbance to the first eigenvalue. The calculation result of this step directly determines the quality of the generation of the third eigenvalue, which in turn affects the discrimination ability and feature compatibility of the new model.

[0117] Step 504: Perform disturbance calculation on the first eigenvalue based on the first disturbance amount to obtain a third eigenvalue corresponding to each sub-data set.

[0118] Step 504 is described in detail below.

[0119] In some embodiments, before applying the disturbance amount, it is usually multiplied by a preset first disturbance amplitude parameter, which is used to control the intensity of the disturbance to ensure that the disturbance is not too large or too small, affecting the quality of the eigenvalue and the stability of the system. The first disturbance amount is multiplied by the disturbance amplitude parameter to obtain an adjusted disturbance amount, and then the adjusted disturbance amount is applied to the corresponding first eigenvalue to obtain a third eigenvalue. The third eigenvalue represents the new position of the sub-data set in the old model feature space after the disturbance.

[0120] Exemplarily, the first disturbance quantity obtained above is Multiply by the preset first disturbance amplitude parameter , and get the third eigenvalue The process is expressed as follows:

[0121]

[0122] Through steps 501 to 504, by calculating and applying the perturbation amount, the discrimination of the feature values ​​of the old model can be improved, thereby providing a learning target for the new model and ensuring that the new model can generate feature representations that are compatible with the old model.

[0123] Reference Figure 6 In some embodiments, step 303 also includes but is not limited to steps 601 to 604.

[0124] Step 601: Based on the second eigenvalue, calculate and obtain the second nearest neighbor sub-dataset of each sub-dataset.

[0125] In some embodiments, based on the second eigenvalue of each sub-dataset, the K second nearest neighbor sub-datasets of the sub-dataset are calculated. Similar to step 501, the purpose of this step is to obtain local structural information in the new model feature space and identify the position and surrounding environment of each sub-dataset in the new model feature space. By finding the second nearest neighbor sub-dataset, the K nearest neighbors of each sub-dataset in the new feature space are determined. For example, the sub-dataset c corresponds to The second eigenvalue corresponding to the nearest neighbor sub-dataset It is expressed as:

[0126]

[0127] Step 602: Calculate a second similarity between the second nearest neighbor sub-dataset and the sub-dataset.

[0128] In some embodiments, after determining the K nearest neighbor sub-datasets of each sub-dataset, it is necessary to calculate the second similarity between each sub-dataset and its second nearest neighbor sub-dataset. Similar to step 502, the second similarity measures the similarity between the two sub-datasets in the new model feature space, and can be calculated using metrics such as Euclidean distance and cosine similarity. For example, for sub-dataset c, calculate its nearest neighbor sub-datasets Similarity with sub-dataset c in the new feature space It is expressed as follows:

[0129]

[0130] Step 603: Calculate and obtain a second disturbance value based on the second similarity.

[0131] In some embodiments, similar to the mapping function construction method in step 503, based on the second nearest neighbor sub-dataset The second similarity with sub-dataset c Calculate the second disturbance It is expressed as follows:

[0132]

[0133] Step 604: Based on the first disturbance amount and the second disturbance amount, perform disturbance calculation on the first eigenvalue to obtain a third eigenvalue corresponding to each sub-data set.

[0134] In some embodiments, the first disturbance amount calculated in step 503 and the second disturbance amount obtained in step 603 may be combined to perform comprehensive disturbance calculation on the first eigenvalue of each sub-data set. Similarly, before applying the second disturbance amount, a preset second disturbance amplitude parameter may also be multiplied.

[0135] Exemplarily, the first disturbance quantity obtained above is Multiply by the preset first disturbance amplitude parameter , the first disturbance Multiply by the preset first disturbance amplitude parameter , and get the third eigenvalue The process is expressed as follows:

[0136]

[0137] Through steps 601 to 604, in the feature space of the old model, the first eigenvalue (old prototype) of each sub-dataset finds the eigenvalues ​​(old neighbor prototypes) of the K other sub-datasets that are most similar to it through nearest neighbor search. The interaction between these old prototypes is reflected by calculating the similarity between them. The higher the similarity, the closer they are in the feature space. At the same time, the second eigenvalue (new prototype) extracted by the new model also finds the most similar sub-dataset through nearest neighbor search. Using the new and old neighbor sub-datasets to calculate the perturbation at the same time not only takes into account the interaction between the old prototypes, but also takes into account the effect of the new prototype on the old prototype, so that the pseudo-old prototype can be kept away from the old prototype and the new prototype with high similarity, thereby effectively reducing the impact of compatible loss on discriminability and improving the discriminability of the new model.

[0138] Reference Figure 7 In some embodiments, step 303 includes but is not limited to steps 701 to 702.

[0139] Step 701: construct a disturbance objective function based on each sub-data set, the first eigenvalue corresponding to each sub-data set, and the second eigenvalue corresponding to each sub-data set.

[0140] Among them, a perturbation objective function is constructed based on each sub-data set, its corresponding first eigenvalue (eigenvalue generated by the old model) and the second eigenvalue (eigenvalue generated by the new model) as follows:

[0141]

[0142] Among them, the objective function aims to provide each old prototype Design a learnable perturbation vector , this perturbation is learned by minimizing the similarity between indistinguishable prototypes, where and are thresholds, which are used to measure the first eigenvalues ​​of the two old prototypes that are difficult to distinguish and the new to old prototypes that are difficult to distinguish. represents other old prototypes in the feature space of the old model, represents the perturbation vector applied to other old prototypes, Represents other new prototypes in the feature space of the new model. This method is also called ODPP (Optimization-Driven Prototype Perturbation), which is a prototype perturbation method based on optimization, namely the first eigenvalue and the second eigenvalue. It aims to adjust the prototypes of the old model by learning a set of perturbation parameters, so that the similarity between prototypes that are too close is reduced to below a certain threshold, thereby increasing the distinction between prototypes.

[0143] Step 702: Solve the disturbance objective function to obtain a third disturbance, and perform disturbance calculation on the first eigenvalue based on the third disturbance to obtain a third eigenvalue.

[0144] Among them, after constructing the perturbation objective function, the next step is to solve the objective function, and solving the perturbation objective function can be performed by optimization algorithms such as gradient descent, Newton method, etc. After obtaining the third perturbation, the first eigenvalue can be perturbed and calculated based on the perturbation to obtain the third eigenvalue.

[0145]

[0146] In this step, the new features generated by the model (i.e., the second eigenvalue) are further guided, and the optimization goal is to reduce the similarity between prototypes to below a certain threshold, so that old prototypes that are too close are far away from each other. By constructing the perturbation objective function, the perturbation calculation problem is transformed into an optimization problem, and the optimal perturbation and third eigenvalue are obtained by solving the optimization problem.

[0147] The above two perturbation methods, steps 501 to 504 and steps 601 to 604, describe a neighbor-driven prototype perturbation (NDPP). When calculating the perturbation for a certain old prototype, other prototypes with a higher similarity ranking with the prototype are considered, which are called the neighbors of the prototype. The vector difference between each neighbor prototype and the prototype is used as one of the perturbation components, and the similarity is used as the weight of the perturbation component. The overall weighted perturbation of the first K neighbors in similarity ranking is calculated as the perturbation value of the prototype. Its advantage is that it utilizes the local structural information between the prototypes. By considering the influence of the neighbor prototypes, a more adaptive and targeted perturbation can be obtained. At the same time, the calculation is simple and efficient. The calculation complexity can be controlled by setting the number of neighbors K, which is suitable for scenes with large data volumes.

[0148] The second calculation method, steps 701 to 702, is optimization-driven prototype perturbation, which uses a set of learnable parameters as the perturbation of the old prototype. The optimization goal is to reduce the similarity between prototypes to below a certain threshold, so that the old prototypes that are too close are far away from each other. By constraining the sample features extracted by the new model to align with the old prototype after perturbation, compatibility between the new and old models can be achieved, thereby realizing the backfill-free upgrade of the retrieval system. Its advantage is that the global optimal perturbation parameters are obtained through optimization solution, and the relationship between all prototypes is considered, which has stronger globality. By setting the similarity threshold, the minimum distance between the prototypes after perturbation can be controlled to balance feature compatibility and discriminability. The NDPP method is more suitable for scenarios with large data volumes, such as large-scale image or video retrieval systems, because of its simple and efficient calculation. The ODPP method, however, has a higher computational complexity due to its optimization solution, and is more suitable for scenarios with smaller data volumes, such as small-scale feature retrieval or recommendation systems.

[0149] Step 304: Calculate the target feature vector corresponding to each data sample based on the second classification model, and construct a negative sample pair according to the target feature vector corresponding to each data sample and the first eigenvalue corresponding to the sub-dataset to which each data sample belongs, and construct a positive sample pair according to the target feature vector corresponding to each data sample and the third eigenvalue corresponding to the sub-dataset to which each data sample belongs, and calculate the first loss function based on the positive sample and the negative sample pairs.

[0150] Among them, in step 304, firstly, the target feature vector corresponding to each data sample is calculated based on the second classification model. Positive sample pairs and negative sample pairs are constructed. The negative sample pairs are composed of the target feature vector corresponding to each data sample and the first feature value corresponding to the sub-dataset to which the data sample belongs. The first feature value here is the feature generated by the old model. The goal is that the new model can distinguish the target feature from the old model feature, so they are formed into negative sample pairs. The positive sample pair is composed of the target feature vector corresponding to each data sample and the third feature value corresponding to the sub-dataset to which the data sample belongs. The third feature value is the feature value after perturbation. It retains the characteristics of the old model to a certain extent, and also approaches the target feature of the new model, so they are formed into positive sample pairs. Finally, comparative learning is performed based on the constructed positive sample pairs and negative sample pairs, and the first loss function is calculated. The purpose of this loss function is to enable the new model to distinguish the target feature from the old model feature (through the negative sample pair), and at the same time associate the target feature with the perturbed feature (through the positive sample pair). The key is to guide the learning direction of the new model by constructing positive and negative sample pairs, so that it retains good classification ability while retaining the characteristics of the old model.

[0151] Reference Figure 8 In some embodiments, the process of constructing positive and negative sample pairs includes but is not limited to steps 801 to 804.

[0152] Step 801: Determine the sub-datasets other than the sub-dataset to which each data sample belongs in the training data set as negative sample data sets.

[0153] Among them, for each data sample in the training data set, the sub-datasets other than the sub-dataset to which it belongs are determined as negative sample data sets. This means that for each data sample, the sub-dataset to which it does not belong is regarded as a negative sample data set. In this way, sub-datasets with different categories or different feature distributions from the current data sample can be selected as negative sample data sets to construct effective negative sample pairs.

[0154] Step 802: construct a negative sample pair according to the target feature vector and the first feature value corresponding to the negative sample data set.

[0155] For each data sample, its target feature vector The first eigenvalue corresponding to each sub-dataset in the negative sample dataset , forming a negative sample pair In this way, multiple negative sample pairs are obtained, each of which consists of the target feature vector of the data sample and the first feature value of a negative sample sub-dataset. These negative sample pairs represent the difference between the representation of the data sample in the new feature space and the representation of the sub-dataset to which it does not belong in the old feature space.

[0156] Step 803: Determine the sub-dataset corresponding to the sub-dataset to which each data sample in the training data set belongs as positive sample data.

[0157] Among them, for each data sample in the training data set, the sub-dataset to which it belongs is determined as the positive sample data set. This means that for each data sample, the sub-dataset to which it belongs is regarded as the positive sample data set. In this way, a sub-dataset with the same category or similar feature distribution as the current data sample can be selected as the positive sample data set to construct a valid positive sample pair.

[0158] Step 804: construct a positive sample pair according to the target feature vector and the third feature value corresponding to the positive sample data set.

[0159] For each data sample, its target feature vector The third eigenvalue corresponding to each sub-dataset in the positive sample dataset , forming a positive sample pair In this way, multiple positive sample pairs are obtained, each of which consists of the target feature vector of the data sample and the third feature value of a positive sample sub-dataset. These positive sample pairs represent the similarity between the representation of the data sample in the new feature space and the representation of the sub-dataset to which it belongs in the old feature space after perturbation.

[0160] Through steps 801 to 804, for each data sample, multiple negative sample pairs and multiple positive sample pairs are constructed. These sample pairs will be used to calculate the first loss function to train the second classification model. By reasonably selecting the negative sample data set and the positive sample data set and constructing the corresponding sample pairs, the second classification model can learn a mapping function that enables it to map the data sample to a target feature vector that is similar to the third eigenvalue of the positive sample sub-dataset and different from the first eigenvalue of the negative sample sub-dataset, thereby achieving improved feature compatibility and discriminability.

[0161] In some embodiments, after obtaining the positive sample pair And negative sample pairs After that, we can construct the loss function of contrastive learning for:

[0162]

[0163] Among them, represents the cosine similarity, is the temperature coefficient. By minimizing , the target feature vector is pushed to the third eigenvalue of the same sub-dataset , that is, the pseudo-old prototype, far away from the first eigenvalue of other sub-datasets , which is the old prototype.

[0164] Step 305: classify the data samples in the training data set based on the second classification model to obtain the classification accuracy of the second classification model, and calculate the second loss function of the second classification model according to the classification accuracy of the second classification model.

[0165] Among them, the trained second classification model can be first used to classify each data sample in the training data set. For each data sample, it is input into the second classification model to obtain the output result of the model, that is, the predicted category label of the data sample. Then, the predicted category label is compared with the true category label of the data sample, the number of correctly classified samples and the total number of samples are counted, and the classification accuracy of the second classification model on the training data set is calculated. The classification accuracy represents the classification performance of the second classification model on the training data set, that is, the proportion of the number of correctly classified samples to the total number of samples. Generally speaking, the higher the classification accuracy, the stronger the classification ability of the second classification model, and the more accurately the data samples can be classified into the correct category. Then, the second loss function of the second classification model is calculated according to the classification accuracy of the second classification model. The second loss function measures the classification error of the second classification model on the training data set, that is, the proportion of the number of samples with incorrect classification to the total number of samples. The role of the second loss function is to evaluate the classification performance of the second classification model on the training data set, reflecting the generalization ability of the model and its adaptability to new data.

[0166] Reference Fig. 9 In some embodiments, step 305 includes but is not limited to steps 901 to 903.

[0167] Step 901: classify each data sample in the training data set based on the second classification model to obtain a classification result for each data sample.

[0168] Among them, for each data sample in the training data set, it is input into the second classification model, and after the forward propagation calculation of the model, the output result of the model is obtained, that is, the predicted category label of the data sample. This predicted category label represents the classification result of the second classification model for the data sample.

[0169] Step 902: Calculate the classification accuracy based on the classification results and the correct classification results preset in the training data set.

[0170] For each data sample in the training data set, the predicted category label obtained by the second classification model is compared with the preset correct category label of the sample, and the number of correctly classified samples and the total number of samples are counted. Then, the classification accuracy is calculated, that is, the ratio of the number of correctly classified samples to the total number of samples.

[0171] Step 903: Based on the classification loss of the cross entropy and the classification accuracy, a second loss function is calculated.

[0172] In some embodiments, based on the classification loss and classification accuracy of the cross entropy, the second loss function is calculated and expressed as follows:

[0173]

[0174] Where N represents the number of samples, Represents the probability that the i-th sample is predicted to be in the sub-dataset c. The cross entropy classification loss measures the difference between the predicted probability distribution of the second classification model and the true probability distribution. For each data sample, the cross entropy loss between its predicted probability distribution and the true label is calculated, and then the cross entropy loss of all samples is averaged to obtain the average cross entropy loss on the entire training dataset. .

[0175] Step 306: construct a target loss function based on the first loss function and the second loss function, and update the model parameters of the second classification model according to the target loss function to obtain an updated second classification model.

[0176] The target loss function consists of two parts: classification loss and compatibility loss. The classification loss corresponds to the second loss function, that is, the loss based on cross entropy, which is used to measure the classification performance of the second classification model on the training data set. The compatibility loss corresponds to the first loss function, that is, the contrastive learning loss, which is used to measure the compatibility between the features extracted by the second classification model and the features of the old model. The target loss function is expressed as:

[0177]

[0178] in, represents the total objective loss function, represents the classification loss (the second loss function), represents the compatibility loss (the first loss function can be an indicator, choose it appropriately), and λ is a weight parameter used to balance the contribution of classification loss and compatibility loss. The larger the value of λ, the more attention is paid to the compatibility loss, and the feature compatibility between the new model and the old model will be more guaranteed. The smaller the value of λ, the more attention is paid to the classification loss, and the discriminative performance of the new model will be more improved.

[0179] In the optimization process, by minimizing the target loss function, the classification performance and feature compatibility of the second classification model can be optimized at the same time. The specific optimization algorithm can be a commonly used gradient descent algorithm, such as stochastic gradient descent (SGD), Adam optimizer, etc. By calculating the gradient of the target loss function to the model parameters and updating the model parameters according to the gradient, the second classification model can be iteratively optimized to continuously improve the classification performance and feature compatibility.

[0180] In general, step 306 combines the classification loss and the compatibility loss by constructing the target loss function, and optimizes the second classification model according to the target loss function, thereby achieving a smooth transition and upgrade between the old and new models. This step balances the discriminative performance and feature compatibility of the model, and provides important technical support for achieving efficient and stable model iteration. By setting the weight parameters reasonably, the contribution of the classification loss and the compatibility loss can be flexibly adjusted to adapt to different task requirements and performance preferences.

[0181] See also Fig.10 In some embodiments, after step 306, the feature backward compatibility learning method between models provided in the embodiment of the present application may also include steps 1001 to 1005. The purpose of this series of steps is to further optimize the second classification model, and to achieve continuous compatibility and performance improvement between the new and old models by iteratively updating feature perturbations, sample pair construction, loss function calculation, and model parameter adjustment.

[0182] Step 1001: Perform perturbation calculation on the first eigenvalue corresponding to each sub-data set based on the updated second classification model to obtain the fourth eigenvalue corresponding to each sub-data set.

[0183] The purpose of this step is to use the feature representation capability of the updated second classification model to perturb the features of the old model again to obtain a more compatible and discriminative feature representation. By inputting the first eigenvalue into the updated second classification model and appropriately transforming and processing the output of the model, the fourth eigenvalue can be obtained as the perturbation result of a new round of iterations. The specific perturbation method is referred to above and will not be repeated here.

[0184] Step 1002: Calculate the target feature vector corresponding to each data sample based on the updated second classification model, and update the negative sample pairs according to the target feature vector corresponding to each data sample and the first eigenvalue corresponding to the sub-dataset to which each data sample belongs, and update the positive sample pairs according to the target feature vector corresponding to each data sample and the fourth eigenvalue corresponding to the sub-dataset to which each data sample belongs.

[0185] It can be understood that the feature representation capability of the updated second classification model is used to reconstruct the positive and negative sample pairs to adapt to the new feature space structure. By inputting each data sample into the updated second classification model to obtain the target feature vector, and constructing negative sample pairs and positive sample pairs with the first eigenvalue and the fourth eigenvalue respectively, a more accurate and effective sample pair representation can be obtained.

[0186] Step 1003: Update the first loss function according to the updated positive samples and the updated negative samples to obtain an updated first loss function, and classify the data samples in the training data set based on the updated second classification model, and update the second loss function of the second classification model according to the classification accuracy of the second classification model to obtain an updated second loss function.

[0187] In step 1003, the updated compatibility loss and classification loss are recombined to construct a new target loss function to guide the next round of model optimization. By reasonably setting the weights of the compatibility loss and classification loss, a more balanced and effective target loss function can be obtained, while taking into account the compatibility and discriminability of the new and old models.

[0188] Step 1004: Update the target loss function based on the updated first loss function and the updated second loss function to obtain an updated target loss function.

[0189] Among them, based on the construction of the new feature space and sample pairs, the compatible loss and classification loss are recalculated to evaluate the performance of the current model. By using the updated positive and negative sample pairs, a more accurate compatible loss (first loss function) can be obtained; by using the updated second classification model to classify the training data set, a more accurate classification loss (second loss function) can be obtained.

[0190] Step 1005: Update the model parameters of the second classification model again based on the updated target loss function to obtain a second classification model that has been updated again.

[0191] The purpose of this step is to fine-tune the parameters of the second classification model according to the new target loss function to further improve the performance of the model. By using a more accurate and effective target loss function, the adjustment of model parameters can be guided so that the new model can continuously improve its own discrimination ability while maintaining compatibility with the old model.

[0192] Steps 1001 to 1005 provide an iterative optimization strategy, which achieves continuous compatibility and performance improvement between the new and old models by repeatedly updating feature perturbations, sample pair construction, loss function calculation, and model parameter adjustment. This series of steps fully utilizes the feature representation capabilities of the updated second classification model, continuously optimizes the feature space structure and sample pair quality, and balances the requirements of compatibility and discriminability. By reasonably setting the number of iterations and convergence conditions, a new classification model with superior performance and strong compatibility can be obtained.

[0193] See also Fig.11 The embodiment of the present application further provides a device 1100 for learning backward-compatible features between models, which can implement the above-mentioned method for learning backward-compatible features between models, including:

[0194] An acquisition unit 1101 is used to acquire a first classification model, a second classification model, and a training data set; wherein the training data set includes a plurality of sub-data sets, and each sub-data set includes at least one data sample;

[0195] A first calculation unit 1102 is used to calculate each sub-dataset based on the first classification model to obtain a first eigenvalue corresponding to each sub-dataset, and to calculate each sub-dataset based on the second classification model to obtain a second eigenvalue corresponding to each sub-dataset;

[0196] A perturbation unit 1103 is used to perform perturbation calculation on the first eigenvalue corresponding to each sub-data set to obtain a third eigenvalue corresponding to each sub-data set;

[0197] A second calculation unit 1104 is used to calculate a target feature vector corresponding to each data sample based on the second classification model, and construct a negative sample pair according to the target feature vector corresponding to each data sample and the first feature value corresponding to the sub-dataset to which each data sample belongs, and construct a positive sample pair according to the target feature vector corresponding to each data sample and the third feature value corresponding to the sub-dataset to which each data sample belongs, and calculate a first loss function according to the positive sample and the negative sample pair;

[0198] The third calculation unit 1105 is used to classify the data samples in the training data set based on the second classification model to obtain the classification accuracy of the second classification model, and calculate the second loss function of the second classification model according to the classification accuracy of the second classification model;

[0199] The updating unit 1106 is used to construct a target loss function based on the first loss function and the second loss function, and update the model parameters of the second classification model according to the target loss function to obtain an updated second classification model.

[0200] The present application proposes a method for backward-compatible learning of features between models, an electronic device, and a storage medium. First, a first classification model, a second classification model, and a training data set are obtained; wherein the training data set includes multiple sub-data sets, and each sub-data set includes at least one data sample; each sub-data set is calculated based on the first classification model to obtain a first eigenvalue corresponding to each sub-data set, and each sub-data set is calculated based on the second classification model to obtain a second eigenvalue corresponding to each sub-data set; a perturbation calculation is performed on the first eigenvalue corresponding to each sub-data set to obtain a third eigenvalue corresponding to each sub-data set; a target feature vector corresponding to each data sample is calculated based on the second classification model, and a target feature vector corresponding to each data sample is obtained according to the target feature vector of each data sample. A negative sample pair is constructed between the target feature vector corresponding to the present invention and the first eigenvalue corresponding to the sub-dataset to which each data sample belongs, a positive sample pair is constructed between the target feature vector corresponding to each data sample and the third eigenvalue corresponding to the sub-dataset to which each data sample belongs, and a first loss function is calculated based on the positive and negative sample pairs; the data samples in the training data set are classified based on the second classification model to obtain the classification accuracy of the second classification model, and a second loss function of the second classification model is calculated based on the classification accuracy of the second classification model; a target loss function is constructed based on the first loss function and the second loss function, and the model parameters of the second classification model are updated according to the target loss function to obtain an updated second classification model.

[0201] With respect to the problems in the background technology, the method for learning backward-compatible features between models provided in the embodiment of the present application first calculates the eigenvalues ​​of each sub-data set using the first classification model and the second classification model respectively to obtain the first eigenvalue and the second eigenvalue. In the prior art, the second eigenvalue is usually required to be strictly aligned with the first eigenvalue to achieve compatibility between models. This approach ignores the influence of the distribution of the feature space of the first classification model itself on the discrimination ability of the second classification model. The present solution relaxes the backward-compatibility constraint of the first classification model by applying a disturbance to the first eigenvalue to obtain the third eigenvalue. The third eigenvalue makes it easier to distinguish the category feature centers that were originally difficult to distinguish in the feature space of the first classification model. Next, Here, a positive sample pair is constructed based on the target feature vector corresponding to each data sample and the third eigenvalue corresponding to the sub-dataset to which each data sample belongs, so as to improve the discrimination ability of the second classification model for categories that were originally difficult to distinguish, and a negative sample pair is constructed between the target feature vector corresponding to each data sample and the first eigenvalue corresponding to the sub-dataset to which each data sample belongs, so as to ensure that the second classification model is still compatible with the first classification model. The first loss function is calculated according to the positive and negative sample pairs, and then the second loss function corresponding to the classification accuracy of the second classification model is calculated. By minimizing the target loss function composed of the first loss function and the second loss function, the second classification model can be made compatible with the old model while having good discrimination performance.

[0202] The present application also provides an electronic device, including:

[0203] at least one memory;

[0204] at least one processor;

[0205] at least one program;

[0206] The program is stored in the memory, and the processor executes at least one program to implement the above-mentioned method for backward-compatible learning of features between models in the present application. The electronic device can be any intelligent terminal including a mobile phone, a tablet computer, a personal digital assistant (PDA), a car computer, etc.

[0207] See also Fig.12 , Fig.12 The hardware structure of an electronic device of another embodiment is illustrated, and the electronic device includes:

[0208] The processor 1201 may be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present application;

[0209] The memory 1202 can be implemented in the form of ROM (Read Only Memory), static storage device, dynamic storage device or RAM (Random Access Memory). The memory 1202 can store operating systems and other applications. When the technical solutions provided in the embodiments of this specification are implemented by software or firmware, the relevant program codes are stored in the memory 1202, and the processor 1201 calls and executes the feature backward-compatible learning method between models in the embodiments of this application;

[0210] Input / output interface 1203, used to implement information input and output;

[0211] The communication interface 1204 is used to realize the communication interaction between the device and other devices. The communication can be realized through a wired manner (such as USB, network cable, etc.) or a wireless manner (such as mobile network, WIFI, Bluetooth, etc.);

[0212] A bus 1205 that transmits information between various components of the device (e.g., the processor 1201, the memory 1202, the input / output interface 1203, and the communication interface 1204);

[0213] The processor 1201 , the memory 1202 , the input / output interface 1203 and the communication interface 1204 are connected to each other in communication within the device via the bus 1205 .

[0214] An embodiment of the present application also provides a storage medium, which is a computer-readable storage medium and stores a computer program. When the computer program is executed by a processor, it implements the feature backward-compatible learning method between the above-mentioned models.

[0215] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely disposed relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0216] The embodiments described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application, and do not constitute a limitation on the technical solutions provided in the embodiments of the present application. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0217] Those skilled in the art will appreciate that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present application, and may include more or fewer steps than shown in the figures, or a combination of certain steps, or different steps.

[0218] The device embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separated, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0219] Those skilled in the art will appreciate that all or some of the steps in the methods disclosed above, and the functional modules / units in the systems and devices may be implemented as software, firmware, hardware, or a suitable combination thereof.

[0220] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0221] It should be understood that in the present application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c. A), can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0222] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the above units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. The mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0223] The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0224] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0225] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including multiple instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, referred to as ROM), random access memory (Random Access Memory, referred to as RAM), disk or optical disk and other media that can store programs.

[0226] The preferred embodiments of the present application are described above with reference to the accompanying drawings, but the scope of the rights of the present application is not limited thereto. Any modification, equivalent substitution and improvement made by those skilled in the art without departing from the scope and essence of the present application should be within the scope of the rights of the present application.

Claims

1. A method for backward-compatible feature learning between models, characterized in that: The method comprises: Acquire a first classification model, a second classification model and a training data set; wherein the training data set includes a plurality of sub-data sets, each of the sub-data sets includes at least one data sample, and the data sample is an image or a feature vector of an image; Calculating each of the sub-datasets based on the first classification model to obtain a first eigenvalue corresponding to each of the sub-datasets, and calculating each of the sub-datasets based on the second classification model to obtain a second eigenvalue corresponding to each of the sub-datasets, including: calculating a first eigenvector of each of the data samples in each of the sub-datasets based on the first classification model, and calculating a second eigenvector of each of the data samples in each of the sub-datasets based on the second classification model; performing mean calculation based on all of the first eigenvectors to obtain the first eigenvalue, and performing mean calculation based on all of the second eigenvectors to obtain the second eigenvalue; Performing perturbation calculation on the first eigenvalue corresponding to each of the sub-data sets to obtain a third eigenvalue corresponding to each of the sub-data sets; Calculate the target feature vector corresponding to each of the data samples based on the second classification model, construct a negative sample pair according to the target feature vector corresponding to each of the data samples and the first feature value corresponding to the sub-dataset to which each of the data samples belongs, construct a positive sample pair according to the target feature vector corresponding to each of the data samples and the third feature value corresponding to the sub-dataset to which each of the data samples belongs, and calculate a first loss function according to the positive sample pairs and the negative sample pairs; Classifying the data samples in the training data set based on the second classification model to obtain the classification accuracy of the second classification model, and calculating the second loss function of the second classification model according to the classification accuracy of the second classification model; A target loss function is constructed based on the first loss function and the second loss function, and the model parameters of the second classification model are updated according to the target loss function to obtain an updated second classification model.

2. The method for backward-compatible feature learning between models according to claim 1, characterized in that: The performing perturbation calculation on the first eigenvalue corresponding to each sub-data set to obtain the third eigenvalue corresponding to each sub-data set includes: Based on the first eigenvalue, a first neighbor sub-dataset of each sub-dataset is calculated; Calculating a first similarity between the first nearest neighbor sub-dataset and the sub-dataset; Calculating a first disturbance amount based on the first similarity; The first eigenvalue is perturbed and calculated based on the first perturbation amount to obtain a third eigenvalue corresponding to each of the sub-data sets.

3. The method for backward-compatible feature learning between models according to claim 2, characterized in that: The method further comprises: Based on the second eigenvalue, calculate and obtain a second nearest neighbor sub-dataset of each of the sub-datasets; Calculating a second similarity between the second nearest neighbor sub-dataset and the sub-dataset; Calculating a second disturbance amount based on the second similarity; Based on the first disturbance amount and the second disturbance amount, a disturbance calculation is performed on the first eigenvalue to obtain a third eigenvalue corresponding to each of the sub-data sets.

4. The method for backward-compatible feature learning between models according to claim 1, characterized in that: The performing perturbation calculation on the first eigenvalue corresponding to each sub-data set to obtain the third eigenvalue corresponding to each sub-data set includes: Constructing a disturbance objective function based on each of the sub-data sets, the first eigenvalue corresponding to each of the sub-data sets, and the second eigenvalue corresponding to each of the sub-data sets; The disturbance objective function is solved to obtain a third disturbance, and the first eigenvalue is disturbed and calculated based on the third disturbance to obtain the third eigenvalue.

5. The method for backward-compatible feature learning between models according to claim 1, characterized in that: The method further comprises: constructing a negative sample pair according to a target feature vector corresponding to each of the data samples and a first feature value corresponding to a sub-dataset to which each of the data samples belongs, and constructing a positive sample pair according to a target feature vector corresponding to each of the data samples and a third feature value corresponding to a sub-dataset to which each of the data samples belongs, including: Determine, in the training data set, a sub-data set other than the sub-data set to which each of the data samples belongs as a negative sample data set; Constructing the negative sample pair according to the target feature vector and the first feature value corresponding to the negative sample data set; Determine the sub-dataset corresponding to the sub-dataset to which each data sample belongs in the training data set as a positive sample data set; The positive sample pair is constructed according to the target feature vector and the third feature value corresponding to the positive sample data set.

6. The method for backward-compatible feature learning between models according to claim 1, characterized in that: The method of classifying the data samples in the training data set based on the second classification model to obtain the classification accuracy of the second classification model, and calculating the second loss function of the second classification model according to the classification accuracy of the second classification model includes: Classifying each of the data samples in the training data set based on the second classification model to obtain a classification result for each of the data samples; Calculating the classification accuracy based on the classification result and the correct classification result preset in the training data set; The second loss function is calculated based on the classification loss of the cross entropy and the classification accuracy.

7. The method for backward-compatible feature learning between models according to claim 1, characterized in that: After constructing a target loss function based on the first loss function and the second loss function, and updating the model parameters of the second classification model according to the target loss function to obtain the updated second classification model, the method further includes: Performing a perturbation calculation on the first eigenvalue corresponding to each of the sub-data sets based on the updated second classification model to obtain a fourth eigenvalue corresponding to each of the sub-data sets; Calculate the target feature vector corresponding to each of the data samples based on the updated second classification model, and update the negative sample pair according to the target feature vector corresponding to each of the data samples and the first feature value corresponding to the sub-dataset to which each of the data samples belongs, and update the positive sample pair according to the target feature vector corresponding to each of the data samples and the fourth feature value corresponding to the sub-dataset to which each of the data samples belongs; The first loss function is updated according to the updated positive sample pairs and the updated negative sample pairs to obtain an updated first loss function, and the data samples in the training data set are classified based on the updated second classification model, and the second loss function of the second classification model is updated according to the classification accuracy of the second classification model to obtain an updated second loss function; Update the target loss function based on the updated first loss function and the updated second loss function to obtain an updated target loss function; The model parameters of the updated second classification model are updated again based on the updated objective loss function to obtain a second classification model that is updated again.

8. An electronic device, characterized in that: It includes a memory and a processor, the memory stores a computer program, and is characterized in that when the processor executes the computer program, it implements the feature backward-compatible learning method between models described in any one of claims 1 to 7.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for backward-compatible learning of features between models described in any one of claims 1 to 7 is implemented.