A pre-training model selection method, related apparatus, device and storage medium

Abstract

The application discloses a pre-training model selection method, which can be applied to various scenes such as cloud technology, artificial intelligence, intelligent transportation and auxiliary driving. The application comprises the following steps: extracting a feature vector set through a pre-training model based on a target data set; determining an inter-class interval matrix and an intra-class interval matrix according to the feature vector set and a category annotation label to which each training sample belongs; determining a regularization coefficient according to the inter-class interval matrix; determining a projection matrix according to the regularization coefficient, the inter-class interval matrix and the intra-class interval matrix; determining a model performance score of the pre-training model according to the projection matrix, the feature vector set and the category annotation label to which each training sample belongs; and selecting a plurality of pre-training models according to the model performance score of each pre-training model. The application also provides a device, equipment and medium. The application can construct an index for evaluating the migration ability of the pre-training model without relying on a classifier, so as to realize model optimization.

Description

Technical Field

[0001] This application relates to the field of deep learning technology, and in particular to a method for selecting pre-trained models, related devices, equipment, and storage media. Background Technology

[0002] The reusability of deep learning models is a crucial objective in real-world business scenarios. Because these scenarios are highly variable, researchers often need to train new models from scratch on each specific business dataset to achieve better performance. For example, different business scenarios require different content moderation and classification models; training each model from scratch would incur significant costs in terms of both human annotation and machine training.

[0003] To address the aforementioned issues, a model selection scheme based on pre-trained model libraries can be employed. In recent years, metrics based on the transferability of pre-trained models have received increasing attention from researchers. Currently, the log expected empirical prediction (LEEP) can be constructed as an indicator of the transferability of pre-trained models based on the modeling relationship between features and labels in downstream tasks.

[0004] The inventors discovered that existing solutions suffer from at least the following problem: although LEEP is computationally easy, it relies on a classifier in a pre-trained model. However, many pre-trained models lack a classifier, such as self-supervised models. Therefore, LEEP can only be used to estimate pre-trained models with a classifier, which limits its application. Summary of the Invention

[0005] This application provides a method, related apparatus, device, and storage medium for selecting pre-trained models. This application enables the construction of metrics for evaluating the transferability of pre-trained models without relying on a classifier, thereby achieving model optimization from a pre-trained model library.

[0006] In view of this, this application provides a method for selecting a pre-trained model, including:

[0007] For each of the M pre-trained models, based on the target dataset, a set of feature vectors is extracted from the pre-trained model. Each training sample in the target dataset corresponds to a category label, and the set of feature vectors includes the feature vectors of each training sample. M is an integer greater than 1.

[0008] For each pre-trained model, the inter-class margin matrix and intra-class margin matrix are determined based on the feature vector set and the class label of each training sample.

[0009] For each pre-trained model, the regularization coefficients are determined based on the inter-class margin matrix;

[0010] For each pre-trained model, the projection matrix is ​​determined based on the regularization coefficient, the inter-class margin matrix, and the intra-class margin matrix.

[0011] For each pre-trained model, a model performance score is determined based on the projection matrix, the set of feature vectors, and the category label of each training sample.

[0012] Based on the model performance score of each pre-trained model, select at least one pre-trained model from the M pre-trained models.

[0013] This application also provides a model selection device, comprising:

[0014] The extraction module is used to extract a set of feature vectors for each of the M pre-trained models based on the target dataset. Each training sample in the target dataset corresponds to a category label, and the set of feature vectors includes the feature vectors of each training sample. M is an integer greater than 1.

[0015] The determination module is used to determine the inter-class margin matrix and the intra-class margin matrix for each pre-trained model, based on the feature vector set and the class label of each training sample.

[0016] The determination module is also used to determine the regularization coefficients for each pre-trained model based on the inter-class margin matrix;

[0017] The determination module is also used to determine the projection matrix for each pre-trained model based on the regularization coefficient, the inter-class margin matrix, and the intra-class margin matrix.

[0018] The determination module is also used to determine the model performance score of each pre-trained model based on the projection matrix, the set of feature vectors, and the category label of each training sample.

[0019] The selection module is used to select at least one pre-trained model from M pre-trained models based on the model performance score of each pre-trained model.

[0020] In one possible design, in another implementation of another aspect of the embodiments of this application,

[0021] The determination module is specifically used to calculate the average of the feature vector set to obtain the overall mean;

[0022] Based on the category label of each training sample, the feature vector set is divided into C feature vector subsets, where each feature vector subset corresponds to the same category label, and C is an integer greater than 1;

[0023] The average value of each feature vector subset is calculated to obtain the category mean value corresponding to each category label.

[0024] The inter-class margin matrix is ​​calculated based on the overall mean and the mean of each category label.

[0025] The intra-class margin matrix is ​​calculated based on each subset of feature vectors and the corresponding class mean.

[0026] In one possible design, in another implementation of another aspect of the embodiments of this application,

[0027] The determination module is specifically used to subtract the overall mean from the category mean for each category label in the C category label to obtain the first feature vector;

[0028] For each category label in the C category label, transpose the first feature vector to obtain the second feature vector;

[0029] For each category label in category C, the first feature vector and the second feature vector are multiplied to obtain the inter-class matrix corresponding to the category label;

[0030] Summing the inter-class matrices corresponding to the label of each category yields the inter-class margin matrix.

[0031] In one possible design, in another implementation of another aspect of the embodiments of this application,

[0032] The determination module is specifically used to subtract the class mean from each feature vector in the feature vector subset for each class label in the C class label to obtain the third feature vector corresponding to each feature vector in the feature vector subset;

[0033] For each category label in the C category label, transpose the third feature vector corresponding to each feature vector in the feature vector subset to obtain the fourth feature vector corresponding to each feature vector in the feature vector subset;

[0034] For each category label in the C category label, the third and fourth feature vectors corresponding to each feature vector in the feature vector subset are multiplied to obtain the first class matrix corresponding to each feature vector in the feature vector subset;

[0035] For each category label in the C category label, sum the first-class inner matrix corresponding to each feature vector in the feature vector subset to obtain the second-class inner matrix corresponding to the category label;

[0036] Summing the second-class intra-matrix corresponding to the label of each category yields the intra-class margin matrix.

[0037] In one possible design, in another implementation of another aspect of the embodiments of this application,

[0038] The determination module is specifically used to obtain at least one set of eigenvalues ​​based on the inter-class margin matrix. Each set of eigenvalues ​​includes an eigenvalue and an eigenvector, where the eigenvector is the eigenvector of the inter-class margin matrix corresponding to the eigenvalue.

[0039] Obtain the maximum eigenvalue from at least one set of eigenvalues;

[0040] The first value is calculated based on the maximum eigenvalue and the first preset parameter;

[0041] The second value is obtained by exponentiation of the first value with the natural constant as the base;

[0042] Based on the second value, the regularization coefficient is calculated using the activation function.

[0043] In one possible design, in another implementation of another aspect of the embodiments of this application,

[0044] The determination module is specifically used to construct the objective function based on the regularization coefficient, the inter-class margin matrix, the intra-class margin matrix, and the identity matrix;

[0045] The projection matrix is ​​calculated based on the objective function.

[0046] In one possible design, in another implementation of another aspect of the embodiments of this application,

[0047] The determination module is specifically used to project the feature vector corresponding to each training sample using a projection matrix to obtain the first target feature vector.

[0048] For each training sample, the sample category score is calculated based on the first target feature vector and the category label corresponding to the training sample.

[0049] For each training sample, the correct classification probability of the training sample is calculated based on the sample category score of the training sample.

[0050] The model performance score of the pre-trained model is calculated based on the correct classification probability corresponding to each training sample.

[0051] In one possible design, in another implementation of another aspect of the embodiments of this application,

[0052] The determination module is also used to determine the number of training samples corresponding to each category label based on the category label to which each training sample belongs;

[0053] The determination module is also used to calculate the prior probability of each category label based on the number of training samples corresponding to each category label and the total number of training samples included in the target dataset.

[0054] The determination module is specifically used to calculate the sample class score of the training sample for each class label based on the first target feature vector corresponding to the training sample, the projection matrix, the class mean corresponding to each class label, and the class prior probability corresponding to each class label.

[0055] In one possible design, in another implementation of another aspect of the embodiments of this application,

[0056] The determination module is specifically used to calculate (N-1) distances based on the first target feature vector corresponding to the training sample and the first target feature vector corresponding to each training sample in (N-1) training samples. The target data set includes N training samples, and the (N-1) training samples are the remaining training samples in the target data set excluding the training samples. N is an integer greater than 1.

[0057] Based on (N-1) distances, determine the K training samples that are closest to the training samples from the (N-1) training samples, where K is an integer greater than 1 and less than (N-1);

[0058] Based on the category label of each of the K training samples, the sample category score for each category label is obtained.

[0059] In one possible design, in another implementation of another aspect of the embodiments of this application,

[0060] The determination module is specifically used to calculate the first transferability score of the pre-trained model based on the correct classification probability corresponding to each training sample.

[0061] The selection module is specifically used to select at least one pre-trained model from M pre-trained models based on the first transferability score of each pre-trained model.

[0062] In one possible design, in another implementation of another aspect of the embodiments of this application,

[0063] The determination module is specifically used to process the noise of the feature vector corresponding to each training sample based on the correct classification probability of the training sample, so as to obtain a mixed feature vector.

[0064] Based on the mixed feature vector corresponding to each training sample and the category label to which each training sample belongs, determine the inter-class margin matrix and the intra-class margin matrix.

[0065] Determine the target regularization coefficient based on the inter-class margin matrix;

[0066] The target projection matrix is ​​determined based on the target regularization coefficient, the inter-class margin matrix, and the intra-class margin matrix.

[0067] The model performance score of the pre-trained model is determined based on the target projection matrix, the mixed feature vector corresponding to each training sample, and the category label of each training sample.

[0068] In one possible design, in another implementation of another aspect of the embodiments of this application,

[0069] The determination module is specifically used to sum the feature vectors in the feature vector subset corresponding to each category label in the (C-1) category label to obtain the category feature vector. Here, the (C-1) category labels are the remaining category labels in the C category labels excluding the category labels corresponding to the training samples. Each feature vector subset corresponds to the same category label, and C is an integer greater than 1.

[0070] The sample mixing noise is calculated based on the category feature vector, the total number of training samples included in the target dataset, and the number of training samples corresponding to the category labels of the training samples.

[0071] The mixed feature vector corresponding to the training sample is calculated based on the correct classification probability of the training sample, the feature vector corresponding to the training sample, and the mixed noise of the samples.

[0072] In one possible design, in another implementation of another aspect of the embodiments of this application,

[0073] The determination module is specifically used to perform feature projection on the mixed feature vector corresponding to each training sample using the target projection matrix to obtain the second target feature vector;

[0074] For each training sample, the target sample category score is calculated based on the second target feature vector and the category label corresponding to the training sample.

[0075] For each training sample, the probability of correct target classification corresponding to the training sample is calculated based on the target sample category score of the training sample.

[0076] The model performance score of the pre-trained model is calculated based on the correct classification probability of the target corresponding to each training sample.

[0077] In one possible design, in another implementation of another aspect of the embodiments of this application,

[0078] The determination module is specifically used to calculate the second transferability score of the pre-trained model based on the correct classification probability of the target corresponding to each training sample.

[0079] The selection module is specifically used to select at least one pre-trained model from M pre-trained models based on the second transferability score of each pre-trained model.

[0080] In one possible design, in another implementation of another aspect of the embodiments of this application,

[0081] The determination module is also used to calculate the complementarity score of each pre-trained model based on the projection matrix and the set of feature vectors corresponding to each pre-trained model.

[0082] The selection module is specifically used to calculate the comprehensive score corresponding to each pre-trained model based on the transferability score and complementarity score of each pre-trained model.

[0083] Based on the comprehensive score corresponding to each pre-trained model, select at least one pre-trained model from the M pre-trained models.

[0084] In one possible design, in another implementation of another aspect of the embodiments of this application,

[0085] The determination module is specifically used to calculate the multi-model feature matrix corresponding to each training sample based on the projection matrix and the feature vector set corresponding to each pre-trained model.

[0086] For each pre-trained model, obtain the first nuclear norm of the feature matrix of the multi-model corresponding to each training sample;

[0087] For each pre-trained model, the multi-model feature matrix corresponding to each training sample is multiplied by the matrix corresponding to the pre-trained model to obtain the third matrix corresponding to each training sample.

[0088] For each pre-trained model, obtain the second nuclear norm of the third matrix corresponding to each training sample;

[0089] For each pre-trained model, the complementarity score of the pre-trained model is calculated based on the first and second nuclear norms corresponding to each training sample.

[0090] In another aspect, this application provides a computer device including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the methods described above.

[0091] Another aspect of this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the methods described above.

[0092] Another aspect of this application provides a computer program product, including a computer program that, when executed by a processor, implements the methods described above.

[0093] As can be seen from the above technical solutions, the embodiments of this application have the following advantages:

[0094] This application provides a method for selecting pre-trained models. Specifically, for each of the M pre-trained models, features are extracted from the target dataset to obtain a set of feature vectors. Then, for each pre-trained model, the inter-class margin matrix and intra-class margin matrix are determined based on the feature vector set and the class label of each training sample. A regularization coefficient is then determined based on the inter-class margin matrix, and a projection matrix is ​​determined using the regularization coefficient, the inter-class margin matrix, and the intra-class margin matrix. Finally, a model performance score for each pre-trained model is determined based on the projection matrix, the feature vector set, and the class label of each training sample. Based on this, at least one pre-trained model is selected from the M pre-trained models by combining the model performance scores of each pre-trained model. By introducing a regularization coefficient based on Fisher discriminant analysis, the intra-class and inter-class distances of different pre-trained models on the target classification task can be better characterized, resulting in better class separability of the projected features. Better class separability is a key feature for fine-tuning training on downstream classification data. Therefore, without relying on a classifier, it is possible to construct an indicator for evaluating the transferability of a pre-trained model, thereby enabling model optimization from a pre-trained model library. Attached Figure Description

[0095] Figure 1 This is a schematic diagram illustrating model selection based on the transferability of pre-trained models in an embodiment of this application;

[0096] Figure 2 This is a schematic diagram of an environment for the model selection system in an embodiment of this application;

[0097] Figure 3 This is a flowchart illustrating the pre-trained model selection method in an embodiment of this application;

[0098] Figure 4 This is a schematic diagram illustrating the extraction of corresponding feature vector sets based on various pre-trained models in an embodiment of this application;

[0099] Figure 5 This is a schematic diagram illustrating the construction of the first mobility score in an embodiment of this application;

[0100] Figure 6 This is a schematic diagram illustrating the construction of a second mobility score in an embodiment of this application;

[0101] Figure 7 This is a schematic diagram of an interface for implementing model optimization in an embodiment of this application;

[0102] Figure 8 This is a schematic diagram of a model selection device in an embodiment of this application;

[0103] Figure 9 This is a schematic diagram of the structure of a computer device in an embodiment of this application. Detailed Implementation

[0104] This application provides a method, related apparatus, device, and storage medium for selecting pretrained models. This application enables the construction of metrics for evaluating the transferability of pretrained models without relying on a classifier, thereby achieving model optimization from a pretrained model library.

[0105] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a particular order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented, for example, in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “corresponding to,” and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0106] This application's embodiments can be applied to various scenarios, including but not limited to cloud technology, artificial intelligence (AI), smart transportation, and assisted driving. AI systems process various types of data, such as images, audio, text, and video. Data is the carrier of information, and classification is the process of determining which category a given data belongs to based on its different characteristics. Different businesses require different content classifications. For example, in a content review scenario, image content can be identified to determine whether an image contains low-quality content, and then images containing low-quality content can be processed. For example, in an e-commerce classification scenario, image content can be identified to determine the content category contained in the image, and then images with the same or similar content can be categorized.

[0107] For example, classifying images or videos typically involves computer vision (CV) technology. CV technology is the science of how to enable machines to "see," or more specifically, machine vision that uses cameras and computers to replace human eyes for target recognition and measurement, and further performs image processing to make the computer-processed images more suitable for human observation or transmission to instruments for detection. As a scientific discipline, CV studies related theories and technologies, attempting to build AI systems capable of extracting information from images or multidimensional data. CV technologies typically include image processing, image recognition, image semantic understanding, image retrieval, optical character recognition (OCR), video processing, video semantic understanding, video content / behavior recognition, 3D object reconstruction, 3D technology, virtual reality, augmented reality, simultaneous localization and mapping (SLAM), and common biometric recognition technologies such as facial recognition and fingerprint recognition.

[0108] For example, classifying sounds typically involves speech technology. Key technologies in speech technology include automatic speech recognition (ASR), text-to-speech (TTS), and voiceprint recognition. Enabling computers to hear, see, speak, and feel is the future direction of human-computer interaction, with speech emerging as one of the most promising methods for future human-computer interaction.

[0109] For example, classifying sounds typically involves Natural Language Processing (NLP) techniques. NLP is an important area of ​​computer science and AI. It studies the theories and methods that enable effective communication between humans and computers using natural language. NLP is a science that integrates linguistics, computer science, and mathematics. Therefore, research in this field involves natural language—the language people use in daily life—and thus it is closely related to linguistic research. NLP techniques typically include text processing, semantic understanding, machine translation, question answering, and knowledge graphs.

[0110] Considering that different business operations require different classification models, training each model from scratch would incur significant costs in terms of annotation manpower and training machines. To address this issue, model optimization schemes based on pre-trained model libraries have gained widespread attention. Specifically, models generated from various business operations can be collected as "pre-trained models," forming a pre-trained model library. When new business requirements arise, the optimal pre-trained model is selected from the library for fine-tuning to obtain the final model. Model optimization is a crucial issue in transfer learning. Given a downstream task, it's necessary to select the optimal pre-trained model from among numerous models—one that achieves the best performance on the downstream task after training and fine-tuning.

[0111] Based on this, the following will combine Figure 1 This section introduces the basic process of model optimization. For easier understanding, please refer to [link / reference needed]. Figure 1 , Figure 1 This is a schematic diagram illustrating model selection based on the transferability of pre-trained models in an embodiment of this application. As shown in the figure, it is assumed that the model library includes M pre-trained models, i.e., the model library is represented as... These pre-trained models are then trained and fine-tuned on an object classification task to obtain performance metrics for different pre-trained models on downstream tasks. This performance metric represents the classification accuracy of each pre-trained model on the downstream task test set after fine-tuning and training on the downstream task. Therefore, ranking these performance metrics reveals the optimal pre-trained model for the current downstream task.

[0112] While this ranking method is accurate, it is inefficient, time-consuming in training and fine-tuning, and difficult to apply in real-world problems. Therefore, model optimization can be performed by constructing a transferability metric for the pre-trained models. The transferability metric for the M pre-trained models is represented as... if and If the order is exactly the same, then the transferability metric can perfectly reflect the transferability of different pre-trained models. In this application, the weighted Kendall coefficient τ can be used. w To measure migration metrics Performance metrics fine-tuned with standard training The correlation coefficient. Where -1≤τ w ≤1, and the weighted Kendall coefficient τ w The closer a value is to 1, the more closely the ranking of the transferability metric approximates the ranking of training fine-tuning performance. In model selection, the Kendall coefficient characterizes the rank correlation between two model sequences. The weighted Kendall coefficient assigns greater weight to the top-ranked models based on the original Kendall coefficient; therefore, it measures the rank correlation of the top-ranked model sequence. This aligns well with the motivation for model selection. Besides the weighted Kendall coefficient, other evaluation coefficients such as the Pearson correlation coefficient or top-k relative accuracy can also be used; no specific limitation is made here.

[0113] The method provided in this application constructs a metric for measuring model transferability, namely, a model performance score. This score allows for the ranking of pre-trained models' transferability on downstream business data, thereby achieving fast and efficient model selection. The method provided in this application can be used... Figure 2 The model selection system shown in the figure may include at least one of a server and a terminal. For example, a model library may be stored on the server, and the terminal sends a target dataset corresponding to the target classification task to the server. The server then evaluates each pre-trained model based on the target dataset to generate a model performance score for each pre-trained model.

[0114] The server involved in this application can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms. The terminal can be a mobile phone, computer, smart voice interaction device, smart home appliance, vehicle terminal, aircraft, etc., but is not limited to these. The terminal and server can be directly or indirectly connected via wired or wireless communication, which is not limited in this application. The number of servers and terminals is also not limited. The solution provided in this application can be completed independently by the terminal, independently by the server, or jointly by the terminal and the server; this application does not specifically limit this.

[0115] Based on the above introduction, the method for selecting the pre-trained model in this application will be described below. Please refer to [link / reference]. Figure 3 The pre-trained model selection method in this application embodiment can be executed by a computer device, which can be a terminal or a server. The method of this application includes:

[0116] 210. For each of the M pre-trained models, extract a set of feature vectors based on the target dataset. Each training sample in the target dataset corresponds to a category label. The set of feature vectors includes the feature vectors of each training sample. M is an integer greater than 1.

[0117] In one or more embodiments, a target data set corresponding to the downstream classification task is obtained, wherein the target data set includes N training samples, and each training sample corresponds to a category label. That is, the target data set is represented as follows: This includes N training samples with class labels, x n Let y represent the nth training sample. n This represents the category label of the nth training sample.

[0118] Specifically, for ease of understanding, please refer to Figure 4 , Figure 4 This is a schematic diagram illustrating the extraction of corresponding feature vector sets based on various pre-trained models in this application embodiment. As shown in the figure, it is assumed that the model library includes M pre-trained models, and the target dataset includes N training samples. The target dataset is input into each pre-trained model, and the feature vector sets are extracted through the pre-trained models. For example, the target dataset is used as the input of "pre-trained model 1," and "pre-trained model 1" outputs the feature vector of each training sample in the target dataset, thus obtaining feature vector set 1. Different pre-trained models extract different feature vector sets.

[0119] It is understood that the pre-trained model in this application refers to a model that has already been trained, which can be used as a feature extraction model or transferred to downstream tasks.

[0120] 220. For each pre-trained model, determine the inter-class margin matrix and the intra-class margin matrix based on the feature vector set and the category label of each training sample.

[0121] In one or more embodiments, an inter-class margin matrix and a intra-class margin matrix need to be calculated separately for each pre-trained model. That is, based on the feature vector set and the class label of each training sample in the target dataset, an inter-class margin matrix and an intra-class margin matrix are calculated respectively. The inter-class margin matrix represents the feature relationship between different classes, and the intra-class margin matrix represents the feature relationship between the same class.

[0122] 230. For each pre-trained model, determine the regularization coefficients based on the inter-class margin matrix;

[0123] In one or more embodiments, a regularization coefficient needs to be calculated separately for each pre-trained model. This regularization coefficient is an adaptive regularization coefficient, the basic principle of which is that if the inter-class margin is smaller, then the supervision signal for the intra-class margin should be larger, and in this case, a smaller regularization strength should be used.

[0124] 240. For each pre-trained model, determine the projection matrix based on the regularization coefficient, the inter-class margin matrix, and the intra-class margin matrix;

[0125] In one or more embodiments, a projection matrix needs to be calculated for each pre-trained model based on the regularization coefficient, the inter-class margin matrix, and the intra-class margin matrix. This projection matrix is ​​used to project the static feature vectors into the Fisher space.

[0126] 250. For each pre-trained model, determine the model performance score based on the projection matrix, the set of feature vectors, and the category label of each training sample.

[0127] In one or more embodiments, for each pre-trained model, a model performance score needs to be calculated based on the projection matrix, the set of feature vectors, and the category label of each training sample.

[0128] 260. Based on the model performance score of each pre-trained model, select at least one pre-trained model from the M pre-trained models.

[0129] In one or more embodiments, after obtaining the model performance score of each pre-trained model, the model performance scores are sorted in descending order, and finally, one or more pre-trained models with the highest ranking are selected as the preferred base models for downstream tasks.

[0130] This application provides a method for selecting pre-trained models. By introducing a regularization coefficient based on Fisher discriminant analysis, the intra-class and inter-class distances of different pre-trained models on the target classification task can be better characterized, resulting in better class separability of the projected features. Better class separability is a key feature for fine-tuning training on downstream classification data. Therefore, without relying on a classifier, an indicator for evaluating the transferability of pre-trained models can be constructed, thereby enabling model optimization from a pre-trained model library.

[0131] Optionally, in the above Figure 3 Based on the corresponding embodiments, in another optional embodiment provided by this application, the inter-class margin matrix and intra-class margin matrix are determined according to the feature vector set and the category label of each training sample. Specifically, this may include:

[0132] The population mean is obtained by averaging the set of eigenvectors.

[0133] Based on the category label of each training sample, the feature vector set is divided into C feature vector subsets, where each feature vector subset corresponds to the same category label, and C is an integer greater than 1;

[0134] The average value of each feature vector subset is calculated to obtain the category mean value corresponding to each category label.

[0135] The inter-class margin matrix is ​​calculated based on the overall mean and the mean of each category label.

[0136] The intra-class margin matrix is ​​calculated based on each subset of feature vectors and the corresponding class mean.

[0137] In one or more embodiments, a method for constructing the overall mean and class mean is described. As can be seen from the foregoing embodiments, this application requires calculating the overall mean and class mean separately for each pre-trained model. Therefore, for ease of explanation, the following description will use any pre-trained model as an example.

[0138] I. Methods for calculating the population mean;

[0139] Specifically, a feature vector is extracted from each training sample in the target dataset using a pre-trained model. This feature vector can be understood as the static features extracted by the pre-trained model on the downstream task. Taking the nth training sample as an example, the feature vector extracted by the mth pre-trained model is represented as follows: in, Let x represent the feature vector of the nth training sample. n Let φ represent the nth training sample.m (·) represents the m-th pre-trained model.

[0140] Based on this, the population mean can be calculated in the following way:

[0141]

[0142] Where μ represents the population mean, and N represents the total number of training samples included in the target dataset. This represents the feature vector of the nth training sample. This represents the set of feature vectors.

[0143] II. Methods for calculating category means;

[0144] Specifically, the target data set can be divided according to categories, that is: Where D represents the target dataset. ∪ represents the union of the datasets. C represents the total number of categories in the category labels. c represents the label for the c-th category. N c This represents the number of training samples for the c-th category. This represents the nth training sample belonging to the cth category. This represents the category label corresponding to the nth training sample belonging to the cth category.

[0145] Therefore, based on the category label of each training sample, the target dataset can be divided into C training sample subsets, and correspondingly, the feature vector set can also be divided into C feature vector subsets. Each feature vector subset corresponds to the same category label, and the same training sample subset corresponds to the same category label. Based on this, the category mean corresponding to each category label can be calculated as follows:

[0146]

[0147] Where, μ c N represents the mean value of the category corresponding to the label of category c. c This represents the number of training samples belonging to the c-th category label. This represents the feature vector corresponding to the nth training sample belonging to the c-th category label (i.e., the nth feature vector in the subset of feature vectors corresponding to the c-th category label). This represents the subset of feature vectors corresponding to the c-th class label.

[0148] Therefore, on the one hand, the inter-class margin matrix is ​​calculated based on the overall mean and the class mean corresponding to each class label; on the other hand, the intra-class margin matrix is ​​calculated based on each feature vector subset and the corresponding class mean.

[0149] Secondly, this application provides a method for constructing the overall mean and category mean. This method provides a feasible approach to implementing the solution, thereby improving its feasibility and operability.

[0150] Optionally, in the above Figure 3 Based on the corresponding embodiments, in another optional embodiment provided by this application, the inter-class interval matrix is ​​calculated according to the overall mean and the category mean corresponding to each category label, which may specifically include:

[0151] For each category label in category C, the overall mean is subtracted from the category mean to obtain the first feature vector;

[0152] For each category label in the C category label, transpose the first feature vector to obtain the second feature vector;

[0153] For each category label in category C, the first feature vector and the second feature vector are multiplied to obtain the inter-class matrix corresponding to the category label;

[0154] Summing the inter-class matrices corresponding to the label of each category yields the inter-class margin matrix.

[0155] In one or more embodiments, a method for calculating the inter-class margin matrix is ​​described. As can be seen from the foregoing embodiments, this application requires calculating the inter-class margin matrix separately for each pre-trained model. Therefore, for ease of explanation, the following description will use any pre-trained model as an example.

[0156] Specifically, the inter-class spacing matrix can be calculated as follows:

[0157]

[0158] Among them, S B This represents the inter-class margin matrix. C represents the total number of categories with category labels. c represents the category label of the c-th category. μ represents the overall mean. c This represents the mean value of the category corresponding to the label of category c. (·) T This represents the transpose operation. (μ-μ) c (μ-μ) represents the first eigenvector. c ) T This represents the second eigenvector. (μ-μ) c )(μ-μ c ) T Represents the inter-class matrix.

[0159] As can be seen, after calculating the overall mean and the category mean corresponding to each category label, the overall mean can be subtracted from the mean of each category to obtain C first eigenvectors. Then, each first eigenvector is transposed to obtain its corresponding second eigenvector. Multiplying each first eigenvector by its corresponding second eigenvector yields the inter-class matrix. Finally, adding the C inter-class matrices gives the inter-class margin matrix.

[0160] Furthermore, this application provides a method for calculating the inter-class spacing matrix. This method provides a feasible approach to implementing the solution, thereby improving its feasibility and operability.

[0161] Optionally, in the above Figure 3 Based on the corresponding embodiments, in another optional embodiment provided by this application, the intra-class margin matrix is ​​calculated according to each subset of feature vectors and the corresponding class mean, which may specifically include:

[0162] For each category label in the C category label, subtract the category mean from each feature vector in the feature vector subset to obtain the third feature vector corresponding to each feature vector in the feature vector subset;

[0163] For each category label in the C category label, transpose the third feature vector corresponding to each feature vector in the feature vector subset to obtain the fourth feature vector corresponding to each feature vector in the feature vector subset;

[0164] For each category label in the C category label, the third and fourth feature vectors corresponding to each feature vector in the feature vector subset are multiplied to obtain the first class matrix corresponding to each feature vector in the feature vector subset;

[0165] For each category label in the C category label, sum the first-class inner matrix corresponding to each feature vector in the feature vector subset to obtain the second-class inner matrix corresponding to the category label;

[0166] Summing the second-class intra-matrix corresponding to the label of each category yields the intra-class margin matrix.

[0167] In one or more embodiments, a method for calculating the intra-class margin matrix is ​​described. As can be seen from the foregoing embodiments, this application requires calculating the intra-class margin matrix separately for each pre-trained model. Therefore, for ease of explanation, the following description will use any pre-trained model as an example.

[0168] Specifically, the intra-class margin matrix can be calculated as follows:

[0169]

[0170] Among them, S W This represents the intra-class margin matrix. C represents the total number of categories for the category labels. c represents the category label for the c-th category. μ c N represents the mean value of the category corresponding to the label of category c. c This represents the number of training samples belonging to the c-th category label. μ represents the feature vector corresponding to the nth training sample belonging to the c-th class label (i.e., the nth feature vector in the subset of feature vectors corresponding to the c-th class label). c This indicates the category label pair of category c.

[0171] Furthermore, this application provides a method for calculating the intra-class interval matrix. This method provides a feasible approach to implementing the solution, thereby improving its feasibility and operability.

[0172] Optionally, in the above Figure 3 Based on the corresponding embodiments, in another optional embodiment provided by this application, the regularization coefficient is determined according to the inter-class spacing matrix, which may specifically include:

[0173] At least one set of eigenvalues ​​is obtained by solving the inter-class margin matrix. Each set of eigenvalues ​​includes an eigenvalue and an eigenvector, where the eigenvector is the eigenvector of the inter-class margin matrix corresponding to the eigenvalue.

[0174] Obtain the maximum eigenvalue from at least one set of eigenvalues;

[0175] The first value is calculated based on the maximum eigenvalue and the first preset parameter;

[0176] The second value is obtained by exponentiation of the first value with the natural constant as the base;

[0177] Based on the second value, the regularization coefficient is calculated using the activation function.

[0178] In one or more embodiments, a method for calculating the regularization coefficient is described. As can be seen from the foregoing embodiments, this application requires calculating the regularization coefficient separately for each pre-trained model, where the regularization coefficient is an adaptive regularization coefficient. The basic principle is that if the inter-class margin is smaller, then the supervision signal for the intra-class margin should be larger. Based on this, for ease of explanation, the following will use any pre-trained model as an example.

[0179] Specifically, the regularization coefficient can be calculated as follows:

[0180]

[0181] Where λ represents the regularization coefficient. S B σ(S) represents the inter-class spacing matrix. B ) represents the maximum eigenvalue of the inter-class margin matrix. e represents the natural constant. a represents the first predefined parameter, for example, a = 2. aσ(S B ) indicates the first value. This indicates the second value.

[0182] Understandably, the eigenvalues ​​are solved as follows: S B x = Tx, where S B Let T represent the inter-class spacing matrix, where S is the inter-class spacing matrix. B Let x be an eigenvalue and x be an eigenvector. Typically, a matrix can be solved to yield at least one set of eigenvalues, each set including an eigenvalue and an eigenvector. Thus, the largest eigenvalue can be determined from at least one set of eigenvalues.

[0183] It should be noted that the activation function can be a function whose value range is between 0 and 1. This application uses the sigmoid function as an example, but this should not be construed as a limitation of this application.

[0184] Secondly, this application provides a method for calculating the regularization coefficient. This method provides a feasible approach to implementing the solution, thereby improving its feasibility and operability.

[0185] Optionally, in the above Figure 3 Based on the corresponding embodiments, in another optional embodiment provided by this application, the projection matrix is ​​determined according to the regularization coefficient, the inter-class margin matrix, and the intra-class margin matrix, which may specifically include:

[0186] Construct the objective function based on the regularization coefficient, inter-class margin matrix, intra-class margin matrix, and identity matrix;

[0187] The projection matrix is ​​calculated based on the objective function.

[0188] In one or more embodiments, a method for calculating the projection matrix is ​​described. As can be seen from the foregoing embodiments, this application requires calculating the projection matrix separately for each pre-trained model. Therefore, for ease of explanation, the following description will use any pre-trained model as an example.

[0189] Specifically, the objective function can be constructed as follows:

[0190] U = argmax U tr((UT [(1-λ)S W +λI]U) -1 U T S B U); Formula (6)

[0191] Where U represents the projection matrix. argmax U This represents finding the projection matrix U of the objective function. B S represents the inter-class spacing matrix. W Let represent the intra-class margin matrix. λ represents the regularization coefficient. I represents the identity matrix. tr(·) represents the trace of the matrix. (·) T This indicates the transpose operation.

[0192] Secondly, this application provides a method for calculating the projection matrix. This method provides a feasible approach to implementing the solution, thereby improving its feasibility and operability.

[0193] Optionally, in the above Figure 3 Based on the corresponding embodiments, in another optional embodiment provided by this application, the model performance score of the pre-trained model is determined according to the projection matrix, the feature vector set, and the category label of each training sample. Specifically, this may include:

[0194] For each training sample, the feature vector corresponding to the training sample is projected using a projection matrix to obtain the first target feature vector;

[0195] For each training sample, the sample category score is calculated based on the first target feature vector and the category label corresponding to the training sample.

[0196] For each training sample, the correct classification probability of the training sample is calculated based on the sample category score of the training sample.

[0197] The model performance score of the pre-trained model is calculated based on the correct classification probability corresponding to each training sample.

[0198] In one or more embodiments, a method for model performance scoring based on Reg-FDA is described. As can be seen from the foregoing embodiments, this application requires calculating a model performance score for each pre-trained model separately. Therefore, for ease of explanation, the following description will use any pre-trained model as an example.

[0199] Specifically, after obtaining the projection matrix corresponding to the pre-trained model, the projection matrix can be used to project the features of each feature vector extracted based on the pre-trained model, that is:

[0200]

[0201] in, Let represent the first target feature vector corresponding to the nth training sample. U represents the projection matrix. (·) T This indicates the transpose operation. This represents the feature vector corresponding to the nth training sample.

[0202] Based on this, according to the first target feature vector corresponding to each training sample and its corresponding class label, its corresponding sample class score can be further calculated. Therefore, the correct classification probability of the training sample can be calculated as follows:

[0203]

[0204] in, y represents the probability of correctly classifying the nth training sample. n This represents the category label of the nth training sample. Let represent the first target feature vector corresponding to the nth training sample. e represents the natural constant. C represents the total number of class labels. c represents the label of the cth class. This represents the sample category score of the category label to which the nth training sample belongs. express The exponent of. This represents the sample category score corresponding to the c-th class label of the nth training sample. express The exponent of.

[0205] After obtaining the correct classification probability for each training sample, the average of these correct classification probabilities is used to obtain the model performance score of the pre-trained model.

[0206] Secondly, this application provides a method for model performance scoring based on Reg-FDA. This method, on the one hand, projects the extracted feature vectors into Fisher space based on Reg-FDA, resulting in projected feature vectors with better class separability. Better class separability is a key feature for fine-tuning training on downstream classification data. On the other hand, Reg-FDA involves only simple matrix operations, thus its computational efficiency is high, making it highly practical and efficient.

[0207] Optionally, in the above Figure 3 Based on the corresponding embodiments, another optional embodiment provided in this application may further include:

[0208] Based on the category label of each training sample, determine the number of training samples corresponding to each category label;

[0209] Calculate the prior probability of each category label based on the number of training samples corresponding to each category label and the total number of training samples included in the target dataset.

[0210] Based on the first target feature vector and the category labels corresponding to the training samples, the sample category score of the training samples is calculated, which may specifically include:

[0211] Based on the first target feature vector, projection matrix, class mean of each class label, and prior probability of each class label corresponding to the training sample, the sample class score of the training sample for each class label is calculated.

[0212] In one or more embodiments, a method for calculating sample category scores based on the Bayes algorithm is described. As can be seen from the foregoing embodiments, this application requires calculating the sample category score for each training sample for each category label for each pre-trained model. Therefore, for ease of explanation, the following will use any pre-trained model as an example and any training sample for illustration.

[0213] Specifically, the sample class score for the training samples labeled with the c-th class can be calculated as follows:

[0214]

[0215] in, This represents the class score of the nth training sample for the cth class. Let μ represent the first target feature vector corresponding to the nth training sample. U represents the projection matrix. c This represents the mean value of the category corresponding to the label of category c. (·) T This represents the transpose operation. q c This represents the prior probability of the class corresponding to the label of class c.

[0216] Based on this, the prior probability of the class corresponding to the class label of class c can be calculated in the following way:

[0217]

[0218] Where, q c N represents the prior probability of the class corresponding to the label of class c. cThis represents the number of training samples corresponding to the category label of class c. N represents the total number of training samples included in the target dataset.

[0219] Understandably, the number of training samples corresponding to each category label can be calculated based on the category label of each training sample. For example, if 80 training samples belong to the category label "kitten", then the number of training samples with the category label "kitten" is 80.

[0220] It should be noted that in practical applications, in addition to using the Bayesian algorithm to calculate sample class scores, other algorithms such as Logistic Regression (LR), Decision Tree, Random Forest, and Support Vector Machine (SVM) can also be used.

[0221] Furthermore, this application provides a method for calculating sample category scores based on a Bayesian algorithm. Because the Bayesian algorithm has high computational efficiency, this method can improve the efficiency of model selection.

[0222] Optionally, in the above Figure 3 Based on the corresponding embodiments, in another optional embodiment provided by this application, the sample category score of the training sample is calculated according to the first target feature vector and the category label corresponding to the training sample, which may specifically include:

[0223] Based on the first target feature vector corresponding to the training sample and the first target feature vector corresponding to each training sample in (N-1) training samples, (N-1) distances are calculated. The target data set includes N training samples, and the (N-1) training samples are the remaining training samples in the target data set excluding the training samples. N is an integer greater than 1.

[0224] Based on (N-1) distances, determine the K training samples that are closest to the training samples from the (N-1) training samples, where K is an integer greater than 1 and less than (N-1);

[0225] Based on the category label of each of the K training samples, the sample category score for each category label is obtained.

[0226] In one or more embodiments, a method for calculating sample class scores based on K-Nearest Neighbor (KNN) is provided. As can be seen from the foregoing embodiments, this application requires calculating the sample class score for each training sample for each class label for each pre-trained model. Therefore, for ease of explanation, the following will use any pre-trained model as an example and any training sample for illustration.

[0227] Specifically, the distance between a training sample and other training samples can be calculated as follows:

[0228]

[0229] Where, d k This represents the distance between the nth training sample and the kth training sample. This represents the first target feature vector corresponding to the nth training sample. Let N represent the first target feature vector corresponding to the k-th training sample. N represents the total number of training samples included in the target dataset.

[0230] Based on this, Sort by size from smallest to largest, and get Therefore, based on the ranking results and the hyperparameter "K" of the KNN classifier, we can record the K training samples that are closest to the nth training sample, that is, obtain the K training samples as follows: Therefore, the sample category score for the labeled class c can be trained in the following way:

[0231]

[0232] in, This represents the sample class score of the nth training sample corresponding to the cth class label. c represents the cth class label. This represents the category label of the nth training sample. #{·} represents the number of elements in the set. After counting the number of elements in the cth category label, this number can be used as the sample category score for the nth training sample corresponding to the cth category label.

[0233] Furthermore, this application provides a method for calculating sample category scores based on KNN. This method offers another approach to calculating sample category scores, thereby improving the flexibility and versatility of the implementation.

[0234] Optionally, in the above Figure 3Based on the corresponding embodiments, in another optional embodiment provided by this application, the model performance score of the pre-trained model is calculated according to the classification correctness probability corresponding to each training sample, which may specifically include:

[0235] The first transferability score of the pre-trained model is calculated based on the correct classification probability corresponding to each training sample.

[0236] Based on the model performance score of each pre-trained model, select at least one pre-trained model from the M pre-trained models, which may specifically include:

[0237] Based on the first transferability score of each pre-trained model, select at least one pre-trained model from the M pre-trained models.

[0238] In one or more embodiments, a method for model optimization based on a first transferability score is described. As can be seen from the foregoing embodiments, this application introduces a regularization coefficient to calculate the model performance score. Based on this, the method can be defined as "regularization Fisher discriminant analysis (Reg-FDA)," and the model performance score calculated based on Reg-FDA is defined as the "first transferability score."

[0239] Specifically, for ease of understanding, please refer to Figure 5 , Figure 5 This is a schematic diagram illustrating the construction of the first transferability score in an embodiment of this application. As shown in the figure, taking a "pre-trained model 1" as an example, assuming the target dataset includes N training samples, each training sample is used as input to "pre-trained model 1," which outputs a feature vector corresponding to each training sample. Based on this, the Reg-FDA method is used to project features onto each feature vector, thereby obtaining the first target feature vector corresponding to each feature vector. The sample class score for each training sample is calculated based on each first target feature vector. Therefore, the corresponding classification correctness probability can be calculated based on the sample class score of each training sample. Finally, the first transferability score of "pre-trained model 1" is calculated based on the N classification correctness probabilities.

[0240] In summary, the first transferability score for each pre-trained model can be calculated as follows:

[0241]

[0242] in, This represents the first transferability score of the m-th pre-trained model. N represents the total number of training samples included in the target dataset. This represents the probability of correctly classifying the nth training sample.

[0243] Understandably, after obtaining the first transferability score of each of the M pre-trained models, the top K pre-trained models with the largest first transferability scores can be selected as the preferred models.

[0244] Furthermore, this application provides a method for model selection based on a first transferability score. Through this method, since the Reg-FDA method introduces a regularization term into the intra-class covariance matrix, Reg-FDA can better characterize the intra-class and inter-class distances of different pre-trained models on the target task. Therefore, the first transferability score can effectively predict the model's transfer learning ability on downstream data.

[0245] Optionally, in the above Figure 3 Based on the corresponding embodiments, in another optional embodiment provided by this application, the model performance score of the pre-trained model is calculated according to the classification correctness probability corresponding to each training sample, which may specifically include:

[0246] For each training sample, noise is processed on the feature vector corresponding to the training sample based on the correct classification probability of the training sample, resulting in a mixed feature vector;

[0247] Based on the mixed feature vector corresponding to each training sample and the category label to which each training sample belongs, determine the inter-class margin matrix and the intra-class margin matrix.

[0248] Determine the target regularization coefficient based on the inter-class margin matrix;

[0249] The target projection matrix is ​​determined based on the target regularization coefficient, the inter-class margin matrix, and the intra-class margin matrix.

[0250] The model performance score of the pre-trained model is determined based on the target projection matrix, the mixed feature vector corresponding to each training sample, and the category label of each training sample.

[0251] In one or more embodiments, a method for reconstructing the feature vectors corresponding to training samples by introducing a self-challenging mechanism is introduced. As can be seen from the foregoing embodiments, the self-challenging mechanism increases the classification difficulty of samples, thereby making different pre-trained models more discriminative on difficult samples. That is, after completing the Reg-FDA method, the correct classification probability of each training sample can be obtained. The lower the probability of correct classification, the greater the difficulty in correctly classifying the current training sample. Therefore, the classification difficulty of the sample can be further increased. This application requires calculation for each pre-trained model separately. Based on this, for ease of explanation, the following will use any pre-trained model as an example and any training sample for illustration.

[0252] Specifically, the confidence-mixed noise (ConfMix) equation can be used to increase the classification difficulty of the samples, thereby generating a mixed feature vector corresponding to each training sample. Taking the nth training sample as an example, its corresponding mixed feature vector can be expressed as: Based on this, the feature vector of the nth training sample can be mixed. The "feature vector" of the nth training sample First, substitute the values ​​into formulas (1) to (4) to obtain the inter-class margin matrix and the intra-class margin matrix. Then, based on the inter-class margin matrix, the target regularization coefficient can be calculated using formula (5). Next, based on the target regularization coefficient, the inter-class margin matrix, and the intra-class margin matrix, the target projection matrix can be calculated using formula (6). Finally, based on the target projection matrix, the mixed feature vector corresponding to each training sample, and the class label of each training sample, the model performance score of the pre-trained model can be calculated using formulas (7) and (8).

[0253] Furthermore, this application provides a method for reconstructing the feature vectors corresponding to training samples by introducing a self-questioning mechanism. Through this method, the self-questioning mechanism allows different pre-trained models to better distinguish difficult samples. During training and fine-tuning, the difference between different pre-trained models lies in their accuracy in classifying difficult samples. Therefore, using a self-questioning mechanism can characterize the classification accuracy of pre-trained models for difficult samples.

[0254] Optionally, in the above Figure 3 Based on the corresponding embodiments, in another optional embodiment provided by this application, noise processing is performed on the feature vector corresponding to the training sample according to the classification correctness probability of the training sample to obtain a mixed feature vector, which may specifically include:

[0255] For each category label in the (C-1) category label, the feature vectors in the feature vector subset corresponding to the category label are summed to obtain the category feature vector. Here, the (C-1) category label is the remaining category label in the C category label except for the category label corresponding to the training sample. Each feature vector subset corresponds to the same category label, and C is an integer greater than 1.

[0256] The sample mixing noise is calculated based on the category feature vector, the total number of training samples included in the target dataset, and the number of training samples corresponding to the category labels of the training samples.

[0257] The mixed feature vector corresponding to the training sample is calculated based on the correct classification probability of the training sample, the feature vector corresponding to the training sample, and the mixed noise of the samples.

[0258] In one or more embodiments, a method for constructing a hybrid feature vector is described. As can be seen from the foregoing embodiments, this application requires calculating the hybrid feature vector for each training sample separately for each pre-trained model. Therefore, for ease of explanation, the following will use any pre-trained model as an example and any training sample for illustration.

[0259] Specifically, the following ConfMix equation can be constructed to increase the classification difficulty of the samples:

[0260]

[0261] in, This represents the mixed feature vector corresponding to the nth training sample. This represents the probability of correctly classifying the nth training sample. Let represent the feature vector corresponding to the nth training sample. C represents the total number of categories in the category labeling. c represents the category label of the cth category. N represents the feature vector corresponding to the nth training sample belonging to the c-th class label (i.e., the nth feature vector in the subset of feature vectors corresponding to the c-th class label). c N represents the number of training samples belonging to the c-th category label. y This represents the number of training samples with the category label of the nth training sample. N represents the total number of training samples included in the target dataset.

[0262] As can be seen, besides the category label of the nth training sample, the feature vector subsets corresponding to the remaining (C-1) other category labels are first summed to obtain the category feature vector. Next, based on the category feature vector, the total number of training samples included in the target dataset, and the number of training samples corresponding to the category labels of the training samples, the sample mixing noise is obtained.

[0263] Furthermore, this application provides a method for constructing hybrid feature vectors. Using this method, training samples are regenerated based on the classification confidence level of the training samples, thereby increasing the classification difficulty of the training samples. The self-questioning mechanism then allows different pre-trained models to better distinguish between training samples with higher classification difficulty. During the training and fine-tuning process, the difference between different pre-trained models lies in their accuracy in classifying difficult samples.

[0264] Optionally, in the above Figure 3 Based on the corresponding embodiments, in another optional embodiment provided by this application, the model performance score of the pre-trained model is determined according to the target projection matrix, the mixed feature vector corresponding to each training sample, and the category label of each training sample. Specifically, this may include:

[0265] For each training sample, the target projection matrix is ​​used to project the mixed feature vector corresponding to the training sample to obtain the second target feature vector;

[0266] For each training sample, the target sample category score is calculated based on the second target feature vector and the category label corresponding to the training sample.

[0267] For each training sample, the probability of correct target classification corresponding to the training sample is calculated based on the target sample category score of the training sample.

[0268] The model performance score of the pre-trained model is calculated based on the correct classification probability of the target corresponding to each training sample.

[0269] In one or more embodiments, a method for calculating model performance scores based on a self-challenging mechanism is introduced. As can be seen from the foregoing embodiments, this application requires calculating the target classification correctness probability for each training sample for each pre-trained model. It is understood that since this application introduces a self-challenging mechanism for calculating model performance scores, this method can be defined as "self-challenging Fisher discriminant analysis (SFDA)". For ease of explanation, the following will use any pre-trained model as an example and any training sample for illustration.

[0270] Specifically, after obtaining the target projection matrix corresponding to the pre-trained model, the target projection matrix can be used to project the mixed feature vectors, that is:

[0271]

[0272] in, Let U represent the second target feature vector corresponding to the nth training sample. U′ represents the target projection matrix. (·) T This indicates the transpose operation. This represents the mixed feature vector corresponding to the nth training sample.

[0273] Based on this, according to the second target feature vector and its corresponding category label for each training sample, the target sample category score can be further calculated. Therefore, the correct classification probability of the training sample can be calculated as follows:

[0274]

[0275] in, y represents the probability of correctly classifying the target for the nth training sample. n This represents the category label of the nth training sample. Let represent the second target feature vector corresponding to the nth training sample. e represents the natural constant. C represents the total number of class labels. c represents the label of the cth class. This represents the target sample category score, which is labeled with the category of the nth training sample. express The exponent of. This represents the target sample category score corresponding to the c-th class label of the nth training sample. express The exponent of.

[0276] As can be seen, by generating a mixed feature vector for each training sample based on its classification confidence level, and then repeatedly using the Reg-FDA method to calculate the probability of each training sample being correctly classified, we obtain the target classification correct probability for each training sample. Finally, the model performance score of the pre-trained model is calculated based on the correct classification probability of the target corresponding to each training sample.

[0277] Furthermore, this application provides a method for calculating model performance scores based on a self-questioning mechanism. Through this method, on the one hand, the extracted feature vectors are projected onto the Fisher space based on the SFDA (Self-Questioning Facing) mechanism, resulting in better class separability of the projected feature vectors. Better class separability is a key feature for fine-tuning training on downstream classification data. Simultaneously, the self-questioning mechanism allows different pre-trained models to better distinguish between training samples with higher difficulty. During training and fine-tuning, the difference between different pre-trained models lies in their accuracy in classifying difficult samples. On the other hand, the SFDA only involves simple matrix operations, therefore, it has high computational efficiency and is highly practical and efficient.

[0278] Optionally, in the above Figure 3 Based on the corresponding embodiments, in another optional embodiment provided by this application, the model performance score of the pre-trained model is calculated according to the correct classification probability of the target corresponding to each training sample, which may specifically include:

[0279] The second transferability score of the pre-trained model is calculated based on the correct classification probability of the target corresponding to each training sample.

[0280] Based on the model performance score of each pre-trained model, select at least one pre-trained model from the M pre-trained models, including:

[0281] Based on the second transferability score of each pre-trained model, select at least one pre-trained model from the M pre-trained models.

[0282] In one or more embodiments, a method for model optimization based on a second transferability score is described. As can be seen from the foregoing embodiments, this application defines the model performance score calculated based on SDFA as the "second transferability score".

[0283] Specifically, for ease of understanding, please refer to Figure 6 , Figure 6 This is a schematic diagram illustrating the construction of a second transferability score in an embodiment of this application. As shown in the figure, taking a "pre-trained model 1" as an example, assuming the target dataset includes N training samples, each training sample is used as input to "pre-trained model 1," which outputs a feature vector corresponding to each training sample. Based on this, the Reg-FDA method is used to project features onto each feature vector, thereby obtaining a first target feature vector corresponding to each feature vector. The sample class score for each training sample is calculated based on each first target feature vector. Therefore, the corresponding correct classification probability can be calculated based on the sample class score of each training sample.

[0284] Based on this, noise is processed on the feature vector corresponding to each training sample using the correct classification probability of each training sample, resulting in a mixed feature vector. The Reg-FDA method is then used to project the features onto each mixed feature vector, thus obtaining the second target feature vector corresponding to each mixed feature vector. Based on each second target feature vector, the target sample class score for each training sample is calculated. Therefore, the corresponding target classification correct probability can be calculated based on the target sample class score for each training sample. Finally, the second transferability score of "pre-trained model 1" is calculated based on the N target classification correct probabilities.

[0285] In summary, the second transferability score for each pre-trained model can be calculated as follows:

[0286]

[0287] in, represents the second transferability score of the m-th pre-trained model. N represents the total number of training samples included in the target dataset. This represents the probability that the target classification of the nth training sample is correct.

[0288] Understandably, after obtaining the second transferability score of each of the M pre-trained models, the top K pre-trained models with the largest second transferability scores can be selected as the preferred models.

[0289] Furthermore, this application provides a method for model optimization based on a second transferability score. Through this method, firstly, SFDA does not rely on the classifier in the pre-trained model, thus making it more general. Secondly, SFDA only involves simple matrix operations, therefore its computational efficiency is very high. Thirdly, SFDA does not rely on strong prior distribution assumptions about features, and the proposed Reg-FDA and self-questioning mechanism can effectively characterize the training fine-tuning process. Therefore, SFDA can select the optimal pre-trained model for each downstream business data, greatly saving the cost of model selection and trial and error.

[0290] Optionally, in the above Figure 3 Based on the corresponding embodiments, another optional embodiment provided in this application may further include:

[0291] Based on the projection matrix and the set of feature vectors corresponding to each pre-trained model, the complementarity score of each pre-trained model is calculated.

[0292] Based on the model performance score of each pre-trained model, select at least one pre-trained model from the M pre-trained models, including:

[0293] Based on the transferability score and complementarity score of each pre-trained model, the comprehensive score corresponding to each pre-trained model is calculated.

[0294] Based on the comprehensive score corresponding to each pre-trained model, select at least one pre-trained model from the M pre-trained models.

[0295] In one or more embodiments, a method for model optimization based on a comprehensive model score is described. As can be seen from the foregoing embodiments, this application requires calculating a comprehensive score for each pre-trained model separately. Therefore, for ease of explanation, the following description will use any pre-trained model as an example.

[0296] For example, one feasible approach is to calculate the comprehensive score corresponding to the pre-trained model in the following manner:

[0297]

[0298] in, This represents the comprehensive score corresponding to the m-th pre-trained model. This represents the complementarity score corresponding to the m-th pre-trained model. This represents the second transferability score of the m-th pre-trained model. α represents the first coefficient, for example, α = 0.5. β represents the second coefficient, for example, β = 0.5.

[0299] For example, another feasible approach is to calculate the comprehensive score corresponding to the pre-trained model in the following manner:

[0300]

[0301] in, This represents the comprehensive score corresponding to the m-th pre-trained model. This represents the complementarity score corresponding to the m-th pre-trained model. This represents the first transferability score of the m-th pre-trained model. α represents the first coefficient, for example, α = 0.5. β represents the second coefficient, for example, β = 0.5.

[0302] Based on this, according to the comprehensive score The top-ranked pre-trained models serve as the foundation for multi-model selection in downstream tasks. For easier understanding, please refer to [link to relevant documentation]. Figure 7 , Figure 7 This is a schematic diagram of an interface for implementing model selection in an embodiment of this application. As shown in the figure, users can upload relevant business data sets through the interface. This business dataset is the "target data set" in this application. Based on this, the pre-trained models in the model library can be sorted by calculating model performance scores (i.e., the first transferability score, the second transferability score, or the comprehensive score). Finally, one or more pre-trained models are selected as the base models for multi-model selection in downstream tasks according to the order of scores from high to low.

[0303] Secondly, this application provides a method for model selection based on comprehensive model scoring. This method takes into account the complementarity between multiple models in multi-model ensemble fine-tuning; models with better complementarity typically achieve greater performance improvements during ensemble fine-tuning. By projecting the features of multiple models into the same Fisher space, all model features have the same dimension, making it easy to obtain the complementarity score of each model with respect to all models. Therefore, introducing a complementarity score can better achieve model selection.

[0304] Optionally, in the above Figure 3 Based on the corresponding embodiments, in another optional embodiment provided by this application, the complementarity score of each pre-trained model is calculated according to the projection matrix and the feature vector set corresponding to each pre-trained model. Specifically, this may include:

[0305] Based on the projection matrix and the set of feature vectors corresponding to each pre-trained model, the multi-model feature matrix corresponding to each training sample is calculated.

[0306] For each pre-trained model, obtain the first nuclear norm of the feature matrix of the multi-model corresponding to each training sample;

[0307] For each pre-trained model, the multi-model feature matrix corresponding to each training sample is multiplied by the matrix corresponding to the pre-trained model to obtain the third matrix corresponding to each training sample.

[0308] For each pre-trained model, obtain the second nuclear norm of the third matrix corresponding to each training sample;

[0309] For each pre-trained model, the complementarity score of the pre-trained model is calculated based on the first and second nuclear norms corresponding to each training sample.

[0310] In one or more embodiments, a method for calculating complementarity scores is described. As can be seen from the foregoing embodiments, this application requires calculating the complementarity score for each pre-trained model separately. Multi-model optimization is a derivative problem of model optimization, and its application scenario is to select multiple pre-trained models for downstream tasks for ensemble fine-tuning. Compared with single-model fine-tuning, ensemble fine-tuning often brings greater improvements to downstream tasks. Multiple models with better complementarity usually achieve greater performance improvements in ensemble fine-tuning. Since the feature vectors of multiple pre-trained models can be projected onto the same Fisher space, in which the features of all pre-trained models have the same dimension, the complementarity score of each pre-trained model with respect to all pre-trained models can be obtained. Based on this, for ease of explanation, the following will use any one pre-trained model as an example.

[0311] Specifically, the complementarity score for each pre-trained model can be calculated as follows:

[0312]

[0313] in, F represents the complementarity score corresponding to the m-th pre-trained model. N represents the total number of training samples included in the target dataset. nThis represents the multi-model feature matrix corresponding to the nth training sample. ||·|| * This represents the nuclear norm of the matrix. 1 m This represents the matrix corresponding to the pre-trained model, i.e., a matrix where the m-th column is 0 and all other columns are 1. ⊙ represents the dot product. ||F n || * F represents the first nuclear norm of the multi-model feature matrix corresponding to the nth training sample. n ⊙1 m Let F represent the third matrix corresponding to the nth training sample. n ⊙1 m || * It represents the second nuclear norm of the third matrix corresponding to the nth training sample.

[0314] Based on this, the first and second nuclear norms corresponding to the nth training sample are subtracted to obtain the nuclear norm difference for the nth training sample. Finally, the nuclear norm differences corresponding to the N training samples are summed to obtain the complementarity score of the mth pre-trained model.

[0315] It is understandable that the multi-model feature matrix corresponding to the nth training sample is:

[0316]

[0317] Among them, F n This represents the multi-model feature matrix corresponding to the nth training sample. This represents the feature representation of the m-th pre-trained model in the Fisher space, i.e., U-projection matrix, This indicates that the feature vector of the nth training sample is extracted based on the mth pre-trained model. This represents the first target feature vector obtained by feature projection after extracting the feature vector of the nth training sample based on the mth pre-trained model.

[0318] Furthermore, this application provides a method for calculating complementarity scores. This method provides a feasible approach to implementing the solution, thereby improving its feasibility and operability.

[0319] The performance score of the model proposed in this application will be explained below with reference to the verification data.

[0320] To verify the validity of the model performance score, this application selected 11 commonly used classification datasets for testing. These datasets include the Aircraft dataset, the Caltech 101 dataset, the Cars dataset, the Canadian Institute for Advanced Research (CF-10) dataset, the CIFAR-100 (CF-100) dataset, the Describable Textures Dataset (DTD), the Flowers dataset, the Food dataset, the Pets dataset, the Scene Understanding (SUN) 397 dataset, and the Visual Object Classes (VOC) 2007 dataset. These datasets cover common scene, texture, coarse-grained, and fine-grained image classification tasks.

[0321] I. Regarding the Reg-FDA approach;

[0322] Reg-FDA can be considered an indicator for model selection and can more accurately predict the transfer learning ability of different pre-trained models for downstream tasks. In the experimental validation process, Reg-FDA was used to predict the transfer learning ability of pre-trained models.

[0323] For ease of explanation, please refer to Table 1, which shows the prediction effects of different methods on the downstream data transfer learning ability of supervised models.

[0324] Table 1

[0325]

[0326] Method (1) represents the method combining the Logarithm of Maximum Evidence (LogME) with ConfMix. Method (2) represents the Reg-FDA method (i.e., the SFDA method excluding ConfMix). Method (3) represents the SFDA method with a regularization coefficient set to 0.5.

[0327] A higher weighted Kendall coefficient indicates better predictive performance. As can be seen, compared with methods (1) and (3), Reg-FDA can achieve a higher Kendall coefficient correlation, proving that Reg-FDA consistently outperforms methods (1) and (3).

[0328] II. Regarding the SFDA's methods;

[0329] Supervised learning factor (SFDA) can be considered an indicator for model selection and can more accurately predict the transfer learning ability of different pre-trained models for downstream tasks. In other words, SFDA is highly effective in predicting the transfer learning ability of pre-trained models for downstream tasks. Furthermore, SFDA involves only simple matrix calculations, making it computationally efficient. SFDA can accurately estimate the transfer learning ability of different types of pre-trained models for downstream tasks. In experimental validation, SFDA was used to predict the transfer learning ability of three different types of pre-trained models: supervised models, self-supervised models, and visual deformers (Transformers). Log expected empirical prediction (LEEP), Log Expected Empirical Prediction with Gaussian Mixture (NLEEP), and LogME were introduced for comparison.

[0330] For ease of explanation, please refer to Table 2, which shows the prediction effects of different methods on the downstream data transfer learning ability of supervised models.

[0331] Table 2

[0332]

[0333]

[0334] A higher weighted Kendall coefficient indicates better predictive performance. Therefore, SFDA achieves a higher Kendall coefficient correlation, demonstrating that SFDA consistently outperforms existing methods such as LEEP, NLEEP, and LogME.

[0335] Please refer to Table 3, which shows the prediction effects of different methods on the transfer learning ability of self-supervised models in downstream data.

[0336] Table 3

[0337]

[0338] A higher weighted Kendall coefficient indicates better predictive performance. Therefore, SFDA achieves a higher Kendall coefficient correlation, demonstrating that SFDA consistently outperforms existing methods like NLEEP and LogME.

[0339] Please refer to Table 4, which shows the prediction effects of different methods on the downstream data transfer learning ability of the visual Transformer.

[0340] Table 4

[0341]

[0342] A higher weighted Kendall coefficient indicates better predictive performance. Therefore, SFDA achieves a higher Kendall coefficient correlation, demonstrating that SFDA consistently outperforms existing methods like NLEEP and LogME.

[0343] Compared to the NLEEP method, SFDA shows better predictive performance on model transfer learning ability, while also offering a computational efficiency improvement of several tens of times. As shown in Tables 2 to 4, the weighted Kendall coefficient of SFDA is on average 63.5% higher than that of NLEEP across the 11 downstream datasets. Furthermore, Table 5 also presents the computation time of NLEEP and SFDA across the 11 downstream datasets. Please refer to Table 5 for a comparison of the computational efficiency of different methods.

[0344] Table 5

[0345]

[0346] As can be seen, SFDA's computational efficiency is 16.4 times higher than that of NLEEP.

[0347] III. Regarding SFDA com Methods;

[0348] SFDA com This metric can be considered a key indicator for model selection and can more accurately predict the transfer learning capabilities of different pre-trained models for downstream tasks. In experimental validation, SFDAcom was used to predict the transfer learning capabilities of pre-trained models. Based on model performance scores, k optimal pre-trained models can be selected and integrated and fine-tuned on target business data to achieve better image classification accuracy.

[0349] For ease of explanation, please refer to Table 6, which shows the image classification accuracy on the target data when the top two best pre-trained models are integrated and fine-tuned based on different methods.

[0350] Table 6

[0351] Aircraft Caltech Cars CF-10 CF-100 DTD Flowers Food Pets SUN VOC LogME 87.23 93.87 91.38 97.86 86.98 77.14 96.87 87.43 94.62 65.47 86.46 NLEEP 85.17 93.45 91.03 97.86 86.98 77.25 97.04 86.62 94.59 65.81 86.64 SFDA 87.71 93.40 91.03 97.86 86.98 76.95 97.59 87.95 94.58 65.81 86.46 SFDA com ]]> 87.71 93.87 91.69 97.91 86.98 76.95 97.24 87.95 94.62 65.81 86.46

[0352] It is evident that the SFDA com It can achieve higher classification accuracy, proving that SFDA com Its classification accuracy surpasses that of existing NLEEP and LogME, as well as the SFDA proposed in this application.

[0353] For ease of explanation, please refer to Table 7, which shows the image classification accuracy on the target data based on the integration and fine-tuning of the top three best pre-trained models selected by different methods.

[0354] Table 7

[0355] Aircraft Caltech Cars CF-10 CF-100 DTD Flowers Food Pets SUN VOC LogME 87.23 93.87 91.80 97.88 86.96 77.68 97.56 87.83 94.70 66.20 86.89 NLEEP 86.98 94.12 91.76 98.02 87.48 78.14 97.65 87.37 94.71 66.95 86.89 SFDA 88.01 93.95 91.76 98.02 87.48 77.68 97.99 88.49 94.82 66.53 86.89 SFDA com ]]> 88.01 93.87 91.95 98.02 87.48 78.14 97.35 88.49 94.92 66.53 86.89

[0356] It is evident that the SFDA com It can achieve higher classification accuracy, proving that SFDA com Its classification accuracy surpasses that of existing NLEEP and LogME, as well as the SFDA proposed in this application.

[0357] The model selection device in this application is described in detail below. Please refer to [link / reference]. Figure 8 , Figure 8 This is a schematic diagram of one embodiment of the model selection device in this application. The model selection device 30 includes:

[0358] Extraction module 310 is used to extract a set of feature vectors for each of the M pre-trained models based on the target data set, where each training sample in the target data set corresponds to a category label, and the set of feature vectors includes the feature vectors of each training sample, where M is an integer greater than 1.

[0359] The determination module 320 is used to determine the inter-class margin matrix and the intra-class margin matrix for each pre-trained model based on the feature vector set and the category label of each training sample.

[0360] The determination module 320 is also used to determine the regularization coefficients for each pre-trained model based on the inter-class margin matrix;

[0361] The determination module 320 is also used to determine the projection matrix for each pre-trained model based on the regularization coefficient, the inter-class margin matrix, and the intra-class margin matrix.

[0362] The determination module 320 is also used to determine the model performance score of each pre-trained model based on the projection matrix, the feature vector set, and the category label of each training sample.

[0363] Selection module 330 is used to select at least one pre-trained model from M pre-trained models based on the model performance score of each pre-trained model.

[0364] This application provides a model selection device. Using this device, a regularization coefficient is introduced based on Fisher discriminant analysis. This allows for a better characterization of the intra-class and inter-class distances of different pre-trained models on the target classification task, resulting in better class separability of the projected features. Better class separability is a key feature for fine-tuning training on downstream classification data. Therefore, without relying on a classifier, an index for evaluating the transferability of pre-trained models can be constructed, thereby enabling model optimization from a pre-trained model library.

[0365] Optionally, in the above Figure 8 Based on the corresponding embodiments, in another embodiment of the model selection device 30 provided in this application,

[0366] Module 320 is specifically used to calculate the average of the feature vector set to obtain the overall mean.

[0367] Based on the category label of each training sample, the feature vector set is divided into C feature vector subsets, where each feature vector subset corresponds to the same category label, and C is an integer greater than 1;

[0368] The average value of each feature vector subset is calculated to obtain the category mean value corresponding to each category label.

[0369] The inter-class margin matrix is ​​calculated based on the overall mean and the mean of each category label.

[0370] The intra-class margin matrix is ​​calculated based on each subset of feature vectors and the corresponding class mean.

[0371] This application provides a model selection device. Using this device provides a feasible way to implement the solution, thereby improving its feasibility and operability.

[0372] Optionally, in the above Figure 8 Based on the corresponding embodiments, in another embodiment of the model selection device 30 provided in this application,

[0373] The determination module 320 is specifically used to subtract the overall mean from the category mean for each category label in the C category label to obtain the first feature vector;

[0374] For each category label in the C category label, transpose the first feature vector to obtain the second feature vector;

[0375] For each category label in category C, the first feature vector and the second feature vector are multiplied to obtain the inter-class matrix corresponding to the category label;

[0376] Summing the inter-class matrices corresponding to the label of each category yields the inter-class margin matrix.

[0377] This application provides a model selection device. Using this device provides a feasible way to implement the solution, thereby improving its feasibility and operability.

[0378] Optionally, in the above Figure 8Based on the corresponding embodiments, in another embodiment of the model selection device 30 provided in this application,

[0379] The determination module 320 is specifically used to subtract the class mean from each feature vector in the feature vector subset for each class label in the C class label to obtain the third feature vector corresponding to each feature vector in the feature vector subset;

[0380] For each category label in the C category label, transpose the third feature vector corresponding to each feature vector in the feature vector subset to obtain the fourth feature vector corresponding to each feature vector in the feature vector subset;

[0381] For each category label in the C category label, the third and fourth feature vectors corresponding to each feature vector in the feature vector subset are multiplied to obtain the first class matrix corresponding to each feature vector in the feature vector subset;

[0382] For each category label in the C category label, sum the first-class inner matrix corresponding to each feature vector in the feature vector subset to obtain the second-class inner matrix corresponding to the category label;

[0383] Summing the second-class intra-matrix corresponding to the label of each category yields the intra-class margin matrix.

[0384] This application provides a model selection device. Using this device provides a feasible way to implement the solution, thereby improving its feasibility and operability.

[0385] Optionally, in the above Figure 8 Based on the corresponding embodiments, in another embodiment of the model selection device 30 provided in this application,

[0386] The determination module 320 is specifically used to obtain at least one set of eigenvalues ​​based on the inter-class margin matrix. Each set of eigenvalues ​​includes an eigenvalue and an eigenvector, where the eigenvector is the eigenvector of the inter-class margin matrix corresponding to the eigenvalue.

[0387] Obtain the maximum eigenvalue from at least one set of eigenvalues;

[0388] The first value is calculated based on the maximum eigenvalue and the first preset parameter;

[0389] The second value is obtained by exponentiation of the first value with the natural constant as the base;

[0390] Based on the second value, the regularization coefficient is calculated using the activation function.

[0391] This application provides a model selection device. Using this device provides a feasible way to implement the solution, thereby improving its feasibility and operability.

[0392] Optionally, in the above Figure 8 Based on the corresponding embodiments, in another embodiment of the model selection device 30 provided in this application,

[0393] Module 320 is specifically used to construct the objective function based on the regularization coefficient, the inter-class margin matrix, the intra-class margin matrix, and the identity matrix.

[0394] The projection matrix is ​​calculated based on the objective function.

[0395] This application provides a model selection device. Using this device provides a feasible way to implement the solution, thereby improving its feasibility and operability.

[0396] Optionally, in the above Figure 8 Based on the corresponding embodiments, in another embodiment of the model selection device 30 provided in this application,

[0397] The determination module 320 is specifically used to perform feature projection on the feature vector corresponding to each training sample using a projection matrix to obtain the first target feature vector;

[0398] For each training sample, the sample category score is calculated based on the first target feature vector and the category label corresponding to the training sample.

[0399] For each training sample, the correct classification probability of the training sample is calculated based on the sample category score of the training sample.

[0400] The model performance score of the pre-trained model is calculated based on the correct classification probability corresponding to each training sample.

[0401] This application provides a model selection device. Using this device, on the one hand, the extracted feature vectors are projected onto the Fisher space based on Reg-FDA, resulting in projected feature vectors with better class separability. Better class separability is a key feature for fine-tuning training on downstream classification data. On the other hand, Reg-FDA only involves simple matrix operations, therefore it has high computational efficiency and is highly practical and efficient.

[0402] Optionally, in the above Figure 8 Based on the corresponding embodiments, in another embodiment of the model selection device 30 provided in this application,

[0403] The determination module 320 is also used to determine the number of training samples corresponding to each category label based on the category label to which each training sample belongs;

[0404] The determination module 320 is also used to calculate the prior probability of each category label based on the number of training samples corresponding to each category label and the total number of training samples included in the target dataset.

[0405] The determination module 320 is specifically used to calculate the sample class score of the training sample for each class label based on the first target feature vector corresponding to the training sample, the projection matrix, the class mean corresponding to each class label, and the class prior probability corresponding to each class label.

[0406] In this embodiment, a model selection device is provided. Using this device, the efficiency of model selection can be improved due to the high computational efficiency of the Bayesian algorithm.

[0407] Optionally, in the above Figure 8 Based on the corresponding embodiments, in another embodiment of the model selection device 30 provided in this application,

[0408] The determination module 320 is specifically used to calculate (N-1) distances based on the first target feature vector corresponding to the training sample and the first target feature vector corresponding to each training sample in (N-1) training samples. The target data set includes N training samples, and the (N-1) training samples are the remaining training samples in the target data set excluding the training samples. N is an integer greater than 1.

[0409] Based on (N-1) distances, determine the K training samples that are closest to the training samples from the (N-1) training samples, where K is an integer greater than 1 and less than (N-1);

[0410] Based on the category label of each of the K training samples, the sample category score for each category label is obtained.

[0411] This application provides a model selection device. Using this device provides another means of calculating sample category scores, thereby improving the flexibility and diversity of the implementation scheme.

[0412] Optionally, in the above Figure 8 Based on the corresponding embodiments, in another embodiment of the model selection device 30 provided in this application,

[0413] The determination module 320 is specifically used to calculate the first transferability score of the pre-trained model based on the classification correctness probability corresponding to each training sample.

[0414] Selection module 330 is specifically used to select at least one pre-trained model from M pre-trained models based on the first transferability score of each pre-trained model.

[0415] In this embodiment, a model selection apparatus is provided. Using this apparatus, because the Reg-FDA method introduces a regularization term into the intra-class covariance matrix, Reg-FDA can better characterize the intra-class and inter-class distances of different pre-trained models on the target task. Therefore, the first transferability score can effectively predict the model's transfer learning ability on downstream data.

[0416] Optionally, in the above Figure 8 Based on the corresponding embodiments, in another embodiment of the model selection device 30 provided in this application,

[0417] The determination module 320 is specifically used to process the noise of the feature vector corresponding to each training sample based on the correct classification probability of the training sample, so as to obtain a mixed feature vector.

[0418] Based on the mixed feature vector corresponding to each training sample and the category label to which each training sample belongs, determine the inter-class margin matrix and the intra-class margin matrix.

[0419] Determine the target regularization coefficient based on the inter-class margin matrix;

[0420] The target projection matrix is ​​determined based on the target regularization coefficient, the inter-class margin matrix, and the intra-class margin matrix.

[0421] The model performance score of the pre-trained model is determined based on the target projection matrix, the mixed feature vector corresponding to each training sample, and the category label of each training sample.

[0422] This application provides a model selection device. Using this device, a self-questioning mechanism allows different pre-trained models to better distinguish difficult samples. During training and fine-tuning, the differences between different pre-trained models are reflected in their accuracy in classifying difficult samples. Therefore, the self-questioning mechanism can characterize the classification accuracy of pre-trained models for difficult samples.

[0423] Optionally, in the above Figure 8 Based on the corresponding embodiments, in another embodiment of the model selection device 30 provided in this application,

[0424] The determination module 320 is specifically used to sum the feature vectors in the feature vector subset corresponding to each category label in the (C-1) category label to obtain the category feature vector. Here, the (C-1) category label is the remaining category label in the C category label except for the category label corresponding to the training sample. Each feature vector subset corresponds to the same category label, and C is an integer greater than 1.

[0425] The sample mixing noise is calculated based on the category feature vector, the total number of training samples included in the target dataset, and the number of training samples corresponding to the category labels of the training samples.

[0426] The mixed feature vector corresponding to the training sample is calculated based on the correct classification probability of the training sample, the feature vector corresponding to the training sample, and the mixed noise of the samples.

[0427] This application provides a model selection device. Using this device, training samples are regenerated based on the classification confidence level of the training samples, thereby increasing the classification difficulty of the training samples. A self-questioning mechanism is then applied, allowing different pre-trained models to better distinguish between training samples with higher classification difficulty. During the training fine-tuning process, the difference between different pre-trained models lies in their accuracy in classifying difficult samples.

[0428] Optionally, in the above Figure 8 Based on the corresponding embodiments, in another embodiment of the model selection device 30 provided in this application,

[0429] The determination module 320 is specifically used to perform feature projection on the mixed feature vector corresponding to the training sample using the target projection matrix for each training sample, so as to obtain the second target feature vector;

[0430] For each training sample, the target sample category score is calculated based on the second target feature vector and the category label corresponding to the training sample.

[0431] For each training sample, the probability of correct target classification corresponding to the training sample is calculated based on the target sample category score of the training sample.

[0432] The model performance score of the pre-trained model is calculated based on the correct classification probability of the target corresponding to each training sample.

[0433] This application provides a model selection device. Using this device, on the one hand, the extracted feature vectors are projected onto the Fisher space based on the SFDA (Special Data Framework), resulting in better class separability of the projected feature vectors. Better class separability is a key feature for fine-tuning training on downstream classification data. Simultaneously, a self-questioning mechanism allows different pre-trained models to better distinguish between training samples with higher difficulty. During training and fine-tuning, the difference between different pre-trained models lies in their accuracy in classifying difficult samples. On the other hand, the SFDA only involves simple matrix operations, therefore, it has high computational efficiency and is highly practical and efficient.

[0434] Optionally, in the above Figure 8 Based on the corresponding embodiments, in another embodiment of the model selection device 30 provided in this application,

[0435] The determination module 320 is specifically used to calculate the second transferability score of the pre-trained model based on the correct classification probability of the target corresponding to each training sample.

[0436] Selection module 330 is specifically used to select at least one pre-trained model from M pre-trained models based on the second transferability score of each pre-trained model.

[0437] This application provides a model selection device. Using this device, firstly, SFDA does not rely on the classifier in the pre-trained model, thus making it more versatile. Secondly, SFDA only involves simple matrix operations, therefore its computational efficiency is very high. Thirdly, SFDA does not rely on strong prior distribution assumptions about features, and the proposed Reg-FDA and self-questioning mechanism can effectively characterize the training and fine-tuning process. Therefore, SFDA can select the optimal pre-trained model for each downstream business data, greatly saving the cost of model selection and trial and error.

[0438] Optionally, in the above Figure 8 Based on the corresponding embodiments, in another embodiment of the model selection device 30 provided in this application,

[0439] The determination module 320 is also used to calculate the complementarity score of each pre-trained model based on the projection matrix and the set of feature vectors corresponding to each pre-trained model.

[0440] Select module 330, specifically used to calculate the comprehensive score corresponding to each pre-trained model based on the transferability score and complementarity score of each pre-trained model;

[0441] Based on the comprehensive score corresponding to each pre-trained model, select at least one pre-trained model from the M pre-trained models.

[0442] This application provides a model selection device. Using this device, considering the complementarity between multiple models in multi-model ensemble fine-tuning, models with better complementarity typically achieve greater performance improvements during ensemble fine-tuning. By projecting the features of multiple models into the same Fisher space, all model features have the same dimension, thus easily obtaining the complementarity score of each model with respect to all models. Therefore, introducing a complementarity score can better achieve model optimization.

[0443] Optionally, in the above Figure 8 Based on the corresponding embodiments, in another embodiment of the model selection device 30 provided in this application,

[0444] The determination module 320 is specifically used to calculate the multi-model feature matrix corresponding to each training sample based on the projection matrix and the feature vector set corresponding to each pre-trained model.

[0445] For each pre-trained model, obtain the first nuclear norm of the feature matrix of the multi-model corresponding to each training sample;

[0446] For each pre-trained model, the multi-model feature matrix corresponding to each training sample is multiplied by the matrix corresponding to the pre-trained model to obtain the third matrix corresponding to each training sample.

[0447] For each pre-trained model, obtain the second nuclear norm of the third matrix corresponding to each training sample;

[0448] For each pre-trained model, the complementarity score of the pre-trained model is calculated based on the first and second nuclear norms corresponding to each training sample.

[0449] This application provides a model selection device. Using this device provides a feasible way to implement the solution, thereby improving its feasibility and operability.

[0450] Figure 9This is a schematic diagram of a computer device structure provided in an embodiment of this application. The computer device 400 can vary significantly due to different configurations or performance. It may include one or more central processing units (CPUs) 422 (e.g., one or more processors) and a memory 432, and one or more storage media 430 (e.g., one or more mass storage devices) for storing application programs 442 or data 444. The memory 432 and storage media 430 can be temporary or persistent storage. The program stored in the storage media 430 may include one or more modules (not shown in the diagram), each module including a series of instruction operations on the computer device. Furthermore, the CPU 422 may be configured to communicate with the storage media 430 and execute the series of instruction operations in the storage media 430 on the computer device 400.

[0451] Computer device 400 may also include one or more power supplies 426, one or more wired or wireless network interfaces 450, one or more input / output interfaces 458, and / or one or more operating systems 441, such as Windows Server. TM Mac OS X TM Unix TM Linux TM FreeBSD TM etc.

[0452] The steps performed by the computer device in the above embodiments can be based on this Figure 9 The computer device structure shown.

[0453] This application also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the methods described in the foregoing embodiments.

[0454] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the methods described in the foregoing embodiments.

[0455] It is understood that in the specific implementation of this application, data such as user information and training samples are involved. When the above embodiments of this application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0456] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0457] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection between apparatuses or units through some interfaces, and may be electrical, mechanical, or other forms.

[0458] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0459] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0460] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing computer programs, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0461] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A method for selecting a pre-trained model, characterized in that, include: For each of the M pre-trained models, a feature vector set is extracted based on the target data set. The target data set includes at least one of image data set, audio data set, text data set, and video data set. Each training sample in the target data set corresponds to a category label. The feature vector set includes the feature vector of each training sample. M is an integer greater than 1. For each pre-trained model, the inter-class margin matrix and the intra-class margin matrix are determined based on the feature vector set and the category label of each training sample. For each pre-trained model, a regularization coefficient is determined based on the inter-class margin matrix; For each pre-trained model, a projection matrix is ​​determined based on the regularization coefficient, the inter-class margin matrix, and the intra-class margin matrix. For each pre-trained model, a model performance score is determined based on the projection matrix, the feature vector set, and the category label of each training sample. Based on the model performance score of each pre-trained model, at least one pre-trained model is selected from the M pre-trained models; Determining the regularization coefficients based on the inter-class spacing matrix includes: At least one set of intrinsic results is obtained by solving the inter-class margin matrix, wherein each set of intrinsic results includes an eigenvalue and an eigenvector, and the eigenvector is the eigenvector of the inter-class margin matrix corresponding to the eigenvalue. Obtain the maximum eigenvalue from the at least one set of eigenvalues; The first value is calculated based on the maximum eigenvalue and the first preset parameter; The second value is obtained by exponentiation of the first value with the natural constant as the base; Based on the second value, the regularization coefficient is calculated using an activation function; Determining the projection matrix based on the regularization coefficient, the inter-class margin matrix, and the intra-class margin matrix includes: Construct an objective function based on the regularization coefficient, the inter-class margin matrix, the intra-class margin matrix, and the identity matrix; The projection matrix is ​​calculated based on the objective function.

2. The selection method according to claim 1, characterized in that, The step of determining the inter-class margin matrix and the intra-class margin matrix based on the feature vector set and the category label of each training sample includes: The average of the feature vector set is calculated to obtain the overall mean. Based on the category label of each training sample, the feature vector set is divided into C feature vector subsets, where each feature vector subset corresponds to the same category label, and C is an integer greater than 1; The average value of each feature vector subset is calculated to obtain the category mean value corresponding to each category label. The inter-class interval matrix is ​​calculated based on the overall mean and the category mean corresponding to each category label. The intra-class margin matrix is ​​calculated based on each subset of feature vectors and the corresponding class mean.

3. The selection method according to claim 2, characterized in that, The step of calculating the inter-class interval matrix based on the overall mean and the category mean corresponding to each category label includes: For each category label in category C, the overall mean is subtracted from the category mean to obtain the first feature vector; For each category label in the C category label, transpose the first feature vector to obtain the second feature vector; For each category label in the C category label, the first feature vector and the second feature vector are multiplied to obtain the inter-class matrix corresponding to the category label; The inter-class interval matrix is ​​obtained by summing the inter-class matrices corresponding to each category label.

4. The selection method according to claim 2, characterized in that, The step of calculating the intra-class margin matrix based on each subset of feature vectors and the corresponding class mean includes: For each category label in the C category label, subtract the category mean from each feature vector in the feature vector subset to obtain the third feature vector corresponding to each feature vector in the feature vector subset; For each category label in the C category label, the third feature vector corresponding to each feature vector in the feature vector subset is transposed to obtain the fourth feature vector corresponding to each feature vector in the feature vector subset; For each category label in the C category label, the third feature vector and the fourth feature vector corresponding to each feature vector in the feature vector subset are multiplied to obtain the first intra-class matrix corresponding to each feature vector in the feature vector subset; For each category label in the C category label, the first intra-class matrix corresponding to each feature vector in the feature vector subset is summed to obtain the second intra-class matrix corresponding to the category label; The intra-class interval matrix is ​​obtained by summing the second intra-class matrix corresponding to each category label.

5. The selection method according to claim 1, characterized in that, The step of determining the model performance score of the pre-trained model based on the projection matrix, the feature vector set, and the category label of each training sample includes: For each training sample, the feature vector corresponding to the training sample is projected using the projection matrix to obtain the first target feature vector; For each training sample, a sample category score is calculated based on the first target feature vector and the category label corresponding to the training sample. For each training sample, the correct classification probability corresponding to the training sample is calculated based on the sample category score of the training sample. The model performance score of the pre-trained model is calculated based on the correct classification probability corresponding to each training sample.

6. The selection method according to claim 5, characterized in that, The method further includes: Based on the category label of each training sample, determine the number of training samples corresponding to each category label; Calculate the prior probability of each category label based on the number of training samples corresponding to each category label and the total number of training samples included in the target dataset; The step of calculating the sample category score of the training sample based on the first target feature vector and the category label corresponding to the training sample includes: Based on the first target feature vector corresponding to the training sample, the projection matrix, the class mean corresponding to each class label, and the class prior probability corresponding to each class label, the sample class score of the training sample for each class label is calculated.

7. The selection method according to claim 5, characterized in that, The step of calculating the sample category score of the training sample based on the first target feature vector and the category label corresponding to the training sample includes: Based on the first target feature vector corresponding to the training sample and the first target feature vector corresponding to each of the (N-1) training samples, (N-1) distances are calculated, wherein the target data set includes N training samples, and the (N-1) training samples are the remaining training samples in the target data set excluding the training samples, and N is an integer greater than 1; Based on the (N-1) distances, determine the K training samples that are closest to the training samples from the (N-1) training samples, where K is an integer greater than 1 and less than (N-1); Based on the category label of each training sample in the K training samples, the sample category score for each category label is calculated.

8. The selection method according to claim 5, characterized in that, The step of calculating the model performance score of the pre-trained model based on the classification correct probability corresponding to each training sample includes: The first transferability score of the pre-trained model is calculated based on the correct classification probability corresponding to each training sample. The step of selecting at least one pre-trained model from the M pre-trained models based on the model performance score of each pre-trained model includes: Based on the first transferability score of each pre-trained model, at least one pre-trained model is selected from the M pre-trained models.

9. The selection method according to claim 5, characterized in that, The step of calculating the model performance score of the pre-trained model based on the classification correct probability corresponding to each training sample includes: For each training sample, noise is processed on the feature vector corresponding to the training sample based on the correct classification probability of the training sample to obtain a mixed feature vector; Based on the mixed feature vector corresponding to each training sample and the category label to which each training sample belongs, determine the inter-class margin matrix and the intra-class margin matrix. The target regularization coefficient is determined based on the target class interval matrix. The target projection matrix is ​​determined based on the target regularization coefficient, the target inter-class margin matrix, and the target intra-class margin matrix. The model performance score of the pre-trained model is determined based on the target projection matrix, the mixed feature vector corresponding to each training sample, and the category label of each training sample.

10. The selection method according to claim 9, characterized in that, The step of processing the feature vector corresponding to the training sample for noise based on the classification correctness probability of the training sample to obtain a mixed feature vector includes: For each category label in the (C-1) category label, the feature vectors in the feature vector subset corresponding to the category label are summed to obtain the category feature vector. The (C-1) category label refers to the remaining category labels in the C category labels excluding the category labels corresponding to the training samples. Each feature vector subset corresponds to the same category label, and C is an integer greater than 1. The sample mixing noise is calculated based on the category feature vector, the total number of training samples included in the target dataset, and the number of training samples corresponding to the category labels of the training samples. The mixed feature vector corresponding to the training sample is calculated based on the classification correctness probability corresponding to the training sample, the feature vector corresponding to the training sample, and the mixed noise of the sample.

11. The selection method according to claim 9, characterized in that, The step of determining the model performance score of the pre-trained model based on the target projection matrix, the mixed feature vector corresponding to each training sample, and the category label of each training sample includes: For each training sample, the target projection matrix is ​​used to project the mixed feature vector corresponding to the training sample to obtain the second target feature vector; For each training sample, the target sample category score is calculated based on the second target feature vector and the category label corresponding to the training sample. For each training sample, the probability of correct target classification corresponding to the training sample is calculated based on the target sample category score of the training sample. The model performance score of the pre-trained model is calculated based on the correct classification probability of the target corresponding to each training sample.

12. The selection method according to claim 11, characterized in that, The step of calculating the model performance score of the pre-trained model based on the correct classification probability of the target corresponding to each training sample includes: The second transferability score of the pre-trained model is calculated based on the correct target classification probability corresponding to each training sample. The step of selecting at least one pre-trained model from the M pre-trained models based on the model performance score of each pre-trained model includes: Based on the second transferability score of each pre-trained model, at least one pre-trained model is selected from the M pre-trained models.

13. The selection method according to any one of claims 1 to 12, characterized in that, The method further includes: Based on the projection matrix and the feature vector set corresponding to each pre-trained model, the complementarity score of each pre-trained model is calculated. The step of selecting at least one pre-trained model from the M pre-trained models based on the model performance score of each pre-trained model includes: Based on the transferability score and complementarity score of each pre-trained model, a comprehensive score corresponding to each pre-trained model is calculated. Based on the comprehensive score corresponding to each pre-trained model, at least one pre-trained model is selected from the M pre-trained models.

14. The selection method according to claim 13, characterized in that, The step of calculating the complementarity score of each pre-trained model based on the projection matrix and the feature vector set corresponding to each pre-trained model includes: Based on the projection matrix and the feature vector set corresponding to each pre-trained model, the multi-model feature matrix corresponding to each training sample is calculated; For each pre-trained model, obtain the first nuclear norm of the feature matrix of the multi-model corresponding to each training sample; For each pre-trained model, the multi-model feature matrix corresponding to each training sample is multiplied by the matrix corresponding to the pre-trained model to obtain the third matrix corresponding to each training sample. For each pre-trained model, obtain the second kernel norm of the third matrix corresponding to each training sample; For each pre-trained model, the complementarity score of the pre-trained model is calculated based on the first and second nuclear norms corresponding to each training sample.

15. A model selection device, characterized in that, include: An extraction module is used to extract a set of feature vectors for each of the M pre-trained models based on a target data set. The target data set includes at least one of image data set, audio data set, text data set, and video data set. Each training sample in the target data set corresponds to a category label. The set of feature vectors includes the feature vectors of each training sample. M is an integer greater than 1. The determination module is used to determine, for each pre-trained model, the inter-class margin matrix and the intra-class margin matrix based on the feature vector set and the category label of each training sample; The determining module is further configured to determine a regularization coefficient for each pre-trained model based on the inter-class margin matrix; The determining module is further configured to determine a projection matrix for each pre-trained model based on the regularization coefficient, the inter-class margin matrix, and the intra-class margin matrix; The determining module is further configured to determine the model performance score of each pre-trained model based on the projection matrix, the feature vector set, and the category label of each training sample. The selection module is used to select at least one pre-trained model from the M pre-trained models based on the model performance score of each pre-trained model. The determining module is specifically used to obtain at least one set of intrinsic results based on the inter-class interval matrix, wherein each set of intrinsic results includes an eigenvalue and an eigenvector, and the eigenvector is the feature vector of the inter-class interval matrix corresponding to the eigenvalue. Obtain the maximum eigenvalue from the at least one set of eigenvalues; The first value is calculated based on the maximum eigenvalue and the first preset parameter; The second value is obtained by exponentiation of the first value with the natural constant as the base; Based on the second value, the regularization coefficient is calculated using an activation function; The determining module is specifically used to construct an objective function based on the regularization coefficient, the inter-class margin matrix, the intra-class margin matrix, and the identity matrix; The projection matrix is ​​calculated based on the objective function.

16. The apparatus according to claim 15, characterized in that, The determining module is specifically used to calculate the average of the feature vector set to obtain the overall mean. Based on the category label of each training sample, the feature vector set is divided into C feature vector subsets, where each feature vector subset corresponds to the same category label, and C is an integer greater than 1; The average value of each feature vector subset is calculated to obtain the category mean value corresponding to each category label. The inter-class interval matrix is ​​calculated based on the overall mean and the category mean corresponding to each category label. The intra-class margin matrix is ​​calculated based on each subset of feature vectors and the corresponding class mean.

17. The apparatus according to claim 16, characterized in that, The determining module is specifically used to subtract the overall mean from the category mean for each category label in the C category label to obtain the first feature vector; For each category label in the C category label, transpose the first feature vector to obtain the second feature vector; For each category label in the C category label, the first feature vector and the second feature vector are multiplied to obtain the inter-class matrix corresponding to the category label; The inter-class interval matrix is ​​obtained by summing the inter-class matrices corresponding to each category label.

18. The apparatus according to claim 17, characterized in that, The determining module is specifically used to subtract the category mean from each feature vector in the feature vector subset for each category label in the C category label to obtain the third feature vector corresponding to each feature vector in the feature vector subset. For each category label in the C category label, the third feature vector corresponding to each feature vector in the feature vector subset is transposed to obtain the fourth feature vector corresponding to each feature vector in the feature vector subset; For each category label in the C category label, the third feature vector and the fourth feature vector corresponding to each feature vector in the feature vector subset are multiplied to obtain the first intra-class matrix corresponding to each feature vector in the feature vector subset; For each category label in the C category label, the first intra-class matrix corresponding to each feature vector in the feature vector subset is summed to obtain the second intra-class matrix corresponding to the category label; The intra-class interval matrix is ​​obtained by summing the second intra-class matrix corresponding to each category label.

19. The apparatus according to claim 15, characterized in that, The determining module is specifically used to perform feature projection on the feature vector corresponding to each training sample using the projection matrix to obtain a first target feature vector. For each training sample, a sample category score is calculated based on the first target feature vector and the category label corresponding to the training sample. For each training sample, the correct classification probability corresponding to the training sample is calculated based on the sample category score of the training sample. The model performance score of the pre-trained model is calculated based on the correct classification probability corresponding to each training sample.

20. The apparatus according to claim 19, characterized in that, The determining module is further configured to determine the number of training samples corresponding to each category label based on the category label to which each training sample belongs; The determining module is further configured to calculate the prior probability of each category label based on the number of training samples corresponding to each category label and the total number of training samples included in the target dataset. The determining module is specifically used to calculate the sample category score of the training sample for each category label based on the first target feature vector corresponding to the training sample, the projection matrix, the category mean corresponding to each category label, and the category prior probability corresponding to each category label.

21. The apparatus according to claim 19, characterized in that, The determining module is specifically used to calculate (N-1) distances based on the first target feature vector corresponding to the training sample and the first target feature vector corresponding to each of the (N-1) training samples, wherein the target data set includes N training samples, and the (N-1) training samples are the remaining training samples in the target data set excluding the training samples, and N is an integer greater than 1; Based on the (N-1) distances, determine the K training samples that are closest to the training samples from the (N-1) training samples, where K is an integer greater than 1 and less than (N-1); Based on the category label of each training sample in the K training samples, the sample category score for each category label is calculated.

22. The apparatus according to claim 19, characterized in that, The determining module is specifically used to calculate the first transferability score of the pre-trained model based on the classification correctness probability corresponding to each training sample. The selection module is specifically used to select at least one pre-trained model from the M pre-trained models based on the first transferability score of each pre-trained model.

23. The apparatus according to claim 19, characterized in that, The determining module is specifically used to perform noise processing on the feature vector corresponding to each training sample based on the correct classification probability of the training sample, so as to obtain a mixed feature vector. Based on the mixed feature vector corresponding to each training sample and the category label to which each training sample belongs, determine the inter-class margin matrix and the intra-class margin matrix. The target regularization coefficient is determined based on the target class interval matrix. The target projection matrix is ​​determined based on the target regularization coefficient, the target inter-class margin matrix, and the target intra-class margin matrix. The model performance score of the pre-trained model is determined based on the target projection matrix, the mixed feature vector corresponding to each training sample, and the category label of each training sample.

24. The apparatus according to claim 23, characterized in that, The determining module is specifically used to sum the feature vectors in the feature vector subset corresponding to each category label in the (C-1) category label to obtain the category feature vector. The (C-1) category labels are the remaining category labels in the C category labels excluding the category labels corresponding to the training samples. Each feature vector subset corresponds to the same category label, and C is an integer greater than 1. The sample mixing noise is calculated based on the category feature vector, the total number of training samples included in the target dataset, and the number of training samples corresponding to the category labels of the training samples. The mixed feature vector corresponding to the training sample is calculated based on the classification correctness probability corresponding to the training sample, the feature vector corresponding to the training sample, and the mixed noise of the sample.

25. The apparatus according to claim 23, characterized in that, The determining module is specifically used to perform feature projection on the mixed feature vector corresponding to each training sample using the target projection matrix to obtain a second target feature vector; For each training sample, the target sample category score is calculated based on the second target feature vector and the category label corresponding to the training sample. For each training sample, the probability of correct target classification corresponding to the training sample is calculated based on the target sample category score of the training sample. The model performance score of the pre-trained model is calculated based on the correct classification probability of the target corresponding to each training sample.

26. The apparatus according to claim 25, characterized in that, The determining module is specifically used to calculate the second transferability score of the pre-trained model based on the target classification correct probability corresponding to each training sample; The selection module is specifically used to select at least one pre-trained model from the M pre-trained models based on the second transferability score of each pre-trained model.

27. The apparatus according to any one of claims 15-26, characterized in that, The determining module is further configured to calculate the complementarity score of each pre-trained model based on the projection matrix and the feature vector set corresponding to each pre-trained model. The selection module is specifically used to calculate the comprehensive score corresponding to each pre-trained model based on the transferability score and complementarity score of each pre-trained model. Based on the comprehensive score corresponding to each pre-trained model, at least one pre-trained model is selected from the M pre-trained models.

28. The apparatus according to claim 27, characterized in that, The determining module is specifically used to calculate the multi-model feature matrix corresponding to each training sample based on the projection matrix and the feature vector set corresponding to each pre-trained model. For each pre-trained model, obtain the first nuclear norm of the feature matrix of the multi-model corresponding to each training sample; For each pre-trained model, the multi-model feature matrix corresponding to each training sample is multiplied by the matrix corresponding to the pre-trained model to obtain the third matrix corresponding to each training sample. For each pre-trained model, obtain the second kernel norm of the third matrix corresponding to each training sample; For each pre-trained model, the complementarity score of the pre-trained model is calculated based on the first and second nuclear norms corresponding to each training sample.

29. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the selection method according to any one of claims 1 to 14.

30. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the selection method as described in any one of claims 1 to 14.

31. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the selection method as described in any one of claims 1 to 14.

Images