Test phase adaptive learning method, device and equipment based on feature alignment

Abstract

The application relates to a test phase adaptive learning method, device and equipment based on feature alignment, wherein the method comprises the following steps: obtaining target domain images and selecting a target domain image queue to be trained from the target domain images; obtaining image pseudo labels of the target domain image queue to be trained, calculating pseudo label probability advantages, screening target domain images meeting preset confidence and pseudo labels corresponding to the target domain images meeting preset confidence, obtaining a pseudo label loss based on a preset pseudo label loss function; and constructing target domain category prototype features, performing feature alignment training on the target domain category prototype features and preset source domain category prototype features, obtaining a feature alignment training loss of the source domain and the target domain, and obtaining a training target of a test phase model according to the feature alignment training loss and the pseudo label loss. Thus, the problem that a test phase model training work of related technologies is not suitable for actual requirements of an industrial scene is solved, and the classification prediction effect of the model is improved.

Description

Technical Field

[0001] This application relates to the field of computer vision technology, and in particular to an adaptive learning method, apparatus and device for the testing phase based on feature alignment. Background Technology

[0002] The industrial model adaptive learning algorithm in the test phase falls under the category of test-time training in the field of computer vision. Test-time training is a branch of the domain adaptation problem, and in this task, the model also needs to overcome the domain gap between the training and test datasets.

[0003] Designing solutions for model training during the actual demand testing phase in the industrial sector presents unique challenges. In actual industrial production, data generated in new production environments (such as new production lines) are similar to data in old production environments in terms of image texture and visual features. However, due to the influence of the production environment on production equipment, the inference from data in the new production environment to the true label is often quite different from the inference in the old production environment.

[0004] Existing technologies focus solely on the overall visual features of images, and the limitations imposed by entropy minimization methods restrict the effectiveness of Tent applications on industrial data. TTT+ directly models different categories of data as Gaussian distributions. While this simplification may be effective for real-world data, it significantly impacts data analysis in industrial settings. Real-world data, influenced by numerous objective conditions, can be approximated as following a Gaussian distribution, but the factors influencing industrial data are likely extremely limited, resulting in a vastly different distribution between industrial and real-world data. Therefore, designing new adaptive testing-phase model training methods suitable for industrial applications is essential. Summary of the Invention

[0005] This application provides a feature-aligned adaptive learning method, device, electronic device, and storage medium for the testing phase, which addresses the problems of related technologies where the simplified data used in the testing phase model training is not suitable for industrial data analysis, and the insufficient utilization of image features makes it unsuitable for the actual requirements of industrial scenarios, resulting in limited application value in the industrial manufacturing field. This application improves the model's classification and prediction performance in the target domain and has high academic and application value.

[0006] The first aspect of this application provides a test-phase adaptive learning method based on feature alignment, comprising the following steps: acquiring target domain images and selecting a queue of target domain images to be trained from the target domain images; acquiring image pseudo-labels of the queue of target domain images to be trained, calculating the image pseudo-label probability advantage of the queue of target domain images to be trained based on the image pseudo-labels, and filtering target domain images that satisfy a preset confidence level and the pseudo-labels corresponding to the target domain images that satisfy the preset confidence level based on the image pseudo-label probability advantage; obtaining a pseudo-label loss based on a preset pseudo-label loss function, according to the target domain images that satisfy the preset confidence level and the pseudo-labels corresponding to the target domain images that satisfy the preset confidence level; constructing target domain category prototype features based on the target domain images that satisfy the preset confidence level and the pseudo-labels corresponding to the target domain images that satisfy the preset confidence level, and performing feature alignment training on the target domain category prototype features and preset source domain category prototype features to obtain the feature alignment training loss of the source domain and the target domain, and obtaining the training objective of the test-phase model based on the feature alignment training loss and the pseudo-label loss, so as to perform test-phase adaptive learning based on the training objective of the test-phase model.

[0007] Optionally, in some embodiments, before performing feature alignment training on the target domain category prototype features and the preset source domain category prototype features, the method further includes: acquiring a source domain image; obtaining source domain data features from the source domain data image through a preset feature extraction network; training the source domain data features using a preset loss minimization loss function and a stochastic gradient descent algorithm to obtain the features of the source domain image and its corresponding classification prediction vector; arranging the source domain image features into a matrix to obtain the feature matrix of the source domain image; arranging the classification prediction vectors into a matrix to obtain the classification prediction matrix of the source domain image; multiplying all the feature matrices of the source domain images with the transpose of the classification prediction matrix of the source domain images to obtain the source domain category prototype matrix; and transposing the source domain category prototype matrix to obtain the source domain category prototype features.

[0008] Optionally, in some embodiments, the preset loss minimization loss function is:

[0009]

[0010] in, The loss function is to minimize the loss. For the standard cross-entropy loss function, C cls For the classification head of the predicted values, N s The number of elements in the source domain image set. For the features of the i-th source domain image, This represents the category label of the i-th source domain image.

[0011] Optionally, in some embodiments, the preset pseudo-label loss function is:

[0012]

[0013] in, For pseudo-label loss function, For the standard cross-entropy loss function, N psd This represents the number of target domain images with confidence scores obtained after filtering in the current queue. For a target domain image with confidence level, These are pseudo-labels for target domain images with confidence levels.

[0014] Optionally, in some embodiments, the training objective of the model during the testing phase is:

[0015]

[0016] in, λ is the training objective of the model during the testing phase. feat For hyperparameters, The feature alignment loss function is... This is the pseudo-label loss function.

[0017] A second aspect of this application provides a test-phase adaptive learning device based on feature alignment, comprising: an acquisition module for acquiring target domain images and selecting a queue of target domain images to be trained from the target domain images; a calculation module for acquiring image pseudo-labels of the queue of target domain images to be trained, calculating the image pseudo-label probability advantage of the queue of target domain images to be trained based on the image pseudo-labels, and filtering target domain images that satisfy a preset confidence level and the pseudo-labels corresponding to the target domain images that satisfy the preset confidence level based on the image pseudo-label probability advantage; and a first generation module for generating images based on a preset pseudo-label loss function, according to the image pseudo-label probability advantage of the queue of target domain images to be trained. A pseudo-label loss is obtained from the target domain image with a pre-set confidence level and the pseudo-labels corresponding to the target domain image that satisfies the pre-set confidence level; and a second generation module is used to construct target domain category prototype features based on the target domain image that satisfies the pre-set confidence level and the pseudo-labels corresponding to the target domain image that satisfies the pre-set confidence level, and to perform feature alignment training on the target domain category prototype features and the preset source domain category prototype features to obtain the feature alignment training loss of the source domain and the target domain, and to obtain the training objective of the test phase model based on the feature alignment training loss and the pseudo-label loss, so as to perform test phase adaptive learning based on the training objective of the test phase model.

[0018] Optionally, in some embodiments, before performing feature alignment training on the target domain category prototype features and the preset source domain category prototype features, the second generation module further includes: an acquisition unit for acquiring a source domain image; a feature extraction unit for obtaining source domain data features from the source domain data image through a preset feature extraction network, and training the source domain data features using a preset loss minimization loss function and a stochastic gradient descent algorithm to obtain the features of the source domain image and its corresponding classification prediction vector; and a generation unit for arranging the source domain image features into a matrix to obtain the feature matrix of the source domain image, arranging the classification prediction vectors into a matrix to obtain the classification prediction matrix of the source domain image, multiplying all the feature matrices of the source domain images by the transpose of the classification prediction matrix of the source domain images to obtain the source domain category prototype matrix, and transposing the source domain category prototype matrix to obtain the source domain category prototype features.

[0019] Optionally, in some embodiments, the preset loss minimization loss function is:

[0020]

[0021] in, The loss function is to minimize the loss. For the standard cross-entropy loss function, C cls For the classification head of the predicted values, N s The number of elements in the source domain image set. For the features of the i-th source domain image, This represents the category label of the i-th source domain image.

[0022] Optionally, in some embodiments, the preset pseudo-label loss function is:

[0023]

[0024] in, For pseudo-label loss function, For the standard cross-entropy loss function, N psd This represents the number of target domain images with confidence scores obtained after filtering in the current queue. For a target domain image with confidence level, These are pseudo-labels for target domain images with confidence levels.

[0025] Optionally, in some embodiments, the training objective of the model during the testing phase is:

[0026]

[0027] in, λ is the training objective of the model during the testing phase.feat For hyperparameters, The feature alignment loss function is... This is the pseudo-label loss function.

[0028] A third aspect of this application provides an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the feature alignment-based adaptive learning method for the test phase as described in the above embodiments.

[0029] A fourth aspect of this application provides a computer-readable storage medium having a computer program stored thereon, which is executed by a processor to implement the feature alignment-based adaptive learning method for the test phase as described in the above embodiments.

[0030] Therefore, this application implements a feature-aligned adaptive learning algorithm for industrial models in the testing phase, specifically addressing the problems of oversimplification of data distribution and insufficient utilization of image features in current testing-phase model training. This further improves the model's classification and prediction performance in the target domain within industrial scenarios. This application has high academic and applied value, and at least the following beneficial effects:

[0031] 1. Introduce the feature alignment paradigm into the model training method during the testing phase in the industrial field, so as to enable the model to make full use of the data features in the industrial field and enhance the model's classification and prediction performance on industrial data during the testing phase.

[0032] 2. An end-to-end industrial scenario testing model training method was implemented, which has high training stability and strong practicality in industrial manufacturing scenarios.

[0033] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description

[0034] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein:

[0035] Figure 1 This is a flowchart of a feature alignment-based adaptive learning method for the testing phase provided in an embodiment of this application;

[0036] Figure 2 A flowchart of a test phase adaptive learning method based on feature alignment according to an embodiment of this application;

[0037] Figure 3This is a block diagram of a feature-aligned adaptive learning device for the testing phase according to an embodiment of this application;

[0038] Figure 4 This is a schematic diagram of the structure of an electronic device provided according to an embodiment of this application. Detailed Implementation

[0039] The embodiments of this application are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain this application, and should not be construed as limiting this application.

[0040] Before introducing the feature alignment-based adaptive learning method for the test phase according to the embodiments of this application, let's first introduce the adaptive learning method for the test phase in related technologies.

[0041] In the application of deep learning models in industrial manufacturing scenarios, collecting data from new production environments and training the model in the target production environment using a domain-adaptive training paradigm often fails to meet the deployment efficiency requirements of industrial manufacturing scenarios. In such cases, the model needs to be trained while collecting data from the new production environment, and the deployed deep model needs to be iteratively optimized during training. Simultaneously, when training in the new production environment, the model cannot access data from the original production environment. The aforementioned problems in the industrial field can be abstracted as a model training problem during the testing phase.

[0042] The test phase model training method of this application aims to train a model that adapts to the characteristics of data in industrial manufacturing scenarios, thereby solving the problem of deploying deep models in a completely new production environment in industrial manufacturing.

[0043] This application innovatively introduces a feature alignment paradigm for feature alignment across different domains. This paradigm is applicable to industrial scenarios where data distribution cannot be considered a standard Gaussian distribution. During the training of the model described in this invention, fully labeled source domain data is input, and the model is pre-trained using the standard Empirical Loss Minimization (ERM) paradigm. Then, target domain data is input in batches. The classification accuracy of the trained target domain data is recorded during the training process. After one iteration on the target domain data, the adaptive classification accuracy of the model on the target domain data is obtained.

[0044] A typical flowchart of industrial model training during the testing phase is shown below. Figure 1As shown. Related technologies employ model training methods during the testing phase that fail to fully utilize data features and oversimplify data distribution to a Gaussian distribution. While these strategies are reasonable for real-world datasets, they represent a degree of oversimplification for industrial data. Therefore, this application aims to introduce a feature alignment method to enhance the model's feature perception capability for industrial scenario data and further improve the model's training performance on such data.

[0045] Specifically, Figure 2 This is a flowchart illustrating a feature-aligned adaptive learning method for the testing phase provided in an embodiment of this application.

[0046] like Figure 2 As shown, this feature-aligned adaptive learning method for the testing phase includes the following steps:

[0047] In step S201, the target domain image is acquired, and a queue of target domain images to be trained is selected from the target domain images.

[0048] It is understood that the embodiments of this application can test the target domain images in mini-batches, record their accuracy, and then perform backpropagation training. During the mini-batch testing process, the network parameters are iteratively improved, and the final target domain classification accuracy reported by the model is the sum of the test accuracies of all batches in one iteration. Improvement Figure 1 The "Calculate relevant loss and perform backpropagation" step in the "Adaptive Training in the Test Phase" shown occurs during model prediction and training for each batch of the target domain.

[0049] As one possible implementation, embodiments of this application can set up a test phase training team, adding a finite length N. q A standard queue data structure for storing a certain number of out-of-bounds target domain images in the current batch is helpful for model training during the testing phase. Controlling the queue length N... q <<N t At the same time, N q >N b , where N b The number of target domain images in each batch. In this embodiment, the queue stores the original image vectors of the target domain images. Before each new batch enters the network for feedforward computation, it is first pushed to the tail of the queue, and at the same time, the corresponding number of images at the head of the queue are dequeued.

[0050] In step S202, image pseudo-labels of the target domain image queue to be trained are obtained, and the image pseudo-label probability advantage of the target domain image queue to be trained is calculated based on the image pseudo-labels. Based on the image pseudo-label probability advantage, target domain images that meet the preset confidence level and pseudo-labels corresponding to the target domain images that meet the preset confidence level are selected.

[0051] Specifically, at the beginning of each batch of training, this embodiment first assigns a pseudo-label to the images in the training queue during the testing phase after the current batch update. The pseudo-label can be set directly using the network's predicted probability distribution vector for image classification. For each target domain image in the queue... The predicted probability distribution vector of the network's classification can be obtained from it. The network obtains the pseudo-labels for the predicted categories.

[0052]

[0053] Because the quality of the network's predicted probability distributions for different images varies, this embodiment selects to filter the pseudo-labels generated by the network. Generally, predictions with lower information entropy from a neural network are considered to have higher reliability. However, directly calculating the information entropy of all images can lead to situations where "two categories have higher predicted probability values, while other categories have lower predicted probability values" are selected as more reliable predictions. However, using such predictions to assign a pseudo-label to an image of a single category is actually of low reliability, because the true category of the image may be one of these two categories, but the network's predicted probabilities for these two categories are similar, which could easily lead to prediction errors. This embodiment attempts to filter reliable pseudo-labels while keeping the above-mentioned pseudo-label setting method unchanged.

[0054] Specifically, for each image in the queue Calculate its pseudo-label probability advantage

[0055]

[0056] in:

[0057]

[0058]

[0059] For network images Predict the most likely class in the probability distribution (i.e., predict the pseudo-label) The corresponding predicted probability, For network images The second most likely category in the predicted probability distribution The corresponding predicted probability. This application's embodiments aim to improve the network's pseudo-label probability advantage. A sufficiently large confidence level, meaning the network has the strongest relative confidence in the most likely class of the image, ensures, to a certain extent, that the information entropy of the predicted probability distribution of the selected images is sufficiently small, while avoiding the occurrence of the network predicting images with two credible classes. This selection process can be represented as filtering target domain images with confidence levels from the queue. and its corresponding pseudo-tags

[0060]

[0061] in, The hyperparameter for the probability advantage threshold of pseudo-labels is set manually.

[0062] In step S203, based on a preset pseudo-label loss function, pseudo-label loss is obtained according to the target domain image that meets the preset confidence level and the pseudo-labels corresponding to the target domain image that meets the preset confidence level.

[0063] Specifically, embodiments of this application can be based on the target domain image with confidence obtained through the above screening and its pseudo-labels. Using pseudo-label loss during the testing phase Pseudo-label training of the model:

[0064]

[0065] in, For the standard cross-entropy loss function, N psd This represents the number of target domain images with confidence scores obtained after filtering in the current queue.

[0066] In step S204, target domain category prototype features are constructed based on the target domain image that meets the preset confidence level and the pseudo-labels corresponding to the target domain image that meets the preset confidence level. Feature alignment training is performed on the target domain category prototype features and the preset source domain category prototype features to obtain the loss of feature alignment training between the source domain and the target domain. The training objective of the test phase model is obtained based on the loss of feature alignment training and the pseudo-label loss, so as to perform adaptive learning in the test phase based on the training objective of the test phase model.

[0067] Specifically, based on the aforementioned target domain image with confidence and its pseudo-labels Construct prototype features for the target domain category. (Note: The original text contains some formatting errors and inconsistencies. A more The features of the above-mentioned filtered target domain images are used to calculate a target domain category prototype feature for each category. Where j = 1, 2, ..., c. In the training process corresponding to each batch, firstly, target domain images with confidence pseudo-labels are selected from the current queue. Then, based on these images, the target domain prototype feature update amount for the current batch is calculated.

[0068]

[0069] Where m is used for calculation Features The number of.

[0070] This application embodiment can use the momentum update method to update the target domain category prototype features. Before training begins, the prototype features of the target domain categories are initialized. Then proceed as follows: Update:

[0071]

[0072] In update At the same time, save a boolean variable. It is initialized to False if and only if After the first update, The value is updated to True. A boolean variable is also stored to indicate the prototype that was updated in the current batch. The value is updated during the training process of each batch if and only if the filtered images in the queue of the current batch are used to update the value in the current batch. hour, The value is True, otherwise The value of is False.

[0073] This application embodiment is based on the above-mentioned target domain category prototype features. With source domain category prototype feature vector Feature alignment training is performed. Since the source and target domains are trained using the same network, and due to model training limitations during the testing phase, the source and target domains are restricted to having the same image category. This embodiment of the application can obtain source domain category prototype vectors with one-to-one category and one-to-one feature vector correspondence. and target domain category prototype vector Therefore, the embodiments of this application can improve the method of Centralized Kernel Alignment (CKA) for feature alignment training.

[0074] First, the feature matrices of the source and target domains are corrected. Using k... s middle The same arrangement will be used to represent the prototype features of the target domain. Arranged into the target domain prototype feature matrix k t In each training batch, k is first removed. s With k t All of them For features that take a value of False, the source domain feature prototype matrix updated in the current batch is obtained. With the target domain feature prototype matrix Calculate the Gram matrix g of the feature prototype matrix updated in the current batch. s With g t ,have:

[0075]

[0076]

[0077] Next, the centered kernel alignment function value CKA(g) of the above Gram matrix is ​​calculated. s g t ), and use the function value for feature alignment training. Let the trace of the matrix be tr(·), then:

[0078]

[0079] The method for calculating the centralization matrix H is as follows:

[0080]

[0081] Where I is the identity matrix and 1 is a column vector with all values ​​of 1.

[0082] Finally, based on the above function values, the loss for feature alignment training between the source and target domains can be obtained.

[0083]

[0084] Combining the pseudo-label loss and feature alignment loss described above, we can obtain the training objective of the model during the testing phase.

[0085]

[0086] Where, λ feat These are hyperparameters that are manually determined during training.

[0087] Optionally, in some embodiments, before performing feature alignment training on the target domain category prototype features and the preset source domain category prototype features, the method further includes: acquiring a source domain image; obtaining source domain data features from the source domain data image through a preset feature extraction network; training the source domain data features based on the source domain data features using a preset loss minimization loss function and a stochastic gradient descent algorithm to obtain the features of the source domain image and its corresponding classification prediction vector; arranging the source domain image features into a matrix to obtain the feature matrix of the source domain image; arranging the classification prediction vectors into a matrix to obtain the classification prediction matrix of the source domain image; multiplying the feature matrices of all source domain images with the transpose of the classification prediction matrix of the source domain image to obtain the source domain category prototype matrix; and transposing the source domain category prototype matrix to obtain the source domain category prototype features.

[0088] Specifically, in this embodiment, a ResNet-50 deep convolutional residual network can be used as the backbone network of the model, and a classification head can be added after the network to perform image classification prediction. The model is trained for image classification by minimizing empirical loss on all source domain data. After training is completed, the model can perform test phase training and prediction for target domain images.

[0089] Before performing feature alignment training on the target domain category prototype features and the preset source domain category prototype features, image classification training with empirical loss minimization is performed on all source domain data.

[0090] This application uses a deep neural network (DNN) to complete the model training task during the testing phase. The model can be built using Python, based on the open-source code libraries PyTorch (version 1.10.2) and torchvision (version 0.11.3). All experiments described in this application can be conducted within these environments. When training image classification with empirical loss minimization on source domain data, a deep model containing a backbone network and a classification head can be used.

[0091] Before detailing the methods proposed in the embodiments of this application, a mathematical definition of the model training problem in the testing phase is first given, along with relevant mathematical symbols used subsequently. A model training problem in the testing phase requires training the model to minimize the empirical loss on a labeled source domain, followed by testing the model on a completely unlabeled target domain. The model is trained specifically for the target domain during the target domain image testing process, and the accuracy of the model test in a single iteration is reported. For the labeled source domain, the images and their category labels are denoted as […]. in Let represent the original image vector of the i-th source domain image. N represents the category label of the i-th source domain image. s This is the number of elements in the aforementioned source domain image set. All unlabeled target domain images are denoted as... in N represents the original image vector of the i-th target domain image. t This is the number of elements in the aforementioned target domain image set. This embodiment of the application follows the requirements of the model training problem in the testing phase, in D... s Train the model and use D during the testing phase. t Perform a single iteration and output the model's prediction accuracy during the iteration process. The above D s D t The two datasets belong to different distributions within the same feature space. The images in both distributions have the same set of labels (belonging to the same set of categories). Therefore, they can be directly compared in D. s A fully trained model is applied to D t It is difficult to achieve the desired effect in a production environment, so special training strategies are required.

[0092] The basic structure of the model designed in this application embodiment. The main architecture of the model includes a feature extraction network f, which adopts a ResNet-50 network structure, and a classification head C for outputting classification prediction values. cls Let f be the feature vector obtained after the image passes through the feature extraction network, and f = f(x). This can be viewed as a column vector of dimension l, where x represents the original image vector after passing through the network. Let p be the category prediction vector obtained after the feature passes through the classification head, and p = C. cls (f) can be viewed as a column vector with dimension c, which is the number of all categories.

[0093] First, the model is trained using labeled source domain images using Empirical Risk Minimization (ERM). Empirical Risk Minimization is a classic model training paradigm in machine learning, generally assuming that models trained on known data tend to perform better on unknown domains. During the testing phase of model training, it can be assumed that a model fully trained on source domain data possesses a certain feature representation ability on unknown target domain data. Following the image classification training paradigm commonly used in deep learning, all source domain images are... Source domain data characteristics are obtained through the network. And through the standard cross-entropy loss function Constructing a loss function that minimizes empirical loss The model was trained using the Stochastic Gradient Descent (SGD) algorithm.

[0094]

[0095] Meanwhile, after training is complete, a portion of the model's information in the source domain is saved to assist in training the model in the target domain. In this embodiment, a source domain category prototype can be constructed using an attention mechanism, and the target domain image training can be guided by this prototype. After completing the above-mentioned empirical loss minimization training, a round of network feedforward computation is performed on all source domain images to obtain the features of all images and their corresponding classification prediction vectors. These vectors are arranged into a matrix, and if... Representing the features of the i-th source domain image, using The classification prediction result for the i-th source domain image can be represented by the feature matrix of all images. And the classification prediction matrix for all images Based on the previous definition, it is not difficult to deduce that matrix p s Can be used with matrix f s transpose Performing matrix multiplication results in a matrix with dimensions c×N. s :

[0096]

[0097] Define the matrix k above s This is the source domain category prototype matrix, where each row represents the transpose of the prototype feature vector for each category in the source domain. Extracting and transposing the vectors from each row yields the source domain category prototype feature vectors. Where i represents the i-th source domain category. In this embodiment, the aforementioned source domain category prototype feature vectors will be used to guide the target domain's learning during the target domain image testing phase.

[0098] According to the feature alignment-based adaptive learning method for the testing phase proposed in this application, the method acquires target domain images, selects a queue of target domain images to be trained from the target domain images, acquires image pseudo-labels for the queue of target domain images to be trained, calculates the probability advantage of the image pseudo-labels for the queue of target domain images to be trained based on the image pseudo-labels, and filters target domain images and their corresponding pseudo-labels that meet a preset confidence level based on the image pseudo-label probability advantage. Based on a preset pseudo-label loss function, the method obtains the pseudo-label loss based on the target domain images and their corresponding pseudo-labels that meet the preset confidence level. The method constructs target domain category prototype features based on the target domain images and their corresponding pseudo-labels that meet the preset confidence level, and performs feature alignment training on the target domain category prototype features and preset source domain category prototype features to obtain the feature alignment training loss for the source and target domains. Based on the feature alignment training loss and the pseudo-label loss, the method obtains the training objective of the testing phase model, and performs adaptive learning for the testing phase based on the training objective of the testing phase model. This solves the problems of model training in the testing phase of related technologies, where the simplified data is not suitable for the analysis of industrial data, and the insufficient utilization of image features makes it unsuitable for the actual requirements of industrial scenarios, resulting in limited application value in the industrial manufacturing field. It improves the classification and prediction performance of the model in the target domain and has high academic and application value.

[0099] Next, referring to the accompanying drawings, we describe the adaptive learning apparatus for the test phase based on feature alignment according to an embodiment of this application.

[0100] Figure 3 This is a block diagram of a feature-aligned adaptive learning device for the testing phase according to an embodiment of this application.

[0101] like Figure 3 As shown, the feature alignment-based test phase adaptive learning device 10 includes: an acquisition module 100, a calculation module 200, a first generation module 300, and a second generation module 400.

[0102] The module 100 is used to acquire target domain images and select a queue of target domain images to be trained from the target domain images; the calculation module 200 is used to acquire image pseudo-labels of the queue of target domain images to be trained, calculate the image pseudo-label probability advantage of the queue of target domain images to be trained based on the image pseudo-labels, and filter to obtain target domain images that meet a preset confidence level and the pseudo-labels corresponding to the target domain images that meet the preset confidence level based on the image pseudo-label probability advantage; the first generation module 300 is used to obtain pseudo-label loss based on a preset pseudo-label loss function, according to the target domain images that meet the preset confidence level and the pseudo-labels corresponding to the target domain images that meet the preset confidence level; the second generation module 400 is used to construct target domain category prototype features based on the target domain images that meet the preset confidence level and the pseudo-labels corresponding to the target domain images that meet the preset confidence level, and perform feature alignment training on the target domain category prototype features and the preset source domain category prototype features to obtain the feature alignment training loss of the source domain and the target domain, and obtain the training objective of the test phase model based on the feature alignment training loss and the pseudo-label loss, so as to perform adaptive learning in the test phase according to the training objective of the test phase model.

[0103] Optionally, in some embodiments, before performing feature alignment training on the target domain category prototype features and the preset source domain category prototype features, the second generation module 400 further includes: an acquisition unit, a feature extraction unit, and a generation unit.

[0104] The acquisition unit is used to acquire the source domain image.

[0105] The feature extraction unit is used to obtain source domain data features from the source domain data image through a preset feature extraction network. Based on the source domain data features, the system is trained using a preset loss minimization loss function and a stochastic gradient descent algorithm to obtain the features of the source domain image and its corresponding classification prediction vector.

[0106] The generation unit is used to arrange the source domain image features into a matrix to obtain the feature matrix of the source domain image, arrange the classification prediction vectors into a matrix to obtain the classification prediction matrix of the source domain image, multiply the feature matrices of all source domain images with the transpose of the classification prediction matrix of the source domain image to obtain the source domain category prototype matrix, and transpose the source domain category prototype matrix to obtain the source domain category prototype features.

[0107] Optionally, in some embodiments, the preset loss minimization loss function is:

[0108]

[0109] in, The loss function is to minimize the loss. For the standard cross-entropy loss function, C cls For the classification head of the predicted values, N sThe number of elements in the source domain image set. For the features of the i-th source domain image, This represents the category label of the i-th source domain image.

[0110] Optionally, in some embodiments, the preset pseudo-label loss function is:

[0111]

[0112] in, For pseudo-label loss function, For the standard cross-entropy loss function, N psd This represents the number of target domain images with confidence scores obtained after filtering in the current queue. For a target domain image with confidence level, These are pseudo-labels for target domain images with confidence levels.

[0113] Optionally, in some embodiments, the training objective of the model during the testing phase is:

[0114]

[0115] in, λ is the training objective of the model during the testing phase. feat For hyperparameters, The feature alignment loss function is... This is the pseudo-label loss function.

[0116] It should be noted that the foregoing explanation of the feature alignment-based adaptive learning method for the test phase also applies to the feature alignment-based adaptive learning device for the test phase in this embodiment, and will not be repeated here.

[0117] According to the feature alignment-based adaptive learning device for the test phase proposed in this application, the device acquires target domain images, selects a queue of target domain images to be trained from the target domain images, acquires image pseudo-labels for the queue of target domain images to be trained, calculates the probability advantage of the image pseudo-labels for the queue of target domain images to be trained based on the image pseudo-labels, and filters target domain images and their corresponding pseudo-labels that meet a preset confidence level based on the image pseudo-label probability advantage. Based on a preset pseudo-label loss function, the device obtains the pseudo-label loss based on the target domain images and their corresponding pseudo-labels that meet the preset confidence level. The device constructs target domain category prototype features based on the target domain images and their corresponding pseudo-labels that meet the preset confidence level, and performs feature alignment training on the target domain category prototype features and preset source domain category prototype features to obtain the feature alignment training loss for the source and target domains. Based on the feature alignment training loss and the pseudo-label loss, the device obtains the training objective of the test phase model, and performs adaptive learning for the test phase based on the training objective of the test phase model. This solves the problems of model training in the testing phase of related technologies, where the simplified data is not suitable for the analysis of industrial data, and the insufficient utilization of image features makes it unsuitable for the actual requirements of industrial scenarios, resulting in limited application value in the industrial manufacturing field. It improves the classification and prediction performance of the model in the target domain and has high academic and application value.

[0118] Figure 4 A schematic diagram of the structure of an electronic device provided in an embodiment of this application. The electronic device may include:

[0119] The memory 401, the processor 402, and the computer program stored on the memory 401 and capable of running on the processor 402.

[0120] When the processor 402 executes the program, it implements the adaptive learning method for the test phase based on feature alignment provided in the above embodiments.

[0121] Furthermore, electronic devices also include:

[0122] Communication interface 403 is used for communication between memory 401 and processor 402.

[0123] The memory 401 is used to store computer programs that can run on the processor 402.

[0124] The memory 401 may include high-speed RAM (Random Access Memory) memory, and may also include non-volatile memory, such as at least one disk storage.

[0125] If the memory 401, processor 402, and communication interface 403 are implemented independently, then the communication interface 403, memory 401, and processor 402 can be interconnected via a bus to complete communication between them. The bus can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus, or an EISA (Extended Industry Standard Architecture) bus, etc. The bus can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 4 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0126] Optionally, in a specific implementation, if the memory 401, processor 402, and communication interface 403 are integrated on a single chip, then the memory 401, processor 402, and communication interface 403 can communicate with each other through an internal interface.

[0127] Processor 402 may be a CPU (Central Processing Unit), an ASIC (Application Specific Integrated Circuit), or one or more integrated circuits configured to implement embodiments of this application.

[0128] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described adaptive learning method for the test phase based on feature alignment.

[0129] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0130] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "N" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0131] Any process or method described in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more N executable instructions for implementing custom logic functions or processes, and the scope of the preferred embodiments of this application includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as should be understood by those skilled in the art to which embodiments of this application pertain.

[0132] It should be understood that the various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, the N steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (FPGAs), field-programmable gate arrays (FPGAs), etc.

[0133] Those skilled in the art will understand that all or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, the program includes one or a combination of the steps of the method embodiments.

[0134] Although embodiments of this application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting this application. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of this application.

Claims

1. A feature alignment based test phase adaptive learning method, characterized in that, Includes the following steps: Acquire target domain images, and select a queue of target domain images to be trained from the target domain images; Obtain the image pseudo-labels of the target domain image queue to be trained, calculate the image pseudo-label probability advantage of the target domain image queue to be trained based on the image pseudo-labels, and filter out the target domain images that meet the preset confidence level and the pseudo-labels corresponding to the target domain images that meet the preset confidence level based on the image pseudo-label probability advantage. Based on a preset pseudo-label loss function, the pseudo-label loss is obtained according to the target domain image that meets the preset confidence level and the pseudo-label corresponding to the target domain image that meets the preset confidence level. as well as Based on the target domain image that satisfies the preset confidence level and the pseudo-label corresponding to the target domain image that satisfies the preset confidence level, construct the target domain category prototype features, and perform feature alignment training on the target domain category prototype features and the preset source domain category prototype features to obtain the loss of feature alignment training between the source domain and the target domain. Based on the loss of feature alignment training and the pseudo-label loss, obtain the training objective of the model in the test phase, so as to perform adaptive learning in the test phase according to the training objective of the model in the test phase.

2. The method according to claim 1, characterized in that, Before performing feature alignment training on the target domain category prototype features and the preset source domain category prototype features, the following steps are also included: Acquire the source domain image; The source domain image is processed through a pre-defined feature extraction network to obtain source domain data features. Based on these features, the system is trained using a pre-defined loss minimization function and a stochastic gradient descent algorithm to obtain the features of the source domain image and its corresponding classification prediction vector. The source domain image features are arranged into a matrix to obtain the feature matrix of the source domain image, and the classification prediction vectors are arranged into a matrix to obtain the classification prediction matrix of the source domain image. The feature matrices of all the source domain images are multiplied by the transpose of the classification prediction matrix of the source domain image to obtain the source domain category prototype matrix, and the source domain category prototype matrix is ​​transposed to obtain the source domain category prototype features.

3. The method according to claim 2, characterized in that, The preset loss minimization loss function is: ； in, The loss function is to minimize the loss. The standard cross-entropy loss function, For the classification head of the predicted values, The number of elements in the source domain image set. For the first Features of Zhang Yuanyu's images Indicates the first Category labeling of Zhang Yuanyu's images.

4. The method according to claim 1, characterized in that, The preset pseudo-label loss function is: ； in, For pseudo-label loss function, The standard cross-entropy loss function, This represents the number of target domain images with confidence scores obtained after filtering in the current queue. For a target domain image with confidence level, These are pseudo-labels for target domain images with confidence levels.

5. The method according to claim 1, characterized in that, The training objective of the model during the testing phase is: in, The training objective for the model during the testing phase. For hyperparameters, The feature alignment loss function is... This is the pseudo-label loss function.

6. A test-phase adaptive learning device based on feature alignment, characterized in that, include: An acquisition module is used to acquire target domain images and select a queue of target domain images to be trained from the target domain images; The calculation module is used to obtain image pseudo-labels of the target domain image queue to be trained, calculate the image pseudo-label probability advantage of the target domain image queue to be trained based on the image pseudo-labels, and filter out target domain images that meet the preset confidence level and pseudo-labels corresponding to the target domain images that meet the preset confidence level based on the image pseudo-label probability advantage. The first generation module is used to obtain pseudo-label loss based on a preset pseudo-label loss function, according to the target domain image that meets the preset confidence level and the pseudo-label corresponding to the target domain image that meets the preset confidence level. as well as The second generation module is used to construct target domain category prototype features based on the target domain image that satisfies the preset confidence level and the pseudo-labels corresponding to the target domain image that satisfies the preset confidence level, and to perform feature alignment training on the target domain category prototype features and the preset source domain category prototype features to obtain the loss of feature alignment training between the source domain and the target domain, and to obtain the training objective of the test phase model based on the loss of feature alignment training and the pseudo-label loss, so as to perform adaptive learning in the test phase based on the training objective of the test phase model.

7. The apparatus according to claim 6, characterized in that, Before performing feature alignment training on the target domain category prototype features and the preset source domain category prototype features, the second generation module further includes: The acquisition unit is used to acquire the source domain image; The feature extraction unit is used to obtain source domain data features from the source domain image through a preset feature extraction network. Based on the source domain data features, the network is trained using a preset loss minimization loss function and a stochastic gradient descent algorithm to obtain the features of the source domain image and its corresponding classification prediction vector. The generation unit is used to arrange the source domain image features into a matrix to obtain the feature matrix of the source domain image, arrange the classification prediction vectors into a matrix to obtain the classification prediction matrix of the source domain image, multiply all the feature matrices of the source domain images by the transpose of the classification prediction matrix of the source domain images to obtain the source domain category prototype matrix, and transpose the source domain category prototype matrix to obtain the source domain category prototype features.

8. The apparatus according to claim 7, characterized in that, The preset loss minimization loss function is: ； in, The loss function is to minimize the loss. The standard cross-entropy loss function, For the classification head of the predicted values, The number of elements in the source domain image set. For the first Features of Zhang Yuanyu's images Indicates the first Category labeling of Zhang Yuanyu's images.

9. An electronic device, characterized in that, include: A memory, a processor, and a computer program stored in the memory and executable on the processor, the processor executing the program to implement the feature alignment-based adaptive learning method for the test phase as described in any one of claims 1-5.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, The program is executed by the processor to implement the feature alignment-based adaptive learning method for the test phase as described in any one of claims 1-5.

Images