A general approach to domain generalization

Abstract

The present invention discloses a universal domain generalization method. The method comprises: obtaining a pre-trained neural network model for a source domain, wherein the pre-trained neural network model comprises a source domain feature extractor and a source domain classifier; and adaptively fine-tuning the pre-trained neural network model through online learning using target domain sample data with the minimization of a set total loss function as the optimization goal for a target domain to obtain an updated neural network model, wherein during the adaptive fine-tuning process, classification prototypes of each category are created as a memory bank, and the entire memory bank is updated and iterated by maximizing the probability similarity between the classification prototypes corresponding to each category of the dynamic learning representation. The present invention can effectively solve the domain shift problem, improve the efficiency of the domain generalization task, and is applicable to various types of neural network models.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and more particularly, to a universal domain generalization method. Background Art

[0002] Domain generalization technology aims to enable neural networks to generalize strongly across a wide range of unseen image domains. Existing domain generalization methods can be categorized into representation learning, backbone network architecture design, and learning strategies, based on different optimization objectives. Representation learning extracts specific features from the source domain and assumes they are robust in the target domain. For example, domain alignment methods can be used to directly extract domain-invariant representations from the source domain. However, this is extremely difficult, requiring feature decoupling methods to relax the constraints of directly extracting domain-invariant features, thereby decomposing them into domain-invariant and domain-dependent representations.

[0003] Different backbone network designs significantly impact domain generalization. These methods optimize mainstream convolutional and Transformer network architectures to address domain generalization. For example, the ViT-based GMoE replaces the FFN layer and employs a mixture of experts model, enabling different experts to focus on different visual attributes.

[0004] Learning strategy approaches leverage machine learning strategies to enhance the model's generalization capabilities on the target domain, such as meta-learning and ensemble learning. Meta-learning splits the training data into a meta-training set and a meta-test set, simulating domain shift and updating parameters during training. Ensemble learning simultaneously learns multiple copies of the same model, extracting features from the source domain and transferring them to the target domain.

[0005] For existing online learning solutions, models trained on the source domain are always faced with unseen domain shift and may not be generalized to the target domain. For example, the existing test-time adaptation (TTA) solution proposes to update model parameters during the test process based on target data. This type of method can be divided into three branches:

[0006] 1) Adversarial learning: With the advancement of generative adversarial networks, deep generative models are used to synthesize target data to improve the ability to handle domain shifts, thereby getting rid of the dependence on source data.

[0007] 2) Method for Batch Normalization (BN) layer: The batch normalization method replaces the batch normalization (BN) statistics of the trained model with the BN statistics estimated on the test data, and only updates the parameters of the BN layer while keeping its backbone network parameters unchanged. It aims to minimize the information entropy during the test process to maximize the generalization ability of the model in unknown fields. Currently, the Wasserstein distance between the source and target statistics is used as the metric.

[0008] 3) Bayesian Learning: We analyze the TTA problem from the perspective of Bayesian probability modeling and propose a regularized entropy minimization process that is achieved by approximating the probability density during training time.

[0009] However, these approaches primarily update a subset of network parameters during test time, resulting in unstable and limited improvements and insufficient adaptability in unknown environments. Furthermore, while large-scale pre-training has achieved significant progress in domain generalization, frequently retraining such large-scale networks to accommodate rapid data updates and iterations is challenging. Furthermore, pre-training also neglects the ability to distinguish and perceive different domains, further limiting their generalization capabilities.

[0010] Take the Transformer network as an example, which has achieved remarkable results in various computer vision tasks such as image classification, object detection, and semantic segmentation. However, when the test data distribution is different from the training data distribution, the model performance may degrade significantly, which is a problem known as domain shift (for example, training the network on a sunny day and testing it on a rainy day).

[0011] The domain shift problem frequently arises in practice, especially as data becomes increasingly accessible and easy to generate. Therefore, how to train deep models to generalize well to new domains has become a common research topic. In this context, domain adaptation (DA) and domain generalization (DG) are two typical research directions to alleviate the domain shift problem. Domain generalization can generalize a model to unseen domains.

[0012] Recently, large-scale pre-training of visual Transformers has had a significant impact on domain generalization, and can even achieve excellent performance through zero-shot learning. The use of pre-training can significantly outperform domain generalization algorithms. For example, CLIP (Contrastive Language-Image Pre-Training), which uses a large number of image-text pairs for pre-training, can demonstrate impressive zero-shot performance on downstream tasks, becoming a powerful and difficult-to-surpass benchmark in domain generalization tasks. At the same time, test-time adaptation (TTA) can alleviate the problem of offset in unknown domains. Based on this technology, researchers expect the model to achieve continuous learning and continuous optimization through online learning methods, and to be able to handle more complex situations, such as continuous changes in data and time correlation.

[0013] However, both pre-training-based methods and test-time adaptation schemes have difficulty in dealing with complex situations caused by very fast data iterations. For example, for pre-training-based methods, if the network cannot generalize well on the new domain, the pre-trained model needs to be frequently retrained from scratch to cope with unseen environments, which requires very high training costs. At the same time, the existing visual pre-training process focuses more on general image understanding and ignores domain perception capabilities. For test-time schemes, they mainly focus on updating the parameters of the batch normalization (BN) layer to adapt the learned classification bias to the new domain. However, since the Transformer architecture does not have a BN layer, this prevents the test-time adaptation method from being directly applied to these emerging Transformer networks with excellent performance and whose performance improves as the amount of data increases. In addition, it has been theoretically proven that updating local parameters is difficult to handle the more complex domain shifts that frequently occur in the 2020s.

[0014] In summary, in order to address the more frequent domain shift problem caused by rapid data updates and iterations, as well as the limitations of online learning methods that are incompatible with existing Transformer networks, it is necessary to improve existing technologies to provide a unified, efficient, and powerful framework that can be used for various existing mainstream neural networks and use test-time adaptation methods to improve the model's generalization ability in unknown domains. Summary of the Invention

[0015] The purpose of the present invention is to overcome the above-mentioned shortcomings of the prior art and provide a universal domain generalization method. The method comprises the following steps:

[0016] For the source domain, obtaining a pre-trained neural network model, wherein the pre-trained neural network model includes a source domain feature extractor and a source domain classifier;

[0017] For the target domain, with the minimization of the set total loss function as the optimization goal, the pre-trained neural network model is adaptively fine-tuned through online learning using the target domain sample data to obtain an updated neural network model. During the adaptive fine-tuning process, classification prototypes of each category are created as a memory library, and the entire memory library is updated and iterated by maximizing the probabilistic similarity between the dynamic learning representations corresponding to the classification prototypes of each category.

[0018] Compared with the existing technology, the advantage of the present invention is that it can bring rapid adaptation to the newly acquired image domain and compatible with the relevant knowledge of source domain training for existing neural networks deployed to terminal devices, computing engines, and ultra-large-scale pre-trained neural networks, which can greatly save the related overhead of training large-scale models from scratch and updating and iterating deployed neural networks. The present invention significantly reduces the high cost of retraining large-scale pre-trained models when processing rapidly iterating new data, and when solving domain generalization problems, it can bring very significant performance improvements to all current mainstream neural networks (such as convolutional networks, multi-layer perceptron networks, Transformer networks, etc.), thereby improving the versatility of existing algorithms.

[0019] Further features and advantages of the present invention will become apparent from the following detailed description of exemplary embodiments of the present invention with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments of the invention and, together with the description, serve to explain the principles of the invention.

[0021] Figure 1 is a flowchart of a general domain generalization method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0022] Various exemplary embodiments of the present invention will now be described in detail with reference to the accompanying drawings. It should be noted that unless otherwise specifically stated, the relative arrangement of components and steps, numerical expressions and numerical values ​​set forth in these embodiments do not limit the scope of the present invention.

[0023] The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the invention, its application, or uses.

[0024] Technologies, methods, and equipment known to ordinary technicians in the relevant art may not be discussed in detail, but where appropriate, the technologies, methods, and equipment should be considered part of the specification.

[0025] In all examples shown and discussed herein, any specific values ​​should be interpreted as merely exemplary and not limiting. Therefore, other examples of the exemplary embodiments may have different values.

[0026] It should be noted that like reference numerals and letters refer to like items in the following figures, and therefore, once an item is defined in one figure, it need not be further discussed in subsequent figures.

[0027] Unlike existing methods that focus on keeping the main neural network parameters unchanged while updating some parameters in the regularization layer, this paper updates the global neural network parameters during testing. Furthermore, to achieve refined and controllable updates of global parameters, a hierarchical coordination module is designed to set different parameter update targets and learning modes for the neural network's feature extractor and classifier. This allows the feature extractor to focus more on the generalizable diversity of the representation space, while the classifier focuses more on the consistency of the classification bias between the current unknown domain and the training source domain.

[0028] Furthermore, the present invention introduces a dynamic memory mechanism to coordinate the knowledge from the fully trained model in the source domain with the bias updates during testing when the model encounters the target domain for adaptation. For example, a memory is created for each category's classification prototypes. This is then used to continuously update and iterate the entire memory by maximizing the probabilistic similarity between the dynamic learning representations corresponding to the classification prototypes of each category. This allows for a dynamic trade-off between the knowledge from the source domain training and the bias in the target domain that requires rapid adaptation.

[0029] Specifically, see Figure 1 As shown, the proposed general domain generalization method includes the following steps:

[0030] Step S110 : designing a domain generalization target, and using a training set to pre-train a neural network model for the source domain to obtain a source domain feature extractor and a source domain classifier.

[0031] When building a machine learning model, a hypothesis function is usually learned from a training set and used to make predictions on new data at test time. However, in the real world, new data may differ from the training data in distribution and characteristics, a situation known as domain shift. To address this issue, domain adaptation techniques can be used to enable the model to generalize across different domains. For example, the domain adaptation technique employed is test-time adaptation, which is an instance-based domain adaptation technique that aims to adapt to new domains by fine-tuning the model at test time. It has a wide range of applications in domain generalization problems, such as in tasks such as object detection, semantic segmentation, and face recognition. By using test-time adaptation, the model can quickly adapt in the new domain and significantly improve performance on the target domain while maintaining performance on other domains.

[0032] Specifically, for a given N different domains Each domain D j Contains images and corresponding labels, where x i Represents an image, y i Represents the corresponding true label. For the domain generalization problem, the domain set is divided into two non-overlapping parts, namely the source domain and inaccessible target domains The goal of domain generalization is to Training on the target domain that the network has never seen It has good generalization ability, where the mapping function of the neural network can be expressed as (z is the feature, C is the number of categories), trained on the source domain. Prediction of the mapping function By feature extractor f: And the representation-based classifier q: Composition, of which represents the Hilbert space, and l(·) represents the loss function:

[0033]

[0034] Among them, θ is the network parameter, represents the input real number domain, Represents the field of labeled real numbers.

[0035] In step S120, the pre-trained neural network model is adaptively fine-tuned through online learning using the target domain sample data with the set total loss function as the optimization goal to obtain an updated neural network model, wherein the adaptive fine-tuning process introduces a dynamic memory library mechanism and adopts a hierarchical update mechanism for the feature extractor and classifier.

[0036] 1) Generalization process of marginal feature extractor

[0037] For example, for the feature extractor f, representation diversity improves its generalization ability to unseen domains. The key lies in gradually introducing representation diversity through gradient descent based on the source knowledge. Therefore, in one embodiment, a generalization method for edge extractors based on representations learned from the source domain and edge constraints formulated using Euclidean distance is proposed. Technically, the gradient is separated, the parameters of the trained source model are frozen, and then used to extract image representations from the target domain as source knowledge.

[0038] Specifically, using the training parameters θ of f * To initialize another new network f′, and pass the Frobenius norm |·| F Describes the representation difference between f and f′. A typical solution is to minimize this representation difference measured by the Frobenius norm. However, this may lead to a model with poor ability to learn the target distribution.

[0039] To solve this problem, preferably, a guarantee margin parameter σ is introduced to control the degree of representation consistency. For example, the loss function is set to:

[0040]

[0041] Where f′ represents the new model that is copied to be the same as the source domain feature extractor f, is the loss for the feature extractor to learn the target distribution.

[0042] By optimizing formula (2), the network model can maintain representation diversity while updating parameters and avoid catastrophic forgetting. The marginal parameter σ introduced here is to make the model more capable of fitting the target data distribution. When σ increases, the model has a stronger ability to overcome domain drift.

[0043] 2) Classifier generalization process

[0044] Unlike training extractors that gradually introduce representation diversity, in one embodiment, classifier parameters are updated directly based on the entropy of the target domain. For example, for the learned representation f′(c), a linear layer is used as the classifier, and a softmax function is used to obtain the probabilistic classification result q′(f′(x)). The entropy is then used as the loss function to update the classifier, thereby introducing the probability distribution of the target domain into the classifier:

[0045]

[0046] Where q′ represents the new model that is copied to be the same as the source domain classifier q, N b is the number of images, is the entropy loss function used to update the classifier.

[0047] 3) Dynamic memory mechanism

[0048] First, the category prototypes are selected as the network memory. For example, the memory bank is set to store the category prototypes for each category in C and D represent the memory bank, the number of categories, and the feature dimension, respectively. For each category j, the classification prototype p j Initialized with the weights of the classifier layer. In the self-supervised adaptation process, the classifier is first used to predict the pseudo label Then, entropy is used as a criterion to screen out the top K representations f′(x) with higher information relevance. Then, the top K samples are used to represent the corresponding category prototypes:

[0049]

[0050] Among them, p j is the classification prototype corresponding to category j.

[0051] For the process of updating iterative classification prototypes, in addition to using the Top-K re-ranking method to select classification prototypes, in order to further unleash the potential of online learning schemes, the memory iterative update process is made learnable and these prototypes are directly optimized by gradient descent. For example, by introducing the learning representation f′(x i ), classification prototype p i and pseudo labels Matrix product of to construct a learnable memory representation.

[0052]

[0053] Among them, BN represents batch normalization operation. It is worth noting that in the present invention, each classification prototype is dynamically updated by the learned representation in each forward propagation step. In short, the memory bank Each prototype p in i Both gradually change with gradient descent to enhance the ability to generalize in unknown areas. i is a learnable memory representation, is the loss of the classification prototype, and i is the image index.

[0054] To further understand the present invention, the process of the general domain generalization method provided by the present invention is summarized as follows:

[0055] Pre-trained model: First, the initialized neural network needs to be trained on basic image classification in the source domain, while strictly ensuring that the neural network is invisible to the target domain image domain labels.

[0056] Fine-tuning on the target domain: During testing, the model is fine-tuned using samples from the target domain. Specifically, for each test sample, it is fed into the model and its prediction is obtained. The loss between the prediction and the true label is then used to update the model's parameters. This update can be done using gradient-based methods such as gradient descent.

[0057] The following is a specific implementation process to further illustrate the idea of ​​the test-time adaptation algorithm provided.

[0058] First, the initial information obtained includes the number of steps T, the feature extractor f and classifier q pre-trained from the source domain, and the unlabeled target domain. The specific execution process includes:

[0059] Step S1: Initialize and set s=0.

[0060] Step S2: Copy a new model that is identical to the source model: f′←f, q′←q, and freeze the parameters of f and q.

[0061] Step S3: For s≤T, execute the following loop:

[0062] Step S4: s=s+1;

[0063] Step S5: Arrival of target data x t ;

[0064] Step S6: Use x t Update the prototype p in formula (4);

[0065] Step S7: Calculate the loss using formula (2)

[0066] Step S8: Calculate the loss using formula (3)

[0067] Step S9: Calculate the loss using formula (5)

[0068] Step S10: Calculate the total loss:

[0069] Step S11: Use the Update the network model f′, q′;

[0070] Step S12: End the loop.

[0071] Step S130: Use the updated neural network model to predict the data to be tested in the target domain.

[0072] After performing domain generalization using the above process, an updated neural network model is obtained. Compared to the pre-trained model in the source domain, this updated neural network model has improved generalization capabilities in the target domain and can effectively address the domain shift problem. Using the updated neural network model, accurate predictions can be obtained for target domain data, for example, for tasks such as image classification, semantic segmentation, or object detection.

[0073] It should be noted that the model pre-training and online learning process involved in the present invention can be performed offline on a server or in the cloud. The updated neural network model obtained can be embedded in an electronic device to achieve real-time image classification, semantic segmentation, or target detection tasks. The electronic device can be a terminal device or a server, and the terminal device includes any terminal device such as a mobile phone, tablet computer, personal digital assistant (PDA), point of sale (POS), in-vehicle computer, smart wearable device (such as smart watch, virtual reality glasses, virtual reality helmet, etc.). The server includes but is not limited to an application server or a web server, and can be a standalone server, a cluster server, or a cloud server.

[0074] In order to further verify the uniformity and efficiency of the present invention in domain generalization tasks, experiments were conducted. In the experiments, the present invention was compared with existing advanced methods using different pre-training methods in terms of performance on DG tasks. The experimental results show that the present invention is significantly better than various classic networks and has good stability. Specifically, on the VLCS, PACS, OfficeHome, Terrain and DomainNet datasets, the average accuracy of the present invention is 68.4%, exceeding the best result to date by 0.5%, and the best experimental results were achieved on VLCS, PACS, terrain and DomainNet respectively. In addition, the performance of the large-scale pre-trained models CLIP and SWAG on the DG task was verified. In general, the present invention achieved significant results, generally exceeding the results of pre-training on ImageNet. In addition, the present invention optimized the ResNet-50 with a smaller number of parameters pre-trained on ImageNet, and the effect was 1.1% higher than the average accuracy of the CARFT model pre-trained based on CLIP. Especially on the terrain dataset with complex domain shift problem, the accuracy of the present invention reaches an astonishing 69.6%, which is 7.8% higher than the current best CARFT, demonstrating strong generalization ability.

[0075] The present invention may be a system, a method and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for causing a processor to implement various aspects of the present invention.

[0076] Computer-readable storage medium can be a tangible device that can keep and store the instructions used by the instruction execution device.Computer-readable storage medium can be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device or any suitable combination thereof.More specific examples (non-exhaustive list) of computer-readable storage medium include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device, for example, a punch card or a convex structure in a groove having instructions stored thereon, and any suitable combination thereof.Computer-readable storage medium used herein is not interpreted as a transient signal itself, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagated by waveguides or other transmission media (for example, light pulses by fiber optic cables), or electrical signals transmitted by wires.

[0077] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions to be stored in the computer-readable storage medium in each computing / processing device.

[0078] The computer program instructions for performing the operation of the present invention can be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, Python, and conventional procedural programming languages ​​such as "C" language or similar programming languages. The computer readable program instructions can be executed entirely on the user's computer, partially on the user's computer, as an independent software package, partially on the user's computer, partially on a remote computer, or completely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (e.g., using an Internet service provider to connect via the Internet). In some embodiments, an electronic circuit, such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), is personalized by utilizing the state information of the computer readable program instructions, and the electronic circuit can execute the computer readable program instructions, thereby realizing various aspects of the present invention.

[0079] Various aspects of the present invention are described herein with reference to flowcharts and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the present invention. It should be understood that each block of the flowcharts and / or block diagrams, and combinations of blocks in the flowcharts and / or block diagrams, can be implemented by computer-readable program instructions.

[0080] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processor of the computer or other programmable data processing device, a device is generated that implements the functions / actions specified in one or more blocks in the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium, where these instructions cause the computer, programmable data processing device, and / or other device to operate in a specific manner. Thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing various aspects of the functions / actions specified in one or more blocks in the flowchart and / or block diagram.

[0081] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operational steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions / actions specified in one or more blocks in the flowchart and / or block diagram.

[0082] The flowcharts and block diagrams in the accompanying drawings show the possible implementation architecture, functions and operations of the systems, methods and computer program products according to multiple embodiments of the present invention. In this regard, each box in the flowchart or block diagram can represent a module, program segment or part of an instruction, and the module, program segment or part of the instruction contains one or more executable instructions for implementing the specified logical function. In some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart, can be implemented by a dedicated hardware-based system that performs the specified function or action, or can be implemented by a combination of dedicated hardware and computer instructions. It is well known to those skilled in the art that implementation by hardware, implementation by software, and implementation by a combination of software and hardware are all equivalent.

[0083] While various embodiments of the present invention have been described above, the foregoing description is intended to be illustrative, non-exhaustive, and not limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is selected to best explain the principles of the embodiments, their practical applications, or technological improvements in the marketplace, or to enable others skilled in the art to understand the embodiments disclosed herein. The scope of the present invention is defined by the appended claims.

Claims

1. A general domain generalization method, comprising the following steps: For the source domain, obtaining a pre-trained neural network model for image data classification, wherein the pre-trained neural network model includes a source domain feature extractor and a source domain classifier; For the target domain, with minimization of a set total loss function as the optimization goal, the pre-trained neural network model is adaptively fine-tuned through online learning using sample data from the target domain to obtain an updated neural network model. During the adaptive fine-tuning process, classification prototypes of each category are created as a memory bank, and the entire memory bank is updated and iterated by maximizing the probabilistic similarity between the classification prototypes corresponding to each category of the dynamic learning representation; For the target domain, using the updated neural network model to predict the category corresponding to the image data to be tested; Wherein, the total loss function is expressed as: in, represents the total loss value, represents the loss of the classification prototype, is the loss of the feature extractor learning the target distribution, expressed as: is the entropy loss function used to update the classifier, expressed as: Among them, D T represents the target domain, σ is the set marginal parameter, x i represents the input image, i is the index, f′ represents the new model that is copied to the same source domain feature extractor f, f represents the source domain feature extractor, N b is the number of images, q′ represents the new model that is copied to be the same as the source domain classifier; Among them, the loss of the classification prototype Expressed as: Among them, q′ represents the new model that is copied to the same source domain classifier q, and p i is a classification prototype, is a pseudo label, N b is the number of images, γ i is a learnable memory representation, and BN(·) represents the batch normalization operation; In the adaptive fine-tuning process, the classification prototype is updated according to the following steps: The memory bank is set up to store the category prototype for each category in C and D represent the memory bank, the number of categories and the feature dimension respectively, j is the classification prototype corresponding to category j; During the adaptive fine-tuning process, we first obtain the predicted pseudo labels Then, entropy is used as a criterion to screen out the top K representations f′(x) with higher information relevance, and then the top K samples are used to represent the corresponding classification prototypes: K represents the number of samples, x i represents an input image, i is the image index, and f′ represents the new model that is copied to the same source domain feature extractor; The classification prototype is updated and iterated.

2. The method according to claim 1, characterized in that The neural network model is a convolutional network, a multi-layer perceptron network or a Transformer network.

3. The method according to claim 1, characterized in that During the adaptive fine-tuning process, different parameter update targets and learning modes are set for the source domain feature extractor and the source domain classifier, so that the feature extractor focuses on the generalizable diversity of the representation space, and the classifier focuses on the consistency of the classification bias in the target domain and the source domain.

4. A computer-readable storage medium having a computer program stored thereon, wherein: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 3 are implemented.

5. A computer device comprising a memory and a processor, wherein a computer program capable of being executed on the processor is stored in the memory, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 3 are implemented.

Images