Robust adaptation method and device based on antagonism explicit task distribution generation

Through the adversarial explicit task distribution generation method, the parameterized meta learner and distribution opponent are used to perform gradient updates, which solves the problem of reducing the adaptability and robustness of the meta learner caused by manual design of prior task distribution, and achieves the improvement High adaptability and robustness.

CN118940805APending Publication Date: 2024-11-12TSINGHUA UNIVERSITY
View PDF 0 Cites 2 Cited by

Patent Information

Application Number
CN202410953379.1
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-16
Publication Date
2024-11-12

AI Technical Summary

Technical Problem

In the prior art, when a distribution offset occurs in the task space using manual design or simple prior task distribution, the adaptability and robustness of the meta-learner are reduced, resulting in catastrophic failures in actual scenarios.

Method used

By using the preset generation model parameterized meta learner and distribution opponent, the parameterized meta learner and distribution opponent are respectively updated to ensure that they meet the preset equilibrium conditions, thereby generating an adversarial explicit task distribution and improving The adaptability and robustness of the meta-learner.

Benefits of technology

It effectively improves the adaptability and robustness of the meta-learner when facing adversarial tasks distribution, avoids catastrophic failures caused by distribution offset, and improves its performance in actual scenarios.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118940805A_ABST
    Figure CN118940805A_ABST
Patent Text Reader

Abstract

The invention relates to the technical field of computers, in particular to a robust adaptation method and device based on antagonism explicit task distribution generation, and the method comprises the steps: parameterizing a meta-learner and a distribution opponent through a preset generation model, and obtaining a parameterized meta-learner and a parameterized distribution opponent; respectively carrying out target gradient updating processing on the parameterized meta-learner and the parameterized distribution opponent, and determining a target equilibrium solution of the parameterized meta-learner and the parameterized distribution opponent meeting a preset equilibrium condition by utilizing a gradient updating result, and enabling the target model generated based on the antagonistic explicit task distribution to meet a preset adaptation condition. Therefore, the problem that the adaptability and robustness of a meta-learner are reduced due to the fact that an obtained manually designed or simple task sampler suffers from catastrophic faults in an actual scene when distribution offset occurs in a task space due to the adoption of manually designed or simple priori task distribution in related technologies is solved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a robust adaptation method and device based on adversarial explicit task distribution generation. Background Art

[0002] Deep learning has made remarkable progress in both academia and industry over the past decade. However, training deep learning models is often time-consuming, and models previously trained on one task may perform poorly when deployed on unseen scenarios.

[0003] In related technologies, meta-learning provides a solution to generalize learned knowledge to unseen scenarios. This learning method uses past experience to extract meta-knowledge as prior knowledge and transfer skills between tasks using some shot examples. In this way, learning from scratch can be avoided and the model can be quickly adjusted to adapt to unseen but similar tasks to meet practical needs, such as fast autonomous driving in various scenarios. Due to these ideal characteristics, this learning paradigm plays an increasingly important role in building basic models.

[0004] However, in the related art, since task release is closely related to the generalization evaluation of the model, task identifiers can be used to configure tasks, such as topic types in a large language model corpus, amplitude and phase in a sine function, or degrees of freedom in a robot manipulator, etc. Artificially designed or simple prior task distributions can be used, such as uniformly distributed task identifiers, etc. However, when a distribution shift occurs in the task space, the resulting artificially designed or simple task samplers will suffer catastrophic failures in actual scenarios, reducing the adaptability and robustness of the meta-learner, which needs to be urgently resolved. Summary of the invention

[0005] The present application provides a robust adaptation method and device based on adversarial explicit task distribution generation to solve the problem that when the distribution shift occurs in the task space, the artificially designed or simple task sampler obtained in the related technology will suffer catastrophic failure in the actual scenario, thereby reducing the adaptability and robustness of the meta-learner.

[0006] The first aspect of the present application provides a robust adaptation method based on adversarial explicit task distribution generation, comprising the following steps: using a preset generation model to parameterize a meta-learner and a distribution opponent to obtain a parameterized meta-learner and a parameterized distribution opponent; performing target gradient update processing on the parameterized meta-learner and the parameterized distribution opponent respectively to obtain a gradient update result; based on the gradient update result, determining a target equilibrium solution in which both the parameterized meta-learner and the parameterized distribution opponent satisfy a preset equilibrium condition, so that the target model generated based on the adversarial explicit task distribution satisfies the preset adaptation condition.

[0007] Optionally, in one embodiment of the present application, the target gradient update processing of the parameterized meta-learner and the parameterized distribution opponent respectively includes: determining a first target gradient of the parameterized meta-learner, and using the first target gradient to perform gradient descent processing on the parameterized meta-learner that meets a first preset condition; determining a second target gradient of the parameterized distribution opponent, and using the second target gradient to perform gradient ascent processing on the parameterized distribution opponent that meets a second preset condition.

[0008] Optionally, in one embodiment of the present application, the fractional equation of the first target gradient is:

[0009]

[0010] where τ represents a task, θ and φ represent the model parameters of the meta-learner and the distribution opponent, respectively, and p φ (τ) represents the task distribution generated by the distribution adversary, Represents the query data set used for testing. represents the support dataset used for fast adaptation, represents the derivative of θ, L represents the meta-learning loss function, K represents the number of tasks sampled by Monte Carlo, dτ represents the integral of the task, and J represents the optimization goal of this application. represents the query data set used to test the Kth task, represents the support dataset for the Kth task to quickly adapt.

[0011] Optionally, in one embodiment of the present application, the fractional equation of the second target gradient is:

[0012]

[0013] in, represents the particles sampled from the initial task allocation, λ represents the Lagrange multiplier, V represents the constant baseline used to reduce the variance, represents the derivative of φ.

[0014] The second aspect of the present application provides a robust adaptation device generated based on adversarial explicit task distribution, including: an acquisition module, used to parameterize a meta-learner and a distribution opponent using a preset generation model to obtain a parameterized meta-learner and a parameterized distribution opponent; a processing module, used to perform target gradient update processing on the parameterized meta-learner and the parameterized distribution opponent respectively to obtain a gradient update result; a control module, used to determine, based on the gradient update result, a target equilibrium solution in which both the parameterized meta-learner and the parameterized distribution opponent satisfy a preset equilibrium condition, so that the target model generated based on the adversarial explicit task distribution satisfies the preset adaptation condition.

[0015] Optionally, in one embodiment of the present application, the processing module includes: a first processing unit, used to determine a first target gradient of the parameterized meta-learner, and use the first target gradient to perform gradient descent processing on the parameterized meta-learner that meets a first preset condition; a second processing unit, used to determine a second target gradient of the parameterized distribution opponent, and use the second target gradient to perform gradient ascent processing on the parameterized distribution opponent that meets a second preset condition.

[0016] Optionally, in one embodiment of the present application, the fractional equation of the first target gradient is:

[0017]

[0018] where τ represents a task, θ and φ represent the model parameters of the meta-learner and the distribution opponent, respectively, and p φ (τ) represents the task distribution generated by the distribution adversary, Represents the query data set used for testing. represents the support dataset used for fast adaptation, represents the derivative of θ, L represents the meta-learning loss function, K represents the number of tasks sampled by Monte Carlo, dτ represents the integral of the task, and J represents the optimization goal of this application. represents the query data set used to test the Kth task, represents the support dataset for the Kth task to quickly adapt.

[0019] Optionally, in one embodiment of the present application, the fractional equation of the second target gradient is:

[0020]

[0021] in, represents the particles sampled from the initial task allocation, λ represents the Lagrange multiplier, V represents the constant baseline used to reduce the variance, represents the derivative of φ.

[0022] The third aspect of the present application provides an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement a robust adaptation method based on adversarial explicit task distribution generation as described in the above embodiment.

[0023] The fourth aspect embodiment of the present application provides a computer-readable storage medium, which stores a computer program, which, when executed by a processor, implements the above-mentioned robust adaptation method based on adversarial explicit task distribution generation.

[0024] The fifth aspect of the present application provides a computer program product, including a computer program, which, when executed, is used to implement the above-mentioned robust adaptation method based on adversarial explicit task distribution generation.

[0025] The embodiments of the present application can use a certain generative model to parameterize the meta-learner and the distributed opponent, and perform target gradient update processing on the parameterized meta-learner and the parameterized distributed opponent respectively, and use the gradient update results to determine the target equilibrium solution in which the parameterized meta-learner and the parameterized distributed opponent both meet the equilibrium conditions, so that the target model generated based on the adversarial explicit task distribution meets certain adaptation conditions, and improves the adaptability and robustness of the meta-learner. Thus, the problem that the artificially designed or simple a priori task distribution used in the related technology leads to the distribution shift in the task space, and the obtained artificially designed or simple task sampler will suffer catastrophic failure in the actual scenario, reducing the adaptability and robustness of the meta-learner.

[0026] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0028] Figure 1 A flowchart of a robust adaptation method based on adversarial explicit task distribution generation provided according to an embodiment of the present application;

[0029] Figure 2 A schematic diagram of generating adversarial task distribution in meta-learning in a specific embodiment of the present application;

[0030] Figure 3 A schematic diagram of adversarial robust meta-learning for a specific embodiment of the present application;

[0031] Figure 4 A schematic diagram of the structure of a robust adaptation device based on adversarial explicit task distribution generation provided according to an embodiment of the present application;

[0032] Figure 5 It is a schematic diagram of the structure of an electronic device provided according to an embodiment of the present application. DETAILED DESCRIPTION

[0033] Embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present application, and should not be construed as limiting the present application.

[0034] The following describes the robust adaptation method and device based on adversarial explicit task distribution generation of the embodiment of the present application with reference to the accompanying drawings. In view of the problem that the artificially designed or simple a priori task distribution is used in the related art mentioned in the above background technology, when the distribution shift occurs in the task space, the artificially designed or simple task sampler obtained will suffer catastrophic failure in the actual scene, reducing the adaptability and robustness of the meta-learner, etc., the present application provides a robust adaptation method based on adversarial explicit task distribution generation, in which a certain generation model can be used to parameterize the meta-learner and the distribution opponent, and the target gradient update processing is performed on the parameterized meta-learner and the parameterized distribution opponent respectively, and the gradient update result is used to determine the target equilibrium solution in which the parameterized meta-learner and the parameterized distribution opponent both meet the equilibrium conditions, so that the target model generated based on the adversarial explicit task distribution meets certain adaptation conditions, and improves the adaptability and robustness of the meta-learner. Thus, the problem that the artificially designed or simple a priori task distribution is used in the related art, when the distribution shift occurs in the task space, the artificially designed or simple task sampler obtained will suffer catastrophic failure in the actual scene, reducing the adaptability and robustness of the meta-learner, etc., is solved.

[0035] Specifically, Figure 1 A flowchart of a robust adaptation method based on adversarial explicit task distribution generation provided in an embodiment of the present application.

[0036] like Figure 1 As shown, the robust adaptation method based on adversarial explicit task distribution generation includes the following steps:

[0037] In step S101, a meta-learner and a distributed opponent are parameterized using a preset generative model to obtain a parameterized meta-learner and a parameterized distributed opponent.

[0038] It can be understood that the embodiments of the present application can utilize preset generative models to parameterize the meta-learner and the distribution opponent. For example, the meta-learner and the distribution opponent can be parameterized using a standard flow model to obtain a parameterized meta-learner and a parameterized distribution opponent, wherein the obtained parameterized meta-learner can effectively learn new tasks from a small amount of data and has strong adaptability to unknown and even adversarial distributions, and the obtained parameterized distribution opponent can be used to continuously evaluate and enhance the performance of the meta-learner.

[0039] For example, as an indispensable element in meta-learning, task distribution is mostly set to be uniformly distributed or manually designed. Such a setting makes it difficult to identify task subgroups that are difficult to solve in practice, and cannot handle task distribution deviation. Therefore, the embodiment of the present application can be based on the two-person Stackelberg game and propose to introduce a distribution opponent in addition to traditional meta-learning, whose optimization goal is the negative value of the traditional meta-learning goal, that is, to try to generate an adversarial task distribution to reduce the utility of the meta-learner, such as Figure 2 As shown in the figure, it is a schematic diagram of generating adversarial task distribution in meta-learning. The initial task distribution p0(τ) is a uniform distribution controlled by two task identifiers [v,μ]. Then, in the standard flow model NF φ With the help of φ (τ).

[0040] Among them, the optimization goal of traditional meta-learning is:

[0041]

[0042] Among them, τ represents a task, θ represents the model parameters of the meta-learner, and p(τ) represents the task distribution in the task domain T. Represents the query data set used for testing. represents the support dataset for fast adaptation, and L represents the meta-learning loss function.

[0043] Therefore, the embodiment of the present application can convert the optimization goal of traditional meta-learning into a min-max optimization problem, namely:

[0044]

[0045] Among them, φ represents the model parameters of the distribution opponent, p0(τ) represents the initial task distribution, and p φ (τ) represents the task distribution generated by the distribution adversary, Represents the query data set used for testing. represents the support dataset used for fast adaptation, the KL divergence constraint term represents the maximum distribution deviation to be tolerated in meta-learning, J represents the optimization goal of this application, σ represents the constraint range, and E represents the expectation.

[0046] The optimization objective can be further written as:

[0047]

[0048] Among them, p 0(π) represents the initial task distribution.

[0049] In addition, this form can be combined with classic meta-learning algorithms such as MAML and CNP.

[0050] Furthermore, the embodiments of the present application can parameterize the meta-learner and the distribution opponent through the standard flow model, wherein the standard flow model has the tractability of the exact log-likelihood, the flexibility of capturing complex distributions, and the direct understanding of the task structure; the standard flow model can transform a simple distribution into a more flexible distribution, using a series of reversible mappings in, Represents a smooth reversible mapping, which can be represented by a neural network NN φ To achieve these mappings, specifically, for the initial task distribution p 0(τ) and a sampled task τ 0 , the embodiment of the present application can use a neural network for mapping, namely:

[0051]

[0052] Therefore, the task distribution of interest is adaptive and adversarially exploits information from the offset task distribution. The transformed density function can be easily computed with the help of the Jacobian function of the function, namely:

[0053]

[0054] in, represents partial derivative, M represents the number of reversible mapping layers of the standard flow model, g i represents the i-th layer smooth reversible mapping.

[0055] In step S102, target gradient update processing is performed on the parameterized meta-learner and the parameterized distribution opponent respectively to obtain a gradient update result.

[0056] It can be understood that the embodiments of the present application can perform target gradient update processing on the parameterized meta-learner and parameterized distributed opponent in the following steps respectively. For example, the embodiments of the present application can use the random alternating gradient descent-ascent algorithm to perform target gradient update processing respectively to obtain gradient update results, so that the meta-learner can perform more robustly when facing adversarial tasks generated by the distributed opponent.

[0057] Among them, in one embodiment of the present application, target gradient update processing is performed on the parameterized meta-learner and the parameterized distribution opponent respectively, including: determining a first target gradient of the parameterized meta-learner, and using the first target gradient to perform gradient descent processing on the parameterized meta-learner that meets the first preset condition; determining a second target gradient of the parameterized distribution opponent, and using the second target gradient to perform gradient ascent processing on the parameterized distribution opponent that meets the second preset condition.

[0058] In some embodiments, the embodiments of the present application can determine the stochastic gradient of a parameterized meta-learner, and use the stochastic gradient to perform gradient descent processing on the parameterized meta-learner that meets the first condition, for example, when the loss under the current task distribution is high or there is a large gap with the optimal state, the parameterized meta-learner is subjected to gradient descent processing; determine the stochastic gradient of a parameterized distribution opponent to perform gradient ascent processing on the parameterized distribution opponent that meets the second condition, for example, when the distribution opponent fails to effectively challenge the meta-learner or does not reach the expected difficulty level, the parameterized distribution opponent is subjected to gradient ascent processing, which helps the meta-learner to quickly adapt and learn on specific tasks, and further improves the robustness and generalization ability of the meta-learner.

[0059] For example, the embodiment of the present application can model the optimization problem of the distributed opponent and the meta-learner as a two-person Stackelberg game. Given two participants with completely different goals, the common strategy for calculating equilibrium is BR (Best Response), that is:

[0060]

[0061] φ t+1 = argmaxJ(θ t+1 ,φ)

[0062] Among them, θ represents the model parameters of the meta-learner, φ represents the model parameters of the distribution opponent, and φ t Represents the optimization of the distribution opponent model parameters at step t.

[0063] Next, the embodiment of the present application can use the random alternating gradient descent-ascent algorithm to apply the gradient update to the meta-learner and the distribution opponent, generating the following iterative equation with index t:

[0064]

[0065] Among them, γ1 represents the learning rate, θ t represents the optimization of the model parameters of the meta-learner at step t, Represents the gradient of the optimization objective with respect to θ at the t-th step.

[0066] At this point, it can be regarded as a gradient approximation of the BR strategy. For the minimum and maximum value problems considered, it will at least be optimized to a local Stackelberg equilibrium.

[0067] First, the embodiment of the present application can use the Monte Carlo method to estimate the stochastic gradient of the meta-learner. The stochastic gradient score equation of the meta-learner is:

[0068]

[0069] where τ represents a task, θ and φ represent the model parameters of the meta-learner and the distribution opponent, respectively, and p φ (τ) represents the task distribution generated by the distribution adversary, Represents the query data set used for testing. represents the support dataset used for fast adaptation, represents the derivative of θ, L represents the meta-learning loss function, K represents the number of tasks sampled by Monte Carlo, dτ represents the integral of the task, and J represents the optimization goal of this application. represents the query data set used to test the Kth task, represents the support dataset for the Kth task to quickly adapt.

[0070] Here, this form of stochastic gradient is with respect to the meta-learner parameters θ and is specific to the meta-learning algorithm or model-dependent.

[0071] Secondly, the embodiment of the present application can use the reinforcement learning algorithm REINFORCE to estimate the stochastic gradient of the distribution opponent, and the derived score equation is as follows:

[0072]

[0073] in, represents the particles sampled from the initial task allocation so that

[0074] This fractional estimate can be verified to be However, this gradient estimate often shows high variance, which weakens the stability of the training process. In order to reduce the variance, a constant baseline can be introduced into the score equation, that is:

[0075]

[0076] The final fractional equation of the stochastic gradient of the distribution opponent is as follows:

[0077]

[0078] in, represents the particles sampled from the initial task allocation, λ represents the Lagrange multiplier, V represents the constant baseline used to reduce the variance, represents the derivative of φ.

[0079] For example, Figure 3 As shown, it is a schematic diagram of adversarial robust meta-learning. In the meta-learning game, the two-player Stackelberg game framework proposed in the embodiment of the present application consists of two participants, namely a distribution opponent and a meta-learner. On the left side of the figure: the distribution opponent attempts to transform the initial task distribution through a neural network parameterized by φ, with the aim of reducing the fast adaptation performance of the meta-learner; on the right side of the figure: the meta-learner parameterized by θ attempts to learn a robust strategy that adapts quickly to the worst-case task in the sampling.

[0080] In other words, in each round of iteration of the embodiment of the present application, the parameters of the meta-learner and the distribution opponent are updated according to their respective loss functions, wherein the meta-learner can minimize its prediction error on the new task through gradient descent; and the distribution opponent can maximize its ability to confuse the meta-learner through gradient ascent, so that the meta-learner further enhances its ability to quickly adapt to new tasks.

[0081] In step S103, based on the gradient update result, a target equilibrium solution is determined in which both the parameterized meta-learner and the parameterized distribution opponent satisfy the preset equilibrium conditions, so that the target model generated based on the adversarial explicit task distribution satisfies the preset adaptation conditions.

[0082] It can be understood that the embodiments of the present application can determine the target equilibrium solution in which the parameterized meta-learner and the parameterized distributed opponent both meet the equilibrium conditions based on the gradient update results in the above steps. For example, the optimization problem of the distributed opponent and the meta-learner can be modeled as a two-person Stackelberg game, and the equilibrium point can be obtained using the random alternating gradient descent-ascent algorithm, that is, the equilibrium solution is solved, so that the meta-learner can perform more robustly when facing adversarial tasks generated by the distributed opponent. In other words, the target model generated based on the adversarial explicit task distribution, such as the meta-learner, meets the robust and rapid adaptation conditions to ensure the robustness and generalization ability of the target model.

[0083] According to the robust adaptation method based on adversarial explicit task distribution generation proposed in the embodiment of the present application, a certain generation model can be used to parameterize the meta-learner and the distribution opponent, and the target gradient update processing is performed on the parameterized meta-learner and the parameterized distribution opponent respectively, and the gradient update result is used to determine the target equilibrium solution in which the parameterized meta-learner and the parameterized distribution opponent both meet the equilibrium conditions, so that the target model generated based on the adversarial explicit task distribution meets certain adaptation conditions, and the adaptability and robustness of the meta-learner are improved. Thus, the problem that the artificially designed or simple a priori task distribution used in the related technology leads to the distribution shift in the task space, and the obtained artificially designed or simple task sampler will suffer catastrophic failure in the actual scenario, reducing the adaptability and robustness of the meta-learner.

[0084] Next, a robust adaptation device based on adversarial explicit task distribution generation proposed in an embodiment of the present application is described with reference to the accompanying drawings.

[0085] Figure 4 It is a block diagram of a robust adaptation device generated based on adversarial explicit task distribution according to an embodiment of the present application.

[0086] like Figure 4 As shown, the robust adaptation device 10 based on adversarial explicit task distribution generation includes: an acquisition module 100, a processing module 200 and a control module 300.

[0087] Specifically, the acquisition module 100 is used to parameterize the meta-learner and the distribution opponent using the preset generation model to obtain the parameterized meta-learner and the parameterized distribution opponent.

[0088] The processing module 200 is used to perform target gradient update processing on the parameterized meta-learner and the parameterized distribution opponent respectively to obtain a gradient update result.

[0089] The control module 300 is used to determine, based on the gradient update result, a target equilibrium solution in which both the parameterized meta-learner and the parameterized distribution opponent satisfy a preset equilibrium condition, so that the target model generated based on the adversarial explicit task distribution satisfies a preset adaptation condition.

[0090] Optionally, in one embodiment of the present application, the processing module 200 includes: a first processing unit and a second processing unit.

[0091] The first processing unit is used to determine a first target gradient of the parameterized meta-learner, and use the first target gradient to perform gradient descent processing on the parameterized meta-learner that meets the first preset condition.

[0092] The second processing unit is used to determine a second target gradient of the parameterized distribution opponent, and perform a gradient ascent process on the parameterized distribution opponent that meets a second preset condition using the second target gradient.

[0093] Optionally, in one embodiment of the present application, the fractional equation of the first target gradient is:

[0094]

[0095] where τ represents a task, θ and φ represent the model parameters of the meta-learner and the distribution opponent, respectively, and p φ (τ) represents the task distribution generated by the distribution adversary, Represents the query data set used for testing. represents the support dataset used for fast adaptation, represents the derivative of θ, L represents the meta-learning loss function, K represents the number of tasks sampled by Monte Carlo, dτ represents the integral of the task, and J represents the optimization goal of this application. represents the query data set used to test the Kth task, represents the support dataset for the Kth task to quickly adapt.

[0096] Optionally, in one embodiment of the present application, the fractional equation of the second target gradient is:

[0097]

[0098] in, represents the particles sampled from the initial task allocation, λ represents the Lagrange multiplier, V represents the constant baseline used to reduce the variance, represents the derivative of φ.

[0099] It should be noted that the aforementioned explanation of the embodiment of the robust adaptation method based on adversarial explicit task distribution generation is also applicable to the robust adaptation device based on adversarial explicit task distribution generation in this embodiment, and will not be repeated here.

[0100] According to the robust adaptation device based on adversarial explicit task distribution generation proposed in the embodiment of the present application, a certain generation model can be used to parameterize the meta-learner and the distribution opponent, and the target gradient update processing is performed on the parameterized meta-learner and the parameterized distribution opponent respectively, and the gradient update result is used to determine the target equilibrium solution in which the parameterized meta-learner and the parameterized distribution opponent both meet the equilibrium conditions, so that the target model generated based on the adversarial explicit task distribution meets certain adaptation conditions, and the adaptability and robustness of the meta-learner are improved. Thus, the problem that the artificially designed or simple a priori task distribution used in the related technology leads to the distribution shift in the task space, and the obtained artificially designed or simple task sampler will suffer catastrophic failure in the actual scenario, reducing the adaptability and robustness of the meta-learner.

[0101] Figure 5 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. The electronic device may include:

[0102] A memory 501 , a processor 502 , and a computer program stored in the memory 501 and executable on the processor 502 .

[0103] When the processor 502 executes the program, the robust adaptation method based on adversarial explicit task distribution generation provided in the above embodiment is implemented.

[0104] Furthermore, the electronic device further comprises:

[0105] The communication interface 503 is used for communication between the memory 501 and the processor 502 .

[0106] The memory 501 is used to store computer programs that can be executed on the processor 502 .

[0107] The memory 501 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0108] If the memory 501, the processor 502 and the communication interface 503 are implemented independently, the communication interface 503, the memory 501 and the processor 502 can be connected to each other through a bus and communicate with each other. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 5 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0109] Optionally, in a specific implementation, if the memory 501, the processor 502 and the communication interface 503 are integrated on a chip, the memory 501, the processor 502 and the communication interface 503 can communicate with each other through an internal interface.

[0110] The processor 502 may be a central processing unit (CPU), or an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application.

[0111] This embodiment also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-mentioned robust adaptation method based on adversarial explicit task distribution generation.

[0112] This embodiment also provides a computer program product, including a computer program, which, when executed, is used to implement the above-mentioned robust adaptation method based on adversarial explicit task distribution generation.

[0113] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or N embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0114] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include at least one of the features. In the description of this application, "N" means at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0115] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, fragment or portion of code comprising one or N executable instructions for implementing the steps of a custom logical function or process, and the scope of the preferred embodiments of the present application includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present application belong.

[0116] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in combination with these instruction execution systems, devices or apparatuses. For the purpose of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in combination with these instruction execution systems, devices or apparatuses. More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or N wirings (electronic devices), a portable computer disk box (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically by optically scanning the paper or other medium and then editing, interpreting or processing in other suitable ways as necessary and then storing it in a computer memory.

[0117] It should be understood that the various parts of the present application can be implemented by hardware, software, firmware or a combination thereof. In the above embodiment, the N steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one or a combination of the following technologies known in the art: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0118] A person skilled in the art may understand that all or part of the steps in the method for implementing the above-mentioned embodiment may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiment.

[0119] In addition, each functional unit in each embodiment of the present application may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.

[0120] The storage medium mentioned above may be a read-only memory, a magnetic disk or an optical disk, etc. Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limiting the present application. A person of ordinary skill in the art may change, modify, replace and modify the above embodiments within the scope of the present application.

Claims

1. A robust adaptation method based on adversarial explicit task distribution generation, characterized in that The following steps are involved: The meta-learner and the distribution opponent are parameterized by using the preset generative model to obtain a parameterized meta-learner and a parameterized distribution opponent; Performing target gradient update processing on the parameterized meta-learner and the parameterized distribution opponent respectively to obtain gradient update results; Based on the gradient update result, a target equilibrium solution is determined in which both the parameterized meta-learner and the parameterized distribution opponent satisfy a preset equilibrium condition, so that a target model generated based on the adversarial explicit task distribution satisfies a preset adaptation condition.

2. The method according to claim 1, characterized in that The performing target gradient update processing on the parameterized meta-learner and the parameterized distribution opponent respectively includes: Determining a first target gradient of the parameterized meta-learner, and performing a gradient descent process on the parameterized meta-learner that meets a first preset condition using the first target gradient; A second target gradient of the parameterized distribution opponent is determined, and a gradient ascent process is performed on the parameterized distribution opponent that meets a second preset condition using the second target gradient.

3. The method according to claim 2, characterized in that The fractional equation of the first objective gradient is: where τ represents a task, θ and φ represent the model parameters of the meta-learner and the distribution opponent, respectively, and p φ (τ) represents the task distribution generated by the distribution adversary, Represents the query data set used for testing. represents the support dataset used for fast adaptation, represents the derivative of θ, L represents the meta-learning loss function, K represents the number of tasks sampled by Monte Carlo, dτ represents the integral of the task, and J represents the optimization goal of this application. represents the query data set used to test the Kth task, represents the support dataset for the Kth task to quickly adapt.

4. The method according to claim 2, characterized in that: The fractional equation of the second objective gradient is: in, represents the particles sampled from the initial task allocation, Ling represents the Lagrange multiplier, V represents the constant baseline used to reduce the variance, represents the derivative of φ.

5. A robust adaptation device based on adversarial explicit task distribution generation, characterized in that: include: An acquisition module, used to parameterize a meta-learner and a distribution opponent using a preset generative model to obtain a parameterized meta-learner and a parameterized distribution opponent; A processing module, used to perform target gradient update processing on the parameterized meta-learner and the parameterized distribution opponent respectively to obtain a gradient update result; A control module is used to determine, based on the gradient update result, a target equilibrium solution in which both the parameterized meta-learner and the parameterized distribution opponent satisfy a preset equilibrium condition, so that a target model generated based on the adversarial explicit task distribution satisfies a preset adaptation condition.

6. The device according to claim 5, characterized in that The processing module comprises: a first processing unit, configured to determine a first target gradient of the parameterized meta-learner, and perform a gradient descent process on the parameterized meta-learner that meets a first preset condition using the first target gradient; The second processing unit is used to determine a second target gradient of the parameterized distribution opponent, and use the second target gradient to perform gradient ascent processing on the parameterized distribution opponent that meets a second preset condition.

7. The device according to claim 6, characterized in that The fractional equation of the first objective gradient is: where τ represents a task, θ and φ represent the model parameters of the meta-learner and the distribution opponent, respectively, and p φ (τ) represents the task distribution generated by the distribution adversary, Represents the query data set used for testing. represents the support dataset used for fast adaptation, represents the derivative of θ, L represents the meta-learning loss function, K represents the number of tasks sampled by Monte Carlo, dτ represents the integral of the task, and J represents the optimization goal of this application. represents the query data set used to test the Kth task, represents the support dataset for the Kth task to quickly adapt.

8. An electronic device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the robust adaptation method based on adversarial explicit task distribution generation as described in any one of claims 1 to 4.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: The program is executed by a processor to implement the robust adaptation method based on adversarial explicit task distribution generation as described in any one of claims 1 to 4.

10. A computer program product, comprising a computer program, characterized in that The computer program is executed by a processor to implement the robust adaptation method based on adversarial explicit task distribution generation as described in any one of claims 1 to 4.

Citation Information

Cited By

  • Metareinforcement learning and domain randomization training method for model prediction task sampling

    CN119940483A

  • Gradient adaptive quantitative perception training method, device and equipment for field generalization

    CN119942304A