Marine battlefield intelligent confrontation method based on online learning rate optimization mechanism

Abstract

The embodiment of the invention relates to a sea battlefield intelligent confrontation method based on an online learning rate optimization mechanism, and the method comprises the steps: calculating the gradient of a network model, updating a learning rate for each layer of neural network through a learning rate optimization model, and updating the parameters of each layer of network according to the updated learning rate; and updating the basic element learning rate, updating parameters of the network model and the learning rate optimization model according to the updated basic element learning rate, and deploying the trained model to a simulation environment to generate a marine battlefield multi-ship intelligent game confrontation strategy. Compared with a traditional combat behavior modeling method, the problem of large-scale agent confrontation task state and action space explosion is solved; on the other hand, the online learning rate optimization mechanism solves the problem of hyper-parameter setting in the training process of the MAML algorithm, the stability of network training is enhanced, the algorithm learning efficiency and final performance are improved, and higher robustness and practicability are achieved.

Description

Technical Field

[0001] The present invention relates to a method for intelligent confrontation in a naval battlefield based on an online learning rate optimization mechanism, and specifically to a method for introducing an online learning rate optimization mechanism into a model-free meta-learning algorithm (MAML) for multi-ship collaborative confrontation tasks in future naval battlefields, and belongs to the technical fields of intelligent decision-making, intelligent task planning and intelligent command and control of military information systems. Background Art

[0002] The naval battlefield refers to the area where the two parties involved in the confrontation use the ocean as the main combat space to achieve their respective combat objectives, accompanied by areas such as the ocean airspace, islands and reefs, and water bodies. The game confrontation tasks in the naval battlefield have the following characteristics: First, the naval battlefield is large in scope and the combat space dimension is high. Unlike the land battlefield, due to the connectivity of water bodies, the spatial scope of the naval battlefield cannot be divided by simple points and lines. The vast sea area brings huge challenges to the research of game confrontation tasks in the naval battlefield. The large scope of the battlefield means that the combat space has multiple states and high dimensions, which directly increases the computational complexity of the multi-ship game confrontation problem in the naval battlefield environment.

[0003] Second, there are many combat units in the naval battlefield, the action space dimension is high, and the coordination between tasks is complex. Multi-ship confrontation tasks in the naval battlefield involve many types and large numbers of combat units. Multi-ship confrontation tasks in the naval battlefield not only include confrontations between multiple ships of the enemy, but also confrontations between unmanned aerial vehicles (UAVs), underwater unmanned boats (UUVs), and surface unmanned boats (USVs). A large number of combat units not only brings about the problem of action space explosion, but also the coordination methods between different combat units are extremely complex.

[0004] Third, the information on the naval battlefield is opaque and the fog of war is thick. Since the naval battlefield involves a very large spatial range, including the sea, the sea surface, and the sea, and the detection range of the enemy's radar, sonar and other devices is always limited, for the two parties involved in the game, unlike the chess game on paper where both parties have complete information, most of the naval battlefield situation is unknown and in the fog of war. This game under incomplete information is also one of the difficulties of the naval battlefield confrontation task.

[0005] Fourth, the real-time requirements for combat in the naval battlefield are high. The naval battlefield environment is complex and changeable, and the highly dynamic environment requires combat units to respond to environmental changes more quickly. In the future naval battlefield environment, ships and their affiliated combat units must be able to fight autonomously around the preset combat objectives based on the real-time situation, and they must also be able to dynamically adjust strategies and engage in autonomous games during combat.

[0006] Traditional methods for modeling naval warfare confrontation behaviors include those based on pure game theory, genetic algorithms, ant colony algorithms, and expert systems. Pure game theory methods are limited to game problems with finite individuals and discrete actions. When the number of intelligent agents increases dramatically and the battlefield is in a highly dynamic and uncertain environment, the computational complexity of pure game theory methods increases exponentially, and cannot meet the timeliness of multi-ship confrontations in naval battlefields. When solving tasks involving a large number of intelligent agents, optimization algorithms such as genetic algorithms and ant colony algorithms have slow convergence speeds, are prone to falling into local optimality, and have long search times. Although the expert system method has a simple and practical model and a reliable rule base, it also has problems with a complex rule base and fixed and rigid strategies.

[0007] In recent years, with the rise of meta-learning method research, it has achieved great success in solving multi-agent confrontation applications under highly dynamic and uncertain conditions. Among them, the model-free meta-learning algorithm is one of the typical representatives of the current meta-learning method, which belongs to the meta-learning method based on the policy gradient type. Although the model-free meta-learning algorithm has unique advantages in solving multi-ship confrontation tasks in the naval battlefield, there is still a defect in the actual training process: the problem of hyperparameter setting solidification. In the training process of traditional model-free meta-learning algorithms, hyperparameters are often manually set to be fixed or simply change linearly over time. This setting method not only introduces subjective factors into the algorithm training, but also does not consider the need for hyperparameter optimization during the actual update of the model, which greatly affects the learning efficiency of the algorithm and even reduces the final performance.

[0008] Therefore, it is necessary to propose a solution to improve one or more problems existing in the above-mentioned related technical solutions.

[0009] It should be noted that the information disclosed in the above background technology section is only used to enhance the understanding of the background of the present disclosure, and therefore may include information that does not constitute the prior art known to ordinary technicians in the field. Summary of the invention

[0010] The embodiment of the present disclosure provides a method for intelligent confrontation in a naval battlefield based on an online learning rate optimization mechanism, comprising the following steps:

[0011] Construct a multi-ship confrontation task simulation environment in the naval battlefield to provide dynamic environmental status and environmental feedback;

[0012] Initialize network parameters, initialize network model and learning rate optimization model;

[0013] Task sampling is performed from the multi-ship confrontation task distribution, and multiple rounds of trajectory sampling are performed under each task;

[0014] Calculate the gradient of the network model, update the learning rate for each layer of the neural network through the learning rate optimization model, and update the network parameters of each layer according to the updated learning rate;

[0015] Update the basic meta-learning rate, and update the parameters of the network model and the learning rate optimization model according to the updated basic meta-learning rate.

[0016] It is determined whether the maximum number of iterations has been reached at the current moment. If not, task sampling will continue. Otherwise, the trained model will be deployed in the simulation environment to generate a multi-ship intelligent game confrontation strategy in the naval battlefield.

[0017] In an exemplary embodiment of the present disclosure, the initialization of network parameters includes: initializing the multi-ship confrontation task distribution, initializing the learning rate, initializing the basic meta-learning rate, and initializing the high-level meta-learning rate.

[0018] In an exemplary embodiment of the present disclosure, the high-level meta-learning rate is a preset first threshold.

[0019] In an exemplary embodiment of the present disclosure, the number of iterations is a preset second threshold.

[0020] In an exemplary embodiment of the present disclosure, the expression for updating the learning rate for each layer of the neural network through the learning rate optimization model is:

[0021]

[0022] Among them, α j represents the learning rate of the j-th layer neural network, {α j} j＝1,...,l Indicates that there are a total of l learning rates, h φ The learning rate generation model, φ is the parameter of the learning rate optimization model,

[0023] Indicates that in T i Task network model f θ The gradient of the loss function, θ is the parameter of the network model.

[0024] In an exemplary embodiment of the present disclosure, the expression for updating the network parameters of each layer is:

[0025]

[0026] Among them, θ j Represents the parameters of the jth layer in the network model, α j represents the learning rate of the j-th layer of the neural network, Represents Hadamard.

[0027] In an exemplary embodiment of the present disclosure, the expression for updating the basic meta-learning rate is:

[0028]

[0029] Among them, β i represents the updated meta-learning rate, β i-1 The current meta-learning rate is the meta-learning rate at the i-th iteration, η β High-level meta-learning rate, T i represents the i-th task sampled from the task distribution ρ(T), ρ(T) represents the task distribution, Represents task T i Based on the parameter θ i 'The loss function of the parameter θ' i-1 The gradient of Represents task T i Based on the parameter θ' i-1 The loss function is about the parameter θ i-2 gradient.

[0030] In an exemplary embodiment of the present disclosure, the expression for updating the parameters of the network model and the learning rate optimization model according to the updated basic meta-learning rate is:

[0031]

[0032] Among them, (θ, φ) represents the set of network model and learning rate optimization model parameters.

[0033] The present invention discloses a method for intelligent confrontation in naval battlefields based on an online learning rate optimization mechanism, which can include the following beneficial effects: on the one hand, compared with the traditional combat behavior modeling method, it solves the problem of state and action space explosion in large-scale intelligent confrontation tasks; on the other hand, the online learning rate optimization mechanism solves the problem of hyperparameter setting in the training process of the MAML algorithm, enhances the stability of network training, improves the learning efficiency and final performance of the algorithm, and has stronger robustness and practicality. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] The accompanying drawings herein are incorporated into the specification and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the specification are used to explain the principles of the present disclosure. Obviously, the accompanying drawings described below are only some embodiments of the present disclosure, and for ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without creative work.

[0035] Figure 1 A schematic diagram showing the steps of a sea battlefield intelligent confrontation method based on an online learning rate optimization mechanism in an exemplary embodiment of the present disclosure;

[0036] Figure 2 A schematic diagram of a model-free meta-learning algorithm framework based on an online learning rate optimization mechanism in an exemplary embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0037] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in a variety of forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that the disclosure will be more comprehensive and complete and to fully convey the concepts of the example embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0038] In addition, the accompanying drawings are only schematic illustrations of the present disclosure and are not necessarily drawn to scale. The same reference numerals in the figures represent the same or similar parts, and their repeated description will be omitted. Some of the block diagrams shown in the accompanying drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software form, or implemented in one or more hardware modules or integrated circuits, or implemented in different networks and / or processor devices and / or microcontroller devices.

[0039] This example implementation provides a method for intelligent confrontation in the sea battlefield based on an online learning rate optimization mechanism. Figure 1 As shown, the following steps may be included:

[0040] Step S101: constructing a multi-ship confrontation task simulation environment in a naval battlefield to provide dynamic environmental status and environmental feedback;

[0041] Step S102: Initialize network parameters, initialize network model and learning rate optimization model;

[0042] Step S103: sampling tasks from the multi-ship confrontation task distribution, and performing multiple rounds of trajectory sampling for each task;

[0043] Step S104: Calculate the gradient of the network model, update the learning rate for each layer of the neural network through the learning rate optimization model, and update the network parameters of each layer according to the updated learning rate;

[0044] Step S105: updating the basic meta-learning rate, and updating the parameters of the network model and the learning rate optimization model according to the updated basic meta-learning rate.

[0045] Step S106: Determine whether the maximum number of iterations has been reached at the current moment. If not, continue to perform task sampling. Otherwise, deploy the trained model into the simulation environment to generate a multi-ship intelligent game confrontation strategy for the naval battlefield.

[0046] In the disclosed embodiment, a method for intelligent confrontation in naval battlefields based on an online learning rate optimization mechanism is proposed, which can include the following beneficial effects: on the one hand, compared with the traditional combat behavior modeling method, it solves the problem of explosion of task states and action spaces of large-scale intelligent agents in confrontation; on the other hand, the online learning rate optimization mechanism solves the problem of hyperparameter setting in the training process of the MAML algorithm, enhances the stability of network training, improves the learning efficiency and final performance of the algorithm, and has stronger robustness and practicality.

[0047] It should be understood that the method of the present application designs a learning rate optimization model for each inner network of the intelligent body, which is used to dynamically generate and adjust the learning rate for each layer of the network. Among them, the learning rate optimization model uses a deep recurrent neural network, the input is the historical gradient information of the network update, and the output is the learning rate of each layer of the network.

[0048] In addition, a meta-learning rate optimization method is designed for the entire network. The meta-learning rate is derived by the gradient descent method, and the differential term is obtained to dynamically update the optimized meta-learning rate:

[0049]

[0050] Among them, β i represents the updated meta-learning rate, β i-1 The current meta-learning rate is the meta-learning rate at the i-th iteration, η β High-level meta-learning rate, T i represents the i-th task sampled from the task distribution ρ(T), ρ(T) represents the task distribution, Represents task T i Based on the parameter θ i 'The loss function of the parameter θ' i-1 The gradient of Represents task T i Based on the parameter θ' i-1 The loss function is about the parameter θ i-2 gradient.

[0051] Below, as Figure 1 , Figure 2 As shown, each step of the above method in this exemplary embodiment is described in more detail.

[0052] Step S101: construct a multi-ship confrontation task simulation environment for a naval battlefield to provide dynamic environmental status and environmental feedback.

[0053] Specifically: Construct a simulation environment for multi-ship confrontation tasks in the naval battlefield. The outputs that the simulation environment can provide include: the state of the environment at each moment; the state of the environment at the next moment and the environmental feedback after the ship agent takes action according to the environmental state.

[0054] Step S102: Initialize network parameters, initialize network model and learning rate optimization model.

[0055] Specific: Initialization of algorithm parameters: Initialization of network model f θ And the learning rate optimization model h φ , initialize the multi-ship confrontation task distribution ρ(T), initialize a set of learning rates {α1,...,α l}, basic meta-learning rate β, high-level meta-learning rate η β .

[0056] Step S103: Task sampling is performed from the multi-ship confrontation task distribution, and multiple rounds of trajectory sampling are performed under each task.

[0057] Specifically: Initialize at time t=1, sample T tasks from the multi-ship confrontation task distribution ρ(T), and sample K1 trajectories for each task.

[0058] Step S104: Calculate the gradient of the network model, update the learning rate for each layer of the neural network through the learning rate optimization model, and update the network parameters of each layer according to the updated learning rate.

[0059] Specifically: Calculate the gradient Generate the model h through the learning rate φ , calculate the learning rate for each layer of the neural network:

[0060]

[0061] Among them, α j represents the learning rate of the j-th layer neural network, {α j} j＝1,...,l Indicates that there are a total of l learning rates, h φ The learning rate generation model, φ is the parameter of the learning rate optimization model, Indicates that in T i Task network model f θ The gradient of the loss function, θ is the parameter of the network model.

[0062] Update the network parameters of each layer: Among them, θ j Represents the parameters of the jth layer in the network model, α j represents the learning rate of the j-th layer of the neural network, Represents Hadamard.

[0063] Step S105: updating the basic meta-learning rate, and updating the parameters of the network model and the learning rate optimization model according to the updated basic meta-learning rate;

[0064] Specifically: Continue to sample K2 trajectories for each task and update the basic meta-learning rate

[0065]

[0066] Among them, β i represents the updated meta-learning rate, β i-1 The current meta-learning rate is the meta-learning rate at the i-th iteration, η β High-level meta-learning rate, T i represents the i-th task sampled from the task distribution ρ(T), ρ(T) represents the task distribution, Represents task T i Based on the parameter θ i 'The loss function of the parameter θ' i-1 The gradient of Represents task T i Based on the parameter θ' i-1 The loss function is about the parameter θ i-2 gradient.

[0067] Update the network model f θ And the learning rate optimization model h φ parameter:

[0068]

[0069] Among them, (θ, φ) represents the set of network model and learning rate optimization model parameters.

[0070] Step S106: Determine whether the number of iterations has been reached at the current moment. If not, continue to perform task sampling. Otherwise, deploy the trained model into the simulation environment to generate a multi-ship intelligent game confrontation strategy for the naval battlefield.

[0071] Specifically: the number of iterations is the preset second threshold, the preset number of iterations, and whether the maximum number of iterations is reached at the current moment. If not, continue to perform task sampling. If not, continue to perform task sampling. Otherwise, the trained network model f θ And the learning rate optimization model h φ Deployed into the simulation environment to generate multi-ship intelligent game confrontation strategies in the naval battlefield.

[0072] Example 1

[0073] Next, taking the game confrontation between multiple ships A and B in a naval battlefield environment as an example, the process of intelligent game confrontation between multiple ships in the naval battlefield based on the model-free meta-learning algorithm based on the online learning rate optimization mechanism is described.

[0074] Step 1: Describe the confrontation mission between multiple ships A and B and construct a simulated sea battlefield environment.

[0075] 1) Description of the multi-ship A and B confrontation mission;

[0076] In a complex sea area, several ships of Party A were found to have invaded. Party B dispatched several ships to attack them. The ships of Party B selected strategies based on the model-free meta-learning algorithm of the online learning rate optimization mechanism, while the ships of Party A output the countermeasure strategy based on the expert system method. There were several unknown reef obstacles in the confrontation sea environment. The task of the ships of Party B was to destroy as many ships of Party A as possible within a limited time and save their own ships.

[0077] 2) Organization and deployment of unmanned watercraft;

[0078] Both A and B have 5 groups of ships, each group is represented by 1 action unit, representing a watercraft. The initial position of the ships of the B side is fixed in the northwest corner of the sea battlefield, and the initial position of the ships of the A side appears randomly near the southeast corner of the sea.

[0079] 3) Termination conditions of the confrontation between A and B;

[0080] The total time step of each round of mission is set to 10,000 time steps. The termination condition of the round mission is that the confrontation time reaches the total time, or the ships of either party A or B are completely destroyed.

[0081] 4) state set, action set and environmental feedback;

[0082] The state s at each time t t for: in Indicates the current position vector of all ships on side B, Indicates the position vector of the A-side ship that appears within the radar detection range of the B-side ship. Indicates the current speed vector of all ships on side B. Represents the current ammunition inventory vector of all ships on side B, Represents the current damage vector of all ships on side B.

[0083] Each action a at time t t For: a t ={m1,m2,m3,m4,m5}, where m1 to m5 represent the maneuvers of the five ships of party B, and the maneuvers include steering, speed, and displacement information.

[0084] Feedback on the sea battlefield environment:

[0085] The reward for hitting a ship is r = +25

[0086] The reward for destroying a ship is r = +200;

[0087] The penalty for a ship being hit is r = -25

[0088] The penalty for destroyed ships is r = -200;

[0089] Penalty for touching obstacles r = -500

[0090] Touching the boundary penalty r = -1000

[0091] Ship collision penalty r = -500;

[0092] Step 2: Initialization of algorithm parameters

[0093] Construct 5 ship agents and initialize the network model f for each agent. θ , f θ A fully connected neural network with 1 hidden layer (16 neurons) was used; the learning rate optimization model was initialized with h φ ,h φ A recurrent neural network with 1 hidden layer (32 neurons) was used; a set of learning rates {α1,...,α l}、basic meta-learning rate β; set high-level meta-learning rate η β =0.005, M=10000.

[0094] Step 3: Initialize time t = 1, sample 1000 tasks from the multi-ship confrontation task distribution ρ(T), and sample 5000 trajectories for each task.

[0095] Step 4: Calculate the gradient for each task Generate the model h through the learning rate φ , calculate the learning rate for each layer of the neural network:

[0096]

[0097] Among them, α j represents the learning rate of the j-th layer neural network, {α j} j＝1,...,l Indicates that there are a total of l learning rates, h φ The learning rate generation model, φ is the parameter of the learning rate optimization model, Indicates that in T i Task network model f θ The gradient of the loss function, θ is the parameter of the network model.

[0098] Update the network parameters of each layer: Among them, θ j Represents the parameters of the jth layer in the network model, α j represents the learning rate of the j-th layer of the neural network, Hadamard

[0099] Step 5: Continue to sample 1000 trajectories for each task and update the basic meta-learning rate:

[0100]

[0101] Among them, β i represents the updated meta-learning rate, β i-1 The current meta-learning rate is the meta-learning rate at the i-th iteration, η β High-level meta-learning rate, T i represents the i-th task sampled from the task distribution ρ(T), ρ(T) represents the task distribution, Represents task T i Based on the parameter θ i 'The loss function of the parameter θ' i-1 The gradient of Represents task T i Based on the parameter θ' i-1 The loss function is about the parameter θ i-2 gradient.

[0102] Update the network model f θ And the learning rate optimization model h φ parameter:

[0103]

[0104] Among them, (θ, φ) represents the set of network model and learning rate optimization model parameters.

[0105] Step 6: If t = 10000, the trained network model f θ And the learning rate optimization model h φ Deploy it to the simulation environment and generate a multi-ship intelligent game confrontation strategy for the naval battlefield. Otherwise, t=t+1, return to step 3.

[0106] In order to alleviate the high dynamic uncertainty problem in the sequential decision-making tasks of multiple ships in the naval battlefield environment, a learning rate optimization model is designed for each ship inner network to dynamically generate and adjust the learning rate for each layer of the network. The learning rate optimization model uses a deep recurrent neural network, with the input as the historical gradient information of the network update, and the output as the learning rate of each layer of the network. A meta-learning rate optimization method is designed for the overall network, and the meta-learning rate is derived by the gradient descent method, which enhances the stability of network training, improves the learning efficiency of the algorithm and the final performance, and has stronger robustness and practicality.

[0107] It should be noted that although several units of the system for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiment of the present disclosure, the features and functions of two or more units described above can be concretized in one unit. Conversely, the features and functions of a unit described above can be further divided into multiple units for concretization. Some or all of the units can be selected according to actual needs to achieve the purpose of the disclosed solution. Those of ordinary skill in the art can understand and implement it without paying creative work.

[0108] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. This application is intended to cover any modification, use or adaptation of the present disclosure, which follows the general principles of the present disclosure and includes common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present disclosure are indicated by the appended claims.

Claims

1. A method for intelligent confrontation in naval battlefield based on online learning rate optimization mechanism, characterized in that: The following steps are involved: Construct a multi-ship confrontation task simulation environment in the naval battlefield to provide dynamic environmental status and environmental feedback; Initialize network parameters, initialize network model and learning rate optimization model; Task sampling is performed from the multi-ship confrontation task distribution, and multiple rounds of trajectory sampling are performed under each task; Calculate the gradient of the network model, update the learning rate for each layer of the neural network through the learning rate optimization model, and update the network parameters of each layer according to the updated learning rate; Update the basic meta-learning rate, and update the parameters of the network model and the learning rate optimization model according to the updated basic meta-learning rate. It is determined whether the number of iterations has been reached at the current moment. If not, task sampling will continue. Otherwise, the trained model will be deployed in the simulation environment to generate a multi-ship intelligent game confrontation strategy in the naval battlefield.

2. The method for intelligent confrontation in the sea battlefield based on the online learning rate optimization mechanism according to claim 1 is characterized in that: The initialization network parameters include: initializing the multi-ship confrontation task distribution, initializing the learning rate, initializing the basic meta-learning rate, and initializing the high-level meta-learning rate.

3. The method for intelligent confrontation in the sea battlefield based on the online learning rate optimization mechanism according to claim 2 is characterized in that: The high-level meta-learning rate is a preset first threshold.

4. According to claim 1, the method for intelligent confrontation in the sea battlefield based on the online learning rate optimization mechanism is characterized in that The number of iterations is a preset second threshold.

5. The method for intelligent confrontation in the sea battlefield based on the online learning rate optimization mechanism according to claim 1 is characterized in that: The expression for updating the learning rate for each layer of the neural network through the learning rate optimization model is: {a j } j＝1,...,l =h φ (▽L Ti (f θ )) Among them, α j represents the learning rate of the j-th layer neural network, {α j } j＝1,...,l Indicates that there are a total of l learning rates, h φ The learning rate generation model, φ is the parameter of the learning rate optimization model, Indicates that in T i Task network model f θ The gradient of the loss function, θ is the parameter of the network model.

6. The method for intelligent confrontation in the sea battlefield based on the online learning rate optimization mechanism according to claim 5 is characterized in that: The expression for updating the network parameters of each layer is: Among them, θ j Represents the parameters of the jth layer in the network model, α j represents the learning rate of the j-th layer of the neural network, Represents Hadamard.

7. The method for intelligent confrontation in the sea battlefield based on the online learning rate optimization mechanism according to claim 1 is characterized in that: The expression for updating the basic meta-learning rate is: Among them, β i represents the updated meta-learning rate, β i-1 The current meta-learning rate is the meta-learning rate at the i-th iteration, η β High-level meta-learning rate, T i represents the i-th task sampled from the task distribution ρ(T), ρ(T) represents the task distribution, Overall representation task T i Based on the parameter θ i 'The loss function is about the parameter θ' i-1 The gradient of Represents task T i Based on the parameter θ' i-1 The loss function is about the parameter θ i-2 gradient.

8. The method for intelligent confrontation in the sea battlefield based on the online learning rate optimization mechanism according to claim 7 is characterized in that: The expression for updating the parameters of the network model and the learning rate optimization model according to the updated basic meta-learning rate is: Among them, (θ, φ) represents the set of network model and learning rate optimization model parameters.

Images