Cluster algorithm training evolution system and method based on architecture model definition

Abstract

The invention discloses a cluster algorithm training evolution system and method based on system architecture model definition. A data acquisition unit acquires task targets and task resources; the system architecture unit analyzes task resources according to a task target and outputs a system architecture model; the algorithm training unit carries out algorithm training according to the system architecture model and outputs a cluster decision strategy and a task allocation scheme; the simulation evaluation unit performs simulation deduction on the cluster decision strategy, evaluates the effect of the task allocation scheme and outputs a corresponding evaluation report; and the result output unit outputs the evaluation report. According to the method, firstly, functions and interfaces of various task resources are defined in a unified mode through the system architecture model, compatibility, expansibility and interoperability among different task resources can be ensured, secondly, the system architecture model and the cluster decision algorithm are combined, decision efficiency is optimized, and traceability and transparency are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of offensive and defensive confrontation simulation and algorithm training. More specifically, it particularly relates to a cluster algorithm training evolution system and method based on architecture model definition. Background Art

[0002] With the continuous evolution of modern combat methods towards informatization and intelligence, the coordinated combat of multiple arms and multiple weapon systems has increasingly become the mainstream trend. The rapid deployment and joint operations of equipment such as swarm drones, autonomous ground platforms, and long-range strike weapons in actual combat scenarios require the rapid generation or iterative optimization of cluster decision-making and control algorithms to cope with the dynamically changing battlefield environment. Traditional methods based on pure rules or machine learning methods limited to single scenarios often lack a unified system standard, making it difficult to organically integrate the functional interfaces, operation logics, and combat processes of different weapon systems and combat units. As a result, in the case of large-scale and multi-system coordination, the simulation deduction results are difficult to have sufficient interpretability and credibility.

[0003] Currently, there has gradually emerged a practice of using architecture models (such as DoDAF, UPDM, etc.) to define the relationships and logics between weapon systems and combat units, and using this as a unified data source for training and simulation, enabling the opportunity to complete confrontation simulation under the premise of standardization and traceability. However, traditional practices usually involve scientific research or engineering teams defining command rules, communication protocols, and functional interfaces of unmanned aerial vehicles (or other weaponry) by themselves, resulting in a lack of a general standardized description method between different projects and platforms. Once the weapon system or combat requirements change, a large number of decision rules or control parameters need to be manually reconfigured or rewritten, greatly limiting the scalability of the system. Secondly, under simple rules or algorithms based on black-box machine learning, it is often difficult to clearly sort out the specific working modes of each weapon system at different combat stages and different functional interfaces from a macroscopic perspective. Especially in large-scale multi-weapon system coordinated combat scenarios, there is a lack of a systematic "system perspective" to understand why the algorithm makes a certain decision, and it is also difficult to explain how to locate and correct problems in a certain link. In addition, there is no unified model for managing links such as combat logic definition, platform modeling, and deduction scenario setting, which is prone to data fragmentation and interface disconnection. The finally generated simulation results are difficult to accurately reflect the real command process or the action rules of troops, and the reference value is limited.

[0004] Therefore, the problems existing in the prior art need to be further improved and developed. Summary of the Invention

[0005] (1) Object of the Invention: To solve the problems existing in the above-mentioned prior art, the object of the present invention is to provide a cluster algorithm training and evolution system and method based on architecture model definition, aiming to solve the deficiencies of existing cluster control, cluster decision-making, and weapon system behavior logic algorithms in practical applications.

[0006] (2) Technical Solution: To solve the above technical problems, the present technical solution provides a cluster algorithm training and evolution system based on architecture model definition, including a data acquisition unit, an architecture unit, an algorithm training unit, a simulation evaluation unit, and a result output unit; the data acquisition unit acquires task objectives and task resources; The architecture unit analyzes the task resources according to the task objectives and outputs an architecture model; The algorithm training unit performs algorithm training according to the architecture model and outputs a cluster decision-making strategy and a task allocation plan; The simulation evaluation unit conducts simulation deduction on the cluster decision-making strategy, evaluates the effect of the task allocation plan, and outputs a corresponding evaluation report; The result output unit outputs the evaluation report.

[0007] Preferably, the data acquisition unit includes a resource acquisition module and a target input module; the resource acquisition module acquires relevant information data of task resources, including the quantity, type, status, current location, energy, payload, etc. of the task resources, and initializes the relevant information data of the task resources to obtain a corresponding task resource vector; The target input module acquires relevant information data of task objectives, including task requirement priorities, task requirement locations, etc., and initializes the relevant information data of the task objectives to obtain a corresponding task objective vector.

[0008] Preferably, the task resource vector is [quantity, type, status, longitude normalization, latitude normalization, energy, payload]; the task objective vector is expressed as [priority, longitude normalization, latitude normalization].

[0009] Preferably, the architecture unit includes a model establishment module and a model optimization module; the model establishment module establishes a framework of the architecture model according to the task objectives and task resources: an architecture pre-model. The architecture pre-model includes functional components, interface protocols, communication relationships, tactics and strategies, state transition rules, etc., covering the interface compatibility and collaborative communication relationships between task resources; The model optimization module dynamically validates the architecture pre-model, including the correctness of task resources, the compatibility of interfaces, and the optimization of tactical logic, to obtain an optimized architecture model: an architecture optimization model.

[0010] Preferably, the model establishment module analyzes the scenarios and specific tasks of the task objectives, and then constructs a pre-model of the system architecture according to the task resources, including key steps such as task resource capacity analysis, system architecture performance design, and task allocation. By analyzing the activity sequence, state transition rules, etc. in different task objectives in detail, a pre-model of the system architecture including multiple views is designed.

[0011] Preferably, the model optimization module adopts a self-verifying logical verification method, trains the algorithm according to the pre-model of the system architecture to obtain a cluster decision-making strategy and a task allocation plan, and the simulation evaluation unit evaluates the effect of the allocation plan, and the evaluation result is fed back to the model optimization module to perform adaptive training and adjustment on the pre-model of the system architecture; finally, according to the real-time feedback of the task resources and task objectives, the correctness decision of the pre-model of the system architecture is continuously adjusted to form a closed-loop feedback mechanism to obtain the optimized model of the system architecture.

[0012] Preferably, the algorithm training unit regards different types of task resources as different agents, and learns to make optimal decisions in complex environments by repeatedly interacting with different environments of the task resources, specifically including: Step 301, perform agent modeling on different task resources; Step 302, train the behavior actions of the agent through a reward mechanism; Step 303, optimize the behavior actions of each agent; Step 304, optimize and adjust the behavior actions of each agent in the cluster to obtain a cluster decision-making strategy.

[0013] Preferably, each agent obtains corresponding rewards or punishments according to the actions it takes and the environmental feedback; when the agent takes the correct action, a positive reward is given, and a punishment is given for a wrong decision. R t = λ1·task completion reward - λ2·resource consumption penalty - λ3·task failure penalty, where λ1 is the reward coefficient, which is the feedback obtained by the agent from the environment after taking a certain action; λ2 and λ3 are the penalty coefficients; According to the historical task execution data, update the Q value and perform policy adjustment. In the Q-learning algorithm, the Q value represents the expected total reward that can be obtained by taking a certain action and continuously executing it in a certain state. , where is the Q value of executing the action in the state ; α is the learning rate; γ is the discount factor; r t is the current reward; st is the current state; ɑ t is the current action; s t+1 is the next state; ɑ' is the next action; is in the next state all possible actions the maximum Q value, representing the expectation of the maximum possible future return; The decision-making strategy is the rule or function for the agent to select actions in a given state; the agent will adjust its behavior actions according to the magnitude of the Q value, select the optimal task allocation plan, thus make decisions on the task resources in the cluster, as well as the corresponding task allocation, to obtain the cluster decision-making strategy. The larger the Q value, the better the performance of the task allocation plan in historical experience and the better the decision-making effect of the agent.

[0014] Preferably, the simulation evaluation unit realizes the coordination and cooperation between different task resources, specifically including: Step 401, the simulation evaluation unit matches the task resources with the task objectives according to the states of the task resources and the requirements of the task objectives; Step 402, the simulation evaluation unit globally coordinates and collaboratively controls the task resources.

[0015] The cluster algorithm training and evolution method defined based on the architecture model is applicable to the cluster algorithm training and evolution system defined based on the architecture model, including the following steps: Step 1, the data acquisition unit acquires the task objectives and task resources; Step 2, the architecture unit analyzes the task resources according to the task objectives and outputs the architecture model; Step 3, the algorithm training unit conducts algorithm training according to the architecture model and outputs the cluster decision-making strategy and the task allocation plan; Step 4, the simulation evaluation unit conducts simulation deduction on the cluster decision-making strategy, evaluates the effect of the task allocation plan, and outputs the corresponding evaluation report; Step 5, the result output unit outputs the evaluation report.

[0016] (3) Beneficial effects: Based on the cluster algorithm training and evolution system and method defined by the architecture model, the present invention first uniformly defines the functions and interfaces of various task resources through the architecture model, making the access of new equipment or new requirements more concise and efficient. Secondly, through the systematic definition and description of task resources and task objectives by the architecture model, the compatibility, scalability, and interoperability between different task resources can be ensured, and it can be quickly adapted to different task objective scenarios without large-scale modification of the existing model. Moreover, the combination of the architecture model and the cluster decision algorithm not only optimizes the decision-making efficiency of the system but also improves its traceability and transparency, avoiding the understanding and debugging problems brought by traditional "black box" algorithms. Finally, the standardized model description and the extensible algorithm training framework can quickly adapt to new task resources or task objectives, reducing the difficulty of large-scale manual adjustment of algorithms and models during system upgrade and scenario switching, and realizing efficient maintenance and management. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 is a schematic structural diagram of the cluster algorithm training and evolution system defined by the present invention based on the architecture model; Figure 2 is a flowchart of the steps of the cluster algorithm training and evolution method defined by the present invention based on the architecture model; Figure 3 is a schematic framework diagram of the pre-architecture model of the present invention; Figure 4 is a flowchart of the operation of the architecture optimization model of the present invention; Figure 5 is a flowchart of the operation of the cluster algorithm training and evolution method defined by the present invention based on the architecture model. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0018] The following further describes the present invention in detail with reference to preferred embodiments. More details are set forth in the following description to facilitate a full understanding of the present invention. However, the present invention is obviously capable of being implemented in many other ways different from those described herein. Those skilled in the art can make similar generalizations and deductions according to the actual application situation without departing from the connotation of the present invention. Therefore, the protection scope of the present invention should not be limited by the content of this specific embodiment.

[0019] The accompanying drawings are schematic diagrams of embodiments of the present invention. It should be noted that this accompanying drawing is only an example and is not drawn under equal-proportion conditions, and should not be used to limit the actual scope of protection required by the present invention.

[0020] The cluster algorithm training and evolution system defined based on the architecture model, as Figure 1As shown in the figure, it includes a data acquisition unit, an architecture unit, an algorithm training unit, a simulation evaluation unit, and a result output unit. The data acquisition unit acquires task objectives and task resources. The architecture unit analyzes the task resources according to the task objectives and outputs an architecture model. The algorithm training unit performs algorithm training based on the architecture model and outputs a cluster decision-making strategy and a task allocation plan. The simulation evaluation unit conducts simulation deduction on the cluster decision-making strategy and evaluates the effect of the task allocation plan, and outputs a corresponding evaluation report. The result output unit outputs the evaluation report.

[0021] The cluster algorithm training evolution method defined based on the architecture model is applicable to the cluster algorithm training evolution system defined based on the architecture model, such as Figure 2 , Figure 5 As shown in the figure, it specifically includes the following steps: Step 1: The data acquisition unit acquires task objectives and task resources.

[0022] Step 2: The architecture unit analyzes the task resources according to the task objectives and outputs an architecture model.

[0023] Step 3: The algorithm training unit performs algorithm training based on the architecture model and outputs a cluster decision-making strategy and a task allocation plan.

[0024] Step 4: The simulation evaluation unit conducts simulation deduction on the cluster decision-making strategy and evaluates the effect of the task allocation plan, and outputs a corresponding evaluation report.

[0025] Step 5: The result output unit outputs the evaluation report.

[0026] In Step 1, the data acquisition unit includes a resource acquisition module and a target input module; the resource acquisition module acquires the relevant information data of the task resources, including the quantity, type, status, current location, energy, payload, etc. of the task resources, and initializes the relevant information data of the task resources to obtain the corresponding task resource vector. Specifically, it includes: Step 1011: Acquire the relevant information data of the task resources from the task resource management system or sensors.

[0027] Specifically, it includes the quantity of the task resources, which may be a scalar value, such as the number of drones, the number of vehicles, etc. The type of the task resources, such as drones, ground platforms, etc. The current status of the task resources, such as "standby", "executing tasks", "damaged", etc. The current geographical location of the task resources, such as GPS coordinates (longitude, latitude). The remaining energy or power of the task resources, such as a percentage. The carrying capacity of the task resources, such as the effective payload of the drone, the ammunition load of the weapon system, etc.

[0028] Step 1012: Initialize the relevant information data for each task resource respectively to obtain the normalized task resource data accordingly.

[0029] Specifically, retain the quantity data of the task resource as a scalar value, which can be directly used when converting it into a vector. Assign a unique integer code to different types of task resources. For example, the drone is coded as 1, the ground platform is coded as 2, etc., and convert them into numerical variables. The status information of the task resource can be represented by category coding. For example, the "standby" status is coded as 0, the "executing task" is coded as 1, the "damaged" is coded as 2, etc. According to the geographical coordinates (longitude, latitude) of the task resource, normalization processing can be performed. For example, normalize the longitude and latitude to the interval [0, 1] respectively. Normalize the remaining energy value of the task resource to the interval [0, 1], usually represented by the ratio of the remaining battery power to the maximum battery power. Perform normalization processing according to the payload capacity of the task resource, usually represented by the ratio of the payload weight to the maximum load-bearing weight.

[0030] Step 1013: Convert the normalized resource data into a unified task resource vector. The task resource vector is represented as [quantity, type, status, normalized longitude, normalized latitude, energy, payload].

[0031] For example, assume a drone resource with the following data: quantity = 3. Type = drone (coded as 1). Status = executing task (coded as 1). Current location = (45.0, 85.0) (longitude and latitude). Energy = 80% (converted to 0.8). Payload = 50% (converted to 0.5). The corresponding task resource vector is [quantity, type, status, normalized longitude, normalized latitude, energy, payload] = [3, 1, 1, 0.75, 0.55, 0.8, 0.5].

[0032] The target input module collects the relevant information data of the task target, including the task requirement priority, task requirement location, etc., and initializes the relevant information data of the task target to obtain the corresponding task target vector. Specifically include: Step 1021: Obtain the relevant information data of the task target from the task management system.

[0033] Specifically include: The urgency (priority) of the task target can be an integer value. For example, a high-priority task is 1, and a low-priority task is 3. The target location for task execution, such as GPS coordinates (longitude, latitude).

[0034] Step 1022: Initialize the relevant information data for each task target respectively to obtain the normalized task target data accordingly.

[0035] Convert the task objective priority into an integer value or a normalized value according to a predefined rule. The geographical coordinates (longitude, latitude) of the task objective requirement location can be normalized in the same way as the current location of the task resource.

[0036] Step 1023: Merge the normalized task objective data into a task objective vector. The task objective vector is represented as [priority, longitude normalization, latitude normalization].

[0037] For example, assume the task objective is: priority = 1 (high priority), requirement location = (50.0, 90.0) (longitude and latitude). The corresponding task objective vector is: [priority, longitude normalization, latitude normalization] = [1, 0.80, 0.60].

[0038] By initializing and converting the relevant information data of each task resource and task objective into a unified vector form, these data can be conveniently input into the algorithm training unit. The vector set is used as input data for task allocation and decision optimization, supporting data processing during algorithm training and simulation deduction.

[0039] In step 2, the architecture unit includes a model establishment module and a model optimization module; the model establishment module establishes the framework of the architecture model: the architecture pre-model according to the task objective and task resources. The architecture pre-model includes functional composition, interface protocol, communication relationship, tactics and strategies, state transition rules, etc., covering content such as interface compatibility and collaborative communication relationship among task resources.

[0040] Specifically, analyze the scenario and specific tasks of the task objective, and then construct the architecture pre-model according to the task resources, including key steps such as task resource capability analysis, architecture performance design, and task allocation. Through detailed analysis of the activity sequence, state transition rules, etc. in different task objectives, such as Figure 3 shown, design an architecture pre-model that includes multiple views such as the global view AV, capability view CV, system view SV, project view PV, standard view StdV, data and information view DIV, service view SvcV, and simulation view OV. Subsequently, optimize and adjust the architecture pre-model through self-verification by the model optimization module to ensure the rationality and feasibility of each view.

[0041] The global view AV includes the architecture content of all views and is used to describe the concept, scope, purpose, actions, events, and environment of the task objective.

[0042] The described Capability View (CV) refers to the capability requirements, delivery time, and capability deployment view. Based on the results of the system mission objectives, the top-level capabilities of the system architecture can be analyzed. For example, if the system architecture needs to perform target perception tasks, the system architecture should have target perception capabilities such as radar perception and optical perception; if it participates in target strike tasks, it should have strike capabilities such as air strikes and surface strikes. These capabilities can be gradually refined. For example, the optical perception capability can be further divided into more detailed sub-capabilities such as visible light perception and infrared perception. Through capability analysis, a complete set of system capability view models can be constructed to display the system's capability concept, capability decomposition, and capability dependency relationships, etc.

[0043] The described System View (SV) refers to the task resources, processes, activities, and their interrelationships. Based on the task objective activities obtained from the system architecture process analysis, after selecting the weapon allocation plan, consider how different weapon systems cooperate to complete the corresponding task objectives. By designing the activities and timings, sort out the state transition rules, interface relationships, and interaction information of each weapon system, and derive the functional requirements and mission tasks of the weapon system.

[0044] The described Project View (PV) refers to the relationship between task resources, task objectives, and implementation projects.

[0045] The described Standard View (StdV) refers to policy standards, guidelines, restrictions, and expectations. The Data and Information View (DIV) refers to the data relationships between system architectures.

[0046] The described Service View (SvcV) includes the task resources, activities, services being executed, and their interrelationships.

[0047] The described Simulation View (OV) includes task resources, processes, activities of task resources, and the requirements of each task objective for task resources. Simulation process analysis is based on the system's task objectives, the results of simulation task analysis, and the results of task resource capability analysis. Associate the task objectives with each task resource, consider how different task resources cooperate to complete the corresponding task objectives, and design the system's simulation process. Consider adversarial factors in the simulation process to make it closer to reality and more meaningful, and finally obtain activities, timings, interface relationships, state transition rules, etc.

[0048] The text input of the task scenarios, compositions, objectives, etc. of the described system architecture pre-model is static. Through the described system architecture pre-model, the coordination and management of multiple task resources and task objectives can be achieved.

[0049] The described system architecture pre-model follows international standards, provides a standardized framework, ensures the compatibility and integration between multiple task resources. The system architecture pre-model guarantees operational consistency and standardized interfaces, providing the necessary framework support for subsequent cluster algorithm task allocation and cooperative control.

[0050] The pre-architecture model preferably adopts standards such as DoDAF and UPDM; however, other architecture frameworks such as MODAF, TOGAF, SysML, or UML Profile can also be used for system modeling and interface definition. As long as the functional structure, interface relationship, and business process of the multi-weapon system can be described standardly and data interaction can be carried out with the algorithm training platform and the simulation platform, the specific selection is not restricted.

[0051] The model optimization module dynamically validates the pre-architecture model, including the correctness of task resources, the compatibility of interfaces, and the optimization of tactical logic, to obtain the optimized architecture model: the architecture optimization model. The model optimization module is optimized according to the feedback results to improve the adaptability of the architecture model in complex environments.

[0052] Self-verification is the main logical verification method of the model optimization module of the present invention. As Figure 4 shown, algorithm training is carried out according to the pre-architecture model to obtain a cluster decision-making strategy and a task allocation plan. The simulation evaluation unit evaluates the effect of the allocation plan, and the evaluation results are fed back to the model optimization module to perform adaptive training adjustment on the pre-architecture model. Finally, through the update of the adaptive calculation training process, the correct decision of the pre-architecture model can be continuously adjusted according to the real-time feedback of new task resources and task objectives, forming a closed-loop feedback mechanism to obtain the architecture optimization model.

[0053] In this embodiment, the self-verification is sufficient to standardize the normativity and logical correctness of the input data for algorithm training and improve the interpretability of the algorithm model. According to the actual use process, the model optimization module includes various optimization and adjustment methods, which are not specifically limited in the present invention.

[0054] The architecture optimization model serves as the only data source for the entire algorithm training and simulation evaluation, and defines and manages entities, attributes, constraints, etc. in the whole process. The design and verification of the architecture model provide a unified data source and a standardized interface for subsequent algorithm training and simulation deduction.

[0055] The architecture optimization model has the ability to guarantee the modeling of the requirement architecture logical model, and supports the verification of the logical relationship correctness of the guarantee requirement design logical view with activity diagrams, state machine diagrams, and sequence diagrams. By deeply analyzing the behavioral graph model elements and parametric graph model elements, and being able to solve and analyze parametric equations. During the simulation analysis process, the software ensures that the underlying logic is rigorous and reasonable, the simulation execution process is clear and smooth, and the simulation analysis results are true and reliable.

[0056] Through the described architecture unit, it is ensured that the tasks and resources can be uniformly managed and scheduled at the global level, ensuring that various task resources can be efficiently coordinated and tasks can be executed.

[0057] The algorithm training unit in step 3 trains different task resources, continuously optimizing the behavior strategies of each task resource. By continuously optimizing the behavior strategies of each task resource in the training cluster, a cluster decision-making strategy and a task allocation plan that meet the task objectives are output. The present invention uses deep reinforcement learning (DRL) technology to enhance the intelligence and adaptability of the algorithm training unit. Through multi-agent reinforcement learning, each task resource acts as an independent agent to execute tasks in different environments and optimizes the cluster decision-making strategy through interaction with the environment.

[0058] The algorithm training unit uniformly defines the functional composition, interface relationship, and operation logic of task resources. The algorithm training unit regards different types of task resources as different agents and learns to make optimal decisions in complex environments through repeated interaction with different environments of task resources.

[0059] The algorithm training unit provides a training framework, including functions such as a reward mechanism, experience replay, and policy update, to support the agent in continuously optimizing its behavior strategy during the training process. In the training stage, the algorithm training unit adopts a distributed training and centralized evaluation method to ensure that the agent can gradually converge in multiple rounds of training and form an efficient cluster collaborative decision-making strategy.

[0060] Specifically, it includes the following steps: Step 301: Perform agent modeling on different task resources.

[0061] Each different type of task resource (such as an unmanned aerial vehicle, a ground platform, etc.) is defined as an independent agent. Each agent selects a suitable action based on its local observations (such as position, friendly and enemy situation, etc.) and executes the action. The actions of the agent include but are not limited to moving, attacking, reconnaissance, etc. Through agent modeling, the agent can dynamically adjust its own strategy according to the real-time battlefield situation, enhancing the flexibility and response ability of the cluster decision-making strategy and task allocation.

[0062] Step 302: Train the behavior actions of the agent through the reward mechanism.

[0063] Each agent obtains corresponding rewards or punishments according to the actions it takes and the environmental feedback. When the agent takes the correct action, it is given a positive reward, and when it makes a wrong decision, it is given a punishment, thereby guiding the agent to continuously adjust its decision-making strategy. As shown in formula (1): R t =λ1·Task completion reward - λ2·Resource consumption penalty - λ3·Task failure penalty (1), Among them, λ1 is the reward coefficient, which is the feedback obtained by the agent from the environment after taking a certain action, helping the agent understand the quality of the current behavior; λ2 and λ3 are the penalty coefficients, and the task completion reward and resource consumption penalty are dynamically adjusted according to the task execution situation.

[0064] In the scenario of task allocation, when an agent (such as a drone) successfully completes a task, it will receive a positive reward; if the task fails or the resource consumption is too high, it will receive a penalty signal. The reward signal tells the agent which behaviors are effective and which are disadvantageous.

[0065] The reward signal is the core information in reinforcement learning, which can help the agent learn which behaviors contribute to achieving the goal. The design of the reward function is the key, which determines the learning efficiency and decision optimization effect of the system. By designing a reasonable reward mechanism, the agent can continuously adjust its own behavior to maximize the task goal while avoiding falling into local optimal solutions in a complex environment.

[0066] Step 303: Optimize the behavioral actions of each agent.

[0067] Through the continuous interaction between the agent and the environment, optimize the action strategy of each agent; then, through techniques such as experience replay and target network in deep reinforcement learning, help the agent extract valuable information from historical experience and accelerate the convergence of the decision-making strategy.

[0068] The valuable information refers to historical data, experience, or feedback that can improve the agent's decision-making and optimize the strategy. Valuable information in deep reinforcement learning usually refers to various data that are helpful for optimizing the strategy and can promote the agent's learning and decision-making, including reward signals, historical experience, state information, Q-values, policy update information, etc. This information comes from the interaction process between the agent and the environment. After time accumulation, it can provide effective learning signals for the agent, helping the agent gradually improve its decision-making strategy, so as to learn how to make better decisions in a complex and changing environment. Through techniques such as experience replay and target network, the agent can extract effective information from historical experience, reduce randomness, and accelerate the convergence of the strategy.

[0069] The target network is a technique in deep reinforcement learning used to stabilize the training process. It is a delayed version of the current training network, usually used to calculate the target Q-value and reduce the fluctuations in the training process. In the Deep Q-Network (DQN), the agent uses the target network to calculate the target of the Q-value. The target network is updated only at regular intervals, thus avoiding the high-frequency fluctuations that may occur when updating the network parameters at each step. The target network provides a stable training target, which helps to reduce the oscillations in policy updates during the training process and ensure the stability and effectiveness of the learning process.

[0070] Historical experience refers to the sequence data of states, actions, and rewards that the agent has collected and experienced during past training. If an agent selected a certain path and successfully completed the task in the last mission, it would record the state transitions and rewards of this path. When faced with a similar situation, it can look up similar state-action pairs from historical experience and make decisions based on past experience. Through experience replay, the agent can utilize past experience to improve current decisions. The accumulation of historical experience helps improve decision-making efficiency and avoid repeated exploration of inefficient strategies.

[0071] In the present invention, the Q-learning or Deep Q-Network (DQN) algorithm is used to select the optimal action according to the state of the agent and the task state. Based on the historical task execution data, the Q-value is updated and the policy is adjusted. The Q-value is the expected cumulative reward for the agent to select a certain action in a certain state, and it is a quantitative value that measures the quality of a certain action in a specific state.

[0072] Specifically, for the calculation of the Q-value, different time mechanism strategies are determined through historical task execution data. The time mechanism strategy splits the time for calculating the Q-values of different agents according to the importance levels of different task objectives, into the total calculation time and the agent calculation time. Thus, when accelerating and optimizing different agents, the effect of optimizing and accelerating the total calculation time can be achieved.

[0073] The total calculation time refers to the total time for calculating the Q-values of all agents relative to each task objective. The agent calculation time refers to the time for each agent to calculate the Q-value for a single task objective. The total calculation time is equal to the sum of the agent calculation times.

[0074] For example, when the task objective is one-to-one with respect to the image processor GPU, that is, when the number of task objectives is less than or equal to the number of GPUs, the central processing unit CPU allocates different task objectives and the task resources (i.e., agents) under the task objective to different graphics processing units GPUs to calculate the Q-values of different agents for the task objective. Thus, the total calculation time can be equal to the agent calculation time, achieving the effect of optimizing and accelerating the total calculation time. When the task objective is many-to-one with respect to the image processor GPU, that is, when the number of task objectives is greater than the number of GPUs, at this time the same GPU may need to process two or more calculation tasks. When calculating the Q-value, the central processing unit CPU determines the importance ranking of the current task objective based on different historical task execution data, marks and assigns the calculation tasks with a high importance level to the GPU. The GPU will perform corresponding sorting on the allocated calculation tasks. Therefore, the GPU will preferentially calculate the Q-value of the agent with a high importance level corresponding to the task objective of the allocated calculation task. By allocating the calculation times of different agents, the optimization and acceleration of the total calculation time are realized. For example, there are 10 calculation tasks for the agent to calculate the Q-value for the task objective, but there are a total of 3 GPUs. Therefore, the first GPU is allocated 4 calculation tasks, that is, it needs to calculate 4 different Q-values. The calculation time for the first GPU is the sum of the calculation times of 4 agents. At the same time, the other three GPUs are each allocated 3 calculation tasks. Therefore, the calculation times of the other 3 GPUs are equal to the sum of the calculation times of 3 agents. Since these 4 GPUs can calculate simultaneously, the total calculation time is equal to the calculation time of the GPU with the longest time (the sum of the calculation times of 4 agents), which is much less than the calculation time when the CPU calculates all agents in sequence (the sum of the calculation times of 10 agents).

[0075] By allocating the time for Q-value calculation through the time mechanism strategy, the time for the agent to calculate the Q-value in the cluster decision-making strategy can be greatly shortened, thereby achieving the effect of accelerating the Q-value calculation in the entire calculation process.

[0076] In the Q-learning algorithm, the Q-value represents the expected total reward that can be obtained by taking a certain action and continuously executing it in a certain state. The Q-value can provide a basis for the agent to select actions, guiding the agent to make decisions that can maximize the cumulative reward. As shown in formula (2): (2), wherein, is the Q-value (action value) of executing action in state ; α is the learning rate; γ is the discount factor; r t is the current reward; s tis the current state; ɑ t is the current action; s t+1 is the next state; ɑ' is the next action; is at the next state all possible actions the maximum Q value among them, representing the expectation of the maximum possible future return.

[0077] Step 304. Optimize and adjust the behavioral actions of each agent in the cluster to obtain the cluster decision-making strategy.

[0078] The training process adopts a distributed reinforcement learning framework, where each agent runs independently in the simulation environment but shares global information for policy coordination. Through repeated training of different agents, the agents continuously optimize their decision-making strategies and finally form an optimal cluster collaborative decision-making strategy.

[0079] The decision-making strategy is a rule or function for an agent to select an action in a given state. The decision-making strategy can be deterministic (each state corresponds to a specific action) or stochastic (select an action according to a certain probability in a given state). In a cluster task, the decision-making strategy may include actions such as scouting, attacking, or defending at a certain moment. These strategies are continuously optimized through training with the aim of maximizing the cumulative reward. By continuously optimizing the strategy, the agent can make more appropriate decisions in a complex environment, thereby improving the efficiency of task completion.

[0080] Based on the current state, use the trained decision-making strategy to determine the tasks executed by each agent. The task assignment priority is determined according to the Q value, so as to make decisions on the task resources in the cluster and the corresponding task assignment, and obtain the cluster decision-making strategy. As shown in formula (3), (3), where ɑ* is the optimal action, that is, the best task assignment decision.

[0081] The larger the Q value, the higher the expected return of the state-action pair under the current policy, representing the total return (i.e., the expected return) that the agent may obtain in the future when taking a certain action in a certain state, indicating that this action is better than other actions in this state, and the agent tends to choose this action. In the task assignment model, the larger the Q value, the better the performance of the task assignment scheme in historical experience, which can bring a higher cumulative return (for example, task completion rate, resource utilization efficiency, etc.), and the better the decision-making effect of the agent obtained. Therefore, the agent will adjust its behavioral actions according to the Q value and select the optimal task assignment scheme, thereby improving the efficiency and success rate of task execution.

[0082] The simulation evaluation unit in step 4 uses the cluster decision strategy to perform simulation deduction, execute tasks and plan paths. During the simulation process, the platform evaluates the task execution effect (task success rate, path planning quality, resource consumption, etc.) and outputs a corresponding evaluation report.

[0083] Specifically, the simulation evaluation unit generates simulation data such as force deployment and tactical strategies based on the system architecture model, simulates the real environment, and evaluates the effectiveness of the cluster decision-making strategy. During the simulation process, the task allocation, path planning, and collaborative behavior between task resources are adjusted through algorithm optimization to ensure a high degree of coordination between task resources.

[0084] The present invention realizes the coordination and collaboration between different task resources through the simulation evaluation unit. The specific operation steps are as follows: Step 401: The simulation evaluation unit matches the task resources with the task objectives according to the status of the task resources and the requirements of the task objectives.

[0085] The simulation evaluation unit calculates and generates a task allocation plan in real time based on the architecture model and cluster decision algorithm. The task allocation plan is dynamically adjusted based on the status, location and other information of the task resources, including location matching, resource matching and priority matching. Each task resource performs corresponding operations according to the assigned tasks and goals. Through dynamic task allocation, the simulation evaluation unit can ensure that all task resources are efficiently coordinated to avoid task conflicts and resource waste.

[0086] In order to design the operation steps for dynamically adjusting the task allocation scheme based on information such as the task resource status and location, as shown in Table 1, it can be described according to the following process. Each step ensures that the task allocation can be adjusted in real time according to the status, location and task objectives of the current task resources, and can effectively avoid task conflicts and resource waste.

[0087] Step 402: The simulation evaluation unit globally coordinates and collaboratively controls task resources.

[0088] During the cluster algorithm training process, each task resource must ensure global coordination while performing its own tasks. The simulation evaluation unit enables each agent to coordinate and cooperate with each other during the training process through a global information sharing mechanism. By obtaining global information, the agent can adjust its own strategy to ensure coordinated training with other agents, ultimately improving the training efficiency of the entire cluster system.

[0089] The simulation results are analyzed through evaluation indicators such as task completion rate, resource consumption, and tactical execution effect to provide feedback for the iterative optimization of the algorithm. The evaluation results will be returned to the training platform to form a closed-loop feedback mechanism of "training-deduction-evaluation-optimization".

[0090] The simulation evaluation unit is responsible for simulating a complex task resource environment and conducting real-time deduction. The simulation evaluation unit combines the output of the system architecture model and the cluster algorithm training to construct multi-scenario and multi-dimensional simulation tasks, adjusts the actions of task resources in real time, and provides feedback based on real-time data. The simulation evaluation unit can evaluate the effectiveness of cluster decisions and, through dynamic updates, help the system continuously optimize strategies and decisions. It not only simulates the enemy and friendly situations and tactical rules but also takes into account physical characteristics (such as flight mechanics, weapon ranges, etc.) to ensure that the simulation results have a high reference value for reality.

[0091] Table 1 Task Assignment Operation Steps

[0092] The simulation evaluation unit includes two parts: the simulation analysis of behavior graph model elements and the simulation analysis of parameter graph model elements. The simulation analysis of behavior graph model elements mainly constructs a simulation model by parsing the elements and relationships in the behavior graph, and then uses a simulation engine to run the model. During the running process, the software will dynamically monitor the state changes of the model, record the running data of the model, and generate a simulation report. The simulation report details the behavior and performance of the model during the simulation process, helping users understand and evaluate the behavior of the system. The simulation analysis of parameter graph model elements constructs a simulation model by parsing the elements and relationships in the parameter graph, and then uses a simulation engine to run the model. During the running process, the software will dynamically monitor the state changes of the model, record the running data of the model, and generate a simulation report. These reports detail the behavior and performance of the model during the simulation process, helping users understand and evaluate the performance of the system.

[0093] In addition, the simulation evaluation unit also supports the solution and analysis of parameter equations. During this process, the simulation evaluation unit will parse the parameter equations and then construct a calculation model based on the parsing results. After the model is constructed, the simulation evaluation unit will start a solution engine to run the solution model according to the logic defined in the equations. During the solution running process, the simulation evaluation unit will dynamically monitor the state changes of the model, record the running data of the model, and generate a solution report.

[0094] The simulation engine integrates the fUML (Foundation UML) engine and a parameter solver. The fUML engine is used to parse and execute SysML-based behavioral models. The parameter solver is used to process parameter models, which can parse and solve models containing mathematical equations and expressions. It provides a function for calculating model parameter metrics, and calculates the parameter metrics by means of a built-in solver and external solvers (such as MATLAB, JavaScript, Python, etc.). After setting the corresponding constraints, it is possible to evaluate whether the parameter results meet the requirements through calculation. The parameter diagram enables the computability and simulation of system parameters. Thus, it supports requirement verification and system metric demonstration, and realizes a well-founded system design.

[0095] The result output unit in step 5 outputs and displays the evaluation report, including the task execution evaluation report, the optimized task allocation and path planning scheme, and the analysis and evaluation of the simulation results.

[0096] The optimized task allocation and path planning scheme includes adjusting weights, learning rates, hyperparameters, etc. The analysis and evaluation of the simulation results include evaluation metrics such as task success rate, resource consumption, and path quality. Through these evaluation metrics, the effects of cluster decision-making and task allocation can be comprehensively analyzed, the performance of the cluster system in different task environments can be evaluated, potential optimization spaces can be discovered, and data support can be provided for subsequent optimization.

[0097] The following is a detailed description with specific embodiments: Suppose there are 5 drones and 3 mission objectives. Then, optimization allocation will be carried out according to the status of each drone (such as battery power, distance from the mission) and mission requirements (such as priority, time-consuming), ensuring that high-priority missions are preferentially executed by the drones, and minimizing the time and resource consumption for mission completion.

[0098] Mission resource information: Mission resource 1: Drone, current status: executing mission, current location: longitude (45.0), latitude (85.0), energy: 80%, payload: 50%. Mission resource 2: Drone, current status: standby, current location: longitude (46.0), latitude (86.0), energy: 60%, payload: 30%. Mission resource 3: Ground platform, current status: executing mission, current location: longitude (47.0), latitude (87.0), energy: 90%, payload: 70%. Mission resource 4: Ground platform, current status: damaged, current location: longitude (48.0), latitude (88.0), energy: 40%, payload: 60%. Mission resource 5: Drone, current status: executing mission, current location: longitude (49.0), latitude (89.0), energy: 70%, payload: 40%. The mission resources are converted into corresponding mission resource vectors through initialization processing, as shown in Table 2.

[0099] Table 2 Task Resource Vector Matrix

[0100] Task objective information: Task objective 1: Priority: High, Location: Longitude (50.0), Latitude (90.0). Task objective 2: Priority: Medium, Location: Longitude (51.0), Latitude (91.0). Task objective 3: Priority: Low, Location: Longitude (52.0), Latitude (92.0). The said task objective information is converted into corresponding task objective vectors through initialization processing, as shown in Table 3.

[0101] Table 3 Task Objective Vector Matrix

[0102] Initialize the state vectors of all drones: task resource vectors, and the state vector of the task: task objective vectors. The state of each drone includes: position, speed, energy, task status, distance from other drones, etc. The task status includes task location, priority, and required resources (battery power). The overall state space of the cluster contains the states of all drones, as shown in Table 4. The actions of the drones include all possible actions of the drones such as moving direction, speed adjustment, task selection, etc.

[0103] Table 4 Drone States

[0104] After that, define the reward function based on task completion, resource consumption, and penalties for task failure. If the task is completed, the reward is based on the task completion rate; assume that when the task completion rate is 100%, the reward is 1. If the resource consumption (battery power) is excessive, there will be a penalty; for example, for every 1 unit of battery power consumed, the penalty coefficient is 0.2. If the task cannot be completed on time or is interrupted, there will be a penalty; assume the penalty coefficient is -1.

[0105] Initialize the starting position, target position, obstacle information, flight speed, and energy limit of each drone, etc. Use the A* algorithm or Dijkstra algorithm to calculate the shortest path for each drone, considering the possible obstacles and the positions of other drones during the flight. For each drone, the path planning algorithm calculates the path from the starting point to the target point.

[0106] As shown in formula (3), the heuristic evaluation function in the A* algorithm is: f(x)=g(x)+h(x) (3), Among them, g(x) is the actual cost from the starting point to the current node, and h(x) is the estimated cost from the current node to the target node (heuristic function).

[0107] Use dynamic programming for path adjustment, and update the path in real time to cope with environmental changes, such as enemy activities or meteorological condition changes. When the battlefield environment changes (for example, enemy interference, the appearance of obstacles, etc.), the path planning needs to be dynamically adjusted. Use dynamic programming to update the path as shown in formula (4): Pnew = Pcurrent + ΔP (4), where: Pnew is the new path after adjustment, and ΔP is the path adjustment amount.

[0108] Calculate the resources consumed during the path execution (such as battery power, time, etc.). As shown in formula (5), the resource consumption formula is:[[]] R consume = λ7 × path length + λ8 × flight time (5), where: λ7 and λ8 are the weight coefficients of the path length and flight time, the path length is the total flight distance, and the flight time is the time required to complete the path.

[0109] If a drone needs to fly from the starting point A to the target B, the A* algorithm will calculate the shortest path, avoid collisions with obstacles, and dynamically adjust the path according to the environmental information to ensure the minimization of time and resource consumption.

[0110] If two or more drones cooperate to complete a task (such as jointly completing reconnaissance), give rewards. If drones collide or overlap excessively, give penalties. Drones that complete the task objective or optimize the task assignment receive rewards. As shown in formula (6): (6), where, λ1, λ2 are adjustment coefficients to ensure the balance between rewards and penalties.

[0111] Use the Q-learning or Deep Q-Network (DQN) algorithm to select the optimal action according to the state of the drone and the task state. Based on the following assumptions: Reward coefficient: task completion reward λ1 = 1, resource consumption penalty λ2 = 0.2, task failure penalty λ3 = -1.

[0112] Learning rate: α = 0.1, Discount factor: γ = 0.9, Update the formula through Q-learning: Q(s,a) ← Q(s,a) + a[R(s,a) + γ * maxQ(s',a) - Q(s,a)].

[0113] During the task assignment process, select the optimal task assignment strategy. Suppose there are two tasks (Task 1 and Task 2), and two drones (Drone A and Drone B). Based on the states of the drones and the tasks, calculate the Q-values and select the optimal task assignment.

[0114] Q(Drone A, Task 1) = 5; Q(Drone A, Task 2) = 3; Q(Drone B, Task 1) = 4; Q(Drone B, Task 2) = 6; From the perspective of Q-values, the Q-value for Drone A to select Task 1 is larger (5 vs 3), while the Q-value for Drone B to select Task 2 is larger (6 vs 4). Therefore, based on the Q-value decision, assign Task 1 to Drone A and Task 2 to Drone B.

[0115] Optimize the collaborative decision-making of swarm drones through multi-agent reinforcement learning (MARL) to ensure efficient collaboration of task resources in a dynamic environment and avoid conflicts. The purpose of training the swarm algorithm is to ensure that each drone in the swarm can make coordinated decisions based on the current environment and the states of other drones, thus completing the tasks.

[0116] Multiple drones cooperate together according to environmental changes and task requirements to avoid conflicts. During this process, the agents will exchange state information with each other and adjust their strategies to coordinate cooperation. As shown in Equation (7), V(s t ) = V(s t ) + α·(r t + γ·V(s t+1 ) - V(s t )) (7), where V(s t ) is the value function, r t is the reward, S t and S t+1 are the current and next states, and α is the learning rate.

[0117] Based on the optimized strategy and reward function, automatically adjust the action decisions of the drones to ensure that each drone can cooperate with other drones, ensure the efficient execution of tasks, and at the same time avoid conflicts and resource waste. Based on the optimization results of multi-agent reinforcement learning, the drones select the optimal collaborative decision-making strategy.

[0118] When multiple drones in a swarm perform reconnaissance missions, they share target information and path planning data with each other to avoid repeating the reconnaissance of the same area and maximize the area coverage rate. The reinforcement learning algorithm adjusts in real time according to the action strategy of each drone to minimize the time and resource consumption of task execution, ensure the task is completed on time, and avoid waste of energy and other resources.

[0119] Suppose the optimal drone is selected between Task 1 and Task 2. After multiple rounds of training, the Q value gradually converges: for Task 1, the optimal drone may be Drone 3 because its state and resources are the most matched and it is close to the task location; for Task 2, the optimal drone may be Drone 1 because it has more power and is close to the task location.

[0120] Finally, after optimization and policy calculation, the final task assignment result is output as shown in Table 5: Table 5 Final Output Task Assignment Result

[0121] Based on the task objectives and environmental information, calculate the optimal flight path of each drone and output the corresponding evaluation report.

[0122] The cluster algorithm training evolution system and method of the present invention are defined based on the architecture model. First, by uniformly defining the functions and interfaces of various task resources in the architecture model, the access of new equipment or new requirements is made more concise and efficient. Second, through the systematic definition and description of task resources and task objectives by the architecture model, the compatibility, scalability, and interoperability between different task resources can be ensured, and it can be quickly adapted to different task objective scenarios without large-scale modification of the existing model. Third, the combination of the architecture model and the cluster decision-making algorithm not only optimizes the decision-making efficiency of the system but also improves its traceability and transparency, avoiding the understanding and debugging problems brought by traditional "black box" algorithms. Finally, the standardized model description and the extensible algorithm training framework can quickly adapt to new task resources or task objectives, reducing the difficulty of large-scale manual adjustment of algorithms and models during system upgrade and scenario switching, and realizing efficient maintenance and management.

[0123] The above content is an illustration of the preferred embodiments of the present invention, which can help those skilled in the art to more fully understand the technical solution of the present invention. However, these embodiments are only examples and cannot be considered that the specific implementation manners of the present invention are limited to the descriptions of these embodiments. For those of ordinary skill in the technical field to which the present invention belongs, without departing from the concept of the present invention, several simple deductions and transformations can be made, which should all be regarded as belonging to the protection scope of the present invention.

Claims

1. A cluster algorithm training evolution system defined based on an architecture model, characterized in that It includes a data acquisition unit, an architecture unit, an algorithm training unit, a simulation evaluation unit, and a result output unit; the data acquisition unit acquires task objectives and task resources; The architecture unit analyzes the task resources according to the task objectives and outputs an architecture model; The algorithm training unit performs algorithm training based on the architecture model and outputs a cluster decision-making strategy and a task allocation plan; The simulation evaluation unit conducts simulation deduction on the cluster decision-making strategy, evaluates the effect of the task allocation plan, and outputs a corresponding evaluation report; The result output unit outputs the evaluation report.

2. The cluster algorithm training evolution system defined based on the architecture model according to claim 1, wherein The data acquisition unit includes a resource acquisition module and a target input module; the resource acquisition module acquires relevant information data of task resources, including the quantity, type, status, current location, energy, load, etc. of task resources, and performs initialization processing on the relevant information data of task resources to obtain a corresponding task resource vector; The target input module acquires relevant information data of task objectives, including task requirement priorities, task requirement locations, etc., and performs initialization processing on the relevant information data of task objectives to obtain a corresponding task objective vector.

3. The cluster algorithm training and evolution system defined based on the architecture model according to claim 2, characterized in that, The task resource vector is [quantity, type, status, longitude normalization, latitude normalization, energy, load]; The task objective vector is expressed as [priority, longitude normalization, latitude normalization].

4. The cluster algorithm training evolution system defined based on the architecture model according to claim 1, characterized in that, The architecture unit includes a model establishment module and a model optimization module; the model establishment module establishes a framework of the architecture model according to the task objectives and task resources: an architecture pre-model; the architecture pre-model includes functional composition, interface protocols, communication relationships, tactics and strategies, state transition rules, etc., covering the interface compatibility and collaborative communication relationships between task resources; The model optimization module conducts dynamic verification on the architecture pre-model, including the correctness of task resources, the compatibility of interfaces, and the optimization of tactical logic, to obtain an optimized architecture model: an architecture optimization model.

5. The cluster algorithm training evolution system defined based on the architecture model according to claim 4, characterized in that, The model establishment module analyzes the scenarios and specific tasks of task objectives, and then constructs an architecture pre-model according to task resources, including key steps such as task resource capability analysis, architecture performance design, and task allocation; by analyzing in detail the activity sequence, state transition rules, etc. in different task objectives, an architecture pre-model including multiple views is designed.

6. The cluster algorithm training evolution system defined based on the architecture model according to claim 4, characterized in that, The model optimization module adopts a self-verifying logical verification method, performs algorithm training according to the architecture pre-model to obtain a cluster decision-making strategy and a task allocation plan, the simulation evaluation unit evaluates the effect of the allocation plan, and the evaluation result is fed back to the model optimization module to perform adaptive training adjustment on the architecture pre-model; finally, according to the real-time feedback of task resources and task objectives, the correctness decision of the architecture pre-model is continuously adjusted to form a closed-loop feedback mechanism to obtain the architecture optimization model.

7. The cluster algorithm training and evolution system defined based on the architecture model according to claim 1, characterized in that The algorithm training unit regards different types of task resources as different agents and repeatedly interacts with different environments of task resources to learn to make optimal decisions in complex environments, specifically including: Step 301: Perform agent modeling on different task resources; Step 302: Train the behavioral actions of the agent through a reward mechanism; Step 303: Optimize the behavioral actions of each agent; Step 304: Optimize and adjust the behavioral actions of each agent in the cluster to obtain a cluster decision-making strategy.

8. The cluster algorithm training evolution system defined based on the architecture model according to claim 7, characterized in that, Each agent obtains corresponding rewards or punishments based on the actions it takes and the environmental feedback; a positive reward is given when the agent takes the correct action, and a punishment is given for wrong decisions. R t = λ1 · Task Completion Reward - λ2 · Resource Consumption Penalty - λ3 · Task Failure Penalty, Among them, λ1 is the reward coefficient, which is the feedback obtained by the agent from the environment after taking a certain action; λ2 and λ3 are the punishment coefficients; According to the historical task execution data, update the Q value and perform policy adjustment. In the Q-learning algorithm, the Q value represents the expected total reward that can be obtained by taking a certain action and continuously executing it in a certain state. ， Among them, is the Q value for executing the action in the state ; α is the learning rate; γ is the discount factor; r t is the current reward; s t is the current state; ɑ t is the current action; s t+1 is the next state; ɑ' is the next action; is the maximum Q value among all actions in the next state ; The decision-making strategy is a rule or function for the agent to select actions in a given state; the agent will adjust its behavioral actions according to the magnitude of the Q value, select the optimal task allocation plan, so as to make decisions on the task resources in the cluster and perform corresponding task allocations to obtain a cluster decision-making strategy. The larger the Q value, the better the performance of the task allocation plan in historical experience and the better the decision-making effect of the agent.

9. The cluster algorithm training and evolution system defined based on the architecture model according to claim 1, characterized in that, The simulation evaluation unit realizes the coordination and cooperation between different task resources, specifically including: Step 401: The simulation evaluation unit matches the task resources with the task objectives according to the state of the task resources and the requirements of the task objectives; Step 402: The simulation evaluation unit globally coordinates and collaboratively controls the task resources.

10. A method for training and evolving a cluster algorithm defined based on an architecture model, applicable to a system for training and evolving a cluster algorithm defined based on an architecture model, characterized in that, It includes the following steps: Step 1: The data acquisition unit acquires the task objectives and task resources; Step 2: The architecture unit analyzes the task resources according to the task objectives and outputs an architecture model; Step 3: The algorithm training unit performs algorithm training according to the architecture model and outputs a cluster decision-making strategy and a task allocation plan; Step 4: The simulation evaluation unit conducts simulation deduction on the cluster decision-making strategy and evaluates the effect of the task allocation plan, and outputs a corresponding evaluation report; Step 5: The result output unit outputs the evaluation report.

Images