A complex system target multi-resource collaborative allocation method based on reinforcement learning

Abstract

The application discloses a complex system target multi-resource collaborative allocation method based on reinforcement learning. The application formalizes the multi-resource collaborative allocation problem of the complex system target as a single-step Markov decision process, outputs a complete "resource type - action point position" matching sequence in a single time step, converts a high-dimensional combined decision problem into a combined optimization problem through a structured discrete action space design, realizes rapid generation and global optimization of resource collaborative allocation, constructs a hierarchical function propagation model and embeds it into a reinforcement learning environment to form a strong closed-loop optimization structure of "resource allocation - function evaluation - reward feedback - strategy update", realizes layer-by-layer aggregation calculation of resource action effects through a logic propagation node and a function retention quantitative rule, and updates the overall function weakening effect and resource efficiency as optimization targets to improve the consistency between the allocation scheme and the overall function weakening target, and realize collaborative optimization between the function effect and the resource consumption.

Description

Technical Field

[0001] This invention relates to the field of complex system optimization and intelligent decision-making technology, specifically to a multi-resource collaborative allocation method for complex system objectives based on reinforcement learning. This invention is applicable to the problem of overall performance degradation and capability suppression of complex system objectives with multi-level functional structures, functional coupling relationships, and redundant design features. Background Technology

[0002] As large-scale engineering systems and networked systems become increasingly complex, modern complex systems typically consist of multiple functional modules, subsystems, and key components, with dependencies and propagation coupling effects existing between different levels. Under limited resources, how to rationally allocate various types of resource units to ensure that the overall function of the target reaches the expected level of weakening in the shortest possible time has become an important research problem in the field of intelligent decision-making.

[0003] Existing methods for allocating target resources in complex systems mainly include: manual experience and template matching, which rely on expert experience to set priorities and allocation rules, resulting in poor adaptability and difficulty in dealing with high-dimensional combinatorial problems; rule-based layer-by-layer allocation, which allocates resources layer by layer according to preset functional importance, lacking dynamic feedback and global optimization capabilities; enumeration search or traditional intelligent optimization algorithms, which use genetic algorithms, particle swarm optimization, etc. for combinatorial search, but under conditions of multiple resources and multiple action points, the combinatorial scale grows exponentially, resulting in high computational complexity; and decoupling optimization and performance evaluation methods, where the generation of allocation schemes and performance evaluation are independent of each other, resulting in weak feedback, slow convergence, and a tendency to get trapped in local optima.

[0004] Although deep learning methods have been introduced in recent years, problems such as step-by-step decision-making processes, insufficient propagation models, and weak optimization loops still exist. A collaborative allocation method for multiple resources in complex systems that can be applied in engineering has not yet been formed.

[0005] In summary, existing methods for allocating target resources in complex systems have the following shortcomings: low computational efficiency under conditions of multiple targets, multiple resource types, and multiple points of action; disconnect between function propagation assessment and resource allocation optimization, with the effectiveness assessment and resource allocation optimization processes mostly in an open-loop or weakly closed-loop state, resulting in a significant deviation between the optimization direction and the actual suppression effect. This makes it impossible to achieve global optimization based on feedback from the weakening of real effectiveness, and it is difficult to meet the needs of rapid, accurate, and dynamic suppression of high-value complex system targets under modern high-intensity confrontation conditions. Summary of the Invention

[0006] In view of this, the present invention provides a method for multi-resource collaborative allocation of complex system targets based on reinforcement learning. It can realize the rapid generation of multi-resource collaborative schemes of system targets in seconds under high-dimensional and multi-constraint conditions, establish a strong closed-loop feedback mechanism between performance evaluation and resource allocation optimization, effectively express multi-dimensional factors such as multi-resource collaboration, priority attack on key functional areas, utilization of spillover effects, and resource constraints, and complete the whole allocation in a single decision. At the same time, while maintaining efficient computation, it supports rapid response to dynamic situations and smooth expansion from single-stage rapid attack to multi-stage progressive damage.

[0007] The present invention provides a method for collaborative allocation of multiple resources for complex systems based on reinforcement learning, comprising:

[0008] S1, the complex system target is decomposed into a structured hierarchical model according to its function to obtain a functional propagation tree model; wherein, the propagation structure type and propagation operator between the upper and lower level nodes are determined according to the functional dependency relationship and coupling strength between the upper and lower level nodes. S2, Constructing a reinforcement learning interactive environment; The state space of reinforcement learning is a joint state composed of a functional state vector, a resource state vector, and candidate action point states. The functional state vector represents the current functional retention of nodes at each level of the complex system target. The functional retention of bottom-level nodes is calculated based on the action parameters of resource units and the action point information acting on that bottom-level node. The functional retention of nodes at other levels is calculated based on the functional propagation tree model constructed using S1. The resource state vector represents the types of resources currently available for allocation and their quantity constraints. The candidate action point states represent the spatial location information and importance characteristics of the action points acting on the bottom-level node structure. The action space is where the agent, for each candidate action point, either does not allocate or allocates a certain type of resource from the current resource state vector; The output is a resource allocation scheme, namely a matching sequence of "resource type - point of action"; The reward function is a comprehensive function that takes into account the overall functional weakening effect, the total resource consumption cost, the priority of key nodes, and the penalty for repeated allocation. S3, a reinforcement learning interactive environment built on top of S2, is used to train the policy network; S4 allocates resources and action points based on the trained policy network.

[0009] Preferably, in S1, the complex system objective is decomposed according to the following five-level structure: overall objective layer—functional requirement layer—subsystem layer—sub-objective layer—key component layer; Among them, the overall target layer represents the overall task capability entity to be weakened; the functional requirement layer represents the first-level functional unit divided according to task requirements; the subsystem layer represents the specific system structure that supports the implementation of related functions; the sub-target layer represents the functional sub-units within the subsystem that undertake specific functions; and the key component layer represents the smallest evaluable unit that constitutes the sub-target function.

[0010] Preferably, in step S1, the propagation structure type and propagation operator between upper and lower layer nodes are determined based on the functional dependency and coupling strength between the upper and lower layer nodes, specifically as follows: If the function of an upper-level node depends on the simultaneous fulfillment of the functions of multiple lower-level nodes, then a logical AND propagation structure is adopted between the upper and lower nodes, and the propagation operator is a logical AND gate. If the function of an upper-level node is contributed by the functions of multiple lower-level nodes and has substitutable or redundant characteristics, then a logical weighted propagation structure is adopted between the upper and lower nodes. The propagation operator is a logical weight gate, and the weight of each child node is determined according to its importance or contribution.

[0011] Preferably, in S2, the importance feature of the candidate action point is determined based on the importance or functional priority of the action point to its corresponding underlying node structure.

[0012] Preferably, the importance is obtained by back-mapping based on the functional propagation tree model in S1.

[0013] Preferably, in step S2, the reward function is:

[0014] in, The current allocation scheme represents the overall functional reduction of the complex system's objectives. The degree of functional reduction expected to be achieved. The total cost of resource consumption. For key node revenue, Penalties are assigned for repetitive or inefficient use.

[0015] Preferably, in S3, the policy network is trained using the Proximal Policy Optimization (PPO), Trust Region Policy Optimization (TRPO), Deep Deterministic Policy Gradient (DDPG), Soft Actor-Commentator (SAC) algorithm, Double Delay Deep Deterministic Policy Gradient (TD3) algorithm, or Policy Gradient Method (DQN).

[0016] Preferably, in S2, the candidate action points are selected from the key positions of the key bottom nodes in the complex system target.

[0017] Preferably, in step S2, the agent assigns a corresponding resource type to the candidate action point according to the matching rule between the action point and the resource type.

[0018] Preferably, in S4, the complex system target is allocated resources and action points multiple times. Each allocation is based on the policy network trained in S3. In the state space of reinforcement learning, the functional state vector is the functional retention degree of each node after the previous allocation; the resource state vector is the types and quantity constraints of the remaining resources that can be allocated after the previous allocation; and the candidate action point is the state of the remaining available candidate points after the previous allocation.

[0019] Beneficial effects: This invention formalizes the multi-resource collaborative allocation problem of complex system objectives into a single-step Markov decision process. Using the overall functional state of the objective and the resource inventory state as input features, it outputs a complete "resource type-action point" matching sequence within a single time step, rather than employing a multi-step time-series decision-making approach for each resource. This mechanism, through structured discrete action space design, transforms the high-dimensional combinatorial decision problem into a combinatorial optimization problem, achieving rapid generation and global optimization of resource collaborative allocation, reducing the number of decision steps and training complexity, improving decision stability and real-time response capability, and avoiding error accumulation and instability issues in multi-step decision-making processes. Simultaneously, through… A hierarchical function propagation model is constructed and embedded into a reinforcement learning environment to form a strong closed-loop optimization structure of "resource allocation - function evaluation - reward feedback - policy update". Through logical propagation nodes and function retention metric rules, the effect of resource utilization is aggregated and calculated from bottom to top. A multi-objective reward function is constructed based on the overall function reduction degree and resource cost, so that the policy network can be updated directly with the overall function reduction effect and resource efficiency as optimization objectives during training. This improves the consistency between the allocation scheme and the overall function reduction objective, realizes the synergistic optimization between function effect and resource consumption, and improves the policy convergence efficiency and global optimization capability. Attached Figure Description

[0020] Figure 1 A method and process for allocating multiple resources to complex system objectives.

[0021] Figure 2 This describes the process of generating allocation schemes based on reinforcement learning. Detailed Implementation

[0022] The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0023] This invention provides a method for multi-resource collaborative allocation of complex system targets based on reinforcement learning. This method is aimed at complex system targets with multi-level structures and functional coupling relationships. By constructing a target-specific functional propagation model, building a closed-loop reinforcement learning environment, formalizing the resource collaborative allocation decision process, and carrying out policy learning and real-time reasoning, it realizes the intelligent generation of multi-resource collaborative allocation schemes that meet the expected functional reduction requirements.

[0024] The flowchart of the method of this invention is as follows Figure 1 As shown, the specific steps include the following: Step 1: Perform structured hierarchical decomposition and functional propagation relationship modeling for the complex system objectives, construct a multi-level functional mapping model from the key component layer to the overall objective layer, establish a quantitative transfer relationship of functional retention between each level, and thus form a unified calculation framework for the overall functional state.

[0025] The functional propagation model is used to characterize the changes in the functional state of each key component after the resource is applied and its cumulative impact in the hierarchical structure, providing a calculable and repeatable mathematical basis for subsequent functional performance evaluation and reward calculation in reinforcement learning environments.

[0026] Step 101: First, systematically decompose the complex system objectives into hierarchical levels and construct a unified multi-level structural model. The complex system objectives are decomposed according to the following five-level structure: overall objective layer—functional requirement layer—subsystem layer—sub-objective layer—key component layer.

[0027] Among them: the overall target layer represents the overall task capability entity to be weakened; the functional requirement layer represents the first-level functional unit divided according to task requirements; the subsystem layer represents the specific system structure that supports the implementation of related functions; the sub-target layer represents the functional sub-units within the subsystem that undertake specific functions; and the key component layer represents the smallest evaluable unit that constitutes the sub-target function.

[0028] The functional requirements layer is abstractly divided according to specific application scenarios and task requirements, and may include, but is not limited to: core control and coordination functional units, task execution functional units, information acquisition and communication functional units, energy and power guarantee functional units, and comprehensive guarantee and support functional units. Each functional requirement unit does not correspond one-to-one with a subsystem layer, but can be achieved through the collaborative support of multiple subsystems. Through this hierarchical decomposition, the complex system objectives are abstracted from a holistic view into a quantifiable and associative multi-level functional structure, providing a structured representation basis for subsequent functional propagation modeling and resource effect calculation.

[0029] Step 102: After completing the hierarchical structure decomposition of the system, a functional propagation tree model is further constructed, and functional propagation rules between nodes at each level are defined to characterize the step-by-step transmission and accumulation process of functional state changes of key components in the hierarchical structure.

[0030] The functional propagation tree is a bottom-up hierarchical computational structure corresponding to the hierarchical structure in step 101. Its nodes are abstracted from functional units in the key component layer, sub-target layer, subsystem layer, functional requirement layer, and overall target layer. The nodes at each level are connected according to functional support relationships, dependency relationships, and functional coupling relationships. The functional propagation direction is from the key component layer to the overall target layer, which is used to characterize the impact of local functional impairment on the upper-level task capabilities.

[0031] Based on the dependencies and coupling strength between different functional units, configure corresponding propagation operators for different nodes.

[0032] When a higher-level function depends on multiple lower-level functions simultaneously, a logical AND-type propagation structure is used. Under this structure, a significant decrease in any critical sub-function will have a significant impact on the higher-level function. This type of node is suitable for functional modules with strong dependencies or serial support structures.

[0033] The formula for calculating a logic AND gate is: In the formula This is the current functional value of the AND-type node; For the first sub-layer i The functional status value of each functional element. n This represents the number of sub-level elements in the current AND gate layer.

[0034] When a certain upper-level function is contributed by multiple lower-level functions and has substitutable or redundant characteristics, a logical weighted propagation structure is adopted. Under this structure, each sub-function participates in the calculation of the upper-level function according to a preset weight, and the weight reflects its importance or contribution.

[0035] The formula for calculating logical weight gates is: In the formula This represents the functional value of the current weighted node. For the first sub-layer i The weight of each functional node.

[0036] Step 2: After completing the functional propagation model of the complex system objectives, in order to reduce the combination scale of the subsequent decision space, a limited set of candidate action points is constructed, and an initial resource matching candidate set is generated to realize the structured constraints and dimensionality reduction of the action space.

[0037] For key components and functional areas within a complex system target, based on the spatial geometric characteristics, structural distribution features, and protection level parameters of the overall physical structure of the target, the spatial location of the key components within the overall target is described using coordinates, and their corresponding spatial coordinates are defined as candidate action points. A finite set of candidate action points is generated within the target space, and the generation of these candidate action points satisfies the following principles: covering the main functional support areas; balancing the scope of resource impact and spatial distribution characteristics; controlling the number of candidates to avoid combinatorial explosion; and ensuring computability and feasibility.

[0038] Resource type adaptation analysis is performed on each candidate action point. For any candidate action point, the corresponding selectable action resource type must meet the preset resource adaptation rules. The resource adaptation rules are established based on factors such as the action radius of the resource unit, action accuracy parameters, energy or effectiveness attenuation model, target protection characteristics, action direction or spatial accessibility, and resource action mechanism matching relationship, and the applicability of resource types is evaluated in combination with historical matching experience models.

[0039] Taking a large building target as an example, the overall target can be a large integrated command building, whose key components include communication equipment rooms, power centers, central control rooms, data server areas, main load-bearing structural nodes, and underground energy security areas. The spatial coordinates of the above-mentioned key components in the overall building structure can be defined as candidate action points. When performing resource type adaptation analysis on each candidate action point, the selectable action resource types corresponding to different action points must meet the preset resource adaptation rules. For example, the data server room located in the core area inside the building is suitable for resource types with penetration or directional action capabilities due to strong structural obstruction and protective isolation; for the communication antenna area located on the top of the building, long-distance high-precision action resources can be prioritized; for the underground energy security area, the spatial accessibility of resources to the underground structure and the action attenuation characteristics need to be considered.

[0040] Only when a resource type meets the adaptation conditions and performance threshold requirements of the corresponding action point can it be included in the candidate resource set for that action point, thus forming a resource type-action point matching candidate set, which constitutes the basic structure of the subsequent reinforcement learning action space.

[0041] Step 3: After completing the construction of the candidate action point set and resource matching candidate set, this step embeds the function propagation model into the reinforcement learning interaction framework to build a simulation environment that can interact with the reinforcement learning agent in real time, providing a closed-loop function effect feedback channel for policy training.

[0042] The functional propagation tree model and functional level quantification criteria established in step one are embedded into the functional performance evaluation module to form a unified environmental computing core. The functional performance evaluation module receives resource allocation schemes and outputs the overall functional status evaluation results of the target.

[0043] A reinforcement learning interactive environment is constructed, comprising: a state input interface; an action parsing module; a functional performance evaluation module; a reward calculation module; and a state update module. This environment is used to simulate the change process of the target functional state of a complex system after resource allocation.

[0044] When the reinforcement learning agent outputs a resource allocation scheme (i.e., a "resource type - action point" matching sequence), the environment performs the following steps: (1) Calculate the degree of functional weakening of the target key component by each point of action based on the action parameters and action point information of the resource unit; (2) Based on the functional propagation tree structure, aggregate from the key component layer upwards layer by layer, and calculate the functional retention rate of each level according to the preset logical propagation rules; (3) Calculate the overall target's functional retention value Φ; (4) According to the functional level mapping criterion, the functional retention value Φ is converted into the corresponding functional reduction level D=1-Φ; (5) Generate reward signals based on the functional weakening results and feed them back to the reinforcement learning agent.

[0045] Step 4: The single-stage multi-resource collaborative allocation problem of complex system objectives is formalized into a single-step Markov decision process, and policy training is completed based on the reinforcement learning interactive environment constructed in Step 3.

[0046] To enable the agent to comprehensively consider the target state, resource constraints, and candidate action region information in a single decision, this invention designs the state space as a joint state composed of functional state, resource state, and candidate action point state, represented as:

[0047] in, For a complex system objective, the functional state vector is... This is the current resource state vector. This is the set of candidate action points.

[0048] The functional state vector is used to characterize the current functional retention status of nodes at each level of the complex system target, and can be represented as:

[0049] in, Indicates the first Functionality retention of each functional node This represents the total number of functional nodes. The functional retention rate of each functional node is calculated by the functional propagation model established in step one and updated in real time by the functional performance evaluation module in step three.

[0050] The resource state vector describes the types of resources currently available for allocation and their quantity constraints, and can be represented as:

[0051] in, Indicates the first The remaining quantity of class resources, Indicates the first The unit cost of this type of resource This represents the total number of resource types. By introducing a resource state vector, the agent can simultaneously consider resource availability and resource consumption constraints when making decisions.

[0052] The set of candidate action points describes the available action locations after screening in step two, and can be represented as:

[0053] in, Indicates the first Candidate action points This represents the total number of candidate action points. Furthermore, each candidate action point can be represented by its spatial location and importance characteristics as follows:

[0054] in,( , , () represents the spatial coordinates of the candidate point of action. This indicates the structural importance or functional priority corresponding to the candidate action point. The importance can be obtained by reverse mapping from the functional propagation tree model in step one, that is, the points in the region corresponding to the key functional nodes have higher importance.

[0055] To enable the agent to output a complete allocation scheme in one go, this invention designs the action space as a two-layer discrete structure of candidate action points and resource type selection. Specifically, for each candidate action point, the agent must select a resource type from a preset resource type set, or choose not to allocate any resource. Therefore, the complete action can be represented as:

[0056] in, Indicates the first Action selection for each candidate action point satisfies:

[0057] in, =0 indicates that for the first... No resources are allocated to each candidate action point. = Indicates the first The allocation of the candidate action point Resource types. Through the above action representation method, the resource selection results at all candidate action points together constitute a multi-dimensional discrete allocation vector, thus representing the complete resource collaborative allocation scheme.

[0058] The reward function is used to guide the reinforcement learning agent to learn the optimal resource collaborative allocation strategy. Unlike traditional methods that only consider a single performance indicator, this invention incorporates the overall function reduction effect, the priority of key nodes, resource consumption, and the penalty for repeated allocation into the reward function, making the policy learning objective more in line with the actual needs of the collaborative allocation of target resources in complex systems.

[0059] Let the overall functional reduction of the complex system objective caused by the current allocation scheme be . The expected functional reduction is The total cost of resource consumption is The key node's revenue is The penalty for duplicate or inefficient allocation is The reward function is then defined as:

[0060] in, 1. 2. 1. These are the weighting coefficients.

[0061] In the above reward function, the first term 1min , () represents an overall functionality reduction reward, used to encourage agents to prioritize generating resource allocation schemes that significantly reduce overall functionality retention; when The target reduction level has not yet been reached. At that time, the award followed This increases the policy network, thus optimizing it towards a stronger weakening effect.

[0062] Second item 2 This represents the benefit of key nodes, used to incentivize resources to prioritize critical components that have a significant impact on overall functionality. The benefit of a key node can be expressed as:

[0063] in, Indicates the first Whether each candidate point of action is selected, and if selected =1, otherwise =0. This option enables the policy network to not only focus on the overall weakening effect, but also to automatically learn to prioritize the allocation of key functional areas.

[0064] Third item 1C represents the resource consumption cost item. The total resource consumption cost can be expressed as:

[0065] in, Indicates the first The number of resources used. Indicates the first The unit cost of a resource class. This item is used to constrain the total amount of resources used, preventing the policy network from overusing resources in pursuit of high weakening power.

[0066] Fourth item This represents a penalty for duplicate or inefficient allocation, used to prevent multiple resources from being concentrated in low-yield areas or causing ineffective overlap. The penalty for duplicate allocation can be expressed as:

[0067] in, ( , ) indicates the first The point and the first Does the effective area of ​​each point have invalid overlap? If duplicate coverage exists, the corresponding penalty will increase.

[0068] The policy network is trained using the Proximal Policy Optimization (PPO) algorithm.

[0069] The PPO algorithm limits the policy update magnitude by shearing the objective function, and its optimization objective is:

[0070] These are the parameters of the policy network, i.e., the parameters of the neural network that need to be trained and updated.

[0071] The expectation operator is used to calculate the average of the sampled state-action data.

[0072] The strategy probability ratio measures the change in the probability of action selection between the old and new strategies. It is defined as follows: ,in The probability of the policy network choosing an action in the current state; This represents the probability of the old policy network choosing an action in a given state.

[0073] The advantage function is used to measure the superiority of the current action relative to the average policy. This indicates that the action is better than average.

[0074] This is a shearing function used to limit the magnitude of policy updates and prevent excessive differences between the old and new policies. This is the shearing threshold parameter, used to control the policy update range, and is typically set to 0.1. 0.2.

[0075] After multiple rounds of iterative training, the policy network can output a complete resource collaborative allocation scheme for any given complex system target state.

[0076] Step 5: Input the real-time state information of the current complex system target into the trained policy network, and generate a multi-resource collaborative allocation scheme through a single forward inference process. The real-time state information includes: the current available resource inventory vector; the functional retention information of each level of the complex system target; the target functional weakening requirements at the current stage; and the set of candidate action points and their spatial structure characteristics. After receiving the above state input, the policy network outputs a complete "resource type - action point" matching sequence through a single forward computation, forming a single-stage resource collaborative allocation scheme. This inference process does not require further iterative optimization or multi-step search; it only relies on the trained policy parameters to make decisions, thereby achieving rapid generation of resource collaborative allocation schemes.

[0077] Step Six: Building upon the single-stage resource collaborative allocation scheme, further expand it into a multi-stage progressive optimization mechanism to achieve dynamic, phased functional reduction planning for complex system objectives. In multi-stage scenarios, proceed as follows: State progression update mechanism: The functional evaluation results after the previous stage are used as the function retention update vectors of each level of the complex system target as the initial state input for the next stage decision.

[0078] Policy reuse mechanism: The same trained policy network is used continuously in each stage, and a new resource collaborative allocation scheme is regenerated based on the updated state information without rebuilding the model or changing the decision structure.

[0079] Phase loop mechanism: Repeat the process of "state input - resource allocation generation - function evaluation - state update" until any of the following conditions are met: the preset target function weakening threshold is reached; the maximum number of phases is reached; the resource inventory is exhausted or other termination constraints are met.

[0080] To ensure the overall consistency of multi-stage planning and the rational use of resources, the present invention further includes: State writing mechanism: After each stage, the latest function retention vector calculated by the function propagation model is written into the state space of the next stage to ensure the continuity and traceability of the decision-making process.

[0081] Global resource constraint mechanism: Resource inventory is cumulatively deducted across stages to maintain the total global resource limit and avoid duplicate counting or over-allocation between stages.

[0082] Target threshold monitoring mechanism: Real-time monitoring of whether the overall function retention value reaches the preset function weakening threshold to determine whether to terminate the phase cycle.

[0083] Step Seven: Define a unified damage assessment standard, mapping the overall functional retention values ​​calculated at each stage to a level of damage understandable to combat personnel. For example, ≥ 0.90 indicates basically intact, 0.70 ≤ <0.90 indicates minor damage, 0.40 ≤ <0.70 indicates moderate damage. <0.40 indicates severe damage.

[0084] In summary, the above are merely preferred embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for collaborative allocation of multiple resources for a complex system based on reinforcement learning, characterized in that, include: S1, the complex system target is decomposed into a structured hierarchical model according to its function to obtain a functional propagation tree model; wherein, the propagation structure type and propagation operator between the upper and lower level nodes are determined according to the functional dependency relationship and coupling strength between the upper and lower level nodes. S2, Constructing a reinforcement learning interactive environment; The state space of reinforcement learning is a joint state composed of a functional state vector, a resource state vector, and candidate action point states. The functional state vector represents the current functional retention of nodes at each level of the complex system target. The functional retention of bottom-level nodes is calculated based on the action parameters of resource units and the action point information acting on that bottom-level node. The functional retention of nodes at other levels is calculated based on the functional propagation tree model constructed using S1. The resource state vector represents the types of resources currently available for allocation and their quantity constraints. The candidate action point states represent the spatial location information and importance characteristics of the action points acting on the bottom-level node structure. The action space is where the agent, for each candidate action point, either does not allocate or allocates a certain type of resource from the current resource state vector; The output is a resource allocation scheme, i.e., a matching sequence of "resource type - point of action"; The reward function is a comprehensive function that takes into account the overall functional weakening effect, the total resource consumption cost, the priority of key nodes, and the penalty for repeated allocation. S3, a reinforcement learning interactive environment built on top of S2, is used to train the policy network; S4 allocates resources and action points based on the trained policy network.

2. The method as described in claim 1, characterized in that, In S1, the complex system objectives are decomposed according to the following five-level structure: overall objective layer — functional requirement layer — subsystem layer — sub-objective layer — key component layer; Among them, the overall target layer represents the overall task capability entity to be weakened; the functional requirement layer represents the first-level functional unit divided according to task requirements; the subsystem layer represents the specific system structure that supports the implementation of related functions; the sub-target layer represents the functional sub-units within the subsystem that undertake specific functions; and the key component layer represents the smallest evaluable unit that constitutes the sub-target function.

3. The method as described in claim 1 or 2, characterized in that, In step S1, the propagation structure type and propagation operator between upper and lower layer nodes are determined based on the functional dependency and coupling strength between them, specifically as follows: If the function of an upper-level node depends on the simultaneous fulfillment of the functions of multiple lower-level nodes, then a logical AND propagation structure is adopted between the upper and lower nodes, and the propagation operator is a logical AND gate. If the function of an upper-level node is contributed by the functions of multiple lower-level nodes and has substitutable or redundant characteristics, then a logical weighted propagation structure is adopted between the upper and lower nodes. The propagation operator is a logical weight gate, and the weight of each child node is determined according to its importance or contribution.

4. The method as described in claim 1, characterized in that, In S2, the importance feature of the candidate action point is determined based on the importance or functional priority of the action point to the corresponding underlying node structure.

5. The method as described in claim 4, characterized in that, The importance is obtained by reverse mapping based on the functional propagation tree model in S1.

6. The method as described in claim 1, characterized in that, In S2, the reward function is: in, The current allocation scheme represents the overall functional reduction of the complex system's objectives. The degree of functional reduction expected to be achieved. The total cost of resource consumption. For key node revenue, Penalties are assigned for repetitive or inefficient use.

7. The method as described in claim 1, characterized in that, In S3, the policy network is trained using the near-end policy optimization algorithm, the trust region policy optimization algorithm, the deep deterministic policy gradient algorithm, the soft actor-commentator algorithm, the double-delay deep deterministic policy gradient TD3 algorithm, or the policy gradient method DQN.

8. The method as described in claim 1, characterized in that, In S2, candidate action points are selected from the key positions of key bottom nodes in the complex system target.

9. The method as described in claim 1 or 8, characterized in that, In step S2, the agent assigns the corresponding resource type to the candidate action point according to the matching rules between the action point and the resource type.

10. The method as described in claim 1, characterized in that, In S4, resources and action points are allocated multiple times for the complex system target. Each allocation is based on the policy network trained in S3. In the state space of reinforcement learning, the functional state vector is the functional retention degree of each node after the previous allocation; the resource state vector is the types and quantity constraints of the remaining resources that can be allocated after the previous allocation; and the candidate action point is the state of the remaining available candidate points after the previous allocation.

Images