A Multi-Domain Multi-Level Cross-Chain Network Collaboration Method and Device

Abstract

The present invention discloses a multi-domain and multi-level cross-chain network collaboration method and device, which includes: obtaining cross-chain network collaboration information, and calculating the cross-chain weight value of each cross-chain network for other cross-chain networks based on the collaboration information; dividing the cross-chain network into regions and levels according to the collaboration information to form multi-level activity functions, element sets and multi-region activity functions and element sets; treating each region as a multi-agent, using multi-agents to link the activity functions and element sets of multiple regions in the cross-chain network to form a multi-agent cross-chain network cluster divided by domains; inputting the multi-agent cross-chain network cluster divided by domains into a trained multi-agent model to obtain the optimal domain weight value of the cross-chain network in each region; inputting the multi-level activity functions and element sets into a trained hierarchical reinforcement learning model to obtain the optimal layer weight value of the cross-chain network at each layer; and determining the final cross-chain network collaboration result.

Description

Technical Field

[0001] The present invention belongs to the field of blockchain cross-chain network collaboration technology, and relates to a multi-domain and multi-level cross-chain network collaboration method and device. Background Art

[0002] Essentially, blockchain is a shared database. The data or information stored within it is characterized by being unforgeable, traceable, open and transparent, and collectively maintained. Based on these characteristics, blockchain technology lays a solid foundation of trust, creates a reliable mechanism for collaboration, and has broad application prospects.

[0003] Among the many challenges facing blockchain, the lack of interoperability between chains significantly limits its application. Cross-chain operations primarily encompass two application scenarios: information interoperability and value interoperability. The rigorous description, standardized implementation, and widespread application of cross-chain interoperability protocols will be key to realizing the "Internet of Value." Cross-chain interoperability technology enables information exchange and value transfer between homogeneous and heterogeneous blockchains, meeting the business scalability requirements of blockchain applications.

[0004] Traditional blockchain cross-chain network collaboration, such as the notary mechanism, the introduction of side chains, relay chains, and hash locking, all conduct cross-chain interactions by determining the type of business collaboration request and whether it meets the execution conditions of the smart contract.

[0005] The existing technology has the following defects: when the cross-chain network is multi-layered and multi-regional, these methods will not be able to dynamically track changes in the actual situation, and will not be able to correctly perform cross-chain collaborative operations, which will cause anomalies. Therefore, the cross-chain network collaboration method needs to be improved. Summary of the Invention

[0006] Purpose: To address the problems and needs of existing technologies, a multi-domain, multi-level cross-chain network collaboration system and method based on multi-agent hierarchical reinforcement learning is introduced. Based on the multi-level cross-chain network, a cross-chain system functional collaboration model with hierarchical reinforcement learning is established to build a two-way collaborative order. A task- and demand-driven funnel-type two-way multi-level hierarchical reinforcement learning strategy is adopted to optimize the performance of the cross-chain system from top to bottom to collaborate on multi-dimensional functions, and to respond to cross-chain business needs and collaborate on multi-dimensional functions from bottom to top. At the same time, multi-agents are used to link the multi-region activity functions and element sets in the cross-chain network, and the collaborative actions and rewards of the agents are designed. The collaborative agents are trained using centralized learning and decentralized decision-making to achieve convenient and high-performance collaboration under the cross-chain system group collaboration.

[0007] Technical solution: To solve the above technical problems, the technical solution adopted by the present invention is:

[0008] In a first aspect, a multi-domain, multi-level, cross-chain network collaboration method is provided, comprising:

[0009] Obtain cross-chain network collaboration information, where the collaboration information includes collaboration data, the level number of the blockchain where the collaboration data is located, collaboration type, location, number of cross-chain interactions, basic weight, and business requirements;

[0010] Calculate the cross-chain weight H of each cross-chain network relative to other cross-chain networks based on the number of cross-chain interactions and basic weight in the collaborative information of each cross-chain network;

[0011] Divide the cross-chain network into regions and levels based on the collaborative information, and classify and divide it based on cross-chain business needs and the actual regions where the cross-chain network is located, to form multi-level activity functions and element sets and multi-region activity functions and element sets;

[0012] Treat each region as a multi-agent, and use multi-agents to link the activity functions and element sets of multiple regions in the cross-chain network to form a multi-agent cross-chain network cluster divided by domains;

[0013] Input the domain-divided multi-agent cross-chain network cluster into the trained multi-agent model, and obtain the optimal domain weight value of the cross-chain network in each region based on the output of the multi-agent model;

[0014] Input the multi-level activity functions and element sets into the trained hierarchical reinforcement learning model, and obtain the optimal layer weight value of each layer of the cross-chain network based on the output of the multi-agent model;

[0015] The final cross-chain network collaboration result is determined based on the optimal domain weight value of the cross-chain network in each region and the optimal layer weight value of the cross-chain network at each layer.

[0016] In some embodiments, the cross-chain weight value H of each cross-chain network relative to other cross-chain networks is calculated based on the number of cross-chain interactions and the basic weight in the collaborative information of each cross-chain network, including:

[0017]

[0018]

[0019]

[0020] Where f(a) is the cross-chain frequency auxiliary function, g(b) is the weight auxiliary function; a is the number of interactions of cross-chain network x with other cross-chain networks, p is the sum of the number of interactions of cross-chain network x itself and the number of interactions initiated by it; k is the basic weight of x, and b is the sum of the weights of the cross-chain networks interacted with by x.

[0021] In some embodiments, the method for constructing the multi-agent model includes:

[0022] The multi-agent model adopts the MAPPO algorithm and the actor-critic architecture, which is a centralized value function.

[0023] Adopt centralized learning and decentralized decision-making;

[0024] Each agent has its own policy network. During training, there is a central controller that collects observations, actions, and rewards from all agents. The central controller helps the agents train their policy networks. After training, each agent makes decisions based on its own observations and using its own policy network, without communicating with the central controller.

[0025] The system uses n agents, each of which has a policy network. The agent interacts with the environment, obtains information elements from the environment, and makes its own decisions to obtain actions a. i , then execute a i ,During training, a central controller is required, and the agent communicates with the central controller,,sending actions, observations and rewards to the central controller;

[0026] The central controller has information about all agents and n value networks Critic i ,Each value network corresponds to an intelligent agent, which trains the value network in the central network and updates the training results using the temporal difference algorithm;

[0027] After training, there is no need for a central controller. Each agent interacts with the environment independently. The agent observes the environment from the local environment. i Each agent has its own decision network, which takes observation as input, and the policy network outputs an action probability distribution, and selects action a according to the probability distribution. i , each agent performs its own action a i , then the environment changes state.

[0028] In some embodiments, the method for constructing the hierarchical reinforcement learning model includes:

[0029] The hierarchical reinforcement learning model adopts the Option-Critic architecture;

[0030] Establish goal-based hierarchical reinforcement learning,

[0031] The upper-layer blockchain network constructs the upper-layer controller. The training principle is to make the output goal satisfy the semantic information we set as much as possible, that is, to make the goal conform to the best state / state transition direction as much as possible.

[0032] The lower-level blockchain network constructs the lower-level controller, and the training principle of the lower-level controller is to make the environment state transition meet the goal as much as possible;

[0033] To this end, the upper blockchain network will pass an internal reward to the lower blockchain network to drive the lower blockchain network to achieve its goals, while the upper blockchain network is usually trained based on external environmental rewards;

[0034] Therefore, the upper-layer blockchain network and the lower-layer blockchain network are actually trained separately. By assigning certain semantic information to the goal and designing reasonable internal rewards based on this semantic information, even when external rewards are very sparse, it is possible to ensure that the lower-layer blockchain network obtains certain internal incentives, thereby enabling the intelligent agent to learn the optimal strategy more efficiently and improve the efficiency of sample utilization.

[0035] In some embodiments, based on the optimal domain weight value of the cross-chain network in each region and the optimal layer weight value of the cross-chain network at each layer, the final cross-chain network coordination result is determined, including:

[0036]

[0037] Result is the cross-chain network collaboration result, H s 、H L are the optimal domain weight value and optimal layer weight value of each cross-chain network respectively, and H is the cross-chain weight value of a cross-chain network for other cross-chain networks.

[0038] In a second aspect, the present invention provides a multi-domain, multi-level, cross-chain network collaboration device, comprising:

[0039] A collaboration information acquisition module is configured to: acquire cross-chain network collaboration information, wherein the collaboration information includes collaboration data, the level number of the blockchain where the collaboration data is located, the collaboration type, the region where it is located, the number of cross-chain interactions, the basic weight, and the business requirements;

[0040] The cross-chain weight value calculation module is configured to: calculate the cross-chain weight value H of each cross-chain network relative to other cross-chain networks based on the number of cross-chain interactions and basic weight in the collaborative information of each cross-chain network;

[0041] A cross-chain network partitioning module is configured to: divide the cross-chain network into regions and levels according to the collaborative information, classify and divide the cross-chain network based on cross-chain business needs and the actual region where the cross-chain network is located, and form multi-level activity functions and element sets and multi-region activity functions and element sets;

[0042] The multi-agent link module is configured to treat each region as a multi-agent, and use multi-agents to link the activity functions and element sets of multiple regions in the cross-chain network to form a multi-agent cross-chain network cluster divided by domains;

[0043] The first cross-chain network collaborative decision module is configured to: input the domain-divided multi-agent cross-chain network cluster into a trained multi-agent model, and obtain the optimal domain weight value of the cross-chain network in each region based on the output of the multi-agent model;

[0044] The second cross-chain network collaborative decision-making module is configured to: input the multi-level activity functions and element sets into the trained hierarchical reinforcement learning model, and obtain the optimal layer weight value of each layer of the cross-chain network based on the output of the multi-agent model;

[0045] The collaboration result acquisition module is configured to determine the final cross-chain network collaboration result based on the optimal domain weight value of the cross-chain network in each region and the optimal layer weight value of the cross-chain network at each layer.

[0046] In a third aspect, the present invention provides a computer device including a processor and a storage medium;

[0047] The storage medium is used to store instructions;

[0048] The processor is configured to operate according to the instructions to perform the steps according to the method.

[0049] In a fourth aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, which implements the steps of the method described in the first aspect when executed by a processor.

[0050] Beneficial effects: The multi-domain, multi-level, cross-chain network collaboration method and device provided by the present invention have the following advantages: by utilizing the interactivity and decision-making capabilities of hierarchical reinforcement learning and the perception capabilities of multiple agents, a solution is designed that can perform cross-chain collaboration based on multi-level and multi-regional blockchain networks, thereby achieving high-performance interactive collaboration under cross-chain system group collaboration. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 Flowchart of a multi-domain, multi-level, cross-chain network collaboration method according to one embodiment of the present invention. DETAILED DESCRIPTION

[0052] The present invention will be further described below in conjunction with the accompanying drawings and examples. The following examples are only used to more clearly illustrate the technical solutions of the present invention and are not intended to limit the scope of protection of the present invention.

[0053] In the description of the present invention, "several" means more than one, "plurality" means more than two, "greater than," "less than," and "exceed" are understood to exclude the number itself, while "above," "below," and "within" are understood to include the number itself. The use of "first" and "second" in the description is solely for the purpose of distinguishing technical features and should not be construed as indicating or implying relative importance, implicitly specifying the number of the indicated technical features, or implicitly specifying the order of the indicated technical features.

[0054] In the description of the present invention, reference to terms such as "one embodiment," "some embodiments," "illustrative embodiments," "examples," "specific examples," or "some examples" means that the specific features, structures, materials, or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the exemplary expressions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.

[0055] Example 1

[0056] like Figure 1 As shown, a multi-domain, multi-level, cross-chain network collaboration method includes:

[0057] Obtain cross-chain network collaboration information, where the collaboration information includes collaboration data, the level number of the blockchain where the collaboration data is located, collaboration type, location, number of cross-chain interactions, basic weight (e.g., transaction volume in a transaction business), and business requirements;

[0058] Calculate the cross-chain weight H of each cross-chain network relative to other cross-chain networks based on the number of cross-chain interactions and basic weight in the collaborative information of each cross-chain network;

[0059] Divide the cross-chain network into regions and levels based on the collaborative information, and classify and divide it based on cross-chain business needs and the actual regions where the cross-chain network is located, to form multi-level activity functions and element sets and multi-region activity functions and element sets;

[0060] Treat each region as a multi-agent, and use multi-agents to link the activity functions and element sets of multiple regions in the cross-chain network to form a multi-agent cross-chain network cluster divided by domains;

[0061] Input the domain-divided multi-agent cross-chain network cluster into the trained multi-agent model, and obtain the optimal domain weight value of the cross-chain network in each region based on the output of the multi-agent model;

[0062] Input the multi-level activity functions and element sets into the trained hierarchical reinforcement learning model, and obtain the optimal layer weight value of each layer of the cross-chain network based on the output of the multi-agent model;

[0063] The final cross-chain network collaboration result is determined based on the optimal domain weight value of the cross-chain network in each region and the optimal layer weight value of the cross-chain network at each layer.

[0064] In some embodiments, the cross-chain weight value H of each cross-chain network relative to other cross-chain networks is calculated based on the number of cross-chain interactions and the basic weight in the collaborative information of each cross-chain network, including:

[0065]

[0066]

[0067]

[0068] Where f(a) is the cross-chain frequency auxiliary function, g(b) is the weight auxiliary function; a is the number of interactions of cross-chain network x with other cross-chain networks, p is the sum of the number of interactions of cross-chain network x itself and the number of interactions initiated by it; k is the basic weight of x, and b is the sum of the weights of the cross-chain networks interacted with by x.

[0069] In some embodiments, the method for constructing the multi-agent model includes:

[0070] The multi-agent model adopts the MAPPO algorithm and the actor-critic architecture, which is a centralized value function.

[0071] Adopt centralized learning and decentralized decision-making;

[0072] Each agent has its own policy network. During training, there is a central controller that collects observations, actions, and rewards from all agents. The central controller helps the agents train their policy networks. After training, each agent makes decisions based on its own observations and using its own policy network, without communicating with the central controller.

[0073] The system uses n agents, each of which has a policy network. The agent interacts with the environment, obtains information elements from the environment, and makes its own decisions to obtain actions a. i , then execute a i ,During training, a central controller is required, and the agent communicates with the central controller,,sending actions, observations and rewards to the central controller;

[0074] The central controller has information about all agents and n value networks Critic i,Each value network corresponds to an intelligent agent, which trains the value network in the central network and updates the training results using the temporal difference algorithm;

[0075] After training, there is no need for a central controller. Each agent interacts with the environment independently. The agent observes the environment from the local environment. i Each agent has its own decision network, which takes observation as input, and the policy network outputs an action probability distribution, and selects action a according to the probability distribution. i , each agent performs its own action a i , then the environment changes state.

[0076] During the learning process of the intelligent agent, the intelligent agent will first observe the state of the current environment, then take actions based on its own observations and strategies, and obtain rewards in the environment. Finally, it will update its own strategy through the time difference algorithm and maximize the cumulative rewards. That is, each round of training updates its own cross-chain weight value.

[0077] Given that agents need to interact simultaneously with the environment and other agents within it, and that when an agent makes a decision, other agents are also taking action, it is difficult to obtain a stable, optimal strategy. Furthermore, multi-agent environments are prone to Markov failures in non-stationary states, making it difficult to guarantee convergence when directly applying single-agent reinforcement learning to a multi-agent environment. Therefore, this model employs the MAPPO algorithm and an actor-critic architecture, which is a centralized value function. Each blockchain agent no longer requires a central controller and interacts independently with the environment, ensuring better convergence performance and sample complexity.

[0078] In some embodiments, the method for constructing the hierarchical reinforcement learning model includes:

[0079] The hierarchical reinforcement learning model adopts the Option-Critic architecture;

[0080] Establish goal-based hierarchical reinforcement learning,

[0081] The upper-layer blockchain network constructs the upper-layer controller. The training principle is to make the output goal satisfy the semantic information we set as much as possible, that is, to make the goal conform to the best state / state transition direction as much as possible.

[0082] The lower-level blockchain network constructs the lower-level controller, and the training principle of the lower-level controller is to make the environment state transition meet the goal as much as possible;

[0083] To this end, the upper blockchain network will pass an internal reward to the lower blockchain network to drive the lower blockchain network to achieve its goals, while the upper blockchain network is usually trained based on external environmental rewards;

[0084] Therefore, the upper-layer blockchain network and the lower-layer blockchain network are actually trained separately. By assigning certain semantic information to the goal and designing reasonable internal rewards based on this semantic information, even when external rewards are very sparse, it is possible to ensure that the lower-layer blockchain network obtains certain internal incentives, thereby enabling the intelligent agent to learn the optimal strategy more efficiently and improve the efficiency of sample utilization.

[0085] Considering the generalization ability and environmental adaptability, this hierarchical reinforcement learning model construction module adopts the Option-Critic framework. It does not require the design of subtasks for the environment and regards the hierarchical blockchain network as a training agent, that is, a computing entity.

[0086] The agent starts from the initial state and selects an option, that is, the cross-chain weight value, and executes the strategy Π of the option until the termination condition is met. It then exits the current option to perform an action or select a new option, and continues to enter the cycle selection until the training task is completed.

[0087] In some embodiments, based on the optimal domain weight value of the cross-chain network in each region and the optimal layer weight value of the cross-chain network at each layer, the final cross-chain network coordination result is determined, including:

[0088]

[0089] Result is the cross-chain network collaboration result, H s 、H L are the optimal domain weight value and optimal layer weight value of each cross-chain network respectively, and H is the cross-chain weight value of a cross-chain network for other cross-chain networks.

[0090] Example 2

[0091] In a second aspect, this embodiment provides a multi-domain, multi-level, cross-chain network collaboration device, including:

[0092] A collaboration information acquisition module is configured to: acquire cross-chain network collaboration information, wherein the collaboration information includes collaboration data, the level number of the blockchain where the collaboration data is located, the collaboration type, the region where it is located, the number of cross-chain interactions, the basic weight, and the business requirements;

[0093] The cross-chain weight value calculation module is configured to: calculate the cross-chain weight value H of each cross-chain network relative to other cross-chain networks based on the number of cross-chain interactions and basic weight in the collaborative information of each cross-chain network;

[0094] A cross-chain network partitioning module is configured to: divide the cross-chain network into regions and levels according to the collaborative information, classify and divide the cross-chain network based on cross-chain business needs and the actual region where the cross-chain network is located, and form multi-level activity functions and element sets and multi-region activity functions and element sets;

[0095] The multi-agent link module is configured to treat each region as a multi-agent, and use multi-agents to link the activity functions and element sets of multiple regions in the cross-chain network to form a multi-agent cross-chain network cluster divided by domains;

[0096] The first cross-chain network collaborative decision module is configured to: input the domain-divided multi-agent cross-chain network cluster into a trained multi-agent model, and obtain the optimal domain weight value of the cross-chain network in each region based on the output of the multi-agent model;

[0097] The second cross-chain network collaborative decision-making module is configured to: input the multi-level activity functions and element sets into the trained hierarchical reinforcement learning model, and obtain the optimal layer weight value of each layer of the cross-chain network based on the output of the multi-agent model;

[0098] The collaboration result acquisition module is configured to determine the final cross-chain network collaboration result based on the optimal domain weight value of the cross-chain network in each region and the optimal layer weight value of the cross-chain network at each layer.

[0099] Example 3

[0100] In a third aspect, the present invention provides a computer device including a processor and a storage medium;

[0101] The storage medium is used to store instructions;

[0102] The processor is configured to operate according to the instructions to perform the steps according to the method.

[0103] Example 4

[0104] In a third aspect, this embodiment provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of the method described in Example 1 are implemented.

[0105] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0106] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the steps in the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0107] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0108] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0109] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A multi-domain, multi-level, cross-chain network collaboration method, characterized by: include: Obtain cross-chain network collaboration information, where the collaboration information includes collaboration data, the level number of the blockchain where the collaboration data is located, collaboration type, location, number of cross-chain interactions, basic weight, and business requirements; Calculate the cross-chain weight H of each cross-chain network relative to other cross-chain networks based on the number of cross-chain interactions and basic weight in the collaborative information of each cross-chain network; Divide the cross-chain network into regions and levels based on the collaborative information, and classify and divide it based on cross-chain business needs and the actual regions where the cross-chain network is located, to form multi-level activity functions and element sets and multi-region activity functions and element sets; Treat each region as a multi-agent, and use multi-agents to link the activity functions and element sets of multiple regions in the cross-chain network to form a multi-agent cross-chain network cluster divided by domains; Input the domain-divided multi-agent cross-chain network cluster into the trained multi-agent model, and obtain the optimal domain weight value of the cross-chain network in each region based on the output of the multi-agent model; Input the multi-level activity functions and element sets into the trained hierarchical reinforcement learning model, and obtain the optimal layer weight value of each layer of the cross-chain network based on the output of the multi-agent model; Determine the final cross-chain network coordination result based on the optimal domain weight value of the cross-chain network in each region and the optimal layer weight value of the cross-chain network at each layer; Among them, according to the number of cross-chain interactions and basic weight in each cross-chain network collaboration information, its cross-chain weight value H for other cross-chain networks is calculated, including: Where f(a) is the cross-chain frequency auxiliary function, g(b) is the weight auxiliary function; a is the number of interactions of cross-chain network x with other cross-chain networks, p is the sum of the number of interactions of cross-chain network x itself and the number of interactions initiated by it; k is the basic weight of x, and b is the sum of the weights of the cross-chain networks interacted with by x; Among them, the construction method of the multi-agent model includes: the multi-agent model adopts the MAPPO algorithm, the actor-critic architecture, and is a centralized value function; centralized learning and decentralized decision-making are adopted; each agent has its own strategy network, and there is a central controller during training. The central controller collects the observations, actions, and rewards of all agents, and the central controller helps the agents train the strategy network. After the training is completed, each agent makes decisions based on its own observations and uses its own strategy network without communicating with the central controller; the system uses n agents, and each agent Actor has a strategy network. The agent interacts with the environment, obtains information elements from the environment, and makes decisions on its own to obtain action a i , then execute a i , a central controller is required during training, and the agent communicates with the central controller and sends actions, observations and rewards to the central controller; the central controller has information about all agents and n value networks Critic i Each value network corresponds to an agent, which trains the value network in the central network and uses the time difference algorithm to update the training results; after the training is completed, there is no need for a central controller, and each agent interacts with the environment independently. The agent observes the environment from the local environment. i Each agent has its own decision network, which takes observation as input, and the policy network outputs an action probability distribution, and selects action a according to the probability distribution. i , each agent performs its own action a i , then the environment changes state; Among them, the construction method of the hierarchical reinforcement learning model includes: the hierarchical reinforcement learning model adopts the Option-Critic architecture; establishing hierarchical reinforcement learning based on goal, the upper blockchain network constructs an upper controller, and the training criterion is to make the output goal satisfy the semantic information we set as much as possible, that is, to make the goal conform to the better state / state transition direction as much as possible; the lower blockchain network constructs a lower controller, and the training criterion of the lower controller is to make the environment state transition satisfy the goal as much as possible; to this end, the upper blockchain network will pass an internal reward to the lower blockchain network to drive the lower blockchain network to complete the goal, and the upper blockchain network is usually trained based on external environment rewards; therefore, the upper blockchain network and the lower blockchain network are actually trained separately; by giving the goal certain semantic information and designing reasonable internal rewards based on this semantic information, even when the external rewards are very sparse, it can ensure that the lower blockchain network obtains certain internal incentives, so that the intelligent agent can learn the optimal strategy more efficiently and improve the efficiency of sample utilization; Among them, based on the optimal domain weight value of the cross-chain network in each region and the optimal layer weight value of the cross-chain network at each layer, the final cross-chain network coordination result is determined, including: Result is the cross-chain network collaboration result, H s 、H L are the optimal domain weight value and optimal layer weight value of each cross-chain network respectively, and H is the cross-chain weight value of a cross-chain network for other cross-chain networks.

2. A multi-domain, multi-level, cross-chain network collaboration device, characterized in that: include: A collaboration information acquisition module is configured to: acquire cross-chain network collaboration information, wherein the collaboration information includes collaboration data, the level number of the blockchain where the collaboration data is located, the collaboration type, the region where it is located, the number of cross-chain interactions, the basic weight, and the business requirements; The cross-chain weight value calculation module is configured to: calculate the cross-chain weight value H of each cross-chain network relative to other cross-chain networks based on the number of cross-chain interactions and basic weight in the collaborative information of each cross-chain network; A cross-chain network partitioning module is configured to: divide the cross-chain network into regions and levels according to the collaborative information, classify and divide the cross-chain network based on cross-chain business needs and the actual region where the cross-chain network is located, and form multi-level activity functions and element sets and multi-region activity functions and element sets; The multi-agent link module is configured to treat each region as a multi-agent, and use multi-agents to link the activity functions and element sets of multiple regions in the cross-chain network to form a multi-agent cross-chain network cluster divided by domains; The first cross-chain network collaborative decision module is configured to: input the domain-divided multi-agent cross-chain network cluster into a trained multi-agent model, and obtain the optimal domain weight value of the cross-chain network in each region based on the output of the multi-agent model; The second cross-chain network collaborative decision-making module is configured to: input the multi-level activity functions and element sets into the trained hierarchical reinforcement learning model, and obtain the optimal layer weight value of each layer of the cross-chain network based on the output of the multi-agent model; The collaboration result acquisition module is configured to determine the final cross-chain network collaboration result based on the optimal domain weight value of the cross-chain network in each region and the optimal layer weight value of the cross-chain network at each layer; wherein, based on the number of cross-chain interactions and basic weight in the collaboration information of each cross-chain network, the cross-chain weight value H of each cross-chain network relative to other cross-chain networks is calculated, including: Where f(a) is the cross-chain frequency auxiliary function, g(b) is the weight auxiliary function; a is the number of interactions of cross-chain network x with other cross-chain networks, p is the sum of the number of interactions of cross-chain network x itself and the number of interactions initiated by it; k is the basic weight of x, and b is the sum of the weights of the cross-chain networks interacted with by x; Among them, the construction method of the multi-agent model includes: the multi-agent model adopts the MAPPO algorithm, the actor-critic architecture, and is a centralized value function; centralized learning and decentralized decision-making are adopted; each agent has its own strategy network, and there is a central controller during training. The central controller collects the observations, actions, and rewards of all agents, and the central controller helps the agents train the strategy network. After the training is completed, each agent makes decisions based on its own observations and uses its own strategy network without communicating with the central controller; the system uses n agents, and each agent Actor has a strategy network. The agent interacts with the environment, obtains information elements from the environment, and makes decisions on its own to obtain action a i , then execute a i , a central controller is required during training, and the agent communicates with the central controller and sends actions, observations and rewards to the central controller; the central controller has information about all agents and n value networks Critic i Each value network corresponds to an agent, which trains the value network in the central network and uses the time difference algorithm to update the training results; after the training is completed, there is no need for a central controller, and each agent interacts with the environment independently. The agent observes the environment from the local environment. i Each agent has its own decision network, which takes observation as input, and the policy network outputs an action probability distribution, and selects action a according to the probability distribution. i , each agent performs its own action a i , then the environment changes state; Among them, the construction method of the hierarchical reinforcement learning model includes: the hierarchical reinforcement learning model adopts the Option-Critic architecture; establishing hierarchical reinforcement learning based on goal, the upper blockchain network constructs an upper controller, and the training criterion is to make the output goal satisfy the semantic information we set as much as possible, that is, to make the goal conform to the better state / state transition direction as much as possible; the lower blockchain network constructs a lower controller, and the training criterion of the lower controller is to make the environment state transition satisfy the goal as much as possible; to this end, the upper blockchain network will pass an internal reward to the lower blockchain network to drive the lower blockchain network to complete the goal, and the upper blockchain network is usually trained based on external environment rewards; therefore, the upper blockchain network and the lower blockchain network are actually trained separately; by giving the goal certain semantic information and designing reasonable internal rewards based on this semantic information, even when the external rewards are very sparse, it can ensure that the lower blockchain network obtains certain internal incentives, so that the intelligent agent can learn the optimal strategy more efficiently and improve the efficiency of sample utilization; Among them, based on the optimal domain weight value of the cross-chain network in each region and the optimal layer weight value of the cross-chain network at each layer, the final cross-chain network coordination result is determined, including: Result is the cross-chain network collaboration result, H s 、H L are the optimal domain weight value and optimal layer weight value of each cross-chain network respectively, and H is the cross-chain weight value of a cross-chain network for other cross-chain networks.

3. A computer device, characterized in that: including processor and storage medium; The storage medium is used to store instructions; The processor is configured to operate according to the instructions to execute the steps of the method according to claim 1 .

4. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to claim 1 are implemented.

Images