Dynamic access method in cross-domain collaborative scene

Abstract

The invention relates to a dynamic access method in a cross-domain cooperative scene, and belongs to the technical field of cross-domain cooperative communication. The method comprises the following steps: dividing cross-domain nodes into multiple clusters, defining intra-cluster and inter-cluster time slot tables, and carrying out initial resource allocation; each cluster head collects node positions, topological relations and flow data in the cluster in real time, a graph neural network and a DQN network are fused, and accumulated rewards of different communication node pairs in the cluster are calculated for each time slot; and based on a dynamic scene state, preferentially selecting a high-yield communication node to access a time slot, updating data traffic to be transmitted, performing a time slot resource allocation decision according to a time sequence until the current data traffic to be transmitted is transmitted, updating a time slot table and synchronizing the whole network. According to the method, the problems of resource competition conflict and poor dynamic topology adaptability in a cross-domain scene are solved, efficient coordination of node dynamic access is realized, the spectrum resource utilization rate and network robustness are improved, and the method is suitable for heterogeneous node cooperative communication scenes such as an unmanned aerial vehicle group and an unmanned fleet.

Description

Technical Field

[0001] The present invention belongs to the technical field of cross-domain collaborative communications, and in particular relates to a dynamic access method in a cross-domain collaborative scenario. Background Art

[0002] As portable communication modules continue to miniaturize and power consumption continues to decrease, drones and unmanned boats equipped with these modules are being widely used in various scenarios. The addition of these low-cost, highly mobile nodes has significantly expanded communication coverage scenarios, broadening the spatial dimensions of ad hoc network architectures. Cross-domain collaboration scenarios are becoming more numerous and the situations they face are becoming increasingly complex.

[0003] In cross-domain collaboration scenarios, the dynamic access of communications between multiple devices in multiple domains is inevitable. This involves conflicts in the scheduling of limited resources. The networks within each domain and cluster are independently managed entities. Different allocation strategies for information data access, such as time slot resources and spectrum resources, may lead to conflicts during channel preemption. In addition, in cross-domain scenarios, dynamic topology and mobility bring about complex changes in channels, and channel quality often drops sharply in the scenario. All of the above unfavorable situations may lead to the inability of each node in the cross-domain scenario to effectively share data and coordinate tasks.

[0004] Existing communication networking methods pay little attention to dynamic changing factors in cross-domain collaboration scenarios, and pay little attention to node requirements and scenario changes. This will inevitably lead to inefficient data access in the scenario and affect the overall communication quality. Summary of the Invention

[0005] In order to address the problems of data access conflicts, low resource utilization, and inability to handle dynamic scene changes that may occur in cross-domain scenarios, the present invention provides a dynamic access method in a cross-domain collaborative scenario based on reinforcement learning. It can perceive and collect data such as node status information, network status, traffic data, etc. in the communication network, and use a deep reinforcement learning network to quickly make dynamic access decisions.

[0006] The present invention provides a dynamic access method in a cross-domain collaboration scenario, comprising the following steps:

[0007] Step 1: The initialization module obtains network nodes in the cross-domain scenario and performs initialization, including: clustering the nodes in the network and designating the cluster head node, generating the intra-cluster time slot table and the inter-cluster time slot table to form a complete time slot table, and loading the complete time slot table to all nodes in the network through broadcasting; communication between nodes in the same cluster follows the same intra-cluster time slot table, and communication between nodes in different clusters follows the inter-cluster time slot table; treating the set of all cluster heads as a cluster numbered 0, and designating a group head from all cluster heads as the cluster head of cluster 0;

[0008] Step 2: The cluster head of each cluster is used as a computing node. For each cluster, each node in the network periodically reports its own sensory data to the cluster head. The cluster head calculates the current network status data within the cluster based on all the sensory data to form dynamic scene data. The sensory data includes the node's current spatial position, the neighboring nodes within the cluster that can directly communicate, and the data flow between neighboring nodes. The current network status data within the cluster includes the cluster topology connection matrix, the node traffic matrix within the cluster, and the node distance matrix within the cluster.

[0009] Step 3: For each cluster, a graph network is constructed based on the current network status data within the cluster, and the graph network and the current time slot number are input into the deep reinforcement learning network for processing; where:

[0010] The graph network constructed for each cluster includes: treating the nodes in the cluster as nodes in the graph network, constructing edges based on the connection relationship between the nodes; setting node features including node number and whether the node is active in the network; setting edge features including node one-hop or multi-hop identification and distance between nodes;

[0011] The deep reinforcement learning network is composed of a graph neural network and a DQN decision network. The graph neural network extracts the network features within the cluster from the input graph network and outputs them to the DQN decision network. The DQN decision network evaluates the Q value of each action selected in the current time slot. The Q value is obtained by calculating the cumulative reward brought by selecting the communication node pair. The action space is the set of legal communication node pairs within the cluster.

[0012] Step 4: The cluster head node makes a time slot resource allocation decision and updates the data flow to be transmitted between nodes in the cluster. It uses the DQN decision network to evaluate the Q value of each action for the next time slot and repeats this step until all the data flow to be transmitted between nodes in the current cluster can be transmitted. The updated intra-cluster time slot table and inter-cluster time slot table are obtained. The updated time slot table is broadcast to the entire network. After each node synchronously updates the time slot table, it accesses the communication according to the time division multiple access mechanism.

[0013] In the step 1, time slot resources are initially allocated to nodes within each cluster in a cyclic equal distribution manner; the initialization module reserves a set number of time slots for inter-cluster communication and management; the complete time slot table is stored in a dictionary, and the time slot table includes: cluster sequence number clus_id, set to 0 for inter-cluster and starting from 1 for different cluster numbers; time slot table length slot_num, recording the total number of time slots; time slot type slot_type, time slots are divided into intra-cluster communication time slots, intra-cluster management time slots, inter-cluster communication time slots, and inter-cluster management time slots; the link path slot_path within the time slot is represented by the array [Tx_clus, Tx_id, Rx_clus, Rx_id], where Tx_clus and Tx_id are the cluster number and number of the sending node, respectively, and Rx_clus and Rx_id are the cluster number and number of the receiving node, respectively.

[0014] In step 3, when the DQN decision network evaluates the Q value of each action in the action space A, the reward of the action is calculated based on the current network state within the cluster and the historical reward is accumulated to obtain a cumulative reward; the reward r of the action is set to be positively correlated with the degree to which the communication requirements are met, an additional reward is given to the action whose communication distance is less than the distance threshold, a negative reward is applied to the action whose communication distance is greater than the longest distance, and a negative reward is applied to the action selected repeatedly.

[0015] In step 4, the cluster head node selects the communication node pair with the largest Q value to access the current time slot with probability β according to the set exploration rate β∈[0,1], and randomly selects a team from the remaining communication node pairs in the cluster except the one with the largest Q value to access the current time slot with probability 1-β.

[0016] The advantages and beneficial effects of the present invention are as follows: the method of the present invention can sense and collect data such as the status information of nodes in the communication network, network status, traffic data, etc., and use a deep reinforcement learning network to quickly make dynamic access decisions. When a node loses network connection or accesses the network during operation, the method of the present invention can still quickly optimize the network access strategy to ensure the operation of the network. While ensuring that each node in a cross-domain scenario accesses according to the time slot plan, the method of the present invention uses an optimized time slot resource allocation strategy to improve the dynamic access capability. The method of the present invention is applicable to cross-domain collaborative communication networks composed of drone swarms and unmanned fleets, and supports real-time updates of time slot tables when nodes dynamically join or exit. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 It is a flow chart of the dynamic access method in the cross-domain collaboration scenario of the present invention. DETAILED DESCRIPTION

[0018] In order to clearly illustrate the purpose, features and advantages of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific implementation methods. It should be noted that the drawings provided in the embodiments of the present invention are to help understand the features and advantages of the present invention more clearly. The drawings are schematic and should not be understood as limiting the present invention in any way. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work. In addition, the embodiments of the present invention and the features in the embodiments can be combined with each other unless there is a conflict. In order to elaborate on the principle of the present invention as detailed as possible, a large number of details in the following implementation methods, in fact, the present invention can also be implemented in other ways different from those described herein, therefore, the scope of protection of the present invention is not limited by the specific embodiments disclosed below.

[0019] Currently, there is little research on dynamic access methods in cross-domain collaborative scenarios in the field of communications. The present invention combines deep reinforcement learning to construct a dynamic access method based on scenario information, using traditional time division multiple access (TDMA) technology as a blueprint to avoid the problem of resource preemption conflicts from the design principle. The present invention divides different nodes in the cross-domain scenario into clusters, and implements a combination of intra-cluster communication and inter-cluster communication. Nodes in the same cluster uniformly follow the same time slot table, which strictly limits the communication nodes in each time slot to avoid resource competition. Data across clusters and domains is exchanged through communication between cluster heads of each cluster. All cluster heads in the scenario also follow the same inter-cluster time slot table to ensure that there is no resource competition in inter-cluster head communication. Therefore, the intra-cluster time slot table used by each cluster and the inter-cluster time slot table used by all clusters are combined into a complete time slot table. Taking into account the limited support of time division multiple access technology for dynamic scenarios and the easy waste of channel resources, the present invention proposes to use reinforcement learning to perform rapid online allocation of dynamic time slot resources. When the scene status changes dynamically, a deep reinforcement learning network is used to quickly re-decide on time slot allocation using inputs such as the collected communication status, and quickly update the time slot table suitable for dynamic scenarios to provide nodes with access.

[0020] like Figure 1 As shown, the dynamic access method in the cross-domain collaborative scenario of the present invention utilizes the node information, network status, and traffic demand in each cluster in the cross-domain scenario as effective input, uses a reinforcement learning network to judge the action of allocating time slots to different nodes for communication, selects time slot allocation according to a certain decision-making method, and completes the purpose of dynamic access by updating the time slot table online. The method implemented in the embodiment of the present invention corresponds to a dynamic access system, in which an initialization module, a status update and data processing module, a deep reinforcement learning network model, and a time slot allocation strategy update module are set in the system, and each module can be implemented as a computer function module. The embodiment of the present invention illustrates the method of the present invention in the following four steps.

[0021] Step S1: Network Initialization. The initialization module obtains network nodes within the cross-domain scenario and performs initialization, including: clustering the nodes within the cross-domain scenario, designating a cluster head node for each cluster; defining the communication time slots between each team of nodes; defining the information exchange method between each node and the functions each node assumes; and separately allocating time slot resources within and between clusters. In the initial state of each cluster, all time slot resources are evenly distributed among the node pairs that need to communicate within the cluster, except for a small number of time slot resources reserved for inter-cluster communication between cluster heads. The time slot usage policy within and between clusters is broadcast and loaded to the entire network at system initialization. This step includes steps S1-1 through S1-4.

[0022] S1-1: Network node statistics and clustering. First, the system will count and pre-cluster all the nodes in the cross-domain scenario, and designate cluster heads for each cluster. In an embodiment of the present invention, the system has two initial clustering criteria, namely the physical location of the node and the functional type. During initialization, nodes in the same domain are clustered according to a certain distance based on the K-Means method. Nodes in the same domain and within a certain distance that have the same task function will be divided into a cluster. Subsequently, the nodes in each cluster designate cluster heads based on the standard of the shortest center distance. In the initial state of each cluster, except for a small number of time slot resources reserved for each cluster head for inter-cluster communication, all other time slot resources are evenly distributed to the node pairs that need to communicate in the cluster. In subsequent steps, each cluster is relatively independent, and the cluster head of each cluster uniformly manages the nodes in the cluster. In this step, the node numbers and cluster head node information in the network are obtained, including:

[0023] node_id: records the node ID, such as node c_b, where c represents the cluster ID and b represents the node ID within the cluster.

[0024] clus_head: used to record the cluster head nodes of each cluster, and its length is the number of divided clusters.

[0025] The embodiment of the present invention further designates a cluster head node from all cluster head nodes, which may be selected based on the shortest center distance.

[0026] S1-2: Set the function of each node and the specified time slot function. After obtaining the global cluster information, the system will set the functional tasks of each node and uniformly divide the specific time slot functions. The data set in this step includes:

[0027] slot_type: This section records the functions of specific time slots in the time division multiple access (TDMA) allocation. This section describes time slots of different functional types, including intra-cluster communication (inclus_communication), intra-cluster management (inclus_management), inter-cluster communication (outclus_communication), and inter-cluster management (outclus_management). Intra-cluster and inter-cluster communication slots occupy the majority of the allocated time slot resources and are used for normal data transmission. Management slots are used for TDMA time alignment, management signaling, and other management operations.

[0028] S1-3: Initial allocation of time slot resources. In the present invention, the cluster head node of each cluster serves as a computing power node, which has both computing and management functions in addition to its original tasks. After setting the functions of each node and the specific number of time slot function types, the system will allocate time slot resources to each cluster node in the initial cross-domain system in a cyclic equivalence manner, where the node number node_id has the same cluster number c, which means that it is a node in the same cluster and will follow the same time slot table. In addition, in order for the system to reserve a fixed position for inter-cluster time slots, all nodes in the network will uniformly follow an inter-cluster time slot table. Inter-cluster time slots reserve a set number of time slots for inter-cluster communication and management. The result of the time slot resource allocation constitutes a time slot table, which is stored in a dictionary and contains the following elements:

[0029] clus_id: This element represents the cluster number, where 0 is defined as between clusters, and starting from 1, it represents different clusters.

[0030] slot_num: This element indicates the length of the time slot table, which is the number of time slots. In this embodiment of the present invention, the time slot table length is set to 500 by default, which can be used to represent the rule of 500 consecutive time slots. The length of each time slot is set to 10ms by default.

[0031] slot_type: This element has been defined and indicates the slot type;

[0032] slot_path: This element represents the specific link path within a time slot. It is represented by a 4-length array [Tx_clus, Tx_id, Rx_clus, Rx_id], where Tx_id is the ID of the sending node, Tx_clus is the cluster number of the sending node, and Rx_id is the ID of the receiving node, Rx_clus is the cluster number of the receiving node. The index is the cluster number followed by the node number within the cluster, in the same format as node_id. For example, if the slot_path for a time slot is [1,4,1,5], it means that node 4 in cluster 1 is the data sending node, and node 5 in cluster 1 is the data receiving node.

[0033] S1-4: Initially, each cluster head distributes the time slot table initially allocated in S1-3 to all nodes within the cluster, ensuring that all nodes in the scenario have loaded their time slot tables. Nodes within the same cluster have identical time slot tables. Next, all nodes in this cross-domain scenario access data in a prescribed manner using TDMA (Time Division Multiple Access).

[0034] Step S2: Dynamic state update and data processing: The state update and data processing module on each cluster head node receives the sensing data sent by the nodes in the cluster and converts it into the current network state data in the cluster.

[0035] S2-1: Each cluster head node records the dynamic information of each node in the cluster. Each node periodically obtains data sensed by its own sensors, including the node's current geographic location, the neighboring nodes in the cluster with which it can directly communicate, and the amount of data to be transmitted between the neighboring nodes, and sends it to the cluster head.

[0036] The current geographical location sent by the node to the cluster head is represented as node_position, and each node is stored in the form of [λ, φ, h], where λ is longitude, φ is latitude, and h is altitude.

[0037] S2-2: Dynamic Data Processing. The cluster head node needs to process and convert the recorded node dynamic data, converting the data reported by each node into information that can represent the current network status, and obtain the current dynamic scene data, including: the intra-cluster topological connection matrix, the inter-node traffic matrix, and the intra-cluster node distance matrix.

[0038] node_distance: Intra-cluster node distance matrix. This matrix is ​​obtained by converting the reported node latitude, longitude, and altitude coordinates to the same spatial coordinate system, such as the geocentric coordinate system, and then calculating the Euclidean distance. The WGS84 ellipsoid model is used, with the semi-major axis a = 6378137 meters and the first eccentricity squared e 2 =0.06694, first convert the geographic coordinates to the geocentric coordinate system:

[0039] Calculate the radius N of the Maoyou circle: Convert [λ,φ,h] to geocentric coordinates [X,Y,Z] as follows:

[0040] X=(N+h)cosφcosλ; Y=(N+h)cosφsinλ; Z=(N(1-e 2 )+h)sinφ.

[0041] The spatial Euclidean distance d between two nodes Node_1 and Node_2 can be calculated using the following formula:

[0042]

[0043] The coordinates of node Node_1 are [X1, Y1, Z1], and the coordinates of node Node_2 are [X2, Y2, Z2].

[0044] Then, according to the positions of all M nodes in the cluster recorded by node_position, the distances between nodes in the cluster are calculated in the above manner and stored as a matrix representing the distances, node_distance.

[0045] clus_topology: The topological connection matrix V within the cluster. An M-dimensional matrix V is used to represent the connection relationship between M nodes in the cluster. The element V in the i-th row and j-th column of the matrix is ij Indicates whether node i and node j are connected, V ij =0 means the node is not connected, V ij =1 indicates that the nodes are connected, i,j∈[1,M]. If two nodes are within each other's communication range and can communicate directly, they are connected, otherwise they are not connected.

[0046] node_data: The inter-node traffic matrix C uses an M-dimensional matrix to represent the data traffic to be transmitted between M nodes in the cluster. The elements C in the matrix ij Indicates the data traffic to be transmitted between node i and node j in the cluster.

[0047] Similarly, each cluster head sends its own perception data and the network status within the cluster to the central cluster head node. The cluster head node can calculate the distance between cluster heads within the cluster based on the perception data, count the traffic to be transmitted in the network, etc. In this embodiment of the present invention, the set of all cluster heads is considered as a cluster numbered 0, and the cluster head is used as the cluster head of cluster 0. The inter-cluster time slot table is updated based on the current network status within cluster 0.

[0048] Step S3: Input dynamic data to call the neural network.

[0049] S3-1: In this system, the designated cluster head and group head nodes undertake computational tasks. All cluster heads can be found using the clus_head table in S1. Simultaneously, during dynamic data updates in step S2, information from the remaining nodes in each cluster is packaged along with the communication information and sent to the cluster head. After each cluster management slot, the cluster head node receives global state information for the cluster.

[0050] S3-2: For each cluster, the intra-cluster node distance matrix node_distance, the intra-cluster node traffic matrix node_data, and the intra-cluster topological connection matrix clus_topology currently obtained in S2 are sent as input to the deep reinforcement learning network to extract the intra-cluster network features.

[0051] The core architecture of the deep reinforcement learning network of the present invention consists of a graph neural network and a DQN decision network.

[0052] Graph neural networks are used to capture topological connectivity information from input. Well-designed graph neural networks can effectively leverage the high-dimensional features of nodes and edges in the topology, improving the ability to represent environmental states. The method proposed in this paper uses graph networks to update node features and encode the connectivity between nodes.

[0053] A graph network is constructed for each cluster, with the nodes in the cluster serving as nodes in the graph network, and the connections between the nodes forming the edges in the graph network. In an embodiment of the present invention, the node feature node_features(i) contains the node number i and whether the node is active in the network inetwork_vector[i]. node_features[i] = np.array([i, inetwork_vector[i]]). Edges indicate whether the nodes are connected and whether they can communicate. The edge feature edge_features[i,j] between node i and node j includes the node one-hop / multi-hop identifier topology_matrix[i][j] and the inter-node distance distance_matrix[i][j]. The node one-hop / multi-hop identifier indicates whether the two nodes can communicate directly or multi-hop; edge_features[i,j] = np.array([topology_matrix[i][j], distance_matrix[i][j]]). np.array is a function that creates an array. At the same time, the sequential nature of time slots will be taken into consideration. The deep reinforcement learning network model needs to know which time slot it is currently in, so that certain sequential characteristics can be included when updating node features.

[0054] The DQN decision network transforms the features extracted by the graph neural network into discrete action space decisions through a linear mapping layer, that is, the selection of legal communication node pairs. A communication node pair represents a group of communicating parties, that is, the specific allocation rules of a certain time slot. Define the action space as A, A contains [a1, a2, ..., a n ], n represents the number of all legal communication pairs in the cluster, and M represents the number of nodes in the cluster, which can be calculated For example, a cluster of four nodes, numbered 1-4, has six possible node connections, and the action space is [1-2, 1-3, 1-4, 2-3, 2-4, 3-4]. Here, the DQN evaluates the Q value of the action in the discrete space A, essentially selecting which communication node pair to connect to in this time slot. The DQN network uses the Adam optimizer to adaptively adjust the learning rate of network parameters based on gradient information. It also uses the MSELoss function to accurately measure the deviation between the predicted Q value of an action and the actual reward. This collaboratively drives the cluster head agent to optimize the time slot allocation strategy during its interaction with the environment, thereby achieving dynamic access.

[0055] S3-3: After the dynamic information of the nodes in the scene is combined through a deep reinforcement learning network with a graph neural network and a DQN decision network as the architecture, a judgment basis for measuring the node link selection in the dynamic scene can be obtained. This basis is a set of values ​​Q, which has been explained in step S3-2. It is given by the DQN decision network in the previous step. DQN evaluates the Q value of the action in the discrete action space A. Q = [q1, q2, ..., q n ], where q i is the cumulative reward that can be obtained by selecting the i-th allocation action in this time slot. The state, action and reward of DQN designed in this embodiment of the present invention are as follows:

[0056] (1) The state has been defined in S2-1, including the distance between nodes in the cluster node_distance, the topology connection clus_topology, and the traffic between nodes node_data. The data consisting of the above three types of cluster node information is used as the input state.

[0057] (2) The action has been defined in S3-2 and represents the set of all available communication pairs in the cluster. During the training phase, the selection of actions is increased by a decay coefficient, from partially selecting the action with the largest Q value and randomly selecting the rest of the actions to gradually selecting only the action with the largest Q value.

[0058] (3) The design considerations of reward r are as follows: the proportion of communication needs met, setting the reward positively correlated with the degree of need satisfaction; giving extra rewards for shorter distance communications and deducting points for long distance communications; and imposing negative rewards for repeated selections.

[0059] Step S4: The time slot allocation strategy update module dynamically updates the time slot access strategy and builds a new time slot table as the dynamic network status of each cluster is updated. The reserved management time slot is broadcasted to the entire network. After each node synchronously updates the time slot table, it accesses the communication according to the time division multiple access mechanism.

[0060] S4-1: Dynamic update strategy. In the initial state, the Q value of the time slot allocation action is set to 0. When each cluster starts to input the dynamic network state, it will start to continuously update the Q value table. Assume that after N rounds of iteration, the deep reinforcement learning network has obtained the cumulative reward quantization table Q under the current network state. now According to this table, we can get the benefits of selecting any pair of communication nodes to access. now Sort by the largest q max The corresponding communication pair is the allocation method that optimizes the benefits. The decision strategy is defined as selecting the allocation method that optimizes the benefits whenever possible. Here, the communication node pairs are selected based on the ε-greedy algorithm. An exploration rate β∈[0,1] is introduced to control the selection of the optimal benefit decision. The cluster head selects the communication pair with the best access time slot with probability β and randomly selects access from the remaining communication pairs with probability 1-β.

[0061] S4-2: Complete the construction of the new time slot table. Every time the cluster head node makes a time slot access selection, it means that a communication is completed, and the corresponding c ij The network traffic, where c ij Calculated by node_distance, clus_topology and channel model. The node_data matrix contains all the traffic states to be transmitted in the network at the current state, so the corresponding c can be subtracted after each transmission. ij Network traffic, when the node_data matrix decreases to 0, it means that all task data transmission is completed, which also means that all dynamic access decisions are completed. The decision records of each step are combined into a new time slot table.

[0062] Through the above steps, each cluster has updated its intra-cluster communication time slot allocation strategy. Similarly, the method of the present invention specifies a cluster head node from all cluster head nodes based on the shortest center distance standard. All cluster head nodes form a graph network. According to the above steps S2-S4, the inter-cluster communication time slot allocation strategy can be updated. The inter-cluster communication time slot can be compared to the description of the intra-cluster communication time slot in the above steps. The cluster head elected between clusters is equivalent to the cluster head within the cluster, and each cluster head is equivalent to an ordinary node within the cluster. In this way, the same method can be used for the upper-level cluster composed of cluster heads of each cluster. In addition, the management time slots within and between clusters have been initialized in step S1, and the number of time slots is small, so this method does not update them. Through this, the method of the present invention can use all time slot tables, and then broadcast the new time slot table to each node in the cluster through the management time slot to complete the strategy update.

[0063] The network status of all clusters including cluster 0 is regularly detected through step S2, and then the intra-cluster time slot table and the inter-cluster time slot table are updated through steps S3 and S4. The time slot table can be updated in real time for dynamically changing network scenarios. The present invention can quickly optimize the network access strategy and ensure the operation of the network.

[0064] Except for the technical features described in the specification, all other technical features are known to those skilled in the art. Descriptions of well-known components and technologies are omitted in this disclosure to avoid redundancy and unnecessary limitation of the present invention. In general, the various exemplary embodiments of the present disclosure can be implemented in hardware or dedicated circuits, software, firmware, logic, or any combination thereof. Certain aspects can be implemented in hardware, while other aspects can be implemented in firmware or software executed by a controller, microprocessor, or other computing device.

Claims

1. A dynamic access method in a cross-domain collaboration scenario, characterized in that: The following steps are involved: Step 1: The initialization module obtains network nodes in the cross-domain scenario and performs initialization, including: clustering the nodes in the network and designating cluster heads, generating a complete time slot table containing intra-cluster time slot tables and inter-cluster time slot tables, and loading the complete time slot table to all nodes in the network via broadcast. Nodes in the same cluster communicate using the same intra-cluster time slot table, while nodes in different clusters communicate using the inter-cluster time slot table. Consider the set of all cluster heads as a cluster numbered 0, and select a cluster head from all cluster heads as the cluster head of cluster 0; Step 2: The cluster head of each cluster is used as a computing node. For each cluster, each node in the network periodically reports its own sensory data to the cluster head. The cluster head calculates the current network status data within the cluster based on all the sensory data. The sensory data includes the current node's geographic location, the neighboring nodes within the cluster that can directly communicate, and the data volume to be transmitted between the neighboring nodes. The current network status data within the cluster includes the cluster topology connection matrix, the inter-node traffic matrix within the cluster, and the node distance matrix within the cluster. Step 3: For each cluster, a graph network is constructed based on the current network state data within the cluster, and the graph network and time slot sequence are input into the deep reinforcement learning network for processing; where: The graph network constructed for each cluster includes: treating the nodes in the cluster as nodes in the graph network, constructing edges based on the connection relationship between the nodes; setting node features including node number and whether the node is active in the network; setting edge features including node one-hop or multi-hop identification and distance between nodes; The deep reinforcement learning network is composed of a graph neural network and a DQN decision network. The graph neural network extracts the network features within the cluster from the input graph network and outputs them to the DQN decision network. The DQN decision network evaluates the Q value of each action selected in the current time slot. The Q value is obtained by calculating the cumulative reward brought by selecting the communication node pair. The action space is the set of legal communication node pairs within the cluster. Step 4: The cluster head node makes a time slot resource allocation decision and updates the data flow to be transmitted between nodes in the cluster. It uses the DQN decision network to evaluate the Q value of each action for the next time slot and repeats this step until all the data flow to be transmitted between nodes in the current cluster can be transmitted. The updated intra-cluster time slot table and inter-cluster time slot table are obtained. The updated time slot table is broadcast to the entire network. After each node synchronously updates the time slot table, it accesses the communication according to the time division multiple access mechanism.

2. The method according to claim 1, characterized in that In step 1, nodes are clustered according to their domain, geographical location and the functions they undertake. Nodes in the same domain and within a set distance that undertake the same function are grouped into a cluster, and a cluster head is designated within the cluster using the shortest center distance standard; a group head is designated from all cluster heads using the shortest center distance standard.

3. The method according to claim 1 or 2, characterized in that Step 1, recording information after clustering: The numbers of all nodes. The node numbers are expressed in the form of c_b, where c is the cluster number and b is the node number within the cluster; The cluster head node of each cluster.

4. The method according to claim 1 or 2, characterized in that In the step 1, time slot resources are initially allocated to the nodes in each cluster in a cyclic equal distribution manner, and a set number of time slots are reserved for inter-cluster communication and management; the generated complete time slot table is stored in a dictionary, and the time slot table includes: a cluster sequence number clus_id, set to 0 for inter-cluster and starting from 1 for different cluster numbers; a time slot table length slot_num, recording the total number of time slots; a time slot type slot_type, where time slots are divided into intra-cluster communication time slots, intra-cluster management time slots, inter-cluster communication time slots, and inter-cluster management time slots; and a link path slot_path within the time slot, represented by an array [Tx_clus, Tx_id, Rx_clus, Rx_id], where Tx_clus and Tx_id are the cluster number and number of the sending node, respectively, and Rx_clus and Rx_id are the cluster number and number of the receiving node, respectively.

5. The method according to claim 1, characterized in that The step 2, calculating the intra-cluster node distance matrix based on the current node geographic location, includes: recording the longitude, latitude, and altitude of the node in the node geographic location, first converting the longitude, latitude, and altitude coordinates of the node into a geocentric coordinate system, and then calculating the Euclidean distance between different nodes in the cluster based on the geocentric coordinates of the node, and storing the calculated distances as the intra-cluster node distance matrix; The topological connection matrix within the cluster is calculated based on whether the nodes can communicate directly. The element in the i-th row and j-th column of the matrix represents whether nodes i and j are connected. If nodes i and j can communicate directly, they are connected, and the element value is 1. Otherwise, the element value is 0. The traffic matrix between nodes in the cluster records the data traffic to be transmitted between different nodes in the cluster.

6. The method according to claim 1, characterized in that In step 3, the action space of the DQN decision network is A=[a1,a2,…,a n ], n is the total number of communication pairs, M is the number of nodes in the cluster; When the DQN decision network evaluates the Q value of each action in the action space A, it calculates the reward of the action based on the current network state within the cluster and accumulates the historical reward to obtain the cumulative reward; the reward r of the action is set to be positively correlated with the degree to which the communication requirements are met, and additional rewards are given to actions with a communication distance less than the distance threshold, negative rewards are applied to actions with a communication distance greater than the longest distance, and negative rewards are applied to actions that are repeatedly selected.

7. The method according to claim 1 or 6, characterized in that In step 4, the cluster head selects the communication node pair with the largest Q value to access the current time slot with probability β according to the set exploration rate β∈[0,1], and randomly selects a team from the remaining communication node pairs in the cluster except the one with the largest Q value to access the current time slot with probability 1-β.

8. The method according to claim 1 or 6, characterized in that In the method, each cluster periodically detects the network status within the cluster, executes step 2, and then updates the communication time slots in the intra-cluster time slot table and the inter-cluster time slot table through steps 3 and 4.

Images