A dynamic access method for cross-domain collaboration scenarios

Abstract

This invention is a dynamic access method for cross-domain collaborative scenarios, belonging to the field of cross-domain collaborative communication technology. The invention includes: dividing cross-domain nodes into multiple clusters, defining intra-cluster and inter-cluster time slot tables, and performing initial resource allocation; each cluster head collects real-time node location, topology relationships, and traffic data within its cluster, fusing graph neural networks and DQN networks to calculate the cumulative reward for different communication node pairs within each time slot; based on the dynamic scenario state, prioritizing high-yield communication node pairs for access to the time slot, updating the data traffic to be transmitted, making time slot resource allocation decisions according to time sequence until the current data traffic to be transmitted is completed, updating the time slot table, and synchronizing across the entire network. This invention solves the problems of resource contention conflicts and poor dynamic topology adaptability in cross-domain scenarios, achieving efficient coordination of dynamic node access, improving spectrum resource utilization and network robustness, and is suitable for heterogeneous node collaborative communication scenarios such as UAV swarms and unmanned ship fleets.

Description

Technical Field

[0001] This invention belongs to the field of cross-domain collaborative communication technology, and in particular relates to a dynamic access method in cross-domain collaborative scenarios. Background Technology

[0002] As portable communication modules continue to become smaller and their power consumption further reduced, drones, unmanned boats, and other devices equipped with communication modules are being widely used in various scenarios. The addition of these low-cost and highly mobile nodes has significantly expanded the communication coverage, broadened the spatial dimensions of ad hoc network architectures, and led to an increasing number of cross-domain collaborative scenarios, which are becoming increasingly complex.

[0003] In cross-domain collaborative scenarios, the dynamic access issues of communication between multiple devices across multiple domains are inevitable. This involves conflicts in the scheduling of limited resources. Each network within a domain and cluster is managed independently, and different strategies for allocating information and data resources, such as time slots and spectrum resources, can lead to conflicts during channel preemption. Furthermore, in cross-domain scenarios, dynamic topologies and mobility bring about complex and varied channel changes, frequently resulting in sudden drops in channel quality. All of these unfavorable conditions can prevent nodes from achieving effective data sharing and collaborative tasks in cross-domain scenarios.

[0004] Existing communication networking methods, in cross-domain collaborative scenarios, rarely consider dynamic changing factors and give little consideration to node requirements and scenario changes. This inevitably leads to inefficient data access in such scenarios and affects the overall quality of communication. Summary of the Invention

[0005] To address issues such as data access conflicts, low resource utilization, and inability to handle dynamic scene changes that may occur in cross-domain scenarios, this invention provides a dynamic access method for cross-domain collaborative scenarios based on reinforcement learning. This method can collect data such as the status information of nodes in the communication network, network status, and traffic data through perception, and make dynamic access decisions quickly using a deep reinforcement learning network.

[0006] This invention provides a dynamic access method for cross-domain collaborative scenarios, comprising the following steps:

[0007] Step 1: The initialization module obtains network nodes within the cross-domain scenario and performs initialization, including: clustering the nodes in the network and designating cluster head nodes, generating intra-cluster time slot tables and inter-cluster time slot tables to form a complete time slot table, and loading the complete time slot table to all network nodes via broadcast; communication between nodes within the same cluster follows the same intra-cluster time slot table, and communication between nodes in different clusters follows the inter-cluster time slot table; the set of all cluster heads is regarded as a cluster numbered 0, and a group head is designated from all cluster heads, and the group head is used as the cluster head of cluster 0;

[0008] Step 2: Using the cluster head as the computing node, each node in the network periodically reports its own perception data to the cluster head. The cluster head calculates the current network state data within the cluster based on all the perception data, forming dynamic scene data. The perception data includes the current spatial location of the node, the neighboring nodes within the cluster that can communicate directly, and the data traffic between the node and its neighboring nodes. The current network state data within the cluster includes the cluster topology connection matrix, the traffic matrix between nodes within the cluster, and the distance matrix between nodes within the cluster.

[0009] Step 3: For each cluster, construct a graph network based on the current network state data within the cluster, and input the graph network and the current time slot number into the deep reinforcement learning network for processing; where:

[0010] The graph network constructed for each cluster includes: treating the nodes in the cluster as nodes in the graph network, constructing edges based on the connection relationships between nodes; setting node features including node number and whether the node is active in the network; setting edge features including node one-hop or multi-hop identifier and distance between nodes.

[0011] The deep reinforcement learning network consists of a graph neural network and a DQN decision network. The graph neural network extracts intra-cluster network features from the input graph network and outputs them to the DQN decision network. The DQN decision network evaluates the Q-value of each action selected in the current time slot. The Q-value is obtained by calculating the cumulative reward brought by selecting a communication node pair. The action space is the set of legal communication node pairs within the cluster.

[0012] Step 4: The cluster head node makes a time slot resource allocation decision and updates the data traffic to be transmitted between nodes within the cluster. It uses the DQN decision network to evaluate the Q value of each action for the next time slot. This step is repeated until all the data traffic to be transmitted between nodes within the current cluster can be transmitted, resulting in the updated intra-cluster time slot table and inter-cluster time slot table. The updated time slot table is then broadcast to the entire network. After each node updates its time slot table synchronously, it accesses the communication via time-division multiple access.

[0013] In step 1, time slot resources are initially allocated to nodes within each cluster in a cyclical and even distribution manner. The initialization module reserves a set number of time slots for inter-cluster communication and management. The complete time slot table is stored as a dictionary, and the time slot table includes: cluster number clus_id, where 0 represents inter-cluster communication and 1 represents different cluster numbers; time slot table length slot_num, which records the total number of time slots; time slot type slot_type, where time slots are divided into intra-cluster communication time slots, intra-cluster management time slots, inter-cluster communication time slots, and inter-cluster management time slots; and the link path within the time slot slot_path, represented by the array [Tx_clus, Tx_id, Rx_clus, Rx_id], where Tx_clus and Tx_id are the cluster number and ID of the sending node, respectively, and Rx_clus and Rx_id are the cluster number and ID of the receiving node, respectively.

[0014] In step 3, when the DQN decision network evaluates the Q value of each action in the action space A, it calculates the cumulative reward by adding the historical reward to the current network state within the cluster. The reward r for the action is set as follows: it is positively correlated with the degree to which the communication requirement is met, and additional rewards are given for actions where the communication distance is less than the distance threshold, negative rewards are given for actions where the communication distance is greater than the longest distance, and negative rewards are given for actions that are repeatedly selected.

[0015] In step 4, the cluster head node selects the communication node pair with the largest Q value to access the current time slot with probability β according to the set exploration rate β∈[0,1]. It also randomly selects a pair of communication nodes from the remaining communication node pairs in the cluster (excluding the pair with the largest Q value) with probability 1-β to access the current time slot.

[0016] The advantages and beneficial effects of this invention are as follows: The method of this invention can collect data such as node status information, network status, and traffic data in the communication network through sensing, and utilize deep reinforcement learning networks to quickly make dynamic access decisions. When nodes experience network drops or additions during operation, the method of this invention can still quickly optimize the network access strategy and ensure network operation. While ensuring that each node accesses according to time slot planning in cross-domain scenarios, the method of this invention improves dynamic access capabilities by using an optimized time slot resource allocation strategy. The method of this invention is applicable to cross-domain collaborative communication networks composed of UAV swarms and unmanned vessel fleets, and supports real-time updates of the time slot table when nodes dynamically join or leave. Attached Figure Description

[0017] Figure 1 This is a flowchart illustrating the dynamic access method for cross-domain collaboration scenarios according to the present invention. Detailed Implementation

[0018] To clearly illustrate the purpose, features, and advantages of this invention, the invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that the accompanying drawings provided in the embodiments of this invention are to help to more clearly understand the features and advantages of this invention. The drawings are illustrative and should not be construed as limiting the invention in any way. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. Furthermore, in the absence of conflict, the embodiments of this invention and the features in the embodiments can be combined with each other. Since the principle of describing the invention as comprehensively as possible is followed, many details of the following embodiments are actually described in other ways different from those described herein. Therefore, the scope of protection of this invention is not limited to the specific embodiments disclosed below.

[0019] Currently, research on dynamic access methods in cross-domain collaborative scenarios in the field of communications is limited. This invention combines deep reinforcement learning to construct a dynamic access method based on scenario information, using traditional Time Division Multiple Access (TDMA) technology as a blueprint to avoid resource contention issues from a design principle perspective. This invention divides different nodes in the cross-domain scenario into clusters, implementing a combination of intra-cluster and inter-cluster communication. Nodes within the same cluster operate according to the same timeslot table, which strictly limits the communication nodes for each timeslot to avoid resource contention. Cross-cluster and cross-domain data is exchanged through communication between the cluster heads of each cluster. All cluster heads within the scenario also follow the same inter-cluster timeslot table, ensuring that no resource contention occurs during communication between cluster heads. Therefore, the intra-cluster timeslot table used by each cluster and the inter-cluster timeslot table shared by all clusters are combined to form a complete timeslot table. Considering that time division multiple access technology has limited support for dynamic scenarios and is prone to wasting channel resources, this invention proposes to use reinforcement learning for fast and dynamic online allocation of time slot resources. When the scenario state changes dynamically, a deep reinforcement learning network is used to quickly re-determine the time slot allocation based on the collected communication status and other inputs, and quickly update the time slot table suitable for the dynamic scenario to enable nodes to complete access.

[0020] like Figure 1 As shown, the dynamic access method for cross-domain collaborative scenarios of the present invention utilizes node information, network status, and traffic demand in each cluster of the cross-domain scenario as effective inputs. It uses a reinforcement learning network to evaluate the actions of allocating time slots to different nodes for communication, selects time slots according to a certain decision-making method, and achieves the purpose of dynamic access by updating the time slot table online. The method implemented in this embodiment corresponds to a dynamic access system, which includes an initialization module, a state update and data processing module, a deep reinforcement learning network model, and a time slot allocation strategy update module. Each module can be implemented as a computer functional module. This embodiment of the invention will describe the method in the following four steps.

[0021] Step S1: Network Initialization. The initialization module acquires network nodes within the cross-domain scenario and performs initialization, including: clustering nodes within the cross-domain scenario, assigning a cluster head node to each cluster; defining communication time slots between each pair of nodes; defining the information exchange methods between each node and the functions undertaken by each node; handling the allocation of time slot resources within and between clusters respectively. In the initial state of each cluster, except for a small portion of time slot resources reserved for inter-cluster communication by each cluster head, all other time slot resources are evenly allocated to the node pairs within the cluster that need to communicate; the time slot usage strategy within and between clusters is broadcast and loaded to the entire network at system initialization. This step includes steps S1-1 to S1-4.

[0022] S1-1: Network Node Statistics and Clustering. First, the system will statistically analyze and pre-cluster all existing nodes in the cross-domain scenario, and assign cluster heads to each cluster. In this embodiment, the system's initial clustering criteria have two aspects: node physical location and functional type. During initialization, nodes within the same domain are clustered according to a certain distance using the K-Means method. Nodes within the same domain and within a certain distance that perform the same task function will be grouped into one cluster. Subsequently, the cluster head is assigned to each node within the cluster based on the shortest center distance. In the initial state, each cluster, except for a small portion of time slots reserved for inter-cluster communication by each cluster head, has all remaining time slots evenly distributed among the node pairs within the cluster that need to communicate. In subsequent steps, each cluster is relatively independent, and each cluster head manages the nodes within its cluster. This step involves obtaining the node IDs and cluster head node information in the network, including:

[0023] node_id: Records the node number, such as node c_b, where c represents the cluster number and b represents the node number within the cluster.

[0024] clus_head: Used to record the cluster head node of each cluster, with a length equal to the number of clusters.

[0025] In this embodiment of the invention, a cluster head node is designated from all cluster head nodes, which can be selected based on the shortest center distance.

[0026] S1-2: Configure the functions of each node and define the functions of each time slot. After obtaining the global cluster information, the system will configure the functional tasks of each node and uniformly allocate specific time slot functions. The data configured in this step includes:

[0027] `slot_type`: Records the specific function and task of the time slots in the allocated Time Division Multiple Access (TDMA) system. This sets different functional types of time slots, including intra-cluster communication (`inclus_communication`), intra-cluster management (`inclus_management`), inter-cluster communication (`outclus_communication`), and inter-cluster management (`outclus_management`). Intra-cluster and inter-cluster communication time slots will occupy the majority of the allocated time slot resources and are used for normal data transmission; management time slots are used for management operations such as TDMA time calibration and management signaling distribution.

[0028] S1-3: Initial allocation of time slot resources. In this invention, the cluster head node of each cluster acts as a computing power node, possessing both computing and management functions in addition to its original task. After configuring the functions of each node and the specific number of time slot function types, the system will allocate time slot resources to each cluster node in the initial cross-domain system in a cyclical and even distribution manner. Nodes with the same cluster number c (node_id) are considered to be in the same cluster and will follow the same time slot table. Furthermore, to reserve fixed positions for inter-cluster time slots, all nodes in the network will uniformly follow a single inter-cluster time slot table. The number of inter-cluster time slots is reserved according to a set number for inter-cluster communication and management. The results of the time slot resource allocation form a time slot table, which is stored as a dictionary and contains the following elements:

[0029] clus_id: This element represents the cluster number, where 0 is defined as inter-cluster, and 1 onwards represents different clusters.

[0030] slot_num: This element represents the length of the time slot table and the number of time slots. In this embodiment of the invention, the time slot table length is set to 500 by default, which can be used to represent a rule of 500 consecutive time slots. The length of each time slot is set to 10ms by default.

[0031] slot_type: This element has been defined and indicates the slot type;

[0032] `slot_path`: This element represents the specific link path within a time slot, represented by an array of length 4: `[Tx_clus, Tx_id, Rx_clus, Rx_id]`. Here, `Tx_id` represents the sending node ID, `Tx_clus` is the cluster number of the sending node, and `Rx_id` represents the receiving node ID, `Rx_clus` is the cluster number of the receiving node. It follows the format `node_id`, indexed by the cluster number followed by the node number within the cluster. For example, if the `slot_path` of a time slot is `[1,4,1,5]`, it means that node 4 in cluster 1 is the sending data node, and node 5 in cluster 1 is the receiving data node.

[0033] S1-4: Initially, each cluster head distributes the time slot table initially allocated in S1-3 to each node within the cluster, ensuring that all nodes in the scenario have their time slot tables loaded. Nodes within the same cluster have identical time slot tables. Subsequently, all nodes in this cross-domain scenario will access data in the prescribed manner according to the TDMA (Time Division Multiple Access) access method.

[0034] Step S2: Dynamic State Update and Data Processing. The state update and data processing module on each cluster head node receives the sensing data sent by nodes within the cluster and converts it into the current network state data within the cluster.

[0035] S2-1: Each cluster head node records the dynamic information of each node within the cluster. Each node periodically acquires data sensed by its own sensors, including the node's current geographical location, neighboring nodes within the cluster with which it can communicate directly, and the data traffic to be transmitted between it and its neighboring nodes, and sends it to the cluster head.

[0036] The current geographical location sent by a node to the cluster head is represented as node_position. Each node stores its position in the format [λ, φ, h], where λ is longitude, φ is latitude, and h is altitude.

[0037] S2-2: Dynamic Data Processing. The cluster head node needs to process and transform the recorded dynamic data of the nodes, converting the data reported by each node into information that can represent the current network state, and obtaining the current dynamic scenario data, including: intra-cluster topology connection matrix, inter-node traffic matrix, and intra-cluster node distance matrix.

[0038] `node_distance`: The intra-cluster node distance matrix. This matrix is ​​obtained by transforming the reported node latitude, longitude, and altitude coordinates to the same spatial coordinate system, such as the geocentric coordinate system, and then calculating the Euclidean distance. The WGS84 ellipsoid model is used, where the semi-major axis a = 6378137 meters and the first eccentricity squared e... 2 =0.06694, first convert the geographic coordinates to the geocentric coordinate system:

[0039] The radius N of the zonal circle is calculated as follows: The [λ,φ,h] coordinates are converted to geocentric coordinates [X,Y,Z] as follows:

[0040] X=(N+h)cosφcosλ; Y=(N+h)cosφsinλ; Z=(N(1-e 2 )+h)sinφ.

[0041] The spatial Euclidean distance d between two nodes Node_1 and Node_2 can be calculated using the following formula:

[0042]

[0043] The coordinates of node_1 are [X1,Y1,Z1], and the coordinates of node_2 are [X2,Y2,Z2].

[0044] Then, based on the positions of all M nodes in the cluster recorded by node_position, the distances between nodes in the cluster are calculated in the above manner and stored as a matrix node_distance representing the distances.

[0045] clus_topology: The intra-cluster topological connectivity matrix V, which uses an M-dimensional matrix V to represent the connectivity relationships between M nodes within a cluster. The element V in the i-th row and j-th column of the matrix is... ij V indicates whether node i and node j are connected. ij =0 indicates that the nodes are not connected, V ij =1 indicates that the nodes are connected, i,j∈[1,M]. If two nodes are within each other's communication range and can communicate directly, then they are connected; otherwise, they are not connected.

[0046] node_data: Inter-node traffic matrix C, using an M-dimensional matrix to represent the data traffic to be transmitted between M nodes within a cluster. The elements in the matrix are C. ij This represents the data flow to be transmitted between node i and node j within the cluster.

[0047] Similarly, each cluster head sends its own sensing data and intra-cluster network status to the central group head node. The group head node can calculate the distance between cluster heads within the group and count the traffic to be transmitted in the network based on the sensing data. In this embodiment of the invention, the set of all cluster heads is regarded as a cluster numbered 0, and the group head is regarded as the cluster head of cluster 0. The inter-cluster time slot table is updated according to the current intra-cluster network status.

[0048] Step S3: Input dynamic data and invoke the neural network.

[0049] S3-1: In this system, the designated cluster head node and group head node undertake the computational tasks. All cluster heads can be found in the `clus_head` table in S1. Simultaneously, during dynamic data updates in step S2, information from the remaining nodes within each cluster is packaged and sent to the cluster head along with the communication information. Each time a cluster management time slot is completed, the cluster head node receives the global state information for that cluster.

[0050] S3-2: For each cluster, the intra-cluster node distance matrix node_distance, intra-cluster inter-node flow matrix node_data, and intra-cluster topology matrix clus_topology obtained in S2 are fed into the deep reinforcement learning network to extract intra-cluster network features.

[0051] The core architecture of the deep reinforcement learning network of this invention consists of a graph neural network and a DQN decision network.

[0052] Graph neural networks are used to capture the topological connectivity information of the input. A well-designed graph neural network can effectively utilize the high-dimensional features of nodes and edges in the topology, improving the ability to represent environmental states. The method of this invention uses a graph network to update node features and encode the connectivity relationships between nodes.

[0053] A graph network is constructed for each cluster, with nodes in the cluster serving as nodes in the graph network, and the connections between nodes forming edges in the graph network. In this embodiment, the node features `node_features(i)` include node number `i`, the node's activity status in the network `inetwork_vector[i]`, and `node_features[i] = np.array([i, inetwork_vector[i]])`. Edges indicate whether nodes have connections and can communicate. The edge feature `edge_features[i, j]` between node `i` and node `j` includes the node's one-hop / multi-hop identifier `topology_matrix[i][j]` and the distance between nodes `distance_matrix[i][j]`. The one-hop / multi-hop identifier indicates whether the two nodes can communicate directly or via multiple hops; `edge_features[i, j] = np.array([topology_matrix[i][j], distance_matrix[i][j]])`. `np.array` is a function for creating arrays. At the same time, the temporal nature of time slots will be taken into account. Deep reinforcement learning network models need to know which time slot they are in so that certain temporal characteristics can be included when updating node features.

[0054] The DQN decision network transforms the features extracted by the graph neural network into discrete action space decisions through a linear mapping layer, namely, the selection of legal communication node pairs. A communication node pair represents a set of two communicating parties, which is essentially the specific allocation rule for a given time slot. This action space is defined as A, containing [a1, a2, ..., a...]. n Let n represent the total number of valid communication pairs within the cluster, and M represent the number of nodes within the cluster. The following can be calculated: For example, a cluster of 4 nodes, numbered 1-4, has 6 possible node linking methods, with an action space of [1-2, 1-3, 1-4, 2-3, 2-4, 3-4]. Here, DQN evaluates the Q-value of actions in the discrete space A, essentially determining "which communication node pair to connect to in this time slot." The DQN network uses the Adam optimizer to adaptively adjust the network parameter learning rate based on gradient information and the MSELoss function to accurately measure the deviation between the predicted action Q-value and the actual reward. This collaboratively drives the cluster head agent to optimize the time slot allocation strategy during its interaction with the environment, thereby achieving dynamic access.

[0055] S3-3: After the dynamic information of nodes in the scene is combined through a deep reinforcement learning network with a graph neural network and a DQN decision network architecture, the judgment criteria for node link selection in a dynamic scene can be obtained. This criterion is a set of numerical values ​​Q, which has been explained in step S3-2. It is given by the DQN decision network in the previous step. DQN evaluates the Q-values ​​of actions in the discrete action space A. Q = [q1,q2,…,q n ], where q i The cumulative reward obtainable by selecting the i-th allocation action in this time slot is determined as follows: In this embodiment of the invention, the states, actions, and rewards of the DQN are designed as follows:

[0056] (1) The state is defined in S2-1 and includes the distance between nodes within the cluster (node_distance), the topology connection (clus_topology), and the flow between nodes (node_data). The data composed of the above three types of node information within the cluster is used as the input state.

[0057] (2) Actions are defined in S3-2 as the set of all available communication pairs within a cluster. During the training phase, the selection of actions increases with a decay coefficient, gradually changing from selecting actions with the largest Q value to randomly selecting the remaining actions, and finally to selecting only the actions with the largest Q value.

[0058] (3) The design considerations for reward r are as follows: the proportion of communication needs met, and the reward is set to be positively correlated with the degree of needs met; additional rewards are given for short-distance communication and deductions are given for long-distance communication; negative rewards are applied to actions that are repeatedly selected.

[0059] Step S4: The time slot allocation strategy update module continuously updates the time slot access strategy and constructs a new time slot table in accordance with the dynamic network status of each cluster. The strategy is broadcast across the entire network via reserved management time slots, and each node accesses the communication via time-division multiple access after synchronously updating its time slot table.

[0060] S4-1: Dynamic Update Strategy. Initially, the Q-values ​​for all time slot allocation actions are set to 0. As each cluster begins inputting dynamic network states, the Q-value table is continuously updated. Assume that after N iterations, the deep reinforcement learning network has obtained the cumulative reward quantization table Q for the current network state. now Based on this table, the benefits of selecting any pair of communication nodes for access can be obtained. The cumulative reward Q... now Sort them, where the largest q is... max The corresponding communication pair is the allocation method that optimizes the benefit. The decision strategy is defined as choosing the allocation method that optimizes the benefit as much as possible. Here, the ε-greedy algorithm is used to select communication node pairs. An exploration rate β∈[0,1] is introduced. β is used to control the decision to select the best benefit. The cluster head will select the communication pair with the best access in the time slot with probability β, and randomly select access from the other communication pairs with probability 1-β.

[0061] S4-2: Complete the construction of the new timeslot table. Each time the cluster head node makes a timeslot access selection, it signifies the completion of a communication cycle, corresponding to the successful transmission of c. ij network traffic, of which c ij The node_data matrix is ​​calculated using node_distance, clus_topology, and the channel model. It contains the states of all traffic to be transmitted in the network under the current condition. Therefore, after each transmission, the corresponding c can be subtracted. ij Network traffic: When the node_data matrix decreases to 0, it signifies the completion of all task data transmission and the completion of all dynamic access decisions. The decision records for each step are then combined into a new timeslot table.

[0062] Through the above steps, each cluster updates its intra-cluster communication time slot allocation strategy. Similarly, the method of this invention designates a group head node from all cluster head nodes based on the shortest center distance criterion. All cluster head nodes form a graph network, and the inter-cluster communication time slot allocation strategy can be updated according to steps S2-S4 above. The inter-cluster communication time slots can be compared with the description of intra-cluster communication time slots in the above steps. The group head elected between clusters is equivalent to the cluster head within the cluster, and each cluster head is equivalent to an ordinary node within the cluster. In this way, the same method can be used for the upper-level clusters composed of each cluster head. In addition, the intra-cluster and inter-cluster management time slots have been initialized in step S1, and the number of time slots is small, so this method does not update them. With this, the method of this invention can use the entire time slot table, and subsequently broadcast the new time slot table to each node within the cluster through the management time slots to complete the strategy update.

[0063] By periodically detecting the network status of all clusters, including cluster 0, in step S2, and then updating the intra-cluster time slot table and inter-cluster time slot table in steps S3 and S4, the time slot table can be updated in real time for dynamically changing network scenarios. This invention can quickly optimize network access strategies and ensure network operation.

[0064] Except for the technical features described in the specification, all other technologies are known to those skilled in the art. Descriptions of well-known components and technologies are omitted in this invention to avoid redundancy and unnecessarily limiting the invention. Generally, various exemplary embodiments of this disclosure can be implemented in hardware or dedicated circuitry, software, firmware, logic, or any combination thereof. Some aspects can be implemented in hardware, while others can be implemented in firmware or software executed by a controller, microprocessor, or other computing device.

Claims

1. A dynamic access method in a cross-domain collaborative scenario, characterized in that, The method comprises the following steps: Step 1: The initialization module obtains network nodes in a cross-domain scene, and performs initialization, including: clustering the nodes in the network and designating cluster heads, generating a complete time slot table including an intra-cluster time slot table and an inter-cluster time slot table, and loading the complete time slot table to all network nodes by broadcasting; nodes in the same cluster follow the same intra-cluster time slot table, and nodes in different clusters follow the inter-cluster time slot table; Regarding a set of all cluster heads as a cluster numbered 0, and designating a group head from all cluster heads as the cluster head of cluster 0; Step 2: Taking the cluster heads of each cluster as computing power nodes, periodically reporting the sensing data of each node in the network to the cluster head, and calculating the current network state data in the cluster according to all the sensing data; the sensing data includes the current geographic position of the node, the intra-cluster neighbor nodes that can directly communicate, and the data traffic to be transmitted between the neighbor nodes; the current network state data in the cluster includes the intra-cluster topology connection matrix, the intra-cluster node traffic matrix, and the intra-cluster node distance matrix; Step 3: For each cluster, constructing a graph network based on the current network state data in the cluster, and inputting the graph network and the time slot sequence into a deep reinforcement learning network for processing; wherein: For each cluster, constructing a graph network includes: regarding the nodes in the cluster as nodes in the graph network, and constructing edges according to the connection relationship between the nodes; setting the node features to include the node number and the activity state of the node; and setting the edge features to include the one-hop or multi-hop identification of the node and the distance between the nodes; The deep reinforcement learning network comprises a graph neural network and a DQN decision network, the graph neural network extracts the intra-cluster network features of the input graph network and outputs them to the DQN decision network, the DQN decision network evaluates the Q values of selecting each action under the current time slot, and the Q values are obtained by calculating the cumulative rewards brought by selecting the communication node pairs; the action space is a set of legal intra-cluster communication node pairs; Step 4: The cluster head node makes a time slot resource allocation decision, and updates the data traffic to be transmitted between the intra-cluster nodes; using the DQN decision network to evaluate the Q values of each action for the next time slot, and repeating the step until the data traffic to be transmitted between the intra-cluster nodes can be completely transmitted, obtaining the updated intra-cluster time slot table and inter-cluster time slot table; broadcasting the updated time slot table to the whole network, and synchronously updating the time slot table by each node before accessing the communication according to the time division multiple access mechanism.

2. The method of claim 1, wherein, In step 1, the nodes are clustered according to the domain, geographic position and task function, the nodes in the same domain and within a certain distance and having the same task function are clustered into a cluster, and a cluster head is designated in the cluster according to the shortest center distance standard; a group head is designated from all cluster heads according to the shortest center distance standard.

3. The method according to claim 1 or 2, characterized in that, In step 1, information is recorded after clustering: The number of all nodes, the node number is represented in the form of c_b, c is the cluster number, and b is the node number in the cluster; The cluster head node of each cluster.

4. The method according to claim 1 or 2, characterized in that, The step 1 is to allocate time slot resources to nodes in each cluster in a cyclically even way, and reserve a set number of time slots for inter-cluster communication and management; the generated complete time slot table is stored in a dictionary, and the time slot table includes: cluster serial number clus_id, 0 for inter-cluster, and 1 for different cluster numbers; time slot table length slot_num, recording the total number of time slots; time slot type slot_type, the time slot is divided into intra-cluster communication time slot, intra-cluster management time slot, inter-cluster communication time slot and inter-cluster management time slot; link path in the time slot slot_path, represented by an array [Tx_clus, Tx_id, Rx_clus, Rx_id], Tx_clus and Tx_id are the cluster number and number of the sending node respectively, and Rx_clus and Rx_id are the cluster number and number of the receiving node respectively.

5. The method of claim 1, wherein, The step 2 is to calculate the intra-cluster node distance matrix according to the geographical position of the current node, including: recording the longitude, latitude and height of the node in the geographical position of the node, first converting the longitude, latitude and height coordinates of the node into the geocentric coordinate system, then calculating the Euclidean distance between different nodes in the cluster according to the geocentric coordinates of the nodes, and storing it as the intra-cluster node distance matrix; The intra-cluster topology connection matrix is calculated according to whether the nodes can directly communicate with each other, and the element in the i-th row and j-th column of the matrix represents whether node i and node j are connected, if node i and node j can directly communicate, it means that they are connected, and the element value is 1, otherwise, the element value is 0; The intra-cluster node flow matrix records the data flow to be transmitted between different nodes in the cluster.

6. The method of claim 1, wherein, In the step 3, the action space A of the DQN decision network , n is the total number of communication pairs, , M is the number of nodes in the cluster; when the DQN decision network evaluates the Q value of each action in the action space A, the cumulative reward of the action is calculated based on the current intra-cluster network state, and the cumulative reward is obtained; the reward r of the action is set as: positively correlated with the degree of meeting the communication demand, additionally rewarded for the action with a communication pair distance less than a distance threshold, negatively rewarded for the action with a communication pair distance greater than the longest distance, and negatively rewarded for the repeatedly selected action.

7. The method according to claim 1 or 6, characterized in that, In step 4, the cluster head selects the communication node pair with the maximum Q value according to the set exploration rate The communication node pair with the maximum Q value is selected according to the exploration rate β to access the current time slot. A team is randomly selected from the remaining communication node pairs except the one with the maximum Q value to access the current time slot.

8. The method of claim 1 or 6, wherein, The method, each cluster periodically detects the network state in the cluster, executes step 2, and then updates the communication time slots in the intra-cluster time slot table and the inter-cluster time slot table through steps 3 and 4.

Images