METHOD, CONTROL PROGRAM, COMPUTER-READY DATA CARRIER, CONTROL UNIT, COMMUNICATION DEVICE AND SYSTEM FOR PROVIDING A NETWORK STRUCTURE, AS WELL AS DEVICE CONFIGURED TO PARTICIPATE AS A NETWORK NODE

Abstract

A method, a control program (11), a computer-readable data carrier (12), a control unit (24), a communication system (1), and a corresponding device (7) are provided, which configure a network structure (N) of a communication system (1) to provide a data connection (C) for transmitting a data object (D) between a sender (A) and at least one receiver (B), in particular a vehicle (1), such as an aircraft, via at least one routing path (R) provided by network nodes (O) of the network structure (N), wherein the method comprises the steps of: assigning a global number (k) of network nodes (O) of the network structure (N) to a global domain (K) controlled by a global controller (H);Providing a global model of the global domain (K) based on routing parameters representing routing capabilities associated with the network nodes (O) assigned to the global domain (K); assigning a local number (I) of respective network nodes (O) assigned to the global domain (K) as subsets of the global number (k) to at least two local domains (L), each controlled by a respective local controller (J); and providing respective local models of the local domains (L) based on routing parameters representing routing capabilities associated with each network node (O) assigned to the respective local domains (L);wherein the global controller (H) interacts with the local controllers (J) to identify at least one of the network nodes (O) that provides routing capabilities according to the global model and / or the local models, enabling the at least one network node (O) to act as a local proxy (P), which allows the data object (D) to be placed along the at least one routing path (R) so that it can be provided from the local proxy (P) to the receiver (B).

Description

Technical field

[0001] The present description relates to the field of communication systems comprising multiple participants in a network structure. In particular, the disclosure relates to a method for configuring a network structure of a communication system to provide a data connection for transmitting a data object between a sender and at least one receiver, in particular a vehicle such as an aircraft, via at least one routing path provided by network nodes of the network structure, to a computer-readable data carrier; to a control unit for providing a data connection between a sender and at least one receiver; and to a communication device configured to participate in a network structure to establish a data connection for transmitting a data object between a sender and at least one receiver, in particular a vehicle such as an aircraft.to provide via at least one routing path provided by network nodes of the network structure, and to a device, in particular a vehicle, such as an aircraft. Technical background

[0002] To improve the performance of mobile networks, state-of-the-art technologies propose various solutions that address data storage at the network edge to minimize data query latency and increase data service reliability. This is typically the case for mobile networks supporting the mobility of pedestrians and vehicles (terrestrial and airborne). However, edge storage is limited, and deployment is challenging in some situations, such as remote areas, disasters, or combat scenarios.

[0003] In these scenarios, the use of assisted, distributed content caching by participating devices, particularly vehicles such as flying drones or aircraft acting as mobile communication devices, becomes a potential solution for content publishing services. Mobile communication devices, such as drones and aircraft, are highly effective in addressing the communication challenges of the aforementioned scenarios due to their rapid deployment capabilities. Their use as airborne network nodes makes it possible to establish a local area network (LAN) and a backbone network, even in locations without existing network infrastructure. This approach is thought to significantly reduce data transmission time compared to relying solely on satellite links, as demonstrated by Liu C, Feng W, Chen Y, et al.„Cell-Free Satellite-UAV Networks for 6G Wide-Area Internet of Things“, IEEE Journal on Selected Areas in Communications, vol.39, no.4, pp.1116-1131,2021 (https: / / ieeexplore.ieee.org / document / 9174846) dargelegt.

[0004] Several state-of-the-art network frameworks that support the use of distributed content caching, such as information-centric networks (ICNs) as described by Loannou A., Weber S., “A Survey of Caching Policies and Forwarding Mechanisms in Information-Centric Networking”, IEEE Commun. Surv. Tutor. 2016;18:2847-2886. doi: 10.1109 / COMST.2016.2565541, allow the caching and retrieval of data objects from any intermediate node in the network, such as an aircraft. Data caching is typically performed at all nodes along the path from the data source to the receivers, without any consideration of the nature of the data being transported or the surrounding context of all nodes in the network.

[0005] Furthermore, according to current best practices, data is typically cached in response to an explicit data request, without any prior action aimed at bringing data closer to potential future consumers. In-network caching is one of the main methods used to reduce network load, increase data availability, and decrease delivery latency for data consumers, especially mobile consumers. The reasons for performing data caching within a network are manifold: On the one hand, if data is only available at the data producer or cached nearby, the network around the data producer can be affected by high traffic load, and data delivery latency can increase.On the other hand, if data objects are only cached near consumers, requests for data objects can be retrieved more quickly. However, if data objects are placed too close to certain data consumers, they may not be available to others located in neighboring network branches or domains. Therefore, a major problem in state-of-the-art data distribution is deciding which intermediate nodes to cache data in.

[0006] In the context of information-centric networks, caching schemes can be divided into the following five categories: (a) popularity-based caching, probabilistic caching, label-based caching, and graph-based caching, as described by Zhang M., Luo H., Zhang H., “A Survey of Caching Mechanisms in Information-Centric Networking”, IEEE Commun. Surv. Tutor.2015:17:1473-1499 described; (b) probabilistic caching, which is based on routers that use a probability value p to make a caching decision; (c) label-based caching, which uses policies regarding content objects that are labeled based on certain properties; (d) graph-based caching, which considers forwarding routes and the network structure to place content objects in the delivery path; and (e) popularity-based caching, in which intermediate nodes decide whether or not to cache a particular data object based on the frequency and request distribution for that data.

[0007] Furthermore, according to the state of the art, several mechanisms exist that attempt to use deep reinforcement learning (DRL) to optimize mobile networks, namely networks comprising an ad-hoc number of aircraft. One example is the proposal to develop a control strategy that utilizes DRL to maximize communication coverage and network connectivity for multiple real-time users within a specific timeframe, as described by GB Tarekegn, R.-T. Juang, H.-P. Lin, YY Munaye, L.-C. Wang, and MA Bitew, “Deep-Reinforcement-Learning-Based Drone Base Station Deployment for Wireless Communication Services,” IEEE Internet of Things Journal, vol. 9, no. 21, November 1, 2022 (https: / / ieeexplore.ieee.org / document / 9794697).

[0008] Other examples of state-of-the-art technology, such as those described by Wang, L.; Zhang, H.; Guo, S.; Yuan, D., “3D UAV Deployment in Multi-UAV Networks with Statistical User Position Information,” IEEE Commun. Lett. vol. 26, no. 6, pp. 1363–1367, 2022 (https: / / ieeexplore.ieee.org / document / 9739696), involve the use of particle swarm optimization algorithms to optimize the deployment positions of multiple drones and aim to improve network coverage, as described by Z. Dai, Y. Zhang, W. Zhang, X. Luo, and Z. He, “A Multi-Agent Collaborative Environment Learning Method for UAV Deployment and Resource Allocation,” IEEE Transactions on Signal and Information Processing over Networks, vol. 8, pp. 120-130, 2022 (https: / / ieeexplore.ieee.org / document / 9712375), or to support decision-making processes regarding deployment positions that may affect transmit power and occupied radio channels, as described by Z. Dai, Y. Zhang, W. Zhang, X.Luo and Z. He, "A Multi-Agent Collaborative Environment Learning Method for UAV Deployment and Resource Allocation," IEEE Transactions on Signal and Information Processing over Networks, vol. 8, pp. 120-130, 2022 (https: / / ieeexplore.ieee.org / document / 9712375).

[0009] In mobile multi-domain networks, such as multi-domain combat cloud systems, the confidentiality of various types of sensitive data must be protected, particularly when collaboration between devices in different domains is required. In this case, the use of Federated Learning (FL) can help maintain confidentiality by transferring only the model, rather than the raw data, between nodes in different domains during the training process. However, updating training models involves a large number of parameters, resulting in high communication costs.

[0010] To address this challenge, a potential state-of-the-art solution involves using a federated learning approach that utilizes adaptive knowledge distillation and dynamic gradient compression techniques, as proposed by Wu, Chuhan and Wu, Fangzhao and Lyu, Lingjuan and Huang, Yongfeng and Xie, Xing, “Communication-efficient federated learning via knowledge distillation”, Nature Communications, 2022, 2032. However, these state-of-the-art approaches and other similar approaches, such as those proposed by Wang HP, Stich S, He Y, et al., “Communication-efficient federated learning via knowledge distillation”, International Conference on Machine Learning, pp. 23034-23054, 2022, have been suggested for specific scenarios and do not aim to improve the overall network utilization problem from the perspective of the amounts of data produced and consumed in different parts of the network.Thus, the known state of the art fails to provide efficient and reliable mechanisms for data provision in a mobile telecommunications infrastructure comprising a range of heterogeneous devices, at least some of which may have limited and changing communication capabilities due to certain technical and / or availability limitations, e.g. caused by changes in their geographical locations. Summary

[0011] In light of the foregoing, the task can be seen as providing an efficient management mechanism capable of coordinating data exchange between multiple heterogeneous network nodes while respecting the confidentiality restrictions of cross-domain communication. In particular, the task can be seen as enabling the efficient and low-latency provision of data objects by and / or to one of the network nodes at the request of senders or receivers, who may participate in fluctuating or fluid network structures, as may be the case when at least some of the participants are mobile. This task is accomplished by the subject matter of the independent claims.

[0012] According to one aspect, a method for configuring a network structure of a communication system to provide a data connection for transmitting a data object between a sender and at least one receiver, in particular a vehicle such as an aircraft, via at least one routing path provided by network nodes of the network structure is provided, wherein the method comprises the following steps: assigning a global number of network nodes of the network structure to a global domain controlled by a global controller; providing a global model of the global domain based on routing parameters representing routing capabilities associated with the network nodes assigned to the global domain;Assigning a local number of respective network nodes, which are assigned to the global domain as subsets of the global number, to at least two local domains, each controlled by a respective local controller; and providing respective local models of the local domains based on routing parameters that represent routing capabilities associated with each network node assigned to the local domains; wherein the global controller interacts with the local controllers to identify at least one of the network nodes that, according to the global model and / or the local models, provides routing capabilities that enable the at least one network node to act as a local proxy, enabling the data object to be placed along the at least one routing path so that it can be delivered from the local proxy to the receiver.

[0013] According to one aspect, a control program is provided for controlling a communication system, which includes instructions that, when the control program is executed by a control unit, cause the control unit to perform a corresponding procedure.

[0014] According to one aspect, a computer-readable data carrier is provided on which a corresponding control program is stored.

[0015] According to one aspect, a control unit is provided for providing a data connection between a sender and at least one receiver, wherein the control unit is configured to execute a corresponding method according to at least one of the claims as domain control and / or global control and / or includes a corresponding computer-readable data carrier.

[0016] According to one aspect, a communication device is provided which is configured to participate in a network structure in order to establish a data connection for transmitting a data object between a sender and at least one receiver, in particular a vehicle such as an aircraft, via at least one routing path provided by network nodes of the network structure, wherein the communication device which is configured to perform a corresponding method according to at least one of the claims comprises a corresponding computer-readable data carrier according to the claim and / or a corresponding control unit.

[0017] According to one aspect, a device is provided, in particular a vehicle such as a satellite, an aircraft, a mobile communication station or a ground station, which comprises a corresponding computer-readable data carrier according to the claim, at least one control unit according to the claim, at least one communication device according to the claim and / or is configured to participate as a network node in a corresponding communication system.

[0018] Data objects can be placed in various autonomous domains, such as groups of nodes within a shared area, region, and / or elevation, in a decentralized manner. This allows data to be cached based on its nature (e.g., popular data) and the properties of the available network nodes. The data link can be configured to send a data stream containing the data object from the sender to the receiver. The data link can also be configured and / or reserved for transmitting mission data.

[0019] At least one network offering sufficient routing capabilities can be designated as a local proxy, positioned along the routing path. The routing path can be designed to incorporate the local proxy. At least the local proxy and / or the receiver can be located on a vehicle, particularly an aircraft.

[0020] The present solution enables the implementation of a multi-domain data management system based on federated learning. The global controller can coordinate cross-domain data distribution between the local domains. Routing parameters can be stored in local experience stores of the local controllers to further train the global model and / or local models, thereby improving the data delivery capabilities of the network structure.

[0021] Consequently, this solution enables collaboration between sets of distributed nodes (e.g., aircraft) to coordinate the best possible decision in selecting network nodes to store a copy of specific data objects, taking into account the nature of the data, the context of the network nodes (e.g., storage capacity and network diversity), and a hierarchical relationship between them. Therefore, it can be assumed that a network comprises heterogeneous nodes (e.g., terrestrial, airborne, space-based) that can be clustered into autonomous local domains which, while willing to cooperate, wish to avoid disclosing confidential operational information.

[0022] Consequently, efficient data management mechanisms are provided that are capable of coordinating multiple heterogeneous network nodes located in different autonomous domains in a decentralized manner. This allows data to be cached based on its nature (e.g., popular data) and the characteristics of the available network nodes, while respecting the confidentiality restrictions of cross-domain communication. The decentralized solution, which comprises a hierarchical structure of mobile devices with heterogeneous capabilities (e.g., drones, aircraft, tankers, high-altitude pseudo-satellites, and satellites) organized in different autonomous network domains, is designed to cooperate to enhance the network's ability to distribute and cache data through a federated learning mechanism. This enables real-time perception of the network status, such as...The proposed algorithm models the overload of transmission links and node storage caused by large data volumes and adjusts the rules for data exchange and storage accordingly, while ensuring the confidentiality of the network state within the domain. Compared to other federated reinforcement learning algorithms, the proposed algorithm aims to reduce transmission overhead while simultaneously accelerating the convergence rate of the learning model.

[0023] Compared to other approaches that aim to enable intelligent data distribution between heterogeneous nodes in different autonomous domains, the proposed solution offers the advantages of a hierarchical, cross-domain data exchange and caching framework that improves network control and simplifies network management. In-network computing can be leveraged by utilizing the computing and communication capabilities of aircraft such as drones and other aircraft to enable the placement of data objects in optimal locations within the network, rather than simply closer to consumers and / or data producers. Heterogeneous storage can be utilized by providing the respective computing and networking capabilities of various aircraft (e.g., tankers and satellites) to serve as domain controllers by deploying efficient data processing algorithms on them.

[0024] Further developments can be derived from the dependent claims and the following description. Many of the features described in relation to a method can be implemented as device features, or vice versa. Therefore, the description provided in the context of a method for establishing a communication channel also applies analogously to a control unit, a communication device, a communication system, or a device. In particular, the steps of a method and the components involved therein can be implemented as functions of a control unit, a communication device, a communication system, and / or a device, and their functions can be implemented as method steps.

[0025] According to one embodiment of the method, the total number of local nodes is less than or equal to the global number. Subsets of network nodes in local domains may or may not interact. This allows local domains to be created as desired or needed to provision and manage individual network nodes, enabling reliable and efficient decentralized data delivery.

[0026] According to one embodiment of the method, the routing parameters include a trajectory parameter, a connection parameter, and / or a storage parameter representing the storage capacity of the network nodes. The connection parameter can represent a connection quality and / or signal strength. The routing parameters can be compared with respective threshold values. This further aids in the selection and management of network nodes to enable reliable and efficient decentralized data delivery.

[0027] According to one embodiment of the method, the method further comprises the step of performing an intra-domain update, wherein each local controller collects the routing parameters to train and / or update the local model and / or routing parameters. A local domain process can be performed using an intra-domain routing algorithm to collect the routing parameters for training and / or updating the local model and / or routing parameters. This can limit the amount of updated data provided by the local controllers to the global controller, which in turn can help improve the performance of the communication system in terms of both data delivery speed and capacity.

[0028] According to one embodiment of the method, each domain controller trains its respective local model based on the routing parameters and stores states that are then aggregated to form a proxy state, which is assigned to the at least one local proxy. Collected local data can include the routing parameters. This further contributes to decentralizing the communication system in a way that allows it to utilize the autonomous storage and social computing capabilities of the participating network nodes.

[0029] According to one embodiment of the method, the method further includes the step of uploading routing parameters from the local controllers to the global controller. Collected local data can include and / or consist of the routing parameters belonging to the local domain. After all agents in a domain have filled the experience memory, the proxy state, as well as the corresponding average strategy, can be calculated, and both can then be uploaded to the global controller. This allows the global controller to contribute to achieving a coherent state of the communication system, thereby providing improved centralized visibility and control in the selection and management of network nodes to enable reliable and efficient decentralized data delivery.

[0030] According to one embodiment of the method, the method further includes the step of aggregating the local models into the global model. For example, after the local proxy experience stores have been uploaded to the global controller, they can then be aggregated. The same proxy states from multiple domains can be combined into a single proxy state. Corresponding policies can then be averaged again. The global controller can train the global model using the aggregated proxy stores to generate global model parameters. A model convergence judgment can be performed before the global model parameters are output to the local controllers and / or network nodes. If the model converges, this means that the global model has been learned and the federated reinforcement learning algorithm can terminate.Otherwise, the algorithm can enter the parameter provisioning phase described below. This further enables the efficient, reliable, and targeted implementation of machine learning algorithms, leading to technically usable results, possibly with functional limitations of the network nodes involved, as described above.

[0031] According to one embodiment of the method, the method further includes the step of providing the global parameters to the domain controllers. In this phase, the global parameters are delivered to each domain controller. The domain controllers can then assign the parameters to the local model and use local data to update the respective model training parameters. This can further contribute to decentralizing the communication system in a way that allows it to leverage the autonomous storage and social computing capabilities of the participating network nodes.

[0032] According to one embodiment of the method, the steps of performing an intra-domain update, uploading routing parameters from the local controllers to the global controller, aggregating the local models into the global model, and / or providing the global parameters to the domain controllers are executed cyclically. This enables real-time, or at least near-real-time, optimization of data provisioning decisions. Brief description of the drawings

[0033] The subject matter of the invention is described below in conjunction with the following drawing figures, where identical numbers denote identical elements, and where: Fig. 1 is a schematic representation of a communication system. Fig. 2 is a schematic representation of the steps of a procedure. Detailed description of the drawings

[0034] The following detailed description is merely exemplary and is not intended to limit the invention or its uses. Furthermore, there is no intention to be bound by any theory presented in the preceding background or in the following detailed description. The illustrations and figures in the drawings are schematic and not to scale. Identical numbers denote identical elements. A better understanding of the subject matter described can be obtained by reviewing the figures together with the following detailed description.

[0035] Fig. Figure 1 shows a schematic representation of a communication system 1 comprising a series of communication devices 2 and respective control units 3, which may be equipped with interface modules 4 to connect the communication devices 2 and / or control units 3 to control elements 5. These control elements 5 can be interconnected via the respective transmission lines 6, which may be configured to transmit any type of information, data, power, and / or energy, including photonic links. Therefore, the transmission lines 6 may include any suitable wired, wireless, and / or optical communication means, including wires, cables, transceivers, antennas, satellite dishes, and the like. In the present example, the communication devices 2 and the respective control unit 3 can be provided to devices 7, such as...Ground stations 8 on a ground G and / or vehicles 9, including aircraft 9a, satellites 9b, unmanned aerial vehicles (UAVs) 9c and / or ground vehicles 9d. The devices 7 may comprise respective computer systems 10, which may include communication devices 2, control units 3, interface modules 4, control elements 5 and / or transmission lines 6, as desired or required for their respective application, e.g., to enable data transmission, storage, calculation, secure communication and / or precision acquisition of certain parameters and / or values, such asfor detecting acceleration, gravity, magnetic effects, photonic effects, radiation, rotation or the like, by means of the control elements 5, which are then calculated by means of the communication devices 2, which are configured to establish a network structure N via respective communication channels C for transmitting data objects D along a respective routing path R, which may include several segments that can be provided by respective communication channels C.

[0036] A computer program 11 for controlling the computing devices 10 can be stored on a computer-readable data carrier 12, which may take the form of a computer-readable medium 13 and / or a data carrier signal 14. The computer system 10 can comprise the communication device 2, the control unit 3, the interface module 4, the control element 5 and / or the transmission lines 7, the computer program 11, and the computer-readable data carrier 12, which may be adapted for data exchange between the respective components mentioned above. Control elements 9 can be any type of data source, such as a measuring element, a sensor, an output device, and / or an actuator of one of the devices 7. The communication devices 2 involved can therefore serve as source communication devices 2 in the sense of a sender A, destination communication devices 2 in the sense of a receiver B, and / or network node O.

[0037] The network nodes O can belong to a global domain K, which comprises local domains L, such as the first local domain L1, a second local domain L2, a third local domain L3, and so on, and which are controlled by a global controller H and / or a local controller J. The global domain K can comprise a global number k of network nodes O. The local domains L can comprise a local number I. Each of the network nodes O can act as a local proxy P, which can store the data object D in its communication device 2 and / or its respective computer system 10 according to a respective storage capacity M, e.g., a specific computer memory or a designated storage space therein.

[0038] The communication system 1 enables a mechanism for federated improved distillation in a hierarchical network of devices 1, such as aircraft, to coordinate the distribution of data objects D among the available storage capacities M, which can provide local caches of the network structure N. These caches aim to reduce the latency experienced by participants. In multi-domain flight networks, each local domain L can generate relatively small amounts of data. This means that each local controller J requires extensive training to obtain a stable model, which significantly increases the model training time and computational power consumption of, for example, flying communication devices 2 acting as local controllers J.

[0039] Since each local controller J can only use data within its own local domain L for training, the trained model has specific characteristics and is only suitable for local data distribution decisions. To overcome these challenges, a hierarchical reinforcement learning data distribution mechanism can be implemented. This mechanism can connect different model training domains and aggregate data from multiple local domains L. Each local controller J can participate in training the full model, thereby expanding the data samples and preventing the leakage of network state information within each local domain L.

[0040] However, to perform tasks within each domain, complex models must be used, which can lead to resource waste. Therefore, in the proposed federated reinforcement distillation approach, the model parameters are not transferred, but rather a proxy of the experience memory in the local domain L. Furthermore, in such multi-domain scenarios, local controllers J with larger neural networks can be used to achieve better task performance, while smaller models can be used in simpler task domains. This allows models in each local domain L to complete tasks within their local domain L more efficiently, thereby reducing the computational energy consumption of the local controller J, which can be a flying device 7, such as an aircraft 9a, a satellite 9b, a UAV 9c, and / or a ground vehicle 9d with limited energy resources.

[0041] The proposed hierarchical multi-domain network structure N framework aims to establish communication channels C for mobile network nodes O, thereby creating a mobile edge computing network system in which aircraft can act as edge controllers. The proposed network framework consists of two main levels: the control plane and the data distribution plane. The control plane includes the global controller H, which is deployed, for example, in a control center at a ground station 8 or in a satellite and high-performance aircraft 7, such as carrier aircraft, as domain controllers, one for each local domain L. The data distribution plane includes multiple flying devices 1 (e.g., UAVs, aircraft, drones) that can provide mobile network and data object D forwarding services to receivers B (e.g., on board the aircraft 9a or on the ground G).

[0042] In the proposed framework, the data distribution matrix is ​​determined directly by the deep reinforcement learning mechanism running in the global controller H and the various local controllers J, with the goal of achieving the best data distribution among all network nodes O in the data plane to achieve very low latency for all receivers B. After executing an action, each local controller J performs local actions that modify the current state of various network nodes O in the data plane of its local domains L. For example, it can be assumed that all local controllers J can communicate with the global controller H via air-to-ground or satellite communication. To ensure the confidentiality of the information in the various local domains L (e.g.,To protect topology, link state, node state, or network traffic state, the entire decision model deploys the federated learning framework on each local controller J and / or on the global controller H to improve data confidentiality and security while reducing network transmission overhead.

[0043] Each local controller J can be responsible for collecting status information within its local domain L and ensuring information consistency across the entire network structure N. When the local controller J receives an intra-domain data request, it begins by identifying the routing path to interested clients and then sends control messages to the network nodes O in the respective local domain L data layer to change their data caching state, thereby enabling data distribution with lower latency. The global controller H interacts with each local controller J to coordinate cross-domain data distribution. Multi-domain routing is facilitated by the local controllers J and / or the global controller H through a federated model.

[0044] The local controllers J can be responsible for maintaining routing information and data distribution within each local domain L and updating this information within the global controller H as parameters for federated learning. The global controller H collects parameters from all domain controllers and maintains a global federated learning model. Because the model parameters are transferred during the learning process, the specific routing path information and data distribution matrices within each local domain L can be protected, thus ensuring confidentiality.

[0045] Fig.Figure 2 shows a schematic representation of the steps S or phases of a procedure for evaluating routing paths R using their respective communication channels C. Certain steps S may involve specific decisions. An exemplary operation of the proposed procedure protocol may comprise four steps S or phases: first, an intra-domain update step S1; second, a parameter upload step S2; third, a global model training / aggregation step S3; and fourth, a parameter deployment step S4, as explained below:

[0046] In the first step, S1, or the intra-domain update phase, a local domain process can use an intra-domain routing algorithm to collect the necessary data to train the local model and update the relevant parameters. Each local domain L trains the model based on collected local data and stores states that are then aggregated to form a proxy state.

[0047] In the second step S2, or in the parameter upload phase, after all agents in a local domain L have filled the experience memory, the proxy state and the corresponding average strategy are calculated, and both are then uploaded to the global controller H.

[0048] In the third step, S3, or in the phase where the local proxy experience stores are uploaded to the global controller H, they can then be aggregated. The same proxy states from multiple domains can be combined into a single proxy state, and the corresponding policies can then be averaged again. The global controller H can train the global model using the aggregated proxy stores to generate global model parameters. Model convergence judgment can be performed before the global model parameters are output. If the model converges, this means that the global model has been learned and the federated reinforcement learning algorithm can terminate. Otherwise, the algorithm enters the parameter deployment phase, S4.

[0049] In the fourth step S4, or in the parameter provisioning phase, the global parameters can be delivered to each local controller J. The local controllers J can then assign the parameters to the local model and use local data to update the model training parameters.

[0050] This four-stage and / or four-phase process can be executed cyclically. During the training process, the experience replay technique can be used to store a series of states, actions, rewards, and next states obtained by the local controllers L through interaction with the environment in an experience replay pool. During training, fixed batches of data can be randomly selected from the experience pool to increase training speed. However, since each data object D stored after interaction with the environment contains the next moment's state, some correlation exists between the samples. To reduce the correlation between data samples and prevent the training process from reaching a local optimum, a random strategy can be applied when selecting the data set.Such an exemplary federated improved distillation algorithm, when combining the cross-domain agent experience store, does not result in the leakage of sensitive intra-domain data, and the proposed algorithm can reduce the amount of data that needs to be transferred, thereby reducing communication overhead.

[0051] While the foregoing detailed description presents at least one exemplary embodiment, it should be noted that a multitude of variations exist. It should also be noted that the exemplary embodiment or embodiments are merely examples and are not intended to limit the scope, applicability, or configuration of the invention in any way. Rather, the foregoing detailed description will provide the person skilled in the art with a convenient way to implement an exemplary embodiment of the invention. It is understood that various modifications to the function and arrangement of the elements described in an exemplary embodiment can be made without deviating from the scope of the claims.

[0052] It is further noted that "comprehensive" or "including" does not exclude any other elements or steps, and "a" or "an" does not exclude a plurality. It is further noted that features or steps described in relation to one of the exemplary embodiments mentioned above may also be used in combination with other features or steps of other exemplary embodiments described above. Reference numerals in the claims are not to be interpreted as limitations. List of reference symbols 1 Communication system 2 Communication device 3 Control unit 4 Interface module 5 Control element 6 transmission line 7 Device 8 Ground stations 9 vehicles 9a Aircraft 9b Satellite 9c UAV / Drone 9d Ground vehicle 10 computer systems 11 Computer / Control Program 12 computer-readable data carriers 13 computer-readable media 14 Data carrier signal k global number I local number A Source / Sender B Target / Recipient C Communication channel D Data object G floor H global control J local control K global domain L local domain M Storage capacity / storage N network structure O network node P local proxy R routing path S step L1 first local domain L2 second local domain L3 third local domain S1 Intra-domain update S2 parameter upload Aggregate / train the S3 global model S4 parameter provision QUOTES INCLUDED IN THE DESCRIPTION

[0000] This list of documents cited by the applicant was automatically generated and is included solely for the reader's convenience. The list is not part of the German patent or utility model application. The DPMA accepts no liability for any errors or omissions. Cited non-patent literature

[0000] Liu C, Feng W, Chen Y, et al. “Cell-Free Satellite-UAV Networks for 6G Wide-Area Internet of Things”, IEEE Journal on Selected Areas in Communications, vol.39, no.4, pp.1116-1131,2021

[0003] https: / / ieeexplore.ieee.org / document / 9174846

[0003] loannou A., Weber S., “A Survey of Caching Policies and Forwarding Mechanisms in Information-Centric Networking,” IEEE Commun. Surv. Tutor. 2016;18:2847-2886. doi: 10.1109 / COMST.2016.2565541

[0004] Zhang M., Luo H., Zhang H, „A Survey of Caching Mechanisms in Information-Centric Networking“, IEEE Commun. Surv. Tutor. 2015:17:1473-1499

[0006] G. B. Tarekegn, R. -T. Juang, H. -P. Lin, Y. Y. Munaye, L. -C. Wang und M. A. Bitew, „Deep-Reinforcement-Learning-Based Drone Base Station Deployment for Wireless Communication Services,“ IEEE Internet of Things Journal, vol. 9, no. 21, 1. Nov. 2022

[0007] https: / / ieeexplore.ieee.org / document / 9794697

[0007] Wang, L.; Zhang, H.; Guo, S.; Yuan, D. „3D UAV Deployment in Multi-UAV Networks with Statistical User Position Information,“ IEEE Commun. Lett. vol.26, no.6, pp.1363-1367, 2022

[0008] https: / / ieeexplore.ieee.org / document / 9739696

[0008] Z. Dai, Y. Zhang, W. Zhang, X. Luo und Z. He, „A Multi-Agent Collaborative Environment Learning Method for UAV Deployment and Resource Allocation“, IEEE Transactions on Signal and Information Processing over Networks, vol. 8, pp. 120-130, 2022

[0008] https: / / ieeexplore.ieee.org / document / 9712375

[0008] Z. Dai, Y. Zhang, W. Zhang, X. Luo und Z. He, „A Multi-Agent Collaborative Environment Learning Method for UAV Deployment and Resource Allocation,“ IEEE Transactions on Signal and Information Processing over Networks, vol. 8, pp. 120-130, 2022

[0008]

Claims

[1] Method for configuring a network structure (N) of a communication system (1) to provide a data link (C) for transmitting a data object (D) between a sender (A) and at least one receiver (B), in particular a vehicle (1), such as an aircraft, via at least one routing path (R) provided by network nodes (O) of the network structure (N), the method comprising the steps: Assigning a global number (k) of network nodes (O) of the network structure (N) to a global domain (K) that is controlled by a global controller (H); Providing a global model of the global domain (K) based on routing parameters that represent routing capabilities associated with the network nodes (O) assigned to the global domain (K); Assigning a local number (I) of respective network nodes (O), which are assigned to the global domain (K) as subsets of the global number (k), to at least two local domains (L), each controlled by a respective local controller (J); and Providing respective local models of the local domains (L) based on routing parameters that represent routing capabilities assigned to the network nodes (O) allocated to each local domain (L); wherein the global controller (H) interacts with the local controllers (J) to identify at least one of the network nodes (O) that provides routing capabilities according to the global model and / or the local models, enabling the at least one network node (O) to act as a local proxy (P) that allows the data object (D) to be placed along the at least one routing path (R) so that it can be provided from the local proxy (P) to the receiver (B). [2] Method according to claim 1, wherein the total local counts (I) are less than or equal to the global count (k). [3] Method according to claim 1 or 2, wherein the routing parameters comprise a trajectory parameter of a trajectory, a connection parameter of the data link (C) and / or a storage parameter of a storage capacity (M) of the network nodes (O). [4] Method according to at least one of claims 1 to 3, further comprising the step of performing an intra-domain update, wherein each local controller (J) collects the routing parameters to train and / or update the local model and / or the routing parameters. [5] Method according to claim 4, wherein each domain controller (J) trains the respective local model based on the routing parameters and stores states which are then aggregated to form a proxy state which is assigned to the at least one local proxy (P). [6] Method according to at least one of claims 1 to 5, further comprising the step of uploading routing parameters from the local controllers (J) to the global controller (H). [7] Method according to at least one of claims 1 to 6, further comprising the step of aggregating the local models into the global model. [8] Method according to at least one of claims 1 to 7, further comprising the step of providing the global parameters to the domain controllers (J). [9] Method according to at least one of claims 1 to 8, wherein the steps (S) of performing an intra-domain update, uploading routing parameters from the local controllers (J) to the global controller (H), aggregating the local models into the global model and / or providing the global parameters to the local controllers (J) are performed cyclically. [10] Control program (11) for controlling a communication system (1) comprising instructions which, when the control program (11) is executed by a control unit (3), cause the control unit (3) to execute a method according to at least one of claims 1 to 9. [11] Computer-readable data carrier (12) on which the control program according to claim 10 is stored. [12] Control unit (24) for providing a data connection (C) between a sender (A) and at least one receiver (B), wherein the control unit (24) is configured to execute a method according to at least one of claims 1 to 9 as a local and / or global control (H) and / or comprises a computer-readable data carrier (12) according to claim 11. [13] Communication device (2) configured to participate in a network structure (N) for establishing a data connection (C) for transmitting a data object (D) between a sender (A) and at least one receiver (B), in particular a vehicle (1), such as an aircraft, via at least one routing path (R) provided by network nodes (O) of the network structure (N), wherein the communication device (2) configured to perform a method according to at least one of claims 1 to 10 comprises a computer-readable data carrier (12) according to claim 11 and / or a control unit (24) according to claim 12. [14] Communication system (2) configured to provide a data link (C) for transmitting a data object (D) between a sender (A) and at least one receiver (B), in particular a vehicle (1), such as an aircraft, via at least one routing path (R) provided by network nodes (O) of a network structure (N), wherein the communication system (2) configured to perform a method according to at least one of claims 1 to 10 comprises a computer-readable data carrier (12) according to claim 11, a control unit (24) according to claim 12 and / or a communication device according to claim 13. [15] Device, in particular a vehicle (9), such as an aircraft, comprising a computer-readable data carrier (12) according to claim 11, at least one control unit (3) according to claim 12, at least one communication device (2) according to claim 13 and / or configured to participate as a network node (O) in a communication system (1) according to claim 14.

Images