Satellite-ground 6G autonomous evolution method driven by cross-layer data graph and system-level storage medium

Abstract

The invention belongs to the technical field of mobile communication and satellite communication, and relates to a satellite-ground 6G autonomous evolution method driven by a cross-layer data graph and a system-level storage medium. The method comprises the following steps: carrying out unified modeling on a clouded 6G wireless access network, a low-orbit satellite constellation and control software thereof, and constructing a cross-layer data graph simultaneously comprising code nodes, topological nodes, configuration nodes and performance index nodes; generating a controller activation and satellite teacher allocation scheme based on an optimization model taking propagation time delay and controller activation quantity as a joint target, and writing the controller activation and satellite teacher allocation scheme into a cross-layer data graph to form configuration nodes and performance nodes with teacher labels; a reconfiguration agent and a code evolution agent group are deployed on a cross-layer data graph, a coordination agent adaptively selects between network reconfiguration and code evolution, and reinforcement learning is carried out, so that cooperative autonomous evolution of satellite-ground 6G network configuration and control software is realized. The operation and maintenance cost is reduced; and the slice delay, the controller resource occupation and the service level agreement achievement rate are improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of mobile communication and satellite communication, and particularly relates to a cross-layer data graph driven satellite-to-ground 6G autonomous evolution method and a system-level storage medium. BACKGROUND

[0002] With the evolution of 6G technology, the access network gradually adopts cloud native, function decoupling, and network slicing technologies. Wireless access network functions are split into several function entities, deployed in a virtualized manner on central cloud, edge cloud, or local data center nodes, and instantiated and migrated on demand according to business types and network slicing requirements, thereby supporting various business scenarios such as ultra-low latency, ultra-high reliability, and large bandwidth.

[0003] At the same time, low-orbit (Low Earth Orbit, LEO) satellite constellations gradually become an important part of 6G non-terrestrial networks. In order to improve the programmability and unified management capability of the constellation network, the industry generally introduces a software-defined network (Software Defined Networking, SDN) architecture: satellites act as data plane nodes to forward traffic, and ground controllers act as control plane nodes to route, allocate resources, and manage overall strategies. Due to the high-speed movement of LEO satellites in orbit, the satellite-to-ground topology changes frequently, and how to dynamically determine "which controllers to activate, and who to manage each satellite" while meeting the latency constraints, and control the number of controllers and energy consumption, is a key problem.

[0004] In the prior art, the controller activation state and satellite-controller allocation relationship are usually modeled as a mixed integer linear programming (MILP) problem, and the propagation delay and the number of controllers are jointly optimized under the premise of a given algorithm implementation to obtain an optimal configuration scheme within a certain time slice. This method can achieve good results at the pure network level, but has the following limitations:

[0005] 1. Network control and software evolution are separated

[0006] The optimization model only acts on configuration parameters such as controller activation and satellite-controller mapping, and does not concern the software implementation behind these decisions. Defects in controller software, access network control software, and configuration generation scripts still need to be maintained through manual troubleshooting and manual modification.

[0007] 2. Lack of unified cross-layer semantic representation

[0008] Cloudification of access network involves multi-layer protocol stack and complex dependency, and satellite-ground network involves topology and physical layer characteristics; large-scale software repository supporting these functions has its own data flow and control flow. In the existing scheme, the software side and the network side usually use independent modeling methods, and there is a lack of a unified structure that can not only depict code-level data transformation, but also reflect network topology, configuration state and performance indicators.

[0009] 3. No double-time-scale autonomous evolution mechanism is formed

[0010] Network configuration adjustment (such as activating / closing controllers, reassigning satellites, migrating access network functions) is usually performed on a time scale of seconds to minutes; and modification, compilation, testing and release of software repository are usually completed on a time scale of hours or even longer. The existing technology generally optimizes only one of the two aspects, and does not establish a coordination mechanism between "fast network reconfiguration" and "slow code evolution", not to mention unified autonomous evolution control.

[0011] With the expansion of satellite-ground integration and the increase of business diversity, manual analysis and scattered tools alone cannot meet the requirements of "zero-touch operation and maintenance, automatic evolution" of 6G network. How to build a unified cross-layer representation and introduce intelligent decision-making on it to realize the co-evolution of network configuration and control software has become a problem to be solved. SUMMARY

[0012] The purpose of the present application is to solve the problems in the prior art that network control optimization and control software evolution are mutually separated, there is a lack of unified cross-layer representation, and there is a lack of double-time-scale autonomous evolution mechanism. A cross-layer data graph driven satellite-ground 6G autonomous evolution method and system-level storage medium are proposed. The code-level data state and transformation relationship of the control software, the topology structure, the controller activation state and the function placement state of the satellite-ground network, the performance indicators and reward signals generated by the network operation are uniformly depicted in the same graph structure, and a multi-agent reinforcement learning system is deployed on the graph to realize the collaborative control between network reconfiguration on a short time scale and software automatic evolution on a long time scale, thereby improving the performance and maintainability of the 6G satellite-ground integrated network. The technical solution of the present application can be understood as follows: on a unified cross-layer data graph, the satellite-ground 6G network topology, control configuration, software code semantics and performance indicators are simultaneously carried, and by introducing an optimization model teacher signal and a double-time-scale multi-agent control, network configuration adjustment and control software evolution form a learnable, constrained and convergent closed loop.

[0013] The technical scheme adopted by the present application is: a cross-layer data graph driven satellite-ground 6G autonomous evolution method, comprising the following steps:

[0014] Step 1: Unified modeling and data collection of satellite-to-ground 6G network and its control software, obtaining the running state information of 6G wireless access network function entities, network slices and their function placement positions deployed in the cloud platform, obtaining the satellite-to-ground network topology and link state information composed of multiple low-orbit satellites and multiple ground candidate controllers, and obtaining the control software source code and its execution information for realizing access network control, satellite-to-ground network control, and controller activation and satellite allocation;

[0015] Step 2: Based on the information obtained in step 1, a cross-layer data graph is constructed to obtain the running state information of 6G wireless access network function entities, network slices and their function placement positions deployed in the cloud platform, obtain the satellite-to-ground network topology and link state information composed of multiple low-orbit satellites and multiple ground candidate controllers, and obtain the control software source code and its execution information for realizing access network control, satellite-to-ground network control, and controller activation and satellite allocation; the cross-layer data graph is stored in a graph database and stored using an attribute graph model, each node and edge has a type identifier and an attribute field to support multi-agent state reading and action writing through a graph database query interface.

[0016] Step 3: Embed the satellite-to-ground control optimization model as a teacher subsystem into the cross-layer data graph, based on the current satellite-to-ground topology and business load, establish an optimization model with propagation delay and controller activation number as joint objectives, solve the optimization model in multiple scenarios to obtain the controller activation state and satellite-controller allocation scheme, write the solved controller activation state and satellite-controller allocation scheme into the configuration node of the corresponding time slice, and write the observed performance indicators when using the scheme into the corresponding performance indicator node, so that the configuration node and the performance indicator node become nodes with teacher labels;

[0017] Step 4: Deploy a double-time-scale multi-agent system on the cross-layer data graph, deploy a fast network reconfiguration agent, a code evolution agent group, and a coordination agent on the cross-layer data graph, the fast network reconfiguration agent is used to adjust the controller activation, satellite reallocation, access network function migration, etc. in a short time scale (seconds to minutes), the code evolution agent group is used to perform warehouse-level automatic repair and optimization of the control software in a long time scale (hours and above), and the coordination agent is used to decide whether to preferentially call the fast network reconfiguration agent, the code evolution agent group, or a combination of the two when detecting performance abnormalities or reaching the optimization period.

[0018] In the above design, the fast loop and the slow loop share the same cross-layer data graph and the same reward node; the fast loop more utilizes the teacher subsystem to speed up decision-making, and the slow loop more performs root cause analysis based on cross-layer tracking, thereby forming a closed loop of mutual cooperation.

[0019] Step 5: Self-evolution decision and execution, specifically, when preset performance anomalies are detected or a preset optimization period is reached, a coordination agent selects to enable a fast network reconfiguration agent and / or a code evolution agent group according to the performance indicator nodes and configuration node states in the cross-layer data graph;

[0020] Through the above steps, the application realizes the collaborative self-evolution of network configuration and control software in the unified cross-layer data graph environment, and gradually converges the system performance to a more optimal state under the premise of ensuring controllable risk.

[0021] Step 6: Unified reward and risk constraint, specifically, based on the changes of performance indicator nodes before and after the configuration changes and patch releases, the latency, controller number, energy consumption, and service level agreement violation rate are combined into a unified reinforcement learning reward value written into the reward node, and the configuration changes and code changes involved are constrained according to the preset risk budget, the candidate actions exceeding the risk budget are pruned or additional penalties are added, so that the multi-agent policy is updated under the unified reward and risk constraint, realizing the collaborative self-evolution of the star-ground 6G network configuration and control software.

[0022] Further, the step 1 specifically comprises: modeling a plurality of 6G wireless access network function entities and their deployment locations on the central cloud, edge cloud, and local nodes on the 6G access network side; modeling different network slices and their associated access network function combinations, resource quotas, and SLA indicators; collecting performance data such as latency, packet loss rate, and throughput of each function entity and each slice during operation.

[0023] On the star-ground control side, the topological relationship of multiple low-orbit satellites, multiple ground candidate controllers, inter-satellite links, and satellite-ground links is modeled; the satellite orbit position, link availability, controller load, and propagation delay are collected.

[0024] On the control software side, source code repositories that implement access network control, constellation control, controller activation and satellite allocation, function placement decision, etc. are selected; code structure, data flow, function call relationship, and execution trajectory of key configuration generation scripts are collected through static analysis and runtime monitoring.

[0025] Further, the step 2 includes: constructing a cross-layer data graph based on the above information, in which the code-level data nodes and code transformation edges are abstracted as code-level data nodes for key variables, objects, and data structures in the control software at different life cycle stages; assignment, function call, remote procedure call, configuration file generation, etc. are abstracted as code transformation edges, and the function, call context, and basic semantic description are recorded on the edges;

[0026] For topology object nodes and configuration state nodes, satellites, controllers, cloud / edge nodes and wireless units are abstracted as topology object nodes, recording location, capability and role information, and controller activation state, satellite-controller mapping relationship, access network function placement state and slice configuration are abstracted as configuration state nodes, and configuration transformation edges are established for controller activation / closure, satellite reassignment, function migration, slice capacity expansion and contraction and other operations,

[0027] For performance indicator nodes and measurement edges, time delay, packet loss rate, throughput, controller number, energy consumption and service level agreement (SLA) violation rate are abstracted as performance indicator nodes through configuration transformation edges, and are modeled separately according to slice or service type; topology object nodes and configuration state nodes are associated with performance indicator nodes through measurement edges, which are used to represent the relationship between performance indicators and topology and configuration changes;

[0028] For reward nodes and reward generation edges, different performance indicators are combined into overall benefits according to preset weights, and reward nodes used to carry reinforcement learning rewards are connected with performance indicator nodes through reward generation edges, so as to simultaneously represent code, topology, configuration and performance information in the same cross-layer data graph;

[0029] Further, the optimization model in step 3 is a mixed integer linear programming model, the objective function of which is the weighted sum of the average propagation delay normalized by the number of satellites and the number of activated controllers normalized by the number of controller candidates, the decision variables of the optimization model include allocation variables representing which controller manages each satellite and activation variables representing whether a controller is activated, and the constraint conditions include that each satellite can only be managed by one activated controller at any time slice, the propagation delay from a satellite to a controller does not exceed a preset threshold, and the load of each controller does not exceed its upper capacity limit, etc.

[0030] Run the optimization model on historical scenarios and typical simulation scenarios to obtain a set of approximately optimal controller activation and satellite allocation schemes, and write the allocation variables and activation variables obtained by optimization directly into the configuration state nodes of the corresponding time slices to form a class of labeled "teacher configuration state"; the performance indicator changes observed when the teacher configuration state is executed are marked on the related performance indicator nodes to form a class of "teacher performance labels";

[0031] In the reinforcement learning training phase, the teacher configuration state and the teacher performance label are used as reward shaping and policy distillation. Specifically, when the configuration selected by the agent in the same scenario is similar to the teacher configuration, the positive reward is increased; when the agent deviates from the teacher configuration but achieves better performance, the improvement space is reserved; and a penalty is given to the decision that is obviously worse than the teacher configuration.

[0032] Further, the fast network reconfiguration agent in step 4 adopts a policy gradient type reinforcement learning algorithm based on graph neural network encoding, whose state representation is a local subgraph embedding vector containing current configuration nodes, partial topology nodes and performance indicator nodes, and whose actions include activating or shutting down a certain controller, migrating a certain satellite from a first controller to a second controller, and migrating a certain access network function instance from a first computing node to a second computing node; its characteristics are: taking the local subgraph composed of current configuration state nodes, topology object nodes and performance indicator nodes as the observation; taking the configuration transformation edge as the main action set, and the action is directly acted on the running network after execution; taking the teacher configuration state as the priority reference area to reduce the exploration cost and speed up the convergence;

[0033] The code evolution agent group is used for warehouse-level automatic repair and optimization of control software in a long time scale (hour level and above), which specifically includes: a context agent for locating the associated configuration state node and code-level data node by traversing the measurement edge, configuration transformation edge and cross-layer association edge in reverse from the abnormal performance indicator node; a repair agent for generating a candidate repair or optimization scheme in the located code-level local subgraph; and an editing agent for converting the candidate repair scheme into specific source code modifications and writing them into a temporary branch of the code repository.

[0034] The coordination of the coordination agent is based on: the duration and severity of performance anomalies, whether similar problems in the history can be effectively alleviated by network reconfiguration, the current risk budget and the allowed change level.

[0035] Further, the step 5 includes:

[0036] The fast network reconfiguration agent selects configuration change actions such as controller activation, satellite redistribution, and access network function migration under the constraint or reference of the configuration node with a teacher label and acts on the running network;

[0037] The code evolution agent group generates candidate patches near the located code nodes and performs building and testing through continuous integration and simulation environment, and performs gray release on the patches that pass the test;

[0038] The effects of the fast network reconfiguration agent in the fast loop and the code evolution agent group in the slow loop are mapped to the reward node, and the reward function includes: the improvement degree of target slice latency and packet loss rate; the change of controller number or energy consumption; the change of SLA violation rate; the influence on key business and key slices;

[0039] The risk budget is set for the intelligent agent change and the code evolution intelligent agent group change of the fast network reconfiguration, budget information is marked in the related configuration state node and the code level node, for the candidate action exceeding the budget, the graph layer is directly pruned or a high penalty is added, so that the intelligent agent strategy is gradually contracted to a high yield action subset in the "acceptable risk range".

[0040] Further, the reinforcement learning reward value in the step 6 includes a weighted combination of the following parts: the improvement degree of the target slice end-to-end delay after the action is performed, the reduction degree of the number of activated controllers or energy consumption, the reduction degree of the service level agreement violation rate, and the penalty item of introducing a negative reward when involving a key service or a key slice; the risk budget is represented by setting a risk level field on the configuration node and the code node, and the coordination intelligent agent preferentially selects the candidate action meeting the risk budget when selecting the action.

[0041] Further, the code level data node at least includes a node type identifier, a symbol name, a data type, a source code file path and line number information; the topology object node at least includes an entity type, position information and capacity information; the configuration node at least includes a time slice identifier, a controller activation list, a satellite-controller mapping relationship, access network function placement information and slice configuration parameters; and the performance indicator node at least includes an indicator type, a belonging slice identifier, a numerical value and a measurement time window information.

[0042] Further, in the step 3, the controller activation state and the satellite-controller allocation scheme are written into the cross-layer data graph by solving the optimization model under multiple scenarios, the configuration node attribute `teacher_flag` is set to true, the `teacher_kpi` label is set in the corresponding performance indicator node, and the `teacher_source_id` identifier is recorded between the configuration node and the performance indicator node, so as to give a reward to the action close to the teacher scheme and good in performance, and give a penalty to the action far from the teacher scheme and poor in performance.

[0043] The application also provides a cross-layer data graph driven star-ground 6G autonomous evolution system, comprising: a cross-layer modeling and data acquisition module for performing step 1; a cross-layer data graph construction module for performing step 2; an optimization model teacher module for performing step 3; a multi-agent decision module for performing step 4; a continuous integration and simulation module for constructing, deploying and testing the code evolution result, and providing performance feedback for steps 4 and 5; a strategy learning and arrangement module for updating the intelligent agent strategy according to the reward node output, and arranging the decision result into an actual executable configuration change and a gray release plan.

[0044] Furthermore, the cross-layer data graph construction module is configured to add a risk budget field and a key business marker field to the configuration node, and a critical path marker field to the code node; when selecting configuration change and code change actions, the multi-agent decision-making module filters or penalizes high-risk actions based on the risk budget field and the critical marker field, thereby achieving autonomous evolution while ensuring the stability of key businesses and key slices.

[0045] The above modules can be deployed as software on one or more servers, or they can be integrated with existing network management systems and automated operation and maintenance platforms.

[0046] The present invention also provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, performs the aforementioned cross-layer data graph-driven autonomous evolution method for 6G satellite-to-ground communication.

[0047] Compared with the prior art, the present invention has the following beneficial effects:

[0048] 1. Form a unified cross-layer data graph representation

[0049] The cross-layer data graph proposed in this invention simultaneously represents code-level data flow, satellite-to-ground network topology, controller and function placement configuration, performance indicators and their reward relationships. It achieves unified modeling of software, network and performance within a single structure, providing a complete observable space for intelligent agents, which facilitates cross-layer root cause analysis and joint optimization.

[0050] 2. Integrating optimization models with reinforcement learning

[0051] This invention embeds the solution results of the optimization model for controller activation and satellite allocation into a cross-layer data graph, and uses it as a teacher signal or reference strategy during reinforcement learning training. This not only preserves the superior performance of the optimization model in small and medium-scale scenarios, but also adapts to large-scale and highly dynamic scenarios through the learning mechanism, thereby achieving a balance between performance and efficiency.

[0052] 3. Construct a dual-timescale autonomous evolution mechanism

[0053] By clearly distinguishing between the two timescales of fast network reconfiguration and slow code evolution, and having a coordinating agent select and coordinate between the two, this invention can not only quickly alleviate performance anomalies in a short period of time, but also fundamentally solve structural problems at the software level through code evolution over a longer timescale, forming a closed-loop autonomous evolution capability.

[0054] 4. Deep integration with 6G cloud-based access networks and space-ground integrated systems

[0055] The application models the access network function entity, network slice and function placement state as a configuration state node, and includes the slice SLA index in the performance index node and the reward generation process, so that the fine optimization for different slices and service scenarios is realized, and the characteristics of high heterogeneity and multiple services of the 6G space-ground integrated network are better adapted.

[0056] 5. It is beneficial to reduce operation and maintenance cost and improve network reliability

[0057] By introducing the multi-agent automatic decision and automatic patch generation mechanism on the cross-layer data graph, the application reduces the frequency of manual troubleshooting and manual upgrading under the premise of ensuring controlled risk, which helps to realize zero-contact operation and maintenance, and improves the overall reliability and sustainable evolution capability of the system. BRIEF DESCRIPTION OF DRAWINGS

[0058] Fig. 1 is a schematic diagram of the overall architecture of the system of the application.

[0059] Fig. 2 is a schematic diagram of the node and edge types of the cross-layer data graph of the application.

[0060] Fig. 3 is a schematic diagram of the controller activation and satellite allocation process in which the optimization model teacher signal is embedded in the application.

[0061] Fig. 4 is a schematic diagram of the double-time-scale multi-agent autonomous evolution process of the application.

[0062] Fig. 5 is a timing diagram of autonomous evolution decision and execution triggered by performance anomaly in the application. DETAILED DESCRIPTION

[0063] The application will be further described below in conjunction with the drawings and specific embodiments. Those skilled in the art can understand that the parameters, module division and deployment methods in the embodiments can be replaced or combined without departing from the concept of the application, and all fall within the protection scope of the application.

[0064] Embodiment one: system deployment architecture

[0065] As shown in Fig. 1, the cross-layer data graph driven space-ground 6G autonomous evolution system provided by the embodiment is deployed on the cloud network management platform of the operator, and physically includes the following three types of computing nodes:

[0066] 1. Management and control cluster

[0067] The cross-layer modeling and data collection module, the cross-layer data graph construction module, the optimization model teacher module, the multi-agent decision module, the strategy learning and arrangement module, etc. are deployed, and the typical form is several x86 servers with GPU or high-performance CPU, which are managed through a container orchestration system.

[0068] 2. CI / CD and simulation cluster

[0069] Deploy continuous integration and simulation modules, including code build services, automatic test services, satellite network simulation environments, and access network simulation environments, to replay real business and fault scenarios in isolated environments.

[0070] 3. Production network cluster

[0071] Including cloud platforms that actually carry 6G access network functions, LEO constellation controller clusters, and satellite access gateways. The production network regularly reports topology status, configuration status, and performance indicators to the management and control cluster through the interface, and receives configuration change instructions issued by the autonomous evolution system.

[0072] Clusters interact through a message bus. Preferably, the message bus uses a distributed message queue, such as a bus system based on a publish / subscribe mechanism, to deliver the following information:

[0073] Configuration change requests and execution results;

[0074] Running logs and performance indicators;

[0075] Simulation task scheduling and result return;

[0076] Multi-agent training data.

[0077] Through the above physical deployment, the autonomous evolution system and the production network interact through standard interfaces, without the need to modify existing network device hardware.

[0078] Embodiment two: data structure and storage method of cross-layer data graph

[0079] As shown in FIG. 2, in this embodiment, the cross-layer data graph is implemented using a property graph model. The property graph is stored in a graph database, such as a graph database based on a distributed key-value store, to support high-concurrency access to large-scale nodes and edges.

[0080] 1. Node type definition

[0081] The cross-layer data graph includes at least the following types of nodes, each with a type identifier `node_type` when stored:

[0082] Code node (CodeNode): used to represent data states in control software. Typical fields include:

[0083] `node_type = "code"`;

[0084] `symbol_name`: Variable or object name;

[0085] `data_type`: Data type, such as integer, struct, message type, etc.

[0086] `file_path` and `line_no`: Source code location;

[0087] `constraints`: Constraints that the data state should meet (optional field).

[0088] Topological Node (TopoNode): Used to represent network entities such as satellites, controllers, cloud / edge nodes, and wireless units. Typical fields include:

[0089] `node_type = "topo"`;

[0090] `entity_type`: Such as "satellite", "controller", "edge_site", "RU", etc.

[0091] `location`: Geographical location or orbital identification;

[0092] `capacity`: Computing / forwarding capacity.

[0093] Configuration Node (ConfigNode): Used to represent the controller activation state, satellite-controller mapping, access network function placement, and slice configuration at a certain time slice. Typical fields include:

[0094] `node_type = "config"`;

[0095] `time_slot`: Time slice identification;

[0096] `controller_state`: Controller activation list;

[0097] `sat2ctrl_map`: Satellite-controller mapping relationship;

[0098] `ranf_placement`: Placement of RAN function entities;

[0099] `slice_config`: Resource configuration of each slice;

[0100] `risk_level`: Risk level label under this configuration.

[0101] KPI Node: used to represent a performance indicator at a certain time window or a certain slice. Typical fields include:

[0102] `node_type = "kpi"`;

[0103] `kpi_type`: such as "latency" "loss" "throughput" "ctrl_count" "energy" "sla_violation" etc.

[0104] `slice_id`: the slice it belongs to.

[0105] `value`: the indicator value.

[0106] `time_window`: the measurement time window.

[0107] Reward Node: used to carry the reward calculated in the reinforcement learning process. Typical fields include:

[0108] `node_type = "reward"`;

[0109] `value`: the reward value.

[0110] `action_id`: the corresponding action identifier.

[0111] `episode_id`: the training round identifier.

[0112] 2. Edge type definition

[0113] At least the following types of edges are defined in the graph, and each edge has a type identifier `edge_type` when stored:

[0114] Code Transformation Edge (CODE_TRANSFORM): from one code node to another code node, representing data transformation such as assignment, function call, return value, or message encoding and decoding, etc.

[0115] Configuration Transformation Edge (CONFIG_TRANSFORM): from one configuration node to another configuration node, representing operations such as controller activation / deactivation, satellite reallocation, function migration, slice scaling, etc.

[0116] Measurement Edge (MEASURE): from a topology node or a configuration node to a KPI node, representing the impact of the corresponding topology / configuration on the performance indicator.

[0117] CROSS_REF: established between code nodes and configuration nodes, code nodes and topology nodes, used to represent the direct association between a piece of code and a specific configuration or entity;

[0118] REWARD_GEN: from KPI nodes to reward nodes, representing the composition source of rewards.

[0119] When the edge is stored, the following attributes can be attached: action type, constraint condition, timestamp, source module, etc.

[0120] 3. Construction process of data graph

[0121] The cross-layer data graph construction module operates according to the following process:

[0122] Step G1: Perform static analysis on the control software repository, obtain the abstract syntax tree from the compiler front end or language service interface, abstract the variables, structures and function parameters in it as code nodes, and abstract the assignment and call relationships as code transformation edges;

[0123] Step G2: In the test or simulation environment, plug in the key modules to collect runtime data flow information, which is used to supplement or correct the code transformation edges obtained by static analysis;

[0124] Step G3: Obtain the current space-ground topology, controller state, RAN function placement and slice configuration from the network management system, and abstract them as topology nodes and configuration nodes;

[0125] Step G4: Obtain various performance indicators from the performance monitoring system and probe system, abstract them as KPI nodes, and connect them with the corresponding topology nodes and configuration nodes through measurement edges;

[0126] Step G5: Analyze the configuration generation script and control logic, and establish cross-layer association edges between the corresponding code nodes and configuration nodes, topology nodes;

[0127] Step G6: According to the preset reward function, combine the values of multiple KPI nodes into a reward value, generate a reward node and connect it through a reward generation edge.

[0128] Through the above steps, the cross-layer data graph is continuously updated in the graph database, realizing the unified and dynamic description of the network state and the control software state.

[0129] Example Three: Engineering implementation of optimization model teacher module

[0130] As shown in FIG. 3, the optimization model teacher module in this embodiment is deployed in the form of an independent service, mainly including a model construction submodule, a solving submodule and a writing submodule.

[0131] 1. Model building submodule

[0132] The model building submodule reads the topology nodes and partial configuration nodes of the current time slice from the cross-layer data graph, and constructs the following optimization model:

[0133] Decision variables:

[0134] `x[i][j] ∈ {0,1}`, indicating whether satellite i is managed by controller j;

[0135] `y[j] ∈ {0,1}`, indicating whether controller j is in an active state.

[0136] Objective function example:

[0137]

[0138] where, is the average propagation delay normalized by the number of satellites, is the number of active controllers normalized by the number of controller candidates, is a weight parameter.

[0139] Main constraints:

[0140] Each satellite is managed by only one active controller:

[0141]

[0142] Delay constraint:

[0143]

[0144] Activation constraint:

[0145]

[0146] Controller load constraint:

[0147]

[0148] The above parameters `d(i,j)`, `load(i)`, `cap(j)` etc. are obtained by estimating the topology node attributes and historical KPIs in the cross-layer data graph.

[0149] 2. Solution submodule

[0150] The solution submodule calls a mixed integer linear programming solver (such as a commercial or open source MILP solving library) to solve the above model under multiple typical scenarios. Typical scenarios include but are not limited to:

[0151] Busy period (high traffic load);

[0152] Night low load period;

[0153] Partial controller failure or maintenance window;

[0154] Partial satellite link anomaly scenario.

[0155] In each scenario, the solving submodule caches the optimization results and attaches performance indicator evaluation results, such as average latency, number of activated controllers, maximum controller load, etc.

[0156] 3. Write submodule

[0157] The write submodule maps the controller activation and satellite allocation scheme obtained by solving back to the cross-layer data graph:

[0158] Create or update a configuration node for each scenario, write the `controller_state` and `sat2ctrl_map` fields, and attach the attribute `teacher_flag = true`;

[0159] Write the performance values obtained from the teacher scheme for the related KPI nodes, and mark `teacher_kpi = true` on the nodes;

[0160] Add edge attributes `teacher_source_id` between the configuration node and the KPI node for reward shaping in the subsequent reinforcement learning process.

[0161] In this way, the fast network reconfiguration agent can directly "see" the teacher scheme and its performance on the graph without separately accessing the optimization module.

[0162] Embodiment Four: Engineering Implementation of Double Time Scale Multi-Agent

[0163] As shown in Figure 4, this embodiment adopts a unified "environment service + multi-agent service" structure to realize double time scale autonomous evolution.

[0164] 1. Environment service

[0165] The environment service encapsulates the read-write interface of the cross-layer data graph, providing a standardized reinforcement learning environment for agents, including:

[0166] `reset()`: Initialize the current configuration node and KPI node according to the specified scenario, and return the initial state;

[0167] `step(action)` : invoke the configuration change or trigger code patch evaluation according to the action content, write the result into the graph, return the new state, reward value and flag of termination;

[0168] `observe()` : generate the observation vector or graph representation for the agent according to the current graph state.

[0169] In implementation, the observation can be encoded by graph neural network, embedding the local subgraph into a fixed dimension vector.

[0170] 2. Fast network reconfiguration agent

[0171] The fast network reconfiguration agent is implemented by policy gradient based reinforcement learning algorithm, such as proximal policy optimization (PPO). Its main features are:

[0172] State representation: intercept the local subgraph containing the current configuration node, related topology node and a small number of KPI nodes, and encode it into state vector by graph neural network;

[0173] Action space: including activating / closing the specified controller, migrating the specified satellite from controller A to controller B, migrating certain RAN function from node A to node B, etc;

[0174] Training time scale: sample at second to minute level time interval, update policy online;

[0175] Teacher fusion: compare the difference between the current action configuration and the teacher configuration when calculating the reward, give reward promotion to the action close to the teacher configuration and with good performance.

[0176] 3. Code evolution agent group

[0177] The code evolution agent group interacts with the simulation cluster through CI / CD and simulation cluster, and adopts offline batch training mode. Its process includes:

[0178] The context agent starts from the abnormal KPI node, and locates to the possible related configuration node and code node along the measurement edge, configuration transformation edge and cross layer association edge, extracts the corresponding local subgraph as the "problem context";

[0179] The repair agent generates candidate patch description based on the context, such as modifying control policy parameters, restructuring decision logic, etc;

[0180] The editing agent writes the candidate patch into the temporary branch of Git repository, and submits the build task to CI / CD system;

[0181] The simulation environment replays the historical business and fault scenarios, measures the KPI difference before and after the patch.

[0182] The policy learning and orchestration module calculates the reward and updates the repair and editing policy based on the KPI improvement degree and whether a new regression is introduced.

[0183] The training cycle of the code evolution intelligent agent group is usually in hours to days, preferably as a batch task running during business off-peak hours.

[0184] 4. Coordination agent

[0185] The coordination agent determines when to enable the fast loop or the slow loop through a combination of preset rules and learning strategies. Typical strategies include:

[0186] When the performance anomaly is a short-term burst, and historical records show that it can be solved by network reconfiguration, the fast loop is preferred;

[0187] When a certain type of anomaly still appears repeatedly after multiple fast loop adjustments, or the optimization model solution cannot meet the SLA for a long time, trigger the slow loop for root cause analysis and code evolution;

[0188] Limit the action amplitude of the fast loop during the execution of the slow loop to avoid mutual interference.

[0189] Embodiment five: autonomous evolution closed-loop operation process

[0190] As shown in FIG. 5, the autonomous evolution closed-loop operation process in this embodiment specifically includes the following steps:

[0191] 1. Abnormality detection:

[0192] The monitoring system in the production network uploads the end-to-end delay, packet loss rate, throughput, controller load and other indicators to the management and control cluster in real time. When it detects that the delay of a certain slice exceeds the threshold or a certain controller is overloaded for a long time, it triggers the autonomous evolution process.

[0193] 2. Decision selection:

[0194] The coordination agent reads the current abnormal KPI node, related configuration node and historical processing record from the cross-layer data graph, and judges whether to prefer fast network reconfiguration according to the rules and strategies.

[0195] 3. Fast adjustment:

[0196] If the fast loop is adopted, the fast network reconfiguration intelligent agent selects configuration change actions under the constraints of the teacher configuration, the environment service writes the changes to the production network, and re-collects KPI after a certain time window, and updates the cross-layer data graph.

[0197] 4. Effect evaluation:

[0198] The policy learning and orchestration module calculates the reward according to the difference between the new and old KPIs, and updates the fast network reconfiguration strategy. When the fast ring cannot control the KPI within the target range after multiple executions, the coordination agent triggers the slow ring.

[0199] 5. Code evolution and release:

[0200] The code evolution agent group automatically repairs and optimizes related codes, performs batch evaluation with the simulation cluster through CI / CD, releases patches with significant performance improvement and within the risk budget to the production network in a gray manner, and continuously monitors the effects.

[0201] 6. Continuous learning and convergence:

[0202] All actions and their results are written into the cross-layer data graph for experience replay in subsequent training processes, so that the multi-agent strategy gradually converges to a stable region that balances performance and risk.

[0203] Embodiment six: risk budget and key business protection

[0204] In this embodiment, to ensure the continuity of key businesses and key slices, the autonomous evolution system supports configuring risk budgets by business, by slice, and by region. Specific measures include:

[0205] 1. Add a field `risk_budget` to the configuration node to represent the maximum change amplitude allowed within the current time window, such as the upper limit of the number of RAN functions allowed to migrate, the upper limit of the number of controllers allowed to change, etc.

[0206] 2. Add a field `critical_flag` to the code node to mark critical path codes. For such codes, automatic patches are only allowed to enter the production network after obtaining higher-level approval or passing the verification in a dedicated test environment.

[0207] 3. Introduce a penalty term in the reward function. When an action involves a high-risk node and causes performance fluctuations, an additional negative reward is given to encourage the agent strategy to automatically avoid such high-risk operations.

[0208] As can be seen from the above embodiments, the present application not only provides a conceptual framework, but also provides a complete engineering implementation path from data structure, model solving, agent training to integration with the existing network, enabling those skilled in the art to implement the cross-layer data graph driven star-ground 6G autonomous evolution method and system accordingly.

Claims

1. A cross-layer data graph-driven autonomous evolution method for 6G satellite-to-ground communication, characterized in that, Includes the following steps: Step 1: Perform unified modeling and data acquisition on the 6G satellite-ground network and its control software. Obtain the operational status information of the 6G wireless access network functional entities, network slices and their functional placement locations deployed in the cloud platform. Obtain the satellite-ground network topology and link status information composed of multiple low-orbit satellites and multiple ground candidate controllers. Obtain the source code and execution information of the control software that realizes access network control, satellite-ground network control, controller activation and satellite allocation. Step 2: Based on the information obtained in Step 1, construct a cross-layer data graph, obtain the operational status information of the 6G wireless access network functional entities, network slices and their functional placement locations deployed in the cloud platform, obtain the topology and link status information of the space-ground network composed of multiple low-orbit satellites and multiple ground candidate controllers, and obtain the source code and execution information of the control software that realizes access network control, space-ground network control, controller activation and satellite allocation. Step 3: Embed the satellite-to-ground control optimization model as a teacher subsystem into the cross-layer data graph. Based on the current satellite-to-ground topology and service load, establish an optimization model with propagation delay and controller activation count as joint objectives. Solve the optimization model in multiple scenarios to obtain the controller activation state and satellite-controller allocation scheme. Write the obtained controller activation state and satellite-controller allocation scheme into the configuration node of the corresponding time slice, and write the performance indicators observed when using the scheme into the corresponding performance indicator node, so that the configuration node and performance indicator node become nodes with teacher tags. Step 4: Deploy a dual-timescale multi-agent system on the cross-layer data graph. Deploy a fast network reconfiguration agent, a code evolution agent swarm, and a coordination agent on the cross-layer data graph. The fast network reconfiguration agent is used to adjust controller activation, satellite reallocation, access network function migration, etc. within a short time scale (seconds to minutes). The code evolution agent swarm is used to perform repository-level automatic repair and optimization of the control software within a longer time scale (hours and above). The coordination agent is used to determine whether to prioritize calling the fast network reconfiguration agent, the code evolution agent swarm, or a combination of both when performance anomalies are detected or the optimization cycle is reached. Step 5: Autonomous evolution decision and execution. Specifically, when a preset performance anomaly is detected or a preset optimization cycle is reached, the coordinating agent selects to enable the fast network reconfiguration agent and / or the code evolution agent group based on the performance index nodes and configuration node status in the cross-layer data graph. Step 6: Unify rewards and risk constraints. Specifically, based on the changes in performance indicators before and after the execution of the configuration changes and patch releases, latency, number of controllers, energy consumption, and service level agreement default rate are combined into a unified reinforcement learning reward value and written into the reward node. The configuration changes and code changes involved are constrained according to a preset risk budget. Candidate actions that exceed the risk budget are pruned or additionally penalized, so that the multi-agent policy is updated under unified rewards and risk constraints, realizing the collaborative autonomous evolution of the space-ground 6G network configuration and control software.

2. The cross-layer data graph-driven autonomous evolution method for 6G satellite-to-ground communication according to claim 1, characterized in that, Step 1 specifically involves: on the 6G access network side, modeling multiple 6G wireless access network functional entities and their deployment locations in the central cloud, edge cloud, and local nodes; modeling different network slices and their associated access network functional combinations, resource quotas, and SLA indicators; and collecting performance data such as latency, packet loss rate, and throughput of each functional entity and each slice during operation. On the satellite-to-ground control side, model the topology relationships of multiple low-Earth orbit satellites, multiple candidate ground controllers, inter-satellite links, and satellite-to-ground links; collect operational status data such as satellite orbital positions, link availability, controller load, and propagation delay. On the control software side, source code repositories that implement functions such as access network control, constellation control, controller activation and satellite allocation, and function placement decisions are selected; the code structure, data flow, function call relationships, and execution trajectory of key configuration generation scripts are collected through static analysis and runtime monitoring.

3. The cross-layer data graph-driven autonomous evolution method for 6G satellite-to-ground communication according to claim 1, characterized in that, Step 2 includes: constructing a cross-layer data graph based on the above information; in the cross-layer data graph, abstracting the states of key variables, objects and data structures in the control software at different lifecycle stages into code-level data nodes and code transformation edges; abstracting assignment, function call, remote procedure call, configuration file generation, etc. into code transformation edges, and recording the function, calling context and basic semantic description on the edges; For topology object nodes and configuration status nodes, satellites, controllers, cloud / edge nodes, and wireless units are abstracted as topology object nodes, recording their location, capabilities, and role information. Controller activation status, satellite-controller mapping relationships, access network function placement status, and slice configuration are abstracted as configuration status nodes. Configuration transformation edges are established for operations such as controller activation / deactivation, satellite reallocation, function migration, and slice scaling. For performance indicator nodes and measurement edges, they are connected by configuration transformation edges. Latency, packet loss rate, throughput, number of controllers, energy consumption, and service level agreement (SLA) default rate are abstracted into performance indicator nodes and modeled separately according to slice or service type. The topology object nodes and configuration status nodes are associated with performance indicator nodes through measurement edges to represent the relationship between performance indicators and topology and configuration changes. For reward nodes and reward generation edges, different performance indicators are combined into the overall revenue according to preset weights. The reward nodes used to carry reinforcement learning rewards and performance indicator nodes are connected through reward generation edges, so that code, topology, configuration and performance information can be represented simultaneously in the same cross-layer data graph.

4. The cross-layer data graph-driven autonomous evolution method for 6G satellite-to-ground communication according to claim 1, characterized in that, The optimization model in step 3 is a mixed integer linear programming model. Its objective function is the weighted sum of the average propagation delay normalized by the number of satellites and the number of active controllers normalized by the number of controller candidates. The decision variables of the optimization model include an allocation variable indicating which controller manages the satellite and an activation variable indicating whether the controller is active. Its constraints include: each satellite can only be managed by one active controller in any time slice, the propagation delay from the satellite to the controller does not exceed a preset threshold, and the load of each controller does not exceed its capacity limit, etc. The optimization model is run on historical scenarios and typical simulation scenarios to obtain a set of near-optimal controller activation and satellite allocation schemes. The optimized allocation variables and activation variables are directly written into the configuration status nodes of the corresponding time slices to form a type of labeled "teacher configuration status". The performance index changes observed when executing the teacher configuration status are marked on the relevant performance index nodes to form a type of "teacher performance label". During the reinforcement learning training phase, the teacher configuration status and teacher performance labels are used as rewards for shaping and policy distillation. Specifically, when the agent selects a configuration similar to the teacher configuration in the same scenario, the positive reward is increased; when the agent deviates from the teacher configuration but achieves better performance, room for improvement is preserved; and decisions that are significantly worse than the teacher configuration are penalized.

5. The cross-layer data graph-driven autonomous evolution method for 6G satellite-to-ground communication according to claim 1, characterized in that, In step 4, the fast network reconfiguration agent employs a policy gradient-based reinforcement learning algorithm based on graph neural network encoding. Its state is represented as a local subgraph embedding vector containing the current configuration node, some topology nodes, and performance indicator nodes. Its actions include activating or deactivating a controller, migrating a satellite from the first controller to the second controller, and migrating an access network function instance from the first computing node to the second computing node. Its characteristics are: observations are based on a local subgraph composed of the current configuration state node, topology object nodes, and performance indicator nodes; configuration transformation edges are the main action set, and actions directly affect the running network; and the teacher's configuration state is used as a priority reference region to reduce exploration costs and accelerate convergence. The code evolution agent swarm is used to perform repository-level automatic repair and optimization of control software over a longer time scale. Specifically, it includes: a context agent that traverses backward from the abnormal performance index node along measurement edges, configuration transformation edges, and cross-layer association edges to locate the associated configuration state nodes and code-level data nodes; a repair agent that generates candidate repair or optimization schemes in the located code-level local subgraph; and an editing agent that converts the candidate repair schemes into specific source code modifications and writes them into a temporary branch of the code repository. The coordination criteria of the coordinating agent are: the duration and severity of the performance anomaly, whether similar problems in the past can be effectively mitigated by network reconfiguration, the current risk budget, and the allowed level of change.

6. The cross-layer data graph-driven autonomous evolution method for 6G satellite-to-ground communication according to claim 1, characterized in that, Step 5 includes: Under the constraints or references of configuration nodes tagged with teachers, the fast network reconfiguration agent selects configuration change actions such as controller activation, satellite reassignment, and access network function migration and applies them to the running network. The code evolution intelligent agent swarm generates candidate patches near the located code nodes and builds and tests them through continuous integration and simulation environment. Patches that pass the test are then released in a canary manner. The effects of fast loop reconfiguration agents and slow loop code evolution agents are uniformly mapped to reward nodes. The reward function includes: the degree of improvement in target slice latency and packet loss rate; changes in the number of controllers or energy consumption; changes in SLA default rate; and the impact on key services and key slices. Risk budgets are set for rapid network reconfiguration agent changes and code evolution agent swarm changes, respectively. Budget information is marked in relevant configuration status nodes and code-level nodes. For candidate actions that exceed the budget, they are pruned directly at the graph level or high penalties are added, so that the agent strategy gradually shrinks to a subset of high-return actions within the acceptable risk range.

7. The cross-layer data graph-driven autonomous evolution method for 6G satellite-to-ground communication according to claim 1, characterized in that, The reinforcement learning reward value in step 6 includes a weighted combination of the following: the degree of improvement in end-to-end latency of the target slice after the action is performed, the degree of reduction in the number of activated controllers or energy consumption, the degree of reduction in the service level agreement default rate, and the penalty item that introduces negative rewards when it involves critical business or critical slices. The risk budget is represented by setting risk level fields on configuration nodes and code nodes, and the coordinating agent prioritizes candidate actions that meet the risk budget when selecting actions.

8. A cross-layer data graph-driven autonomous satellite-to-ground 6G evolution system according to claims 1-7, characterized in that, include: The cross-layer modeling and data acquisition module is used to execute step 1; the cross-layer data graph construction module is used to execute step 2. Optimize the teacher module of the model to perform step 3; The multi-agent decision-making module is used to execute step 4; the continuous integration and simulation module is used to build, deploy and test the code evolution results, and provide performance feedback for steps 4 and 5. The strategy learning and orchestration module is used to update the multi-agent policy based on the output changes of performance indicator nodes and reward nodes, orchestrate the decision results into an actual executable network configuration change plan and patch canary release plan, and distribute them to the production network.

9. A cross-layer data graph-driven autonomous satellite-to-ground 6G evolution system according to claim 8, characterized in that, The cross-layer data graph construction module is configured to add a risk budget field and a key business marker field to the configuration node, and a critical path marker field to the code node; when selecting configuration change and code change actions, the multi-agent decision-making module filters or penalizes high-risk actions based on the risk budget field and the critical marker field, thereby achieving autonomous evolution while ensuring the stability of key businesses and key slices.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it performs the cross-layer data graph-driven autonomous evolution method for 6G satellite-to-ground communication as described in any one of claims 1 to 7.

Images