A large-area sea area beidou base station control network adaptive layout method

Abstract

The present application relates to the technical field of machine learning, in particular to a method for adaptive layout of a Beidou base station control network in a large sea area, which deeply integrates machine learning and reinforcement learning techniques: firstly, the stability of the system is evaluated using supervised learning, and the operating mode of the control network is identified through a variational autoencoder and unsupervised clustering, and a deep reinforcement learning decision-making agent is constructed, which takes the system state and environmental characteristics as input and is trained under the guidance of a composite reward function with adaptive risk perception. The present application uses a deep Q network structure (DQN) to construct a decision-making neural network, and iteratively optimizes the state-action value function, so that the agent can learn to generate optimal node layout, topology reconstruction and parameter adjustment instructions, and realize efficient, autonomous and intelligent operation of the control network in complex marine environments.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of machine learning, in particular to a large-area sea area Beidou base station control network adaptive layout method. BACKGROUND

[0002] In the field of sea area Beidou base station control network layout, the existing machine learning method based on neural network has several key technical bottlenecks. In terms of dynamic environment adaptability, the traditional static training paradigm cannot meet the real-time response demand, and the mainstream time series modeling method cannot balance the calculation efficiency and long-term dependence. In the small sample learning scene, there are challenges such as data scarcity and feature nonlinearity, which limit the model generalization ability. In the multi-objective optimization aspect, there are problems such as fixed target weight and low optimization efficiency of high-dimensional solution space. These technical bottlenecks collectively reflect the adaptability challenges of neural network models in complex sea area environment applications.

[0003] Therefore, innovation breakthroughs are needed in algorithm architecture and training paradigm to realize the adaptive layout of the control network. SUMMARY

[0004] The present application aims to solve the problem of poor survivability and low intelligence of large-area sea area Beidou base station control network in harsh environments. Through the adaptive adjustment of the operation strategy of the control network by the deep reinforcement learning agent, an intelligent, efficient and highly survivable sea area positioning is constructed.

[0005] A large-area sea area Beidou base station control network adaptive layout method, comprising the following steps:

[0006] The multi-dimensional system running state parameters and environmental feature vectors are taken as inputs, and a state evaluation neural network is trained by a supervised learning method to fit and generate a system stability index that can represent the running state of the system in a specific environment;

[0007] A deep feature extraction network model is used to learn the underlying features of the collected data channel between nodes, generate a core state vector reflecting the system health, and perform unsupervised clustering analysis on the core state vector to realize automatic classification and identification of the current running mode of the system;

[0008] A deep reinforcement learning decision-making agent is loaded, the state space of the agent includes the system stability index, environmental feature vector, system node resource occupation rate and data transmission path efficiency index, the action space of the agent includes adjustment instructions for system node layout, connection topology and running parameters, and the agent is trained through an adaptive risk perception composite reward function to iteratively optimize the decision-making neural network and generate adaptive system configuration and adjustment strategies.

[0009] Preferably, the system operating state parameters at least select three types of key indicators: the first type is the inter-node interaction signal quality factor reflecting data stream clarity, the second type is the data interaction delay index reflecting processing efficiency, and the third type is the data block error rate reflecting transmission fidelity; the environment feature vector includes sea wave level data, salt mist concentration index, and refractive index profile parameter of atmospheric waveguide effect.

[0010] Preferably, the step of extracting features by using the deep feature extraction network model specifically comprises: selecting a variational autoencoder as the deep feature extraction network model, the variational autoencoder being composed of an encoder and a decoder in a symmetrical manner; inputting the collected data channel bottom layer features into the encoder, compressing the input data through multi-layer nonlinear transformation, mapping to a low-dimensional hidden space, and outputting parameters for defining a local probability distribution in the hidden space, the parameters specifically being a mean vector and a variance vector of a Gaussian distribution; applying a reparameterization technique to randomly sample from the Gaussian distribution, and determining the sampling result as a core state vector.

[0011] Preferably, the unsupervised clustering analysis comprises: according to the set core point, boundary radius and minimum sample number parameters, adopting a density-based noise application spatial clustering algorithm to divide the core state vector into different dense regions, and establishing a mapping rule between the dense regions and the system physical operation mode; mapping a high-density sample cluster to a continuous system macroscopic operation state, and identifying a discrete low-density sample as an isolated incidental event.

[0012] Preferably, the action space of the deep reinforcement learning decision-making agent contains adjustment instructions specifically comprising: at least three different levels of operation strategies, the first level being adjustment of node physical interaction parameters, dynamically adjusting the output signal energy level of a certain node; the second level being data stream logical path reselection, when the performance of a certain key data transmission path is about to deteriorate, the agent can actively decide to switch the data stream carried by the path to one or more predefined redundant data paths; the third level being resource scheduling of system topology structure, when the system load is at a peak or a key node fails, the agent can issue instructions to remotely activate a backup node connection to reconstruct the system connection topology.

[0013] Preferably, the composite reward function comprises: a positive core performance reward term positively correlated with the increase amplitude or maintenance level of the system stability index; a positive service quality reward term triggered when the performance indicator of the key data transmission path is maintained above a preset threshold; and a negative resource consumption penalty term related to the system resource or energy consumption generated by executing the adjustment instruction; by configuring different weight coefficients for the core performance reward term, the service quality reward term and the resource consumption penalty term, a composite reward function for balancing multiple objectives is constructed.

[0014] Preferably, the generation of the adaptive system configuration and adjustment strategy comprises: the decision neural network can dynamically adjust the decision priority according to the environmental risk level reflected by the real-time values of the system stability index and the data transmission path performance indicator, and when the indicator deteriorates due to increased environmental risk, the adjustment instruction that can maximize the sum of core performance and service quality rewards is preferentially selected; on the contrary, when the indicator state is good and the environmental risk is low, the instruction that can minimize the resource consumption penalty term is preferentially selected, and an adaptive system configuration and adjustment strategy is generated.

[0015] Preferably, the implementation and optimization of the decision neural network specifically comprises: a deep Q network structure DQN is used to construct the decision neural network to fit the state-action value function of the deep reinforcement learning decision agent; during the training process, the agent generates experience data through interaction with the environment, and samples the experience data through an experience replay mechanism and sets up a parameter update delay target network for calculating target Q values, the calculation of the target Q values fuses the composite reward function; the loss function is minimized by minimizing the time difference error between the Q values predicted by the decision neural network and the target Q values calculated by the target network, and the weight parameters of the decision neural network are updated using a gradient descent algorithm until the network model converges.

[0016] Compared with the prior art, the beneficial effects of the present application are embodied in:

[0017] 1. The method can be used in a wide and complex environment, and the core idea of deep Q network is adopted to convert the complex deployment problem into a sequence decision problem, so that the model learns an optimal state-action value function through repeated interaction with the environment, and autonomously finds a base station deployment strategy close to the optimal one through self-learning and exploration. Instead of relying on fixed rules or formulas, it can dynamically adjust according to real-time feedback during deployment, realize truly adaptive deployment, and ultimately obtain a high-quality layout scheme.

[0018] 2. By storing the interaction experience into the data pool and randomly sampling, the time correlation between the training data is broken, so that the data can be reused, greatly improving the learning efficiency and the value of the sample, and improving the training efficiency and data utilization, which significantly shortens the time required for model training, and can make full use of the data generated by each simulation layout. Avoid the problem of low learning efficiency caused by strong data correlation, so that the model can learn from historical experience more quickly.

[0019] 3. The method effectively solves the problem of model parameter shock or non-convergence caused by changing learning goals during training, sets up a parameter delay update target network, separates the network used to calculate the target Q value from the decision network being updated, and delays the update of the target network, providing a relatively fixed "target" for training, thereby avoiding the instability of "pursuing a moving target" type training, ensuring reliable convergence of the algorithm. Ensure that the training process is more stable and reliable, and ultimately a high-performance decision model can be stably obtained. BRIEF DESCRIPTION OF DRAWINGS

[0020] Fig. 1 A step flowchart of a large-area sea area Beidou base station control network adaptive layout method is provided for the present application.

[0021] Fig. 2 A flowchart of the deep reinforcement learning decision-making agent action space adjustment instruction is provided for the embodiment of the present application.

[0022] Fig. 3 A step flowchart of the decision-making neural network is provided for the embodiment of the present application. DETAILED DESCRIPTION

[0023] The technical solutions in the embodiments of the present application will be described clearly and completely in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, not all. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor are within the scope of protection of the present application.

[0024] Please refer to Figs. 1 to 3 The present application provides a large-area sea area Beidou base station control network adaptive layout method, and the technical solutions are as follows:

[0025] The multi-dimensional system running state parameters and environmental feature vectors are taken as inputs, and the state evaluation neural network is trained by a supervised learning method to fit and generate a system stability index that can represent the running state of the system in a specific environment;

[0026] The deep feature extraction network model is used to perform feature learning on the collected data channel bottom features between nodes, to generate a core state vector reflecting the system health, and to perform unsupervised clustering analysis on the core state vector, to realize automatic classification and identification of the current operation mode of the system.

[0027] A deep reinforcement learning decision-making agent is loaded, the state space of the agent includes the system stability index, the environmental feature vector, the system node resource occupation rate and the data transmission path efficiency index, the action space of the agent includes the adjustment instruction of the system node layout, the connection topology and the operation parameter, and the agent is trained through a composite reward function with adaptive risk perception, the decision-making neural network is iteratively optimized, and an adaptive system configuration and adjustment strategy is generated.

[0028] Embodiment one:

[0029] The implementation scenario takes a certain deep sea area as an example, which has typical complexity, extreme weather, complex hydrogeology, variable atmospheric environment, high salinity sea wind, limited communication conditions and high maintenance cost. In such an environment, once encountering bad weather or equipment failure, it will cause large-area navigation service interruption, seriously affecting offshore operations. Therefore, the core goal of the embodiment method is to create a network intelligent agent that can realize real-time perception, intelligent diagnosis, autonomous decision-making and adaptive execution.

[0030] The Beidou base station control network in the implementation scenario is composed of several floating or semi-submersible base station platforms distributed in a certain deep sea area. Each base station platform integrates Beidou signal receiving, processing and forwarding module, high-precision atomic clock, communication module, various environmental sensors, internal state monitoring unit and energy supply system. All base stations perform bidirectional data interaction and instruction issuance with the central intelligent control platform located on land through high-reliability satellite communication link or long-distance microwave link. Referring to Fig. 1 A step flowchart of an adaptive layout method of a Beidou base station control network in a large-area sea area is provided for the present application.

[0031] The method integrates data fusion, state evaluation, pattern recognition, decision optimization and instruction generation five functions, each step is closely connected before and after, combines the control network itself with environmental characteristics, makes adjustment scheme through deep reinforcement learning agent decision-making, and forms a closed-loop perception from analysis to decision-making to execution process.

[0032] Further, the base station end data collection continuously collects and encrypts transmission of each base station node, specifically, the network performance data includes inter-node interaction signal quality factor SQF, data interaction delay index DID, data block error rate DBER; environmental feature data includes sea wave level data, salt spray concentration index, atmospheric waveguide effect refractive index profile parameters; node resource state data includes base station CPU memory occupation, power voltage and current, internal temperature, storage space, equipment running time; data channel bottom layer features include instantaneous channel gain, phase noise, multipath effect characteristics, frequency selective fading mode, bottom noise power spectral density, demodulation error vector amplitude high-dimensional original signal data;

[0033] The intelligent control platform is responsible for receiving and integrating various data uploaded by the base station, and distributing to different AI modules for processing. Each AI module works cooperatively to generate the optimal adjustment strategy, and transmits it back to the relevant base station node through a secure communication link to execute control network configuration and parameter adjustment. This way of receiving and issuing instructions for execution at any time can more intuitively perceive the current state of the control network and take the most suitable adjustment action for the current state.

[0034] Further, the input parameters and data acquisition include inter-node interaction signal quality factor SQF reflecting data stream clarity: the communication module of each base station is built-in with a high-performance digital signal processor, which monitors the signal strength RSSI, signal-to-noise ratio SNR, and bit error rate BER between itself and other base stations or satellite links in real time, and calculates the comprehensive quality factor percentage. When the base station platform shakes due to wind and waves in a certain sea area, or the signal fades due to atmospheric waveguide effect, SQF will decrease significantly.

[0035] The data interaction delay index DID reflecting processing efficiency is obtained by measuring the round-trip time and end-to-end delay of data packets between base stations. When the communication link is congested, the base station CPU load is too high, or the routing hop count increases, DID will increase.

[0036] The data block error rate DBER reflecting transmission fidelity uses the report of the cyclic redundancy check or forward error correction mechanism of the data packet to count the proportion of error bits in the received data block. When there is strong interference, severe multipath effect, or channel fading, DBER will increase significantly.

[0037] These system operating parameters are collected by the network monitoring agent of the base station at a frequency of once per second in real time, and by taking different dimensions of data in real time and packing them for uploading to the system for analysis and processing, the running state of each module in the base station is comprehensively controlled, and the running state is transmitted to the system to realize full-angle monitoring, ensuring that the real-time parameters of the control network are perceived by humans and recorded.

[0038] Further, the environmental feature vector includes sea wave level data, which measures sea wave height, period, etc. in real time, and automatically divides into 0-9 sea wave levels according to the international sea state standard. When a typhoon passes, the sea wave level will quickly rise from 3-4 to 7-9.

[0039] Salt fog concentration index: In summer, high temperature and high humidity, salt fog concentration is high all year round, which seriously corrodes equipment and affects the wave transmission performance of the radio frequency window.

[0040] Refractive index profile parameters of atmospheric waveguide effect: According to historical meteorological data and numerical prediction model, the profile curve of atmospheric refractive index changing with height is calculated. Key parameters such as waveguide height, waveguide strength, and waveguide layer top are extracted. When strong evaporation waveguide occurs, signals of certain frequency bands will propagate far on the sea surface, while signals at slightly higher altitudes will be severely attenuated.

[0041] These parameters directly affect the stability and fading characteristics of over-the-horizon communication links. The sea conditions are complex and change rapidly. By monitoring these three types of environmental feature vectors, the current sea environment can be remotely understood. Further, the future sea changes can be predicted according to the current environmental characteristics of the sea, and the risk of sea environment changes can be judged according to the prediction results, providing more forward-looking suggestions for parameter regulation to avoid uncontrollable sudden conditions caused by bad weather.

[0042] Further, at the land control center, a large amount of historical operation data is collected in advance, covering various sea conditions, weather conditions and system operation modes, and an expert team manually annotates the stability score of the system overall stability corresponding to each group of data. These data are used to train a multi-layer perception MLP neural network. The training process is to let the state evaluation neural network learn how to accurately predict system stability according to the input of 6 dimensions, including SQF, DID, DBER, sea wave level, salt fog concentration, and waveguide parameters. When the training is completed, this state evaluation neural network can receive current data in real time and immediately generate a quantitative system stability index SSI. For example, when a typhoon arrives, the sea wave level rises rapidly, accompanied by deterioration of DID and DBER, and the SSI will quickly drop from 85 to 40, indicating that the system is in an unstable state.

[0043] Further, the underlying features of the real-time captured inter-node data channel are extracted, including but not limited to: instantaneous channel gain, reflecting the real-time amplification or attenuation in the process of signal propagation; phase noise, measuring the degree of random jitter of signal phase, affecting the accuracy of data demodulation; multipath effect features, in complex sea areas, signals form multiple paths to the receiving end through sea surface reflection, atmospheric refraction, etc., and VAE analyzes the time delay, intensity and phase difference of these multipath signals; frequency selective fading refers to the fact that under certain channel conditions, signals in certain frequency bands will suffer more severe fading; noise floor power spectral density is used to measure the intensity distribution of background noise, which directly affects the signal-to-noise ratio; demodulation error vector magnitude is used to measure the deviation between the actual modulated signal and the ideal modulated signal, and directly reflects the signal transmission quality. These features collectively reflect the smoothness of the inter-node data channel. These original feature markers are high-sampling-rate, high-dimensional time-series data, for example, 100 signal snapshots are collected per second, and each snapshot contains different underlying physical parameters.

[0044] To extract comprehensive indicators to reflect the smoothness of the inter-node data channel and improve computational efficiency, a variational autoencoder is selected as the deep feature extraction network. It consists of two main parts: an encoder and a decoder.

[0045] Specifically, the encoder is responsible for compressing and mapping high-dimensional original channel feature data, such as a 100-dimensional vector composed of ten signal snapshots, as input. Through multiple layers of complex nonlinear transformations, these large data are compressed and mapped to a low-dimensional hidden space. In this process, the encoder does not directly output a fixed point, but outputs parameters that define a "local probability distribution" in this hidden space. For a Gaussian distribution, a series of mean vectors and variance vectors are output, representing the center position and spread of the distribution, respectively.

[0046] Specifically, the decoder reconstructs the original input data from the sampling results in the hidden space. The decoder also consists of multiple layers of nonlinear transformations, roughly symmetric with the encoder structure.

[0047] Further, during learning, VAE does not directly sample the mean and variance, but uses a reparameterization trick. This trick allows the system to indirectly implement sampling during the learning process by introducing a random, pre-set distribution noise. This noise is combined with the mean and variance output by the encoder to produce a specific vector drawn from the hidden space probability distribution. The low-dimensional vector obtained by sampling and containing the most essential features of the original channel data, the core state vector CSV, is mapped to a 16-dimensional vector, with each dimension representing a clear abstract feature of channel health.

[0048] By integrating the predicted environmental trends, real-time base station status and historical failure data as inputs to the risk assessment neural network, a comprehensive perception and proactive warning of the Beidou base station control network risk is achieved, rather than just passive response. The training data combines expert experience, preset rules and real historical records to ensure the accuracy and reliability of the risk labels, making the quantitative risk level output by the neural network more realistic and more meaningful. This not only improves the accuracy and forward-looking of risk assessment, but also provides a high-quality basis for subsequent deep reinforcement learning agents to make timely and effective adaptive deployment and risk avoidance decisions, significantly enhancing network resilience and management efficiency.

[0049] Further, a density-based noise application spatial clustering algorithm DBSCAN is used for unsupervised clustering analysis. The advantage of DBSCAN is that it can identify data clusters of any shape and effectively distinguish "dense" data points and "sparse" abnormal data points,

[0050] Specifically, two key parameters of DBSCAN are pre-set: the boundary radius is defined as the "neighborhood" range around a data point. For example, in the 16-dimensional space of CSV, a specific distance value of 0.5 is set, indicating that if the distance between two CSVs is less than 0.5, they are considered to be adjacent; the minimum sample size is defined as the minimum number of data points in the neighborhood required for a point to be a "core point". For example, setting it to 5 means that a CSV must have at least 5 other CSVs within a radius of 0.5 to be considered a core point.

[0051] Further, the massive core state vectors CSV generated by VAE during system operation are input into the DBSCAN algorithm. DBSCAN will automatically identify high-density regions in the CSV space based on the set boundary radius and minimum sample size, and divide the CSVs in these regions into different clusters.

[0052] Specifically, each identified high-density CSV cluster is mapped to a persistent system macroscopic operating state. The normal operating mode corresponds to a dense cluster with a large number of CSVs, indicating healthy channels and stable performance; the atmospheric duct interference mode corresponds to another CSV cluster, which is highly related to abnormal changes in the refractive index profile parameters, causing specific changes in signal propagation characteristics; the local channel degradation mode is caused by base station antenna icing, salt deposition or local hardware aging, and is characterized by collective deterioration of specific channel parameters; the mild electromagnetic interference mode is related to nearby ship radars or illegal signal sources, and is characterized by specific patterns of background noise.

[0053] While the discrete low-density samples are identified by DBSCAN as isolated CSV points that do not belong to any high-density cluster. These points are identified as isolated sporadic events or transient anomalies. Such as a short burst of strong impulse interference, a momentary communication link interruption or a transient jump in sensor readings.

[0054] Further, during system operation, each newly generated CSV is compared in real-time with the learned clusters. If it falls into a known cluster, the system can automatically identify that the current mode of operation is in that cluster. If it is identified as a noise point by DBSCAN, an anomaly alert is triggered, indicating that an unknown or transient anomaly has occurred.

[0055] Further, the state space of the agent, the snapshot of the agent's perception of the environment, is the comprehensive basis for making decisions. It is composed of the following real-time data: the system stability index SSI reflects the overall health of the system; the environmental feature vector reflects the challenge level of the external environment; the system node resource occupancy rate reflects the running load and hardware health of each base station; and the data transmission path performance index reflects the quality of network services.

[0056] These multi-dimensional data are integrated into a unified, high-dimensional vector as the input for each decision of the agent. Different data dimensions are used as comprehensive indicators to save computing resources, and a more explicit numerical value is displayed.

[0057] The action space of the agent stores a set of executable discrete operation instructions. In order to achieve fine-grained and multi-level network adjustment, the action space of this agent contains the following three different levels of operation strategies, which can cover a wide range of adjustments from the physical layer to the network layer:

[0058] Specifically, the first level: adjustment of node physical interaction parameters. When a signal in a particular direction is severely attenuated by atmospheric waveguide, the agent can instruct base station X to increase the transmission power by 3 decibels to penetrate the attenuation area; when the sea conditions are good and the signal is sufficient, instruct base station Y to reduce the transmission power to save energy and reduce potential interference to other communications; for the pointing deviation caused by the floating platform swaying in the sea waves, the agent can instruct base station Z to fine-tune the azimuth or elevation angle of the antenna to ensure accurate beam pointing to the target node or satellite.

[0059] The second level: when the performance of a certain key data transmission path is about to deteriorate, it is predicted that a typhoon will affect the path, or the delay continues to exceed the preset threshold, the packet loss rate increases, the agent can actively decide to switch the navigation data stream or control instruction stream carried by the path to one or more predefined redundant data paths with better current performance. This is achieved by modifying the routing table on the network router or the flow table rules in the software-defined network (SDN) controller, ensuring the continuity and reliability of key data transmission; when multiple paths are available, the agent can intelligently distribute data traffic among multiple paths according to the real-time load and bandwidth of each path, avoiding single-point congestion and improving overall network throughput.

[0060] The third level: during system load peaks, key node failures, such as main base station failures due to wind and waves or equipment aging, and dramatic deterioration of specific sea environments, the agent can issue instructions to remotely activate standby base station nodes that are pre-deployed but in low-power standby state. When a typhoon approaches and a main base station is expected to be temporarily out of contact, the agent can activate nearby standby base stations in advance, actively establish new communication links, quickly reconfigure the system connection topology, and ensure network coverage and redundancy; unnecessary nodes are turned off to save energy, and the agent can adjust the periodic sleep and wake-up strategy of the base stations according to the predicted traffic and environmental conditions, further optimizing energy consumption.

[0061] Reference Fig. 2 The flowchart of the deep reinforcement learning decision-making agent action space adjustment instruction proposed for the embodiments of the present application; the decision-making action defines three different action strategies, greatly enhancing the fine and multi-dimensional control ability of the deep reinforcement learning agent for the Beidou base station control network. Through the three-level hierarchical and comprehensive action space, the agent is given flexible and powerful adaptive deployment and management capabilities to effectively cope with complex and dynamic sea environments.

[0062] Further, the adaptive risk-aware composite reward function is a guide signal for the agent to learn good behavior. In order to balance multiple goals such as network stability, service quality and resource consumption, the method constructs a comprehensive composite reward function. This reward function is a weighted combination of multiple terms:

[0063] A positive core performance reward term: when the system stability index (SSI) improves compared to before or remains at a high level, the agent will receive a positive reward. This reward encourages the agent to take actions that maintain or improve the overall health of the network.

[0064] A positive service quality reward term: when the performance indicators of the critical data transmission path, the average delay is less than 100 milliseconds, the packet loss rate is less than 0.5% and is maintained above the preset high-quality threshold, the agent will also obtain a positive reward. This reward ensures that the agent optimizes the overall network performance while not affecting the transmission quality of the critical business.

[0065] A negative resource consumption penalty term: when the agent executes any adjustment instruction, it will consume corresponding system resources or energy, and increasing the transmission power and activating the standby node will consume more power; frequent switching of routes will increase the calculation and communication overhead. This penalty term is proportional to the amount of resources consumed, encouraging the agent to choose more efficient and energy-saving strategies.

[0066] The three reward terms are weighted and summed according to the preset weight coefficients to construct the final composite reward value. These weight coefficients are adjustable, for example, in the early stage of training, the weights can be evenly distributed; and after actual deployment, the weights can be fine-tuned according to the running strategy preference.

[0067] A single reward function only considers one dimension and cannot achieve balanced optimization of multiple objectives. By constructing a composite reward function, the positive and negative rewards and penalties are considered comprehensively, and the core performance reward and service quality reward encourage the agent to continuously improve network performance and user experience. At the same time, the negative resource consumption penalty term guides the agent to achieve performance targets while considering energy efficiency and operating costs. By configuring adjustable weight coefficients for each reward term, this method can flexibly adjust the optimization focus according to actual business needs or environmental changes, so that the strategies learned by the agent are more in line with actual needs, achieving fine-grained control of intelligent decision-making and optimal resource allocation.

[0068] Further, a deep Q network (DQN) structure is used to construct the decision-making neural network. The feedforward neural network includes multiple hidden layers. The input layer receives the current complete state vector, and the output layer has the same size as the action space. Each output neuron represents a Q value of an action, i.e., the long-term cumulative reward expected to be obtained after executing the action. This reward is updated through Q value backpropagation to update the network parameters, achieving long-term positive feedback. This combines gradient optimization of deep learning with value iteration of reinforcement learning to achieve efficient and stable strategy learning. Referring to Fig. 3 The step flowchart of the decision-making neural network proposed for the embodiments of the present application.

[0069] The training process is as follows: each experience generated by the agent during interaction with the simulated environment, including the current state, the action taken, the reward obtained, and the next state, is stored in an experience replay buffer. During training, the system randomly selects a small batch of data from this buffer for learning, rather than following the actual chronological order, which effectively breaks the correlation between data and improves the stability and efficiency of training.

[0070] Further, to solve the problem of unstable Q value calculation target during training, the system additionally sets up a target network with the same structure as the main decision-making neural network. The parameters of the target network are periodically copied from the main decision-making network once, but remain fixed between two copies. In this way, when calculating the target Q value, a more stable reference point can be used, thereby stabilizing the training process.

[0071] The purpose of training is to make the Q value predicted by the decision-making neural network approach the true Q value. This true Q value is calculated by the target network. The difference between the two is set as the loss function, and the system will try to minimize this difference. This difference is measured by calculating the temporal difference error between the two.

[0072] To minimize the loss function, the system uses the gradient descent algorithm to iteratively update the "weight parameters" inside the decision-making neural network. By continuously adjusting the parameters, the predicted Q value becomes more and more accurate, allowing the agent to choose the optimal action. Training will continue until the network model converges, i.e., the predicted Q value tends to be stable and no longer changes significantly.

[0073] Deep Q Network (DQN) fits the state-action value function to enable the agent to evaluate the long-term return of different actions. The experience replay mechanism effectively breaks the temporal correlation between training samples, improving the stability and data utilization of training. The design of the target network further stabilizes the Q value update process and alleviates the oscillation problem during training. By minimizing the temporal difference error and iteratively updating the network weights using the gradient descent algorithm, the decision-making neural network can efficiently converge and learn the optimal decision-making strategy, providing accurate and reliable adaptive configuration and adjustment instructions for the Beidou base station control network, which is a key technical guarantee for intelligent decision-making.

[0074] Further, the process of generating risk-aware adaptive system configuration and adjustment strategies is as follows:

[0075] Strategy in high-risk mode: When the SSI deteriorates to below 60 due to extreme environment such as typhoon, strong waveguide, or the performance indicators of the critical data transmission paths continue to deteriorate, the agent will recognize it as a high-risk mode. In this mode, the decision-making neural network will preferentially select the adjustment instructions that can maximize the sum of core performance rewards and service quality rewards. This means that even if the instruction will produce a higher resource consumption penalty, the agent will execute it without hesitation to ensure that the core functions of the network and the transmission of critical services are not interrupted. At this time, the weights assigned to the core performance reward and the service quality reward in the composite reward function will be dynamically increased, while the weight of the resource consumption penalty term will be relatively reduced.

[0076] Strategy in low-risk mode: Conversely, when the SSI is greater than 80 and the environment is low-risk, the sea conditions are stable, the salt spray concentration is low, there is no obvious waveguide effect, and all the performance indicators of the critical data transmission paths are in a healthy state, the agent recognizes it as a low-risk mode. In this mode, the decision-making neural network will preferentially select instructions that can minimize the resource consumption penalty term. It will tend to adopt a more energy-saving and conservative strategy, such as moderately reducing the transmit power of some nodes to save energy, or shutting down standby nodes that are not needed temporarily, so as to maximize resource utilization efficiency while ensuring good performance. At this time, the weight of the resource consumption penalty term in the composite reward function will be dynamically increased, while the weights of the core performance reward and the service quality reward will be relatively reduced.

[0077] Through this dynamic, risk perception-based weight adjustment mechanism, the agent can exhibit high resilience and adaptability, ensuring core functions in severe challenges, and striving for optimization of efficiency in stable environments, achieving true intelligence and adaptability of the Beidou base station control network. The decision-making neural network can dynamically adjust the decision priority to generate more environmentally adaptive system configuration and adjustment strategies. This embodiment first analyzes the control network operating state and environmental characteristics to determine the current risk level, and then implements priority adjustment based on the adaptive decision of the risk level. The intelligent judgment of the system reduces the dependence on human, so that the Beidou base station control network can intelligently balance performance and efficiency in different environments, can adjust at any time according to the environment, reasonably allocate resources, avoid unnecessary resource waste, and consider both system intelligence and economic benefits.

[0078] Embodiment two:

[0079] There are 15 Beidou base station nodes deployed in a certain deep sea area, in a distributed mesh structure. Among them, 5 base stations (numbered BS01-BS05) located on the typhoon path are the key areas affected by this typhoon. The central intelligent control platform is located on Hainan Island.

[0080] The scenario event is a strong typhoon approaching, which is divided into three stages to realize the response of the agent.

[0081] Stage one: typhoon approaching warning and early stage, response intelligent sensing and preventive adjustment, at this time the typhoon center is about 72 hours away from the BS01-BS05 area. The barometric pressure sensor detects that the air pressure continues to drop, the wind speed sensor starts to capture the wind speed enhancement, and the wave buoy shows that the sea wave level gradually rises from 2 level to 4 level. The atmospheric refractive index profile parameter shows that the evaporation waveguide effect begins to enhance, indicating that the signal propagation characteristics near the sea surface will change.

[0082] Specifically, the input end of the SSI evaluation module, although the SQF, DID, DBER has not deteriorated significantly at this time, but the change of environmental feature vector has begun to cause the SSI score to decrease from 88 to 80, prompting that the system stability is threatened.

[0083] The VAE continues to process the channel bottom features, and the CSV clustering model still mainly identifies the "normal operation mode" cluster, but occasionally CSV falls near the boundary of "unstable propagation mode" or "channel fluctuation mode".

[0084] Further, the intelligent control platform receives the updated SSI and environmental features, and identifies that the system is transitioning from "low risk" to "medium risk". Based on the trained decision neural network, the DRL agent predicts that the future system state will further deteriorate. Under the adaptive risk perception mechanism, the reward function weight at this time begins to tilt towards "core performance" and "service quality".

[0085] The agent makes a decision and issues preventive adjustment instructions: instruct BS01-BS05 and 2-3 nodes around them to increase the transmission power by 1-2 decibels to enhance the stability of the signal link and cope with the channel fading that will appear, and proactively identify the several key data transmission paths that will be most severely affected on the typhoon path, and pre-switch some non-critical data streams on these paths, such as device logs and secondary monitoring data, to other redundant paths to free up more bandwidth for the primary path and prepare for subsequent critical data transmission.

[0086] The instructions are located outside the typhoon path, but the two backup base stations (e.g. BS06, BS07) that can establish a connection with BS01-BS05 enter a pre-activated state, so that their communication modules are preheated to quickly go online when needed.

[0087] Stage two: typhoon core influence area, resilience maintenance in high-risk mode, at this time the typhoon center directly passes through BS02 and BS03. The sea wave level soars to 9 level, the salt fog concentration rises sharply, and the atmospheric waveguide effect will be temporarily weakened or become extremely unstable due to strong wind shear.

[0088] SQF from 90 sharply dropped to 40, DID from 50ms soared to 300ms, DBER from 0.1% sharply increased to more than 5%, SSI score from 80 points fell to less than 20 points, and the system entered an extremely high-risk mode. The internal temperature or power supply voltage of BS02 base station may have abnormal fluctuations, indicating that the device is under great pressure.

[0089] The VAE-generated CSVs fall into the cluster of severe channel fading mode or high noise interference mode, and a large number of CSVs are identified by DBSCAN as isolated sporadic events, i.e., short-term disconnection and recovery.

[0090] Further, the intelligent control platform identifies that the system is in an extremely high-risk mode. The adaptive risk perception mechanism is maximally activated, the reward function weight is completely reversed to the core performance and service quality, and the punishment weight of resource consumption is minimized. The DRL agent quickly evaluates the current state and decides to implement high-intensity and protective adjustments:

[0091] The transmission power of all available links of BS01-BS05 is raised to the highest limit to try to maintain the communication connection. The data flow logical path is reselected:

[0092] When it is detected that the key microwave link between BS02 and BS03 is temporarily interrupted due to strong winds and huge waves, the agent immediately decides to switch all the Beidou differential correction data streams passing through this link to the standby satellite backhaul link through BS01 and BS04. The agent decides to compress the non-critical data stream to the maximum extent, or even suspend its transmission, to ensure that the key navigation and positioning data can be transmitted with minimal delay.

[0093] BS06 and BS07 are instructed to fully online and try to establish new redundant links with all affected base stations to form new communication paths and build a temporary and more robust network topology.

[0094] Phase three: typhoon passing and recovery, intelligent optimization and energy saving, about 24 hours after the typhoon center has moved away from the BS01-BS05 area. The sea wave level gradually decreases to 5-level large waves, the salt spray concentration begins to decrease, and the atmospheric duct effect tends to be stable.

[0095] The SQF, DID, DBER and other indicators gradually improve, the SSI score from about 20 points steadily rises to more than 75 points, and the system from high risk to low risk. The CSV clustering model begins to identify more "normal operation mode" clusters, and the sporadic events decrease.

[0096] Further, the intelligent control platform identifies that the system risk is reduced. The adaptive risk perception mechanism begins to tilt towards the priority of resource consumption, and the reward function weight gradually returns to a balanced state.

[0097] The DRL agent evaluates the current state and decides to implement optimization and energy-saving adjustments:

[0098] Instruct BS01-BS05 and related nodes to gradually adjust the transmission power back to normal or slightly lower than normal level to save energy.

[0099] The agent re-evaluates the load and performance of all links and finds that the link between BS02 and BS03, which was previously interrupted, has been restored. The decision is to switch the key data streams switched out during the typhoon back to the original path with better performance or more directness, while optimizing traffic distribution to ensure network efficiency.

[0100] The agent detects that the network redundancy and performance have returned to a healthy level, and decides to downgrade the standby base stations such as BS06 and BS07 activated during the typhoon to low-power standby mode, or completely hibernate if the traffic allows, to maximize energy saving.

[0101] The agent reviews the temporary links and topologies established during the typhoon to deal with emergency situations when making decisions, evaluates long-term benefits, and for temporary links that are no longer needed or are not efficient, the agent will issue instructions to close or downgrade them, restoring a more streamlined and energy-efficient topology.

[0102] Through the highly coordinated and adaptive mechanism of artificial intelligence technology, this embodiment realizes the performance improvement, risk avoidance, resilience maintenance and energy efficiency optimization of the Beidou base station control network in complex sea environment, improves network availability, greatly shortens service interruption time, and shortens fault recovery time from hours to minutes, significantly reduces operation and maintenance cost and improves service quality. Through the above dynamic and phased adaptive response, the Beidou base station control network can exhibit unprecedented resilience, reliability and intelligent level in large-scale complex sea areas, especially in extreme weather, ensuring the continuity of Beidou services and effectively balancing performance and resource consumption.

[0103] Although embodiments of the present application have been shown and described, it will be understood by those of ordinary skill in the art that various changes, modifications, substitutions and alterations can be made without departing from the principles and spirit of the present application, and the scope of the present application is defined by the appended claims and their equivalents.

Claims

1. A method for adaptive layout of a large-area Beidou base station control network in a sea area, characterized in that, The method comprises the following steps: a multi-dimensional system running state parameter and an environment characteristic vector are taken as inputs, a state evaluation neural network is trained by a supervised learning method, and a system stability index capable of representing the running state of the system in a specific environment is fitted and generated; a deep feature extraction network model is used to perform feature learning on the collected underlying features of the data channel between nodes, a core state vector reflecting the health degree of the system is generated, and unsupervised clustering analysis is performed on the core state vector to realize automatic classification and identification of the current running mode of the system; a deep reinforcement learning decision-making agent is loaded, the state space of the agent includes the system stability index, the environment characteristic vector, the resource occupation rate of the system node, and the data transmission path efficiency index, the action space of the agent includes adjustment instructions for the system node layout, connection topology, and running parameters, and the agent is trained through a composite reward function with adaptive risk perception, the decision-making neural network is iteratively optimized, and an adaptive system configuration and adjustment strategy is generated; the system running state parameter selects at least three types of key indicators: the first type is a node interaction signal quality factor reflecting data flow clarity, the second type is a data interaction delay index reflecting processing efficiency, and the third type is a data block error rate reflecting transmission fidelity; the environment characteristic vector includes sea wave level data, salt mist concentration index, and refractive index profile parameters of atmospheric waveguide effect; the adaptive system configuration and adjustment strategy includes: the decision-making neural network can dynamically adjust the decision priority according to the environmental risk level reflected by the real-time values of the system stability index and the efficiency index of the data transmission path, when the efficiency index deteriorates due to increased environmental risk, the adjustment instruction that can maximize the sum of core performance and service quality rewards is preferentially selected; on the contrary, when the efficiency index is in good condition and the environmental risk is low, the instruction that can minimize the resource consumption penalty term is preferentially selected to generate an adaptive system configuration and adjustment strategy; the implementation and optimization of the decision-making neural network specifically include: a deep Q network structure DQN is used to construct the decision-making neural network to fit the state-action value function of the deep reinforcement learning decision-making agent; during the training process, the agent generates experience data through interaction with the environment, samples the experience data through an experience replay mechanism, and sets up a parameter update delay target network for calculating target Q values, and the calculation of the target Q values fuses the composite reward function; the loss function is obtained by minimizing the time difference error between the Q value predicted by the decision-making neural network and the target Q value calculated by the target network, and the weight parameters of the decision-making neural network are updated by using the gradient descent algorithm until the network model converges.

2. The method according to claim 1, wherein, the step of using a deep feature extraction network model for feature extraction specifically includes: a variational autoencoder is selected as the deep feature extraction network model, and the variational autoencoder is composed of an encoder and a decoder. The collected data channel bottom features are input into the encoder, the input data is compressed through multi-layer nonlinear transformation, mapped to a low-dimensional hidden space, and parameters for defining a local probability distribution in the hidden space are output, specifically the mean vector and variance vector of the Gaussian distribution; The reparameterization technique is applied to randomly sample from the Gaussian distribution, and the sampling result is determined as the core state vector.

3. The method according to claim 1, wherein, The unsupervised clustering analysis includes: According to the set core point, boundary radius and minimum sample number parameters, the noise application space clustering algorithm based on density is used to divide the core state vector into different dense areas, and a mapping rule between the dense areas and the system physical operation mode is established; High-density sample clusters are mapped to persistent system macro operation states, while discrete low-density samples are identified as isolated incidental events.

4. The method according to claim 1, wherein, The action space of the deep reinforcement learning decision-making agent contains adjustment instructions, specifically: At least three different levels of operation strategies, the first level is the adjustment of node physical interaction parameters, dynamically adjusting the output signal energy level of a node; the second level is data flow logical path reselection, when the performance of a key data transmission path is about to deteriorate, the agent can actively decide to switch the data flow carried by the path to a pre-defined redundant data path; the third level is the resource scheduling of system topology structure, when the system load is at peak or a key node fails, the agent can issue instructions to remotely activate the backup node connection and reconstruct the system connection topology.

5. The method according to claim 1, wherein, The composite reward function includes: A positive core performance reward item, positively related to the improvement amplitude or maintenance level of the system stability index; A positive service quality reward item triggered when the performance index of the key data transmission path is maintained above the preset threshold; And a negative resource consumption penalty item related to the system resource or energy consumption generated by executing the adjustment instructions; By configuring different weight coefficients for the core performance reward item, service quality reward item and resource consumption penalty item, a composite reward function for balancing multiple objectives is constructed.

Images