Dynamic spectrum sensing and covert communication method for unmanned aerial vehicles against environment

Abstract

The application discloses a UAV dynamic spectrum sensing and covert communication method suitable for resisting environment, and relates to the technical field of UAV communication; the application extracts a sensing quality index and an interference influence index based on initial interference signal characteristics, calculates a UAV single-machine fusion weight by combining an entropy weight method, dynamically weights and fuses single-machine sensing data of each UAV, and can further introduce an interference suppression factor to generate interference-spectrum double-feature fusion features to extract single-machine sensing data; the application can adaptively distribute fusion weights according to interference levels and sensing accuracy, avoids fusion errors caused by strong interference UAVs, fully develops the advantages of airspace diversity of multiple machines, significantly improves spectrum sensing accuracy and robustness, reduces multiple-machine data redundancy, and improves sensing real-time performance.

Description

Technical Field

[0001] This invention belongs to the field of UAV communication technology, specifically a dynamic spectrum perception and covert communication method for UAVs applicable to adversarial environments. Background Technology

[0002] With the widespread application of drones in reconnaissance, communication, and other tasks in adversarial scenarios, complex electromagnetic environments, illegal signal monitoring, and dynamic spectrum occupancy have become key factors restricting the reliable operation of drones. In adversarial environments with strong interference and high monitoring risks, drones need to identify idle spectrum through dynamic spectrum sensing and complete data transmission in a covert manner, which places extremely high demands on sensing accuracy, anti-interference capabilities, and communication stealth.

[0003] Existing UAV collaborative spectrum sensing and communication technologies mostly use fixed weights to fuse multi-UAV and single-UAV sensing data. At the same time, spectrum sensing and interference suppression are independent of each other, making it difficult to effectively reduce the pollution of sensing data by interference. This leads to large multi-UAV data fusion errors, insufficient sensing robustness, and high data redundancy. It is unable to adapt to dynamic changes in the adversarial environment and cannot meet the reliable operation requirements of UAVs in complex adversarial scenarios.

[0004] This invention provides a method for dynamic spectrum perception and covert communication of unmanned aerial vehicles (UAVs) for combating environmental challenges, in order to solve the aforementioned technical problems. Summary of the Invention

[0005] The present invention aims to solve at least one of the technical problems existing in the prior art; to this end, the present invention proposes a dynamic spectrum perception and covert communication method for unmanned aerial vehicles (UAVs) suitable for adversarial environments.

[0006] To achieve the above objectives, a first aspect of the present invention provides a method for dynamic spectrum sensing and covert communication of unmanned aerial vehicles (UAVs) suitable for adversarial environments, comprising: Initial sensing data is collected by UAVs based on collaborative sensing; the initial sensing data includes spectrum state characteristics, initial interference signal characteristics, and adversarial environment state characteristics. Extract individual UAV perception data from the initial perception data; dynamically fuse the individual perception data of each UAV to obtain target perception data; wherein, the dynamic fusion is achieved based on the individual fusion weights generated from the initial interference signal characteristics; Target perception data is input into a pre-trained reinforcement learning model to obtain communication decision results; covert communication is completed based on the communication decision results; the communication decision results include spectrum selection, transmission power and UAV scheduling strategy, and the reinforcement learning model is bound to a dynamic reward function.

[0007] In one possible implementation, the individual fusion weights of the UAV are calculated based on the characteristics of the initial interference signal, including: Extract the perception quality indicators and interference impact indicators of the UAV; among which, the perception quality indicators include the single-unit perception accuracy and the reliability of the spectrum characteristics, and the interference impact indicators include the interference level and the amplitude of the interference signal at the location of the UAV. Based on the perception quality index and interference impact index, the individual fusion weight of each UAV is calculated by combining the entropy weight method.

[0008] In one possible implementation, an interference suppression factor is introduced, and the interference suppression factor is combined with the initial perception data to extract the UAV's single-unit perception data, including: Generate interference suppression factors based on the characteristics of the initial interference signal; Dual-feature fusion features are generated based on interference suppression factors and spectral state features; single-unit perception data of UAVs are extracted based on dual-feature fusion features.

[0009] In one possible implementation, an interference suppression factor is generated based on the characteristics of the initial interference signal, including: The interference core parameters are extracted from the initial signal features, and an interference feature matrix is ​​constructed based on the interference core parameters; the interference core parameters include the interference signal amplitude, power spectral density, and interference duration. The interference feature matrix is ​​input into the minimum mean square error algorithm, and the interference suppression factor is obtained by iterative calculation according to the preset iterative objective. The iterative objective is set to maximize the energy of the idle spectrum signal and minimize the energy of the interference signal.

[0010] In one possible implementation, dual-feature fusion features are generated based on interference suppression factors and spectral state features, including: The perceptual suppression features are obtained by performing point-to-point weighted operations on the spectral state features and the interference suppression factor. The residual interference features after weighted operation are extracted, and the perceptual suppression features and residual interference features are concatenated by dimension to obtain the dual-feature fusion features of interference-spectrum.

[0011] In one possible implementation, a dynamic reward function is used to dynamically assign weights to the target optimization parameters; wherein the target optimization parameters include maximizing the covert rate of the legitimate receiver, maximizing the error probability of the illegitimate monitor, and minimizing the communication energy consumption.

[0012] In one possible implementation, the reinforcement learning model includes an online network and a target network; the online network is responsible for real-time inference, and the target network is used to stabilize the training process; the network structures of the online network and the target network are consistent.

[0013] In one possible implementation, an environmental state evaluation factor is introduced to construct a dynamic reward function, including: Calculate environmental status evaluation factors based on the characteristics of the adversarial environment. The weight coefficients of each objective optimization parameter are matched according to the environmental status evaluation factors; a dynamic reward function for the objective optimization parameters is constructed based on these weight coefficients; wherein, the matching of weight coefficients is achieved based on a pre-established weight mapping relationship.

[0014] In one possible implementation, environmental status evaluation factors are calculated based on the characteristics of the adversarial environment, including: Extract the state characteristics of the adversarial environment; among which, the state characteristics of the adversarial environment include interference intensity, spectrum change rate and monitoring risk level; The environmental status evaluation factor is obtained by weighting and summing the basic weights and standard values ​​of the indicators corresponding to the characteristics of the environmental status; the basic weights are set by expert scoring, and the standard values ​​of the indicators are normalized values.

[0015] Compared with the prior art, the beneficial effects of the present invention are: 1. Based on the characteristics of the initial interference signal, this invention extracts perception quality indicators and interference impact indicators, and calculates the fusion weight of individual UAVs using the entropy weight method. It then dynamically weights and fuses the perception data of each UAV. Furthermore, it can introduce an interference suppression factor to generate interference-spectrum dual-feature fusion features to extract individual UAV perception data. This invention can adaptively allocate fusion weights according to the interference level and perception accuracy, avoid fusion errors caused by strong interference UAVs, fully leverage the advantages of multi-UAV spatial diversity, significantly improve spectrum perception accuracy and robustness, reduce multi-UAV data redundancy, and enhance perception real-time performance.

[0016] 2. This invention calculates environmental state evaluation factors based on the characteristics of the adversarial environment, dynamically allocates weights for three-dimensional optimization parameters—stealth rate, monitoring error probability, and communication energy consumption—based on a weight mapping relationship, constructs an adaptive dynamic reward function, and binds it to a reinforcement learning model. This invention can adjust target weights in real time according to interference intensity and monitoring risk, prioritizing communication stealth and anti-monitoring capabilities in adversarial environments, while taking energy consumption control into account in stable environments, improving the adaptability of reinforcement learning decisions, accelerating model convergence, and meeting the real-time stealth communication needs of UAVs. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1This is a schematic diagram of the method flow for the UAV dynamic spectrum perception and covert communication method in an embodiment of the present invention; Figure 2 This is a schematic diagram of the calculation process for single-machine fusion weights in an embodiment of the present invention; Figure 3 This is a schematic diagram illustrating the construction process of the dynamic reward function in an embodiment of the present invention. Detailed Implementation

[0019] The technical solution of the present invention will be clearly and completely described below with reference to the embodiments. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0020] Please see Figure 1 The first aspect of the present invention provides a method for dynamic spectrum sensing and covert communication of unmanned aerial vehicles (UAVs) suitable for adversarial environments, comprising: Initial sensing data is collected by UAVs based on collaborative sensing; the initial sensing data includes spectrum state characteristics, initial interference signal characteristics, and adversarial environment state characteristics. Extract individual UAV perception data from the initial perception data; dynamically fuse the individual perception data of each UAV to obtain target perception data; wherein, the dynamic fusion is achieved based on the individual fusion weights generated from the initial interference signal characteristics; Target perception data is input into a pre-trained reinforcement learning model to obtain communication decision results; covert communication is completed based on the communication decision results; the communication decision results include spectrum selection, transmission power and UAV scheduling strategy, and the reinforcement learning model is bound to a dynamic reward function.

[0021] The adversarial environment refers to the complex interference and monitoring scenarios that UAVs face when conducting spectrum sensing and communication. It includes three key elements: the presence of illegal monitors (attempting to detect UAV communication signals in order to intercept and locate them), the presence of various types of spectrum interference (interfering with UAV spectrum sensing and communication links), and dynamic changes in spectrum occupancy status (not fixed idle / occupied, requiring real-time sensing).

[0022] In existing technologies, dynamic spectrum sensing and interference suppression are mostly implemented independently and simply superimposed, failing to achieve deep fusion at the feature level. Furthermore, multi-machine data fusion uses fixed weights, which cannot adapt to the dynamic interference characteristics of adversarial environments, making it difficult to balance sensing accuracy and real-time performance. This improvement abandons the traditional separate approach, jointly modeling adaptive interference suppression factors and spectrum sensing features. It enhances effective feature extraction through an attention mechanism, while optimizing the differential privacy federated transfer learning framework and introducing a dynamic weight allocation strategy to achieve synergistic linkage between "sensing-interference suppression-data fusion." This not only solves the problems of low sensing accuracy and multi-machine data redundancy caused by strong interference in adversarial environments but also achieves a synergistic improvement in sensing robustness and real-time performance.

[0023] Due to the complex scenarios in adversarial environments, a multi-drone collaborative sensing scheme is chosen to achieve dynamic spectrum sensing. This scheme effectively counteracts problems such as localized signal interference, multipath fading, and sensing blind spots in adversarial environments through multi-drone diversity and frequency band allocation. It achieves coverage of a wide airspace and all target frequency bands, and combined with data fusion, significantly improves the accuracy of spectrum occupancy assessment and interference source localization. Furthermore, the scheme is highly reliable; even if some drones are interfered with, others can still supplement the sensing spectrum data.

[0024] It should be noted that multi-drone diversity refers to the use of multiple drones to form different signal receiving spatial channels by dispersing them in space. This is a core technology principle of multi-drone collaborative sensing.

[0025] Initial sensing data includes spectral state characteristics, initial interference signal characteristics, and adversarial environment state characteristics. Spectral state characteristics include frequency energy characteristics, signal periodicity characteristics, frequency domain amplitude characteristics, and phase characteristics. Initial interference signal characteristics include interference type, interference intensity, and interference amplitude. Adversarial environment state characteristics mainly refer to adversarial environment state-related features.

[0026] The initial sensing data mentioned above is mainly extracted from the following data collected by UAVs. The specific data types include: time-domain waveform and sampling sequence of the target sensing frequency band signal, frequency domain amplitude and power spectral density data of each channel signal, original data of signal phase offset and periodic fluctuation, original energy, amplitude and duration data of interference signal, original basis for single-channel spectrum occupancy determination (idle / occupied / interfered), and original characteristic data of floor noise intensity and multipath fading.

[0027] The workflow for acquiring target perception data using UAVs based on collaborative perception is as follows: G01: Determine the frequency bands for collaborative sensing of UAVs and targets; Multi-drone collaborative perception involves using multiple drones as collaborative perception drones, with one collaborative perception drone acting as the master and the others as slaves; target perception frequency band refers to the target frequency range pre-defined for drone privacy communication, which is the legal communication frequency band that drones can use.

[0028] G02: The target sensing frequency band is divided into several sensing sub-bands according to the number of slave devices, and each slave device is associated with a sensing sub-band. The sensing sub-bands are spliced ​​together to form the target sensing frequency band. The master device is responsible for full-band scanning sensing of the target sensing frequency band, and the slave devices are responsible for scanning sensing of the corresponding sensing sub-bands to obtain initial sensing data. The initial sensing data includes spectrum occupancy status and initial interference signal characteristics. When acquiring initial sensing data through the host and slave devices, multi-coset preprocessing of sub-Nyquist sampling can be used, reducing the sampling rate to 1 / 3 of the Nyquist frequency. This can reduce the sampling pressure in adversarial environments, improve real-time performance, and also adapt to low-computing-power hardware.

[0029] G03: Calls a pre-trained feature extraction model to extract single-machine sensing data from single-machine sensing data; the feature extraction model is built and trained based on an artificial intelligence model. Single-machine perception data consists of spectrum characteristics, interference characteristics, and perception quality indicators obtained from local perception and processing by a single UAV, serving as the data foundation for multi-machine data fusion.

[0030] The feature extraction model can utilize existing mature models, such as convolutional neural network models, principal component analysis models, and energy detection feature models. This invention does not impose any restrictions on the type, structure, or number of feature extraction models.

[0031] The feature extraction model can also embed a softmax attention mechanism module to assign weights to the initial sensing data. When assigning specific weights, high weights (e.g., 0.92) are assigned to the spectral features of suspected idle channels, and low weights (e.g., 0.08) are assigned to the redundant features of interference signals. This strengthens the extraction of effective features and suppresses interference features, which can effectively reduce feature extraction errors.

[0032] G04: Based on the initial interference signal characteristics, calculate the single-unit fusion weight of each UAV using the entropy weight method; and perform weighted fusion of the single-unit perception data based on the single-unit fusion weight to obtain the target perception data. Target perception data includes idle spectrum information and interference signal information. Idle spectrum information includes idle spectrum bands, stable available duration, spectrum purity, etc., while interference signal information includes interference type, interference intensity, interference level, etc.

[0033] G05: Input the target perception data into the reinforcement learning model to obtain the communication decision result.

[0034] Communication decision outcomes include spectrum selection, transmit power, and UAV scheduling strategies. The reinforcement learning model is built and trained based on the DDQN model framework, and incorporates techniques such as online transfer learning, knowledge distillation, and online fine-tuning to improve the learning efficiency of the reinforcement learning model.

[0035] The lower the interference level and the higher the perception accuracy of the drone's location, the greater the weight of the single-drone fusion; conversely, the higher the interference level and the lower the perception accuracy of the drone's location, the smaller the weight of the single-drone fusion. Therefore, by using the single-drone weight coefficient as a basis, weighted fusion can obtain target perception data, which can give full play to the advantages of airspace diversity and avoid fusion errors caused by excessive single-drone interference.

[0036] Please see Figure 2 The process of generating the single-machine weight coefficient is as follows: Q01: Extract the perception quality indicators and interference impact indicators for each UAV; the perception quality indicators include the single-unit perception accuracy and the reliability of the spectrum characteristics, and the interference impact indicators include the interference level and the amplitude of the interference signal at the location of the UAV. Sensing quality indicators can be extracted from single-machine sensing data, meaning that sensing quality is analyzed simultaneously when acquiring single-machine sensing data. Interference impact indicators are determined based on the characteristics of the adversarial environment or the characteristics of the initial interference signal.

[0037] Q02: Standardize the perceived quality indicators and interference impact indicators to obtain the standardized matrix. This ensures that all indicator values ​​are within the range of [0,1], eliminating differences in indicator dimensions; among them, Number the perceived quality indicators. To perceive the quantity of quality indicators. Number the indicators that affect interference. To perceive the number of influencing indicators; Q03: Based on the standardized matrix, calculate the information entropy of each indicator; the formula for calculating the information entropy of an indicator is: ; Q04: Calculate the indicator difference coefficient and indicator weight coefficient; the formula for calculating the indicator difference coefficient is: The formula for calculating the indicator weight coefficient is: ; Q05: The initial weight score of each UAV is obtained by weighting the indicator weight coefficient and the indicator standardized value. After normalizing the initial weight score of each UAV, the individual weight coefficient of each UAV is obtained.

[0038] Interference suppression factors are designed to specifically mitigate the contamination of initial sensing data by factors such as illegal monitoring interference, environmental clutter, and multipath fading interference in the anti-interference environment. They eliminate feature shifts and noise redundancy caused by interference signals while preserving the effective features of the idle spectrum. Furthermore, interference suppression factors can dynamically match changes in interference intensity, avoiding the mismatch between interference capability and suppression capability that can occur with a fixed interference suppression factor.

[0039] The calculation process for the interference suppression factor is as follows: Y01: Extract core interference parameters from the initial interference signal features and construct an interference feature matrix based on the core interference parameters; among which, the core interference parameters include interference signal amplitude, power spectral density and interference duration; Y02: Input the interference feature matrix into the minimum mean square error algorithm, with the iterative objective of "maximizing the energy of the idle spectrum signal and minimizing the energy of the interference signal". The core iterative formula is: Calculate the optimal interference suppression factor , The value range of is [0,1], and the greater the interference intensity, the closer the interference suppression factor is to 1; where, The interference suppression factor to be iterated is... For ideal idle spectrum signals, The actual sensed signal contains interference. For mathematical expectation, The statistical mean used to calculate the error.

[0040] The process of generating dual-feature fusion features based on interference suppression factors and spectral state features includes: S01: Extract spectral state features from the initial sensing data. The spectral state features include spectral energy features, signal periodic features, frequency domain amplitude features, and phase features. The initial sensing data was not processed for interference suppression and noise reduction, and retained the characteristics of idle spectrum signals and superimposed interference and noise.

[0041] S02: Perform point-to-point weighted operation on the spectral state features and interference suppression factors to obtain the perceptual suppression features; extract the interference residual features after weighted operation, and concatenate the perceptual suppression features and interference residual features according to dimensions to obtain the interference-spectrum dual feature fusion features.

[0042] The weighted operation uses the interference suppression factor as a weighting coefficient and multiplies it one by one with the feature parameters in the initial sensing features to achieve targeted weakening of strong interference components while preserving the core effective features of the idle spectrum. Residual interference features refer to the weak interference signal features, noise features, and multipath fading features that are not completely weakened after weighted suppression by interference suppression factors. Residual interference features reflect the current suppression strength and residual interference strength, providing parameter basis for subsequent attention mechanism feature screening and multi-machine dynamic weight fusion, and are used to further accurately remove interference components while avoiding excessive suppression of effective features in the idle spectrum.

[0043] Perceptual suppression feature extraction after weighted operation of residual interference features. Specifically, this is achieved through a threshold screening method: a threshold value for the amplitude of residual interference features is preset, and all weak feature components in the weighted signal that are below the strong interference judgment threshold and above the base noise threshold are extracted. Valid features in the idle spectrum and completely eliminated interference components are removed to obtain a normalized residual interference feature vector. The dimension of this vector matches that of the weighted feature vector to ensure a smooth subsequent merging process.

[0044] In one example, the process of acquiring target perception data based on a collaborative perception scheme is as follows: Set up a confrontational environment scenario: 3 collaborative drones (1 master and 2 slaves), the target sensing frequency band is the 2.4GHz-2.483GHz civilian wireless frequency band, the environment has narrowband suppression interference emitted by an illegal monitoring party, the interference intensity is -65dBm (strong interference level), there is spurious noise interference in the surrounding area, and the spectrum occupancy status dynamically changes.

[0045] Step 1: The main unit is responsible for full-band scanning, and two slave units are responsible for the 2.4GHz-2.44GHz and 2.44GHz-2.483GHz sub-band sensing respectively. Multi-co-set preprocessing and sub-Nyquist sampling are adopted, and the sampling rate is reduced to 1 / 3 of the Nyquist frequency. Raw sensing data is collected. It is found that 3 of the 6 channels in the frequency band are occupied by licensed users, 2 are under strong suppression interference, and 1 is suspected to be idle. The interference signal is initially extracted as continuous wave suppression type, with an intensity range of -60dBm to -70dBm.

[0046] Step 2: Based on the initial interference characteristics, an adaptive interference suppression factor is generated through iterative calculation using the minimum mean square error algorithm. The initial factor weight is 0.85, which is dynamically fine-tuned to 0.82 as the interference intensity fluctuates. This suppression factor is directly embedded into the spectrum sensing feature vector and jointly modeled with the idle spectrum energy characteristics and periodic characteristics to construct a dual-feature fusion feature of interference-spectrum, simultaneously eliminating the fundamental wave feature offset error of the interference signal.

[0047] Step 3: Embed a softmax attention mechanism into the feature extraction network, assign weights to the fused feature vectors, assign a weight of 0.92 to the spectral features of suspected idle channels, and assign a weight of 0.08 to the redundant features of interference signals, thereby enhancing effective feature extraction, suppressing interference features, and reducing the feature extraction error from 18% in the traditional scheme to less than 6%.

[0048] Step 4: Real-time collection of perception quality indicators and interference impact indicators of the three UAVs; the main unit is in a weak interference area with a perception accuracy of 94%; slave 1 is in a strong interference area with a perception accuracy of 82%; slave 2 is in a moderate interference area with a perception accuracy of 88%; calculate the fusion weights using the entropy weight method: main unit 0.42, slave 1 0.26, slave 2 0.32; fuse to obtain target perception data.

[0049] Step 5: Output target perception data, specifically including: ① Idle spectrum information: The 2.451GHz-2.456GHz frequency band is locked as a stable idle frequency band with an availability time of ≥1.2s and a spectrum purity of 91%; ② Interference-related data: The location coordinates of the interference source (35° azimuth relative to the host, distance 1200m), the interference type is narrowband suppression interference, and the intensity level is strong; ③ Perception auxiliary data: The overall perception accuracy after multi-machine fusion is 92.5%, with no redundant interference features remaining, and is directly transmitted to the subsequent enhancement solution module as decision input.

[0050] In existing technologies, reinforcement learning applications in UAV communication decision-making often employ single-objective solutions or fixed-weight multi-objective solutions, which cannot adapt to the dynamic changes in interference intensity, spectrum status, and monitoring risks in adversarial environments. Furthermore, the algorithms suffer from slow training convergence and poor adaptability, making it difficult to meet real-time decision-making requirements. This invention improves the design of three-dimensional optimization objectives (stealth rate, monitoring error probability, and UAV energy consumption), introduces a dynamic reward function, and can adaptively adjust the objective weights according to the real-time state of the adversarial environment. Simultaneously, it combines transfer learning and online distillation strategies to achieve historical experience reuse and algorithm performance improvement, solving the core pain points of traditional reinforcement learning in dynamic adversarial scenarios.

[0051] The process of solving communication decision results based on target perception data is as follows: T01: Determine the target optimization parameters and target constraints; The target optimization parameters include: 1) Maximizing the covert rate of the legitimate receiver, with a covert rate constraint of ≥2Mbps; 2) Maximizing the error probability of the illegal monitor, with a probability constraint of ≥90%; 3) Minimizing communication energy consumption, with an energy consumption constraint of ≤0.8W. T02: Calculate the environmental status evaluation factor, assign weights to the target optimization parameters based on the dynamic environmental evaluation factor, and construct a dynamic reward function based on the weights assigned to the target optimization parameters; T03: Initialize the DDQN reinforcement learning model, set the state space dimensions, corresponding to idle frequency band parameters, stable availability time, spectrum purity, interference type, interference intensity, monitoring risk level, remaining UAV battery power, flight altitude, and communication distance; the action space includes frequency selection range, transmit power levels, and UAV scheduling strategies; the transmit power levels include three levels: 10dBm, 12dBm, and 14dBm, and the UAV scheduling strategies include the main unit maintaining hover, slave 1 lateral avoidance, and slave 2 auxiliary monitoring; When initializing the DDQN reinforcement learning model, transfer learning and online distillation strategies can be introduced to retrieve the parameters of pre-trained models from historical data with strong interference and high monitoring risk scenarios and transfer them to the DDQN reinforcement learning model, skipping the initial random training stage. This can reduce the number of iterations, shorten the convergence time, and meet the real-time decision-making needs of UAVs. T04: Bind the dynamic reward function to the reinforcement learning model. After the reinforcement learning model is fine-tuned, input the target perception data and the environment perception state into the DDQN reinforcement learning model to solve for the communication decision results. The communication decision results include spectrum selection, transmission power and UAV scheduling strategy.

[0052] Please see Figure 3 The calculation process for environmental status assessment factors and the construction process for dynamically establishing functions are as follows: H01: Extract the state characteristics of the adversarial environment; among which, the state characteristics of the adversarial environment include interference intensity, spectrum change rate and monitoring risk level; Interference intensity is categorized into weak interference (≤-75dBm), medium interference (-75dBm to -60dBm), and strong interference (≥-60dBm); spectrum change rate is categorized into slow (channel occupancy handover time ≥1s), medium (channel occupancy handover time between 0.5s and 1s), and fast (channel occupancy handover time ≤0.5s); monitoring risk level is categorized into low, medium, and high, determined comprehensively based on the distance to the interference source and the strength of the illegal monitoring signal. H02: An expert scoring method will be adopted, combined with engineering experience, to assign basic weights to various indicators in the characteristics of the environmental state, such as interference intensity of 0.4, monitoring risk level of 0.4, and spectrum change rate of 0.2; the actual value of each indicator will be converted into a standard value of the indicator in the interval [0,1]. The standard value of the indicator represents the strength of the corresponding indicator, eliminating the difference in dimensions. H03: The environmental status assessment factor is obtained by weighted summation of the basic weights and the standard values ​​of the indicators; the calculation formula for the environmental status assessment factor is: ;in, As an environmental status assessment factor, To number the indicators in the characteristics of the environmental state. The basic weights for the corresponding indicators, The standard value of the indicator; H04: Establish the weight mapping relationship between environmental status assessment factors and target optimization parameters; construct a dynamic reward function based on the weight mapping relationship; the dynamic reward function is: ,in, , and The weight coefficients are dynamically assigned based on the weight mapping relationship. As a stealth speed bonus, To monitor the error probability reward, This is an energy consumption penalty item.

[0053] The core logic of the weight mapping relationship is: the more severe the environment (environmental status evaluation factor) The larger the value, the higher the weight of the covert rate of the legitimate receiver and the probability of monitoring errors by illegal monitors, and the lower the weight of energy consumption; the environment tends to be stable (environmental state assessment factor). The smaller the value, the more appropriate the weight of energy consumption control should be, while also taking into account the drone's endurance; In one example, the weight mapping relationship is used to dynamically allocate weights to each objective optimization parameter based on environmental state factors. The weight allocation can refer to the following scheme: 1) High risk of strong interference (E≥0.7): Coverage rate weight 0.4-0.45, monitoring error probability weight 0.35-0.4, energy consumption weight 0.1-0.15; 2) Medium interference risk (0.3 < E < 0.7): Coverage rate weight 0.35-0.4, monitoring error probability weight 0.3-0.35, energy consumption weight 0.25-0.3; 3) Low interference and low risk (E≤0.3): Concealment rate weight 0.3-0.35, monitoring error probability weight 0.25-0.3, energy consumption weight 0.35-0.4.

[0054] The reinforcement learning model used in this invention is the DDQN model. The basic framework of the DDQN model includes an online network and a target network. The online network is the core network responsible for real-time inference and real-time parameter updates. In this invention, it is used to receive target-aware data and output communication decision results in combination with a dynamic reward function. The target network is used to stabilize the training process. Its parameters are not updated in real time. Its core function is to avoid training oscillations and Q-value estimation biases caused by frequent updates of the online network, and to provide a stable target reference for the parameter optimization of the online network.

[0055] The online network and the target network have the same network structure, both including: 1) State space design: including idle frequency band parameters, stable availability time, spectrum purity, interference type, interference intensity, monitoring risk level, UAV remaining battery power, flight altitude and communication distance; 2) Action space design: including spectrum selection strategy, power configuration strategy and UAV scheduling strategy; spectrum selection strategy is used to lock the target's idle frequency band, power configuration strategy is used to divide the communication transmission power into multiple levels, and UAV scheduling strategy includes hovering, avoidance and guarding, etc. 3) Initialization of basic parameters: Common hyperparameters, such as the initial value of the learning rate, discount factor, and capacity of the experience replay pool, are set according to the common values ​​in the field.

[0056] The process of initializing the DDQN model with pre-trained parameters based on transfer learning is as follows: C01: Retrieve the parameters of the source model obtained from offline pre-training; the source model is a DDQN model that has been pre-trained offline in various typical adversarial scenarios and has converged. This source model stores the optimal network weights, bias terms and other model parameters, and has basic spectrum decision and anti-interference decision capabilities. C02: Transfer the parameters of the source model to the corresponding network layer (online network and target network) of the DDQN model. During the transfer process, only the dimension alignment of the state space and action space is performed. The parameter values ​​are not modified and gradient calculation is not performed. C03: The DDQN model that has completed online migration is used as the teacher model. A lightweight student model is built based on the core architecture of the teacher model. The student model is a simplified version of the teacher model. It retains the core structure of the online network and the target network. It is lightweight only by reducing the number of hidden layer neurons, simplifying the complexity of fully connected layers, and compressing the convolution kernel dimension. It does not change the network input and output dimensions, the matching rules of state space and action space, and ensures compatibility with the previous and subsequent processes. C04: Through knowledge distillation, the decision logic, feature extraction rules, and pre-trained knowledge of the teacher model are synchronously transferred to the student model. This completes the parameter initialization of the online network and the target network within the student network, enabling the transfer of mature knowledge from the large model to the small model. This preserves the core decision-making capabilities while reducing computational consumption. After the transfer is completed, all parameters of the teacher model remain fixed, gradient backpropagation is turned off, parameter modification is prohibited, and the student model is deployed in a drone.

[0057] The online fine-tuning process for the initialized DDQN model (student model) is as follows: WO1: Bind the dynamic reward function to the student model, freeze the parameters of the feature extraction layer of the student model, and only iteratively optimize the weights and biases of the decision output layer. This ensures that the general feature extraction capability remains unchanged, can quickly adapt to the current real-time adversarial scenario, and reduces the amount of computation and shortens the convergence time, thus adapting to the low computing power and low latency requirements of drones. W02: Call the target perception data and update the student model in combination with the dynamic reward function. When the reward value tends to stabilize and no longer fluctuates greatly, or after reaching the preset number of iterations, the fine-tuning is terminated, and a reinforcement learning model that can be used for communication decision-making is obtained.

[0058] The update rules for the student model include: the online network is responsible for receiving target perception data in real time, outputting communication decision results, calculating the loss function based on the dynamic reward value, and updating the top-level parameters in real time through gradient backpropagation; the network parameters of the target network are not updated in real time, but the network parameters in the online network are copied to the target network at each fixed iteration step to keep the target network parameters stable in stages and avoid training oscillations.

[0059] In one example, the process of solving for the communication decision result is illustrated using the previous example, as follows: In the previous example, three collaborative drones (one master and two slaves) were operating in the 2.4GHz-2.483GHz frequency band. There was narrowband suppression interference from an illegal monitoring party (intensity -65dBm, strong interference, high monitoring risk). The sensing link had locked the idle frequency band 2.451GHz-2.456GHz, and the sensing data was complete and usable. The distance between the legitimate receiver and the master drone was 800m. The rated power consumption limit for a single frame of communication of the drone was 0.8W. The preset concealment rate threshold was ≥2Mbps. The detection error probability threshold of the illegal monitor was ≥90%.

[0060] Step 1: Based on target perception data, determine the three-dimensional optimization objectives and hard constraints: ① Objective 1: Maximize the covert rate of the legitimate receiver, constraint ≥ 2 Mbps; ② Objective 2: Maximize the detection error probability of the illegal monitor, constraint ≥ 90%; ③ Objective 3: Minimize the power consumption of UAV communication transmission, constraint ≤ 0.8 W; Simultaneously import perception data: idle frequency band 2.451 GHz-2.456 GHz, interference source azimuth 35° distance 1200 m, fusion perception accuracy 92.5%.

[0061] Step 2: Introduce environmental status assessment factors and calculate using fuzzy comprehensive evaluation: Given strong current interference, high monitoring risk, and slow spectral change rate, the environmental status assessment factor score is 0.89. Dynamically allocate three-dimensional target weights: concealment rate 0.45, monitoring error probability 0.40, and energy consumption control 0.15, prioritizing concealment and anti-monitoring performance while considering energy consumption control. If subsequent interference weakens and monitoring risk decreases, automatically switch the weights to concealment rate 0.35, monitoring error probability 0.30, and energy consumption control 0.35. Construct a dynamic reward function based on the allocated weights. Step 3: Initialize the DDQN reinforcement learning model, set the state space dimensions, including idle frequency band parameters, interference intensity, monitoring risk level, remaining UAV battery power, flight altitude, and communication distance; the action space includes three types of selectable actions: ① Spectrum selection: lock onto 2.451GHz-2.456GHz; ② Transmit power: three levels of 10dBm, 12dBm, and 14dBm; ③ UAV scheduling: host UAV maintains hovering, slave UAV 1 lateral avoids, and slave UAV 2 assists in monitoring; embed the dynamic reward function into the model, the reward value is positively correlated with stealth rate and monitoring error probability, and negatively correlated with energy consumption.

[0062] Step 4: Introduce transfer learning and online distillation strategies to retrieve pre-trained model parameters from historical scenarios with strong interference and high monitoring risks, and directly transfer them to the current model initialization, skipping the initial random training stage; compress the number of model parameters through online distillation technology, reducing the number of iterations from the traditional 2000 to 600, and shortening the model convergence time to 0.3s, meeting the real-time decision-making needs of UAVs.

[0063] Step 5: Input target perception data, and the model solves to output the optimal communication decision result: ① Spectrum selection: Fixed use of the 2.451GHz-2.456GHz idle frequency band; ② Transmit power: Configured to 12dBm (lower than the illegal monitor signal detection threshold, balancing concealment and communication quality); ③ UAV scheduling: The main unit hovers to maintain communication, slave 1 moves 50m in the opposite direction of the interference source to avoid it, and slave 2 stays in place to provide backup; Finally, the concealment rate is 2.8Mbps, the illegal monitor detection error probability is 93.2%, and the single-frame communication power consumption is 0.52W, all of which meet the constraints. The result is directly output to the subsequent adaptive parameter configuration module, completing the entire closed loop from perception to decision.

[0064] It should be understood that the descriptions of technical features, technical solutions, beneficial effects, or similar language in this application do not imply that all features and advantages can be achieved in any single embodiment. Rather, it is understood that the description of a feature or beneficial effect means that a specific technical feature, technical solution, or beneficial effect is included in at least one embodiment. Therefore, the descriptions of technical features, technical solutions, or beneficial effects in this specification do not necessarily refer to the same embodiment. Furthermore, the technical features, technical solutions, and beneficial effects described in this embodiment can be combined in any suitable manner. Those skilled in the art will understand that embodiments can be implemented without one or more specific technical features, technical solutions, or beneficial effects of a particular embodiment. In other embodiments, additional technical features and beneficial effects may be identified in specific embodiments that do not embody all embodiments.

[0065] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any other combination thereof. When implemented using a software program, it can be implemented entirely or partially in the form of a computer program product. This computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium accessible to a computer or a data storage device containing one or more servers, data centers, etc., that can be integrated with the medium. The available media can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media (e.g., solid-state disks (SSDs)).

[0066] The above embodiments are only used to illustrate the technical methods of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical methods of the present invention without departing from the spirit and scope of the technical methods of the present invention.

Claims

1. A method for dynamic spectrum sensing and covert communication of unmanned aerial vehicles (UAVs) suitable for adversarial environments, characterized in that, include: Initial sensing data is collected by UAVs based on collaborative sensing; the initial sensing data includes spectrum state characteristics, initial interference signal characteristics, and adversarial environment state characteristics. Extract individual UAV perception data from the initial perception data; dynamically fuse the individual UAV perception data to obtain target perception data; wherein, the dynamic fusion is achieved based on the individual fusion weights generated from the initial interference signal characteristics; The target perception data is input into a pre-trained reinforcement learning model to obtain communication decision results; covert communication is completed based on the communication decision results; wherein, the communication decision results include spectrum selection, transmission power and UAV scheduling strategy, and the reinforcement learning model is bound to a dynamic reward function; An interference suppression factor is introduced, and the interference suppression factor is combined with the initial perception data to extract the single-unit perception data of the UAV, including: An interference suppression factor is generated based on the characteristics of the initial interference signal; A dual-feature fusion feature is generated based on the interference suppression factor and the spectral state features; single-unit perception data of the UAV is extracted based on the dual-feature fusion feature; Interference suppression factors are generated based on the characteristics of the initial interference signal, including: The interference core parameters are extracted from the initial interference signal features, and an interference feature matrix is ​​constructed based on the interference core parameters; wherein, the interference core parameters include the interference signal amplitude, power spectral density, and interference duration; The interference feature matrix is ​​input into the minimum mean square error algorithm, and the interference suppression factor is obtained by iterative calculation according to the preset iterative objective; wherein, the iterative objective is set to maximize the energy of the idle spectrum signal and minimize the energy of the interference signal. Based on the interference suppression factor and the spectral state features, a dual-feature fusion feature is generated, including: The perceptual suppression features are obtained by performing point-to-point weighted operations on the spectral state features and the interference suppression factor. The residual interference features after weighted operation are extracted, and the perceptual suppression features and residual interference features are concatenated by dimension to obtain the dual-feature fusion features of interference-spectrum.

2. The method for dynamic spectrum sensing and covert communication of unmanned aerial vehicles (UAVs) in adversarial environments according to claim 1, characterized in that, The individual fusion weights of the UAV are calculated based on the characteristics of the initial interference signal, including: Extract the perception quality indicators and interference impact indicators of the UAV; among which, the perception quality indicators include the single-unit perception accuracy and the reliability of the spectrum characteristics, and the interference impact indicators include the interference level and the amplitude of the interference signal at the location of the UAV. Based on the perception quality index and the interference impact index, the individual fusion weight of each UAV is calculated using the entropy weight method.

3. The method for dynamic spectrum sensing and covert communication of unmanned aerial vehicles (UAVs) in adversarial environments according to claim 1, characterized in that, The dynamic reward function is used to dynamically allocate weights to the target optimization parameters; wherein, the target optimization parameters include maximizing the covert rate of the legitimate receiver, maximizing the error probability of the illegal monitor, and minimizing the communication energy consumption.

4. The UAV dynamic spectrum sensing and covert communication method suitable for adversarial environments according to claim 1, characterized in that, The reinforcement learning model includes an online network and a target network; the online network is responsible for real-time inference, and the target network is used to stabilize the training process; the network structures of the online network and the target network are consistent.

5. The UAV dynamic spectrum sensing and covert communication method suitable for adversarial environments according to claim 3, characterized in that, A dynamic reward function is constructed by introducing environmental status evaluation factors, including: The environmental state evaluation factor is calculated based on the characteristics of the adversarial environment. The weight coefficients of each objective optimization parameter are matched according to the environmental state evaluation factor; a dynamic reward function for the objective optimization parameter is constructed based on the weight coefficients; wherein the matching of weight coefficients is based on a pre-established weight mapping relationship.

6. The UAV dynamic spectrum sensing and covert communication method suitable for adversarial environments according to claim 5, characterized in that, The environmental state evaluation factor is calculated based on the characteristics of the adversarial environment, including: Extract the state characteristics of the adversarial environment; among which, the state characteristics of the adversarial environment include interference intensity, spectrum change rate and monitoring risk level; The environmental status evaluation factor is obtained by weighting and summing the basic weights and standard values ​​of the indicators corresponding to the characteristics of the adversarial environment; wherein the basic weights are set by expert scoring and the standard values ​​of the indicators are normalized values.

Images