Unmanned cluster message forwarding time control method based on communication data security modeling

Abstract

The invention discloses an unmanned cluster message forwarding time control method based on communication data security modeling, and the method comprises the steps: obtaining communication data and selectable behaviors of an unmanned cluster at a current moment, the communication data comprising a standardized position matrix and a communication data matrix, and the behaviors representing forwarding time; the method comprises the following steps: taking communication data and a behavior of an unmanned cluster as input of a pre-constructed and trained integrated model in an unmanned aerial vehicle in an unmanned cluster center, respectively acquiring average waiting delay output by the integrated model and corresponding to each behavior under current communication data, and selecting the behavior with the minimum delay as forwarding time under the current communication data; the integrated model comprises an integrated vector generation model deployed on each unmanned aerial vehicle and a behavior reward prediction model deployed on the central unmanned aerial vehicle. According to the method, the forwarding time with the minimum message delay is determined by predicting the waiting delay of the communication data at different forwarding times, the method can adapt to the dynamic change of the unmanned cluster, and the overall communication delay is reduced.

Description

Technical Field

[0001] The present invention relates to the technical field of unmanned cluster control, and in particular to an unmanned cluster message forwarding time control method based on communication data security modeling. Background Art

[0002] Unmanned swarms use communication satellite technology to connect remote drones and ground stations into a whole to achieve two-way long-distance communication. Since the communication process with the ground station is long, unmanned swarms usually select a drone as the central node according to the system topology, and the central node acts as a relay between the ground station and the communication satellite. The central node usually packages the same type of data and instructions in the cluster and forwards them to the ground station via the communication satellite, reducing its own burden through batch information processing to maintain smooth communication. However, since the communication behavior of each drone in the unmanned swarm is not synchronized, there is a time delay between each message. In the same channel, if the forwarding frequency is too high, most nodes will be in a state of waiting for forwarding, resulting in low delays for individual nodes and high overall delays; if the forwarding frequency is too low, it will directly lead to too long waiting time for all nodes. Therefore, the forwarding time of the central node directly determines the smoothness of the communication of the remote unmanned swarm and is an important factor affecting the message delay of cluster members.

[0003] The existing remote unmanned cluster message forwarding time control mainly adopts a static control scheme, which uniformly controls the forwarding time based on the number of messages or forwarding intervals and other characteristics. Among them, the static control scheme based on the number of messages is to forward a message once the number of messages to be forwarded received by the drone central node reaches a threshold; the static control scheme based on the forwarding interval is to forward a message once at a fixed interval. The above static control scheme treats all communication conditions equally, fails to fully consider the actual operation of unmanned cluster communications, lacks flexibility, and is difficult to adapt to the dynamic changes of unmanned clusters, which can easily lead to more communication resource occupation and higher overall communication delay. For example, when cluster messages are sparse, the waiting time for the number of messages to reach the forwarding threshold is long; when messages are dense, the forwarding strategy based on fixed time intervals is likely to cause a sharp increase in instantaneous data throughput, causing communication congestion.

[0004] In addition, although there are dynamic control solutions for similar scenarios, such as the dynamic control solution for message forwarding in distributed Internet of Things, although it can make appropriate responses according to changes in the scenario, it lacks attention to historical communication behaviors and only considers the communication messages at the current moment, resulting in a limited data analysis field of view. It is a local optimization solution with poor forwarding control and delay optimization effects. It lacks recognition and perception of the system, and cannot give a scientific and reasonable forwarding time node based on data and instruction behaviors. The design is relatively simple and crude. Moreover, it directly relies on the distributed structure to share all node data, which can easily lead to single point breakthroughs and global losses, and has poor information security. Summary of the invention

[0005] In order to solve some or all of the technical problems existing in the above-mentioned prior art, the present invention provides an unmanned cluster message forwarding time control method based on communication data security modeling.

[0006] The technical solution of the present invention is as follows:

[0007] A method for controlling the time of unmanned cluster message forwarding based on communication data security modeling is provided, the method comprising:

[0008] Obtain the communication data and optional behavior of the unmanned cluster at the current moment. The unmanned cluster includes multiple drones. The communication data includes a standardized position matrix and a communication data matrix. The behavior represents the forwarding time. The position matrix is ​​determined according to the set unmanned cluster center drone and the topological structure of the unmanned cluster. The communication data matrix is ​​obtained by filling the corresponding drone's communication behavior data in the corresponding position of the position matrix.

[0009] The communication data and a behavior of the unmanned cluster are used as inputs of the pre-built and trained integrated model in the unmanned cluster center drone. The average waiting delay output by the integrated model corresponding to each behavior under the current communication data is obtained respectively, and the behavior with the smallest average waiting delay is selected as the forwarding time under the current communication data. The integrated model includes an integrated vector generation model deployed on each drone and a behavior reward prediction model deployed on the unmanned cluster center drone. The input of the integrated vector generation model on drones other than the unmanned cluster center drone is the communication behavior data of the corresponding drone. The input of the integrated vector generation model on the unmanned cluster center drone is the communication behavior data and the selected behavior of the unmanned cluster center drone. The output of the integrated vector generation model is an integrated vector. The input of the behavior reward prediction model is the integrated vector output by all integrated vector generation models. The output of the behavior reward prediction model is the average waiting delay.

[0010] In some optional embodiments, the communication behavior data of the drone includes but is not limited to: message type, transmission and reception frequency, and data size.

[0011] In some optional embodiments, the integrated vector generation model processes input data by utilizing a convolutional neural network and a self-attention mechanism to obtain and output an integrated vector.

[0012] In some optional implementations, the integrated vector generation model includes:

[0013] A first convolutional neural network, whose input is the input of the integrated vector generation model, and whose output is connected to the maximum pooling layer, is used to perform data compression through convolution operation and extract local features of the input data;

[0014] The output of the maximum pooling layer is connected to the first activation function layer, and is used to perform maximum pooling processing on the input features;

[0015] The output of the first activation function layer is connected to the second convolutional neural network, and is used to perform nonlinear transformation on the input features using a preset activation function;

[0016] The output of the second convolutional neural network is connected to the average pooling layer to encode deep communication behavior features and extract more abstract features;

[0017] The average pooling layer, whose output is connected to the second activation function layer, is used to perform average pooling processing on the input features;

[0018] The second activation function layer, whose output is connected to the flattening layer, is used to perform nonlinear transformation on the input features using a preset activation function;

[0019] The output of the flattening layer is connected to the first fully connected layer, and is used to map the input multi-dimensional features into a one-dimensional vector;

[0020] The output of the first fully connected layer is connected to the second fully connected layer, which is used to compress the input features and strengthen the interaction between the features;

[0021] The output of the second fully connected layer is connected to the self-attention mechanism module to further compress the input features and complete the dimensionality transformation;

[0022] The self-attention mechanism module is used to fuse input features and obtain and output an integrated vector.

[0023] In some optional implementations, the self-attention mechanism module performs feature fusion to obtain an integrated vector in the following manner:

[0024]

[0025] Wherein, W1, W2, b1 and b2 represent pre-trained parameters, F represents the input of the self-attention mechanism module, tanh represents the tanh function operation, superscript T represents the transposition operation, softmax represents the softmax function operation, θ1 and μ1 represent the intermediate vectors, μ i represents the i-th component of μ1, f i represents the i-th column of F, and z represents the integrated vector.

[0026] In some optional embodiments, the behavior reward prediction model obtains and outputs the average waiting delay by utilizing a gated attention mechanism to perform information interaction and deeply encode an integrated vector of the input.

[0027] In some optional embodiments, the behavior reward prediction model includes:

[0028] A vector compression module, whose input is the input of the behavior reward prediction model, whose output is connected to the gate attention mechanism module, is used to calculate the correlation between the input integrated vectors and compress the integrated vectors, and obtain and output the compressed vector;

[0029] The gated attention mechanism module, whose input is the compressed vector output by the vector compression module and the integrated vector corresponding to the unmanned cluster center drone in the input of the behavior reward prediction model, is used to process the input vector by utilizing the gated attention mechanism to obtain and output the average waiting delay.

[0030] In some optional implementations, the vector compression module calculates the correlation between the input integrated vectors and compresses the integrated vectors to obtain the compressed vectors in the following manner:

[0031]

[0032] Among them, W κ represents the pre-training parameters, z represents the integration vector corresponding to the center drone of the unmanned cluster, z1 represents the integration vector corresponding to the first drone in the set of neighboring drones of the center drone of the unmanned cluster, z2 represents the integration vector corresponding to the second drone in the set of neighboring drones of the center drone of the unmanned cluster, the superscript T represents the transpose operation, softmax represents the softmax function operation, κ represents the correlation weight vector, z i represents the integration vector corresponding to the i-th drone in the set of neighboring drones of the center drone of the unmanned cluster, κ i represents the i-th component of κ, represents the set of neighboring drones of the center drone of the unmanned cluster, and q represents the compression vector.

[0033] In some optional implementations, the gate attention mechanism module processes the input vector in the following manner to obtain the average waiting delay:

[0034]

[0035] Among them, W g , W r and b r represents the pre-trained parameters, sigmoid represents the sigmoid function operation, g represents the gate value, u represents the intermediate vector, Represents the predicted value of the average wait delay.

[0036] In some optional embodiments, the integrated model is trained in the following manner:

[0037] Obtain a training data set, where the training data includes communication data and selected behaviors of an unmanned cluster at a moment, and an actual average waiting delay obtained based on the selected behaviors;

[0038] The communication behavior data of each UAV in the training data is used as the input of the integrated vector generation model on each UAV, the communication behavior data and the selected behavior of the unmanned cluster center UAV in the training data are used as the input of the integrated vector generation model on the unmanned cluster center UAV, the integrated vector output by all integrated vector generation models is used as the input of the behavior reward prediction model on the unmanned cluster center UAV, the actual average waiting delay in the training data is used as the output of the behavior reward prediction model, the integrated vector generation model and the behavior reward prediction model are trained, and the model parameters of the integrated vector generation model and the model parameters of the behavior reward prediction model are obtained;

[0039] The model parameters of each integrated vector generation model and the model parameters of the behavior reward prediction model are aggregated by group learning to determine the final model parameters and obtain the integrated model.

[0040] The main advantages of the technical solution of the present invention are as follows:

[0041] The unmanned cluster message forwarding time control method based on communication data security modeling of the present invention takes the communication data of the unmanned cluster as the state, the forwarding time as the behavior, and the average waiting delay after forwarding based on the selected forwarding time as the reward of the behavior, and constructs an integrated model to approximate the Q-value function. The integrated model can be used to quickly predict the average waiting delay under different forwarding times under given communication data, and then determine the forwarding time with the minimum message delay. It can fully consider the actual operation of the unmanned cluster communication, adapt to the dynamic changes of the unmanned cluster, reduce the communication resource occupation, and reduce the overall communication delay; and by dividing the integrated model into two parts and deploying and co-training them separately, it can fully adapt to the distributed structural characteristics of the communication data of the unmanned cluster, reduce data transmission, and reduce communication costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] The drawings described herein are used to provide a further understanding of the embodiments of the present invention and constitute a part of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0043] Figure 1 A flow chart of an unmanned cluster message forwarding time control method based on communication data security modeling provided by an embodiment of the present invention;

[0044] Figure 2 A schematic diagram of an unmanned cluster message forwarding structure model provided by an embodiment of the present invention;

[0045] Figure 3 A schematic diagram of the structural principle of the integrated model provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0046] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the specific embodiments of the present invention and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0047] The technical solution provided by the embodiments of the present invention is described in detail below with reference to the accompanying drawings.

[0048] See also Figure 1 The embodiment of the present invention provides an unmanned cluster message forwarding time control method based on communication data security modeling, the method comprising the following steps:

[0049] Step 1: Obtain the communication data and optional behaviors of the unmanned cluster at the current moment;

[0050] In an embodiment of the present invention, the unmanned cluster includes multiple drones, the communication data includes a standardized position matrix and a communication data matrix, the behavior represents the forwarding time, the position matrix is ​​determined according to the set unmanned cluster center drone and the topological structure of the unmanned cluster, and the communication data matrix is ​​obtained by filling the corresponding drone's communication behavior data in the corresponding position of the position matrix.

[0051] refer to Figure 2 , it is assumed that at time t, the center drone of the unmanned cluster corresponds to several adjacent drones, and the center drone maintains communication connection with each drone. Based on the topological structure of the unmanned cluster, a drone is regarded as a node, and the adjacent relationship is represented by the corresponding number to construct the location matrix of the unmanned cluster n represents the number of drones in the unmanned cluster; collect the communication behavior data of each drone in the unmanned cluster, fill the communication behavior data of each drone into the corresponding position of the drone in the position matrix, and obtain the communication data matrix of the unmanned cluster l represents the number of data types in the communication behavior data; the position matrix M is processed using the maximum normalization method p and the communication data matrix M v To obtain the standardized position matrix and communication data matrix, and then obtain the communication data of the unmanned cluster

[0052]

[0053] In the embodiment of the present invention, the number representing the adjacency relationship may be 0 or 1, where 0 indicates that there is no adjacency relationship and 1 indicates that there is an adjacency relationship.

[0054] In the embodiment of the present invention, the communication behavior data of the drone includes but is not limited to: message type, sending and receiving frequency, and data size.

[0055] Step 2: Use the communication data and a behavior of the unmanned cluster as the input of the pre-built and trained integrated model in the center drone of the unmanned cluster, obtain the average waiting delay output by the integrated model corresponding to each behavior under the current communication data, and select the behavior with the smallest average waiting delay as the forwarding time under the current communication data.

[0056] refer to Figure 3 ,In an embodiment of the present invention, the integrated model includes an integrated vector generation model deployed on each UAV and a behavior reward prediction model deployed on the UAV at the center of the UAV cluster;

[0057] refer to Figure 3In an embodiment of the present invention, the input of the integrated vector generation model on other drones other than the unmanned cluster center drone is the communication behavior data of the corresponding drone, the input of the integrated vector generation model on the unmanned cluster center drone is the communication behavior data and the selected behavior of the unmanned cluster center drone, the output of the integrated vector generation model is the integrated vector, the input of the behavior reward prediction model is the integrated vector output by all integrated vector generation models, and the output of the behavior reward prediction model is the average waiting delay.

[0058] The unmanned cluster message forwarding time control method based on communication data security modeling provided by the embodiment of the present invention takes the communication data of the unmanned cluster as the state, the forwarding time as the behavior, and the average waiting delay after forwarding based on the selected forwarding time as the reward of the behavior, and constructs an integrated model to approximate the Q-value function. The integrated model can be used to quickly predict the average waiting delay under different forwarding times under given communication data, and then determine the forwarding time with the minimum message delay. It can fully consider the actual operation of the unmanned cluster communication, adapt to the dynamic changes of the unmanned cluster, reduce the communication resource occupation, and reduce the overall communication delay; and by dividing the integrated model into two parts and deploying and co-training them separately, it can fully adapt to the distributed structural characteristics of the communication data of the unmanned cluster, reduce data transmission, and reduce communication costs.

[0059] The unmanned cluster message forwarding time control method based on communication data security modeling provided by the embodiment of the present invention combines the mechanism principle of deep reinforcement learning, takes the communication data of the unmanned cluster as the state, and the forwarding time as the behavior. Taking into account the complexity of the unmanned cluster, it is necessary to avoid overlap and select appropriate indicators to design rewards without omission. Since the forwarding delay can measure the smoothness of message transmission and reflect the communication status of the unmanned cluster, the average waiting delay is used as a reward.

[0060] Furthermore, in an embodiment of the present invention, an integrated vector generation model is used to compress the communication behavior data of the drone to obtain an integrated vector containing rich information.

[0061] In an embodiment of the present invention, the integrated vector generation model specifically processes input data by utilizing a convolutional neural network and a self-attention mechanism to obtain and output an integrated vector.

[0062] refer to Figure 3 In an embodiment of the present invention, the integrated vector generation model includes:

[0063] The first convolutional neural network, whose input is the input of the integrated vector generation model, and whose output is connected to the maximum pooling layer, is used to compress data through convolution operations and extract local features of the input data;

[0064] The maximum pooling layer, whose output is connected to the first activation function layer, is used to perform maximum pooling processing on the input features;

[0065] The first activation function layer, whose output is connected to the second convolutional neural network, is used to perform nonlinear transformation on the input features using a preset activation function;

[0066] The second convolutional neural network, whose output is connected to the average pooling layer, is used to encode deep communication behavior features and extract more abstract features;

[0067] Average pooling layer, the output is connected to the second activation function layer, which is used to perform average pooling on the input features;

[0068] The second activation function layer, whose output is connected to the flattening layer, is used to perform nonlinear transformation on the input features using a preset activation function;

[0069] Flattening layer, whose output is connected to the first fully connected layer, is used to map the input multi-dimensional features into a one-dimensional vector;

[0070] The output of the first fully connected layer is connected to the second fully connected layer to compress the input features and strengthen the interaction between features;

[0071] The second fully connected layer, whose output is connected to the self-attention mechanism module, is used to further compress the input features and complete the dimension transformation;

[0072] The self-attention mechanism module is used to fuse the input features and obtain and output the integrated vector.

[0073] In an embodiment of the present invention, the input of the integrated vector generation model on the drone is the communication behavior data of the drone. Since the behavior taken by the unmanned cluster will have a significant impact on the subsequent communication behavior, for the central drone of the unmanned cluster, it is necessary to integrate the selected behavior into the communication behavior data and then use the communication behavior data integrated with the behavior as the input of the integrated vector generation model of the central drone.

[0074] Specifically, integrating the selected behavior into the communication behavior data can be expressed as:

[0075] Y=Y(kW) T ;

[0076] Among them, Y represents the communication behavior data of the central drone, k represents the selected behavior, that is, the selected forwarding time, and W represents the transfer matrix, which is preset.

[0077] In the embodiment of the present invention, the communication data for the unmanned cluster is The sliding window size of the first convolutional neural network is [4×4], the number of channels is (l+1), the step size is [2×2], and the filling mode is "SAME"; the sliding window size of the second convolutional neural network is [2×2], the number of channels is 8, the step size is [2×2], and the filling mode is "SAME"; the first activation function layer uses the tanh activation function to perform nonlinear transformation on the input features; the second activation function layer uses the tanh activation function to perform nonlinear transformation on the input features.

[0078] In the embodiment of the present invention, each column vector in the output of the second fully connected layer corresponds to the communication behavior data of an adjacent drone and has the characteristic of symmetry. Therefore, it is appropriate to use a self-attention mechanism to complete the fusion of vectors.

[0079] Specifically, in this embodiment of the present invention, the self-attention mechanism module performs feature fusion in the following manner to obtain an integrated vector:

[0080]

[0081] Among them, W1, W2, b1 and b2 represent the pre-trained parameters, F represents the input of the self-attention mechanism module, that is, the output of the second fully connected layer, tanh represents the tanh function operation, the superscript T represents the transposition operation, softmax represents the softmax function operation, θ1 and μ1 represent the intermediate vectors, μ i represents the i-th component of μ1, f i represents the i-th column of F, and z represents the integrated vector.

[0082] It should be noted that the pre-trained parameters are determined by training the model.

[0083] In the embodiment of the present invention, the communication behavior data of the UAV is processed based on the integrated vector generation model set above, and the obtained integrated vector condenses all the communication behavior data of the UAV.

[0084] Furthermore, in an embodiment of the present invention, after obtaining the integrated vector, the drone can upload the integrated vector to the central server of the unmanned cluster through a wired link for subsequent retrieval, thereby avoiding repeated transfer of data and reducing the communication burden.

[0085] Furthermore, in an embodiment of the present invention, the behavior reward prediction model is used to aggregate the integrated vector data of multiple drones and fit the mapping relationship between state-behavior and reward to achieve data analysis with a wide field of view.

[0086] In an embodiment of the present invention, the behavior reward prediction model specifically obtains and outputs the average waiting delay by utilizing the gated attention mechanism to perform information interaction and deeply encode the input integrated vector.

[0087] refer to Figure 3 In an embodiment of the present invention, the behavior reward prediction model includes:

[0088] The vector compression module has the input of the behavior reward prediction model and the output is connected to the gate attention mechanism module, which is used to calculate the correlation between the input integrated vectors and compress the integrated vectors, and obtain and output the compressed vectors;

[0089] The gated attention mechanism module takes as input the compressed vector output by the vector compression module and the integrated vector corresponding to the unmanned cluster center drone in the input of the behavior reward prediction model. It is used to process the input vector by utilizing the gated attention mechanism to obtain and output the average waiting delay.

[0090] In the embodiment of the present invention, the vector compression module first calculates the correlation between the integrated vectors and normalizes the correlation into a weight, and then performs weighted fusion of the integrated vectors according to the weight to obtain a compressed vector.

[0091] Specifically, the processing process of the vector compression module can be expressed as:

[0092]

[0093] Among them, W κ represents the pre-training parameters, z represents the integration vector corresponding to the center drone of the unmanned cluster, z1 represents the integration vector corresponding to the first drone in the set of neighboring drones of the center drone of the unmanned cluster, z2 represents the integration vector corresponding to the second drone in the set of neighboring drones of the center drone of the unmanned cluster, the superscript T represents the transpose operation, softmax represents the softmax function operation, κ represents the correlation weight vector, z i represents the integration vector corresponding to the i-th drone in the set of neighboring drones of the center drone of the unmanned cluster, κ i represents the i-th component of κ, represents the set of neighboring drones of the center drone of the unmanned cluster, and q represents the compression vector.

[0094] In an embodiment of the present invention, the gated attention mechanism module first uses the relevance weight to calculate the gated value of the gated attention mechanism to deepen the impact of the behavior on the decision, and then uses the gated attention mechanism to convert the integrated vector of the central drone and the compressed vector obtained by the vector compression module into an intermediate variable based on the calculated gated value. Finally, the intermediate variable is transformed into a predicted reward through a layer of dimensional transformation, and the predicted reward is the predicted value of the average waiting delay.

[0095] Specifically, the processing process of the gate attention mechanism module can be expressed as:

[0096]

[0097] Among them, W g , W r and b r represents the pre-trained parameters, z represents the integrated vector corresponding to the drone at the center of the unmanned cluster, the superscript T represents the transposition operation, sigmoid represents the sigmoid function operation, g represents the gate value, q represents the compressed vector, and u represents the intermediate vector. represents the predicted reward, i.e., the predicted value of the average waiting delay.

[0098] It should be noted that the pre-trained parameters are determined by training the model.

[0099] In an embodiment of the present invention, the model is divided into two parts: an integrated vector generation model and a behavior reward prediction model, and the integrated vector generation model is deployed on each drone, and the behavior reward prediction model is deployed on a central drone. The integrated vector generation model is used to achieve compression of original large-scale data for lightweight knowledge sharing. The central drone receives compressed data from each drone and uses the behavior reward prediction model to predict the reward of the behavior. This can not only expand the data field of view, promote information fusion, and improve the accuracy of the fitted state-behavior to reward mapping relationship, but also fully adapt to the distributed structural characteristics of the communication data of the unmanned cluster, reduce data transmission, and reduce communication costs.

[0100] Furthermore, in an embodiment of the present invention, the integrated model is trained in the following manner:

[0101] Step 201, obtaining a training data set, where the training data includes communication data and selected behaviors of an unmanned cluster at a moment, and an actual average waiting delay obtained based on the selected behaviors;

[0102] Step 202, using the communication behavior data of each drone in the training data as the input of the integrated vector generation model on each drone, using the communication behavior data and the selected behavior of the unmanned cluster center drone in the training data as the input of the integrated vector generation model on the unmanned cluster center drone, using the integrated vector output by all integrated vector generation models as the input of the behavior reward prediction model on the unmanned cluster center drone, using the actual average waiting delay in the training data as the output of the behavior reward prediction model, training the integrated vector generation model and the behavior reward prediction model, and obtaining model parameters of the integrated vector generation model and the model parameters of the behavior reward prediction model;

[0103] Step 203, using group learning to aggregate the model parameters of each integrated vector generation model and the model parameters of the behavior reward prediction model, determine the final model parameters, and obtain the integrated model.

[0104] In the embodiment of the present invention, since the communication behaviors of the unmanned cluster have a temporal dependency, the training data set can be obtained by collecting multiple historical data.

[0105] Furthermore, in the embodiment of the present invention, in step 202, when the integrated vector generation model and the behavior reward prediction model are collaboratively trained, the gradient descent method is used to perform training and update of the model parameters.

[0106] Specifically, referring to Table 1, Table 1 shows the specific process of model collaborative training in step 202.

[0107] Table 1 Model collaborative training process

[0108]

[0109]

[0110] In Table 1 above, M1 represents the integrated vector generation model, z represents the integrated vector corresponding to the center UAV, None represents the empty set, η i Represents the set of adjacent drones The i-th drone in z i represents the integrated vector corresponding to the i-th drone in the adjacent drone set, M2 represents the behavior reward prediction model, represents the predicted reward output by the behavior reward prediction model, r t represents the actual reward in the training data, which is the average waiting delay, Indicates reward based on prediction and the actual reward r t Calculate the loss function loss, G,G c =η.compute_G(M1;M2|loss) means calculating the critical gradient G according to the loss function loss c and model full gradient G, which is used to update the parameters of the model deployed in the central drone, critical gradient G c Updates the parameters of the model deployed in neighboring drones.

[0111] During the collaborative model training process, the integrated vector generation model of each drone converts the communication data Y i Compressed into an integrated vector z i , and uploaded to the temporary central server Γ, stored in the vector set Center drone downloads vector set from temporary center server Γ The behavior reward prediction model deployed on the central drone is based on the received integrated vector z and vector set Complete the state-behavior and reward fitting and get the predicted reward; then, according to the predicted reward and the actual reward r t Calculate the loss function loss, and calculate the critical gradient G according to the loss function loss c And the model full gradient G, use the model full gradient G to update the parameters of the model deployed by itself, and change the critical gradient G c Shared to each neighboring drone so that the neighboring drone can utilize the critical gradient G c Update the parameters of a deployed model.

[0112] In the embodiment of the present invention, the specific type of the loss function is set according to actual needs, and a conventional loss function can be selected.

[0113] Furthermore, in the embodiment of the present invention, considering that the data distribution of different drones is different, such as a drone is always congested, while a drone is always unobstructed, this will hinder the training progress and reduce the generalization ability of the model. To this end, group learning is introduced to restrict the state space of the fitting through the interaction of parameters and optimize the training process of the model. At the same time, in order to avoid centralized training and protect user privacy, each drone only shares parameters without transmitting data.

[0114] In the embodiment of the present invention, in step 203, the integrated model is obtained by training the updated parameters through the group learning fusion step 202.

[0115] Specifically, referring to Table 2, Table 2 shows the specific process of using group learning to aggregate model parameters in step 203.

[0116] Table 2 Parameter integration process based on group learning

[0117]

[0118]

[0119] In the above Table 2, None represents an empty set. Represents a collection of drones The i-th drone in .

[0120] In the process of using group learning to aggregate model parameters, the parameters in the parameter set Ψ And the loss function set The losses in the above mentioned model collaborative training process are derived from each drone’s own parameters. Passed to the temporary central server Γ, the temporary central server Γ receives the parameters of all drones Then, the parameters are aggregated by:

[0121]

[0122] The above aggregation formula calculates the integrated model parameter Ψ based on the lossagg , giving greater weight to local parameters with high accuracy is conducive to improving the fitting effect; then, the temporary central server Γ assigns the integrated model parameter Ψ agg The module is deployed by publishing it to each drone in the domain through a wired link. Subsequently, whenever a signal switch is required, the drone evaluates each behavior in the current state based on the integrated model, predicts the cumulative reward, and thus determines the forwarding time with the minimum communication delay.

[0123] In the embodiment of the present invention, group learning is used to fuse the models of different nodes to obtain the final integrated model, which can fully adapt to the distributed structural characteristics of the communication data of the unmanned cluster, while avoiding centralized training, optimizing the model training process, and improving information security.

[0124] It should be noted that, in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

[0125] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An unmanned cluster message forwarding time control method based on communication data security modeling, characterized in that: The method comprises: Obtain the communication data and optional behavior of the unmanned cluster at the current moment. The unmanned cluster includes multiple drones. The communication data includes a standardized position matrix and a communication data matrix. The behavior represents the forwarding time. The position matrix is ​​determined according to the set unmanned cluster center drone and the topological structure of the unmanned cluster. The communication data matrix is ​​obtained by filling the corresponding drone's communication behavior data in the corresponding position of the position matrix. The communication data and a behavior of the unmanned cluster are used as inputs of the pre-built and trained integrated model in the unmanned cluster center drone. The average waiting delay output by the integrated model corresponding to each behavior under the current communication data is obtained respectively, and the behavior with the smallest average waiting delay is selected as the forwarding time under the current communication data. The integrated model includes an integrated vector generation model deployed on each drone and a behavior reward prediction model deployed on the unmanned cluster center drone. The input of the integrated vector generation model on drones other than the unmanned cluster center drone is the communication behavior data of the corresponding drone. The input of the integrated vector generation model on the unmanned cluster center drone is the communication behavior data and the selected behavior of the unmanned cluster center drone. The output of the integrated vector generation model is an integrated vector. The input of the behavior reward prediction model is the integrated vector output by all integrated vector generation models. The output of the behavior reward prediction model is the average waiting delay.

2. The unmanned cluster message forwarding time control method based on communication data security modeling according to claim 1 is characterized in that: The communication behavior data of drones includes but is not limited to: message type, sending and receiving frequency, and data size.

3. The unmanned cluster message forwarding time control method based on communication data security modeling according to claim 1 is characterized in that: The integrated vector generation model processes input data by utilizing a convolutional neural network and a self-attention mechanism to obtain and output an integrated vector.

4. The unmanned cluster message forwarding time control method based on communication data security modeling according to claim 3 is characterized in that: The integrated vector generation model includes: A first convolutional neural network, whose input is the input of the integrated vector generation model, and whose output is connected to the maximum pooling layer, is used to perform data compression through convolution operation and extract local features of the input data; The output of the maximum pooling layer is connected to the first activation function layer, and is used to perform maximum pooling processing on the input features; The output of the first activation function layer is connected to the second convolutional neural network, and is used to perform nonlinear transformation on the input features using a preset activation function; The output of the second convolutional neural network is connected to the average pooling layer to encode deep communication behavior features and extract more abstract features; The average pooling layer, whose output is connected to the second activation function layer, is used to perform average pooling processing on the input features; The second activation function layer, whose output is connected to the flattening layer, is used to perform nonlinear transformation on the input features using a preset activation function; The output of the flattening layer is connected to the first fully connected layer, and is used to map the input multi-dimensional features into a one-dimensional vector; The output of the first fully connected layer is connected to the second fully connected layer, which is used to compress the input features and strengthen the interaction between the features; The output of the second fully connected layer is connected to the self-attention mechanism module to further compress the input features and complete the dimensionality transformation; The self-attention mechanism module is used to fuse input features and obtain and output an integrated vector.

5. The unmanned cluster message forwarding time control method based on communication data security modeling according to claim 4 is characterized in that: The self-attention mechanism module performs feature fusion in the following way to obtain an integrated vector: Wherein, W1, W2, b1 and b2 represent pre-trained parameters, F represents the input of the self-attention mechanism module, tanh represents the tanh function operation, superscript T represents the transposition operation, softmax represents the softmax function operation, θ1 and μ1 represent the intermediate vectors, μ i represents the i-th component of μ1, f i represents the i-th column of F, and z represents the integrated vector.

6. The unmanned cluster message forwarding time control method based on communication data security modeling according to claim 1 or 5 is characterized in that: The behavior reward prediction model obtains and outputs the average waiting delay by utilizing the gated attention mechanism to perform information interaction and deeply encode the integrated vector of the input.

7. The unmanned cluster message forwarding time control method based on communication data security modeling according to claim 6 is characterized in that: The behavior reward prediction model includes: A vector compression module, whose input is the input of the behavior reward prediction model, whose output is connected to the gate attention mechanism module, is used to calculate the correlation between the input integrated vectors and compress the integrated vectors, and obtain and output the compressed vector; The gated attention mechanism module, whose input is the compressed vector output by the vector compression module and the integrated vector corresponding to the unmanned cluster center drone in the input of the behavior reward prediction model, is used to process the input vector by utilizing the gated attention mechanism to obtain and output the average waiting delay.

8. The unmanned cluster message forwarding time control method based on communication data security modeling according to claim 7 is characterized in that: The vector compression module calculates the correlation between the input integrated vectors and compresses the integrated vectors in the following manner to obtain a compressed vector: Among them, W κ represents the pre-training parameters, z represents the integration vector corresponding to the center drone of the unmanned cluster, z1 represents the integration vector corresponding to the first drone in the set of neighboring drones of the center drone of the unmanned cluster, z2 represents the integration vector corresponding to the second drone in the set of neighboring drones of the center drone of the unmanned cluster, the superscript T represents the transpose operation, softmax represents the softmax function operation, κ represents the correlation weight vector, z i represents the integration vector corresponding to the i-th drone in the set of neighboring drones of the center drone of the unmanned cluster, κ i represents the i-th component of κ, represents the set of neighboring drones of the center drone of the unmanned cluster, and q represents the compression vector.

9. The unmanned cluster message forwarding time control method based on communication data security modeling according to claim 8 is characterized in that: The gated attention mechanism module processes the input vector in the following way to obtain the average waiting delay: Among them, W g , W r and b r represents the pre-trained parameters, sigmoid represents the sigmoid function operation, g represents the gate value, u represents the intermediate vector, Represents the predicted value of the average wait delay.

10. The unmanned cluster message forwarding time control method based on communication data security modeling according to claim 1 is characterized in that: The ensemble model is trained in the following way: Obtain a training data set, where the training data includes communication data and selected behaviors of an unmanned cluster at a moment, and an actual average waiting delay obtained based on the selected behaviors; The communication behavior data of each UAV in the training data is used as the input of the integrated vector generation model on each UAV, the communication behavior data and the selected behavior of the unmanned cluster center UAV in the training data are used as the input of the integrated vector generation model on the unmanned cluster center UAV, the integrated vector output by all integrated vector generation models is used as the input of the behavior reward prediction model on the unmanned cluster center UAV, the actual average waiting delay in the training data is used as the output of the behavior reward prediction model, the integrated vector generation model and the behavior reward prediction model are trained, and the model parameters of the integrated vector generation model and the model parameters of the behavior reward prediction model are obtained; The model parameters of each integrated vector generation model and the model parameters of the behavior reward prediction model are aggregated by group learning to determine the final model parameters and obtain the integrated model.

Images