Unmanned cluster message forwarding time control method based on communication data security modeling

Abstract

The application discloses a kind of unmanned cluster message forwarding time control method based on communication data security modeling, comprising: obtaining the communication data of current time of unmanned cluster and optional behavior, communication data includes standardized position matrix and communication data matrix, behavior indicates forwarding time;With the communication data of unmanned cluster and a behavior as the input of integrated model pre-constructed and trained in unmanned cluster center unmanned plane, the average waiting delay corresponding to each behavior under current communication data output by integrated model is obtained respectively, the behavior with minimum delay is selected as the forwarding time under current communication data, and the integrated model includes integrated vector generation model deployed on each unmanned plane and behavior reward prediction model deployed on center unmanned plane.The application can adapt to the dynamic change of unmanned cluster by predicting the waiting delay under different forwarding time under communication data, determine the forwarding time with minimum message delay, and reduce overall communication delay.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of unmanned cluster control, and particularly relates to a message forwarding time control method for unmanned cluster based on communication data security modeling. BACKGROUND

[0002] The unmanned cluster uses communication satellite technology to connect remote unmanned aerial vehicles and ground stations as a whole to realize two-end long-distance communication. Due to the long communication process with the ground station, a unmanned aerial vehicle is usually selected as a center node according to the topological structure of the system, and the center node acts as a relay between the ground station and the communication satellite. The center node usually packages the same type of data and instructions in the cluster and forwards them to the ground station through the communication satellite, thereby reducing the burden of the center node through batch information processing to maintain smooth communication. However, since the communication behaviors of each unmanned aerial vehicle in the unmanned cluster are not synchronized, there is a time delay between messages. Under the same channel, if the forwarding frequency is too high, most nodes will be in a waiting state for forwarding, resulting in low delay for individual nodes and high overall delay; if the forwarding frequency is too low, it will directly result in too long waiting time for all nodes. Therefore, the forwarding time of the center node directly determines whether the remote unmanned cluster communication is smooth or not, and is an important factor affecting the message delay of the cluster members.

[0003] The existing message forwarding time control of the remote unmanned cluster mainly adopts a static control scheme, which specifically controls the forwarding time according to the number of messages or the forwarding interval. Among them, the static control scheme based on the number of messages specifically refers to performing message forwarding once when the number of messages received by the unmanned aerial vehicle center node reaches a threshold; the static control scheme based on the forwarding interval specifically refers to performing message forwarding once every fixed time interval. The above static control scheme treats all communication conditions equally, fails to fully consider the actual operation of the unmanned cluster communication, lacks flexibility, and is difficult to adapt to the dynamic changes of the unmanned cluster, which easily leads to more communication resource occupation and higher overall communication delay. For example, when the cluster messages are sparse, the waiting time for the number of messages reaching the forwarding threshold is long; when the messages are dense, the forwarding strategy based on the fixed time interval easily leads to a sharp increase in instantaneous data throughput, causing communication congestion.

[0004] In addition, although there are currently dynamic control schemes for similar scenarios, such as dynamic control schemes for message forwarding of distributed Internet of Things, which can respond appropriately according to changes in the scenario, they lack attention to historical communication behavior and only consider communication messages at the current time, resulting in limited data analysis vision and local optimization schemes, with poor forwarding control and delay optimization effect; lack of system identification and perception, unable to give scientific and reasonable forwarding time nodes according to data and instruction behavior, with a simple and crude design; and directly relying on distributed structure to share all node data, which is easy to cause single-point breakthrough and global loss, with poor information security. SUMMARY

[0005] To solve the above-mentioned technical problems in the prior art, the present application provides a message forwarding time control method for an unmanned cluster based on communication data security modeling.

[0006] The technical scheme of the present application is as follows:

[0007] A message forwarding time control method for an unmanned cluster based on communication data security modeling is provided, which comprises:

[0008] Communication data and an optional behavior of the unmanned cluster at the current time are obtained, the unmanned cluster comprising a plurality of unmanned aerial vehicles, the communication data comprising a standardized position matrix and a communication data matrix, the behavior representing the forwarding time, the position matrix being determined according to a set unmanned cluster center unmanned aerial vehicle and the topology of the unmanned cluster, and the communication data matrix being obtained by filling in the corresponding communication behavior data of the unmanned aerial vehicles at the corresponding positions of the position matrix;

[0009] The communication data and the behavior of the unmanned cluster are taken as inputs of an integrated model pre-constructed and trained in the unmanned cluster center unmanned aerial vehicle, the average waiting delay output by the integrated model corresponding to each behavior under the current communication data is obtained respectively, and the behavior with the minimum average waiting delay is selected as the forwarding time under the current communication data, the integrated model comprising an integrated vector generation model deployed on each unmanned aerial vehicle and a behavior reward prediction model deployed on the unmanned cluster center unmanned aerial vehicle, the input of the integrated vector generation model on the unmanned aerial vehicles other than the unmanned cluster center unmanned aerial vehicle being the communication behavior data of the corresponding unmanned aerial vehicles, the input of the integrated vector generation model on the unmanned cluster center unmanned aerial vehicle being the communication behavior data of the unmanned cluster center unmanned aerial vehicle and the selected behavior, the output of the integrated vector generation model being an integrated vector, and the input of the behavior reward prediction model being the integrated vectors output by all integrated vector generation models, the output of the behavior reward prediction model being the average waiting delay.

[0010] In some optional embodiments, the communication behavior data of the unmanned aerial vehicle includes but is not limited to message type, transmission frequency and data size.

[0011] In some optional embodiments, the integrated vector generation model obtains and outputs the integrated vector by processing the input data using a convolutional neural network and a self-attention mechanism.

[0012] In some optional embodiments, the integrated vector generation model comprises:

[0013] a first convolutional neural network, which is inputted with the input of the integrated vector generation model, and is connected with a max-pooling layer to perform data compression through convolution operation and extract local features of the input data;

[0014] the max-pooling layer, which is connected with a first activation function layer to perform max-pooling processing on the input features;

[0015] the first activation function layer, which is connected with a second convolutional neural network to perform nonlinear transformation on the input features using a preset activation function;

[0016] the second convolutional neural network, which is connected with an average-pooling layer to encode deep communication behavior features and extract more abstract features;

[0017] the average-pooling layer, which is connected with a second activation function layer to perform average-pooling processing on the input features;

[0018] the second activation function layer, which is connected with a flattening layer to perform nonlinear transformation on the input features using a preset activation function;

[0019] the flattening layer, which is connected with a first fully connected layer to map the input multi-dimensional features to a one-dimensional vector;

[0020] the first fully connected layer, which is connected with a second fully connected layer to compress the input features and strengthen the interaction between the features;

[0021] the second fully connected layer, which is connected with a self-attention mechanism module to further compress the input features and complete dimension transformation;

[0022] the self-attention mechanism module, which is used to fuse the input features to obtain and output the integrated vector.

[0023] In some optional embodiments, the self-attention mechanism module fuses the features to obtain the integrated vector in the following manner:

[0024]

[0025] wherein W1, W2, b1 and b2 represent pre-trained parameters, F represents the input of the self-attention mechanism module, tanh represents a tanh function operation, T represents a transposition operation, softmax represents a softmax function operation, θ1 and μ1 represent intermediate vectors, μ i represents the i-th component of μ1, f i represents the i-th column of F, and z' represents an integrated vector.

[0026] In some optional embodiments, the behavior reward prediction model obtains and outputs the average waiting delay by utilizing the gate attention mechanism to interact information and deeply encode the integrated vector of the input.

[0027] In some optional embodiments, the behavior reward prediction model comprises:

[0028] a vector compression module, which is inputted with the input of the behavior reward prediction model, and is connected with the gate attention mechanism module, is used to calculate the correlation between the integrated vectors of the input and compress the integrated vectors to obtain and output a compressed vector;

[0029] the gate attention mechanism module, which is inputted with the compressed vector outputted by the vector compression module and the integrated vector corresponding to the UAV of the UAV cluster center in the input of the behavior reward prediction model, is used to process the inputted vectors by utilizing the gate attention mechanism to obtain and output the average waiting delay.

[0030] In some optional embodiments, the vector compression module calculates the correlation between the integrated vectors of the input and compresses the integrated vectors to obtain the compressed vector by:

[0031]

[0032] wherein W κ represents pre-trained parameters, z represents the integrated vector corresponding to the UAV of the UAV cluster center, z1 represents the integrated vector corresponding to the first UAV in the adjacent UAV set of the UAV of the UAV cluster center, z2 represents the integrated vector corresponding to the second UAV in the adjacent UAV set of the UAV of the UAV cluster center, T represents a transposition operation, softmax represents a softmax function operation, κ represents a correlation weight vector, z i represents the integrated vector corresponding to the i-th UAV in the adjacent UAV set of the UAV of the UAV cluster center, κ i represents the i-th component of κ, represents the adjacent UAV set of the UAV of the UAV cluster center, and q represents the compressed vector.

[0033] In some optional embodiments, the gate attention mechanism module processes the input vector to obtain the average waiting delay in the following manner:

[0034]

[0035] wherein W g , W r and b r represent pre-trained parameters, sigmoid represents a sigmoid function operation, g represents a gating value, u represents an intermediate vector, represents a predicted value of the average waiting delay.

[0036] In some optional embodiments, the integrated model is trained in the following manner:

[0037] obtaining a training data set, the training data including communication data of the unmanned cluster at a time instant and selected behaviors, and actual average waiting delays obtained based on the selected behaviors;

[0038] respectively taking the communication behavior data of each unmanned aerial vehicle in the training data as the input of the integrated vector generation model on each unmanned aerial vehicle, taking the communication behavior data and the selected behaviors of the center unmanned aerial vehicle of the unmanned cluster in the training data as the input of the integrated vector generation model on the center unmanned aerial vehicle of the unmanned cluster, taking the integrated vectors output by all integrated vector generation models as the input of the behavior reward prediction model on the center unmanned aerial vehicle of the unmanned cluster, taking the actual average waiting delays in the training data as the output of the behavior reward prediction model, training the integrated vector generation model and the behavior reward prediction model, obtaining model parameters of the integrated vector generation model and model parameters of the behavior reward prediction model;

[0039] aggregating the model parameters of each integrated vector generation model and the model parameters of the behavior reward prediction model by swarm learning, determining final model parameters, and obtaining an integrated model.

[0040] The main advantages of the technical scheme of the present application are as follows:

[0041] The unmanned cluster message forwarding time control method based on communication data security modeling provided by the embodiment of the present application can quickly predict the average waiting delay under different forwarding times under given communication data by constructing an integrated model to approximate the Q value function by taking the communication data of the unmanned cluster as the state, taking the forwarding time as the behavior, and taking the average waiting delay after forwarding based on the selected forwarding time as the reward of the behavior, and then determining the forwarding time with the minimum message delay, can fully consider the actual operation of the unmanned cluster communication, adapt to the dynamic change of the unmanned cluster, reduce the occupation of communication resources, and reduce the overall communication delay; and by dividing the integrated model into two parts and deploying and cooperatively training them separately, the communication data of the unmanned cluster can be fully adapted to the distributed structural characteristics, data transmission can be reduced, and communication cost can be reduced. BRIEF DESCRIPTION OF DRAWINGS

[0042] The accompanying drawings, which are included to provide a further understanding of the embodiments of the present application, constitute a part of the present application and illustrate the embodiments of the present application and the description thereof serve to explain the present application, and do not constitute an improper limitation on the present application. In the drawings:

[0043] Figure 1 The flowchart of the unmanned cluster message forwarding time control method based on communication data security modeling provided by the embodiment of the present application is shown in the figure.

[0044] Figure 2 The schematic diagram of the unmanned cluster message forwarding structure model provided by the embodiment of the present application is shown in the figure.

[0045] Figure 3 The schematic diagram of the structure principle of the integrated model provided by the embodiment of the present application is shown in the figure. DETAILED DESCRIPTION

[0046] To make the objectives, technical solutions and advantages of the present application clearer, the technical solutions of the present application will be described in detail below with reference to the embodiments of the present application and the corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of the present application, but not all the embodiments. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0047] The technical solutions provided by the embodiments of the present application will be described in detail below with reference to the drawings.

[0048] Reference Figure 1 The embodiment of the present application provides an unmanned cluster message forwarding time control method based on communication data security modeling, which comprises the following steps:

[0049] Step 1: obtaining the communication data of the unmanned cluster at the current time and the optional behavior;

[0050] In the embodiment of the present application, the unmanned cluster includes a plurality of unmanned aerial vehicles, the communication data includes a standardized position matrix and a communication data matrix, the behavior represents a forwarding time, the position matrix is determined according to a set unmanned cluster center unmanned aerial vehicle and a topology of the unmanned cluster, and the communication data matrix is obtained by filling in the communication behavior data of the corresponding unmanned aerial vehicle at the corresponding position of the position matrix.

[0051] Reference Figure 2 At time t, the unmanned cluster center unmanned aerial vehicle corresponds to a plurality of adjacent unmanned aerial vehicles, and the center unmanned aerial vehicle maintains communication connection with each unmanned aerial vehicle. Based on the topology of the unmanned cluster, one unmanned aerial vehicle is regarded as a node, and a corresponding number is used to represent the adjacent relationship, and a position matrix of the unmanned cluster is constructed n represents the number of unmanned aerial vehicles in the unmanned cluster; the communication behavior data of each unmanned aerial vehicle in the unmanned cluster is collected, and the communication behavior data of each unmanned aerial vehicle is filled in the position corresponding to the unmanned aerial vehicle in the position matrix to obtain a communication data matrix of the unmanned cluster l represents the number of data types in the communication behavior data; the position matrix M is processed by using a maximum value normalization method p and the communication data matrix M v to obtain a standardized position matrix and a communication data matrix, and further obtain the communication data of the unmanned cluster

[0052] In the embodiment of the present application, the number representing the adjacent relationship can be 0 or 1, 0 representing no adjacent relationship and 1 representing an adjacent relationship.

[0053] In the embodiment of the present application, the communication behavior data of the unmanned aerial vehicle includes but is not limited to: message type, transmission frequency, and data size.

[0054] Step 2, using the communication data of the unmanned cluster and one behavior as the input of the integrated model pre-constructed and trained in the unmanned cluster center unmanned aerial vehicle, the average waiting delay output by the integrated model corresponding to each behavior under the current communication data is obtained respectively, and the behavior with the minimum average waiting delay is selected as the forwarding time under the current communication data.

[0055] Reference Figure 3 In the embodiment of the present application, the integrated model includes an integrated vector generation model deployed on each unmanned aerial vehicle and a behavior reward prediction model deployed on the unmanned cluster center unmanned aerial vehicle.

[0056] Reference Figure 3In the embodiment of the present application, the input of the integrated vector generation model on the other unmanned aerial vehicles than the unmanned aerial vehicle cluster center unmanned aerial vehicle is the communication behavior data of the corresponding unmanned aerial vehicle, the input of the integrated vector generation model on the unmanned aerial vehicle cluster center unmanned aerial vehicle is the communication behavior data of the unmanned aerial vehicle cluster center unmanned aerial vehicle and the selected behavior, the output of the integrated vector generation model is an integrated vector, the input of the behavior reward prediction model is the integrated vector output by all integrated vector generation models, and the output of the behavior reward prediction model is an average waiting delay.

[0057] The unmanned aerial vehicle cluster message forwarding time control method based on communication data security modeling provided by the embodiment of the present application can quickly predict the average waiting delay under different forwarding times under given communication data by constructing an integrated model to approximate a Q value function by taking the communication data of the unmanned aerial vehicle cluster as a state, taking the forwarding time as a behavior, and taking the average waiting delay after forwarding based on the selected forwarding time as the reward of the behavior, and then determining the forwarding time with the minimum message delay, can fully consider the actual operation of the unmanned aerial vehicle cluster communication, adapt to the dynamic change of the unmanned aerial vehicle cluster, reduce the occupation of communication resources, and reduce the overall communication delay; and by dividing the integrated model into two parts and deploying and cooperatively training the two parts, the communication data of the unmanned aerial vehicle cluster can be fully adapted to the distributed structural characteristics, data transmission can be reduced, and communication cost can be reduced.

[0058] The unmanned aerial vehicle cluster message forwarding time control method based on communication data security modeling provided by the embodiment of the present application combines the mechanism principle of deep reinforcement learning, takes the communication data of the unmanned aerial vehicle cluster as a state, takes the forwarding time as a behavior, and simultaneously considers the complexity of the unmanned aerial vehicle cluster, needs to avoid overlapping and missed selection of appropriate index design reward, and since the forwarding delay can measure whether the message transmission is smooth or not and reflects the communication condition of the unmanned aerial vehicle cluster, the average waiting delay is taken as the reward.

[0059] Further, in the embodiment of the present application, the integrated vector generation model is used to compress the communication behavior data of the unmanned aerial vehicle to obtain an integrated vector containing rich information.

[0060] In the embodiment of the present application, the integrated vector generation model specifically obtains and outputs the integrated vector by processing the input data by using a convolutional neural network and a self-attention mechanism.

[0061] Reference Figure 3 In the embodiment of the present application, the integrated vector generation model includes:

[0062] The first convolutional neural network takes the input of the integrated vector generation model as the input, and is connected with the max-pooling layer to perform data compression by convolution operation and extract local features of the input data;

[0063] The maximum pooling layer is connected with the first activation function layer and is used for maximum pooling processing on the input feature.

[0064] The first activation function layer is connected with the second convolutional neural network and is used for nonlinear transformation on the input feature by using a preset activation function.

[0065] The second convolutional neural network is connected with the average pooling layer and is used for encoding deep communication behavior features and extracting more abstract features.

[0066] The average pooling layer is connected with the second activation function layer and is used for average pooling processing on the input feature.

[0067] The second activation function layer is connected with the flattening layer and is used for nonlinear transformation on the input feature by using a preset activation function.

[0068] The flattening layer is connected with the first full connection layer and is used for mapping the input multi-dimensional feature into a one-dimensional vector.

[0069] The first full connection layer is connected with the second full connection layer and is used for compressing the input feature and strengthening the interaction between features.

[0070] The second full connection layer is connected with the self-attention mechanism module and is used for further compressing the input feature and completing dimension transformation.

[0071] The self-attention mechanism module is used for fusing the input feature, obtaining and outputting an integrated vector.

[0072] In the embodiment of the application, the input of the integrated vector generation model on the unmanned aerial vehicle is the communication behavior data of the unmanned aerial vehicle, and since the behavior adopted by the unmanned aerial vehicle cluster has an important influence on the subsequent communication behavior, for the center unmanned aerial vehicle of the unmanned aerial vehicle cluster, the selected behavior needs to be integrated into the communication behavior data, and then the communication behavior data fused with the behavior is used as the input of the integrated vector generation model of the center unmanned aerial vehicle.

[0073] Specifically, the integration of the selected behavior into the communication behavior data can be represented as:

[0074] Y=Y(kW) T ;

[0075] Wherein, Y represents the communication behavior data of the center unmanned aerial vehicle, k represents the selected behavior, that is, the selected forwarding time, and W represents a transition matrix, which is pre-set.

[0076] In the embodiment of the application, the communication data of the unmanned aerial vehicle cluster is The first convolutional neural network has a sliding window size of [4x4], a channel number of (l+1), a step of [2x2] and a padding mode of "SAME"; the second convolutional neural network has a sliding window size of [2x2], a channel number of 8, a step of [2x2] and a padding mode of "SAME"; the first activation function layer performs nonlinear transformation on the input features by using a tanh activation function; and the second activation function layer performs nonlinear transformation on the input features by using a tanh activation function.

[0077] In the embodiment of the application, each column vector in the output of the second fully connected layer corresponds to the communication behavior data of an adjacent unmanned aerial vehicle, and has the characteristic of symmetry, so it is appropriate to use the self-attention mechanism to fuse the vectors.

[0078] Specifically, in the embodiment of the application, the self-attention mechanism module fuses features by the following method to obtain an integrated vector:

[0079]

[0080] wherein W1, W2, b1 and b2 represent pre-trained parameters, F represents the input of the self-attention mechanism module, i.e., the output of the second fully connected layer, tanh represents a tanh function operation, T represents a transpose operation, softmax represents a softmax function operation, θ1 and μ1 represent intermediate vectors, μ i represents the i-th component of μ1, f i represents the i-th column of F, and z' represents the integrated vector.

[0081] It should be noted that the pre-trained parameters are determined by training the model.

[0082] In the embodiment of the application, the integrated vector generated by the integrated vector generation model set above is used to process the communication behavior data of the unmanned aerial vehicle, and the integrated vector obtained condenses all the communication behavior data of the unmanned aerial vehicle.

[0083] Further, in the embodiment of the application, after obtaining the integrated vector, the unmanned aerial vehicle can upload the integrated vector to the central server of the unmanned aerial vehicle cluster through a wired link for subsequent use, thereby avoiding repeated transfer of data and reducing communication burden.

[0084] Further, in the embodiment of the application, the behavior reward prediction model is used to integrate the integrated vector data of multiple unmanned aerial vehicles, fit the mapping relationship between the state-behavior and the reward, and realize large field of view data analysis.

[0085] In the embodiment of the application, the behavior reward prediction model specifically performs information interaction and deep encoding on the input integrated vector by using the gate attention mechanism, and obtains and outputs the average waiting delay.

[0086] Reference Figure 3 In the embodiment of the present application, the behavior reward prediction model comprises:

[0087] a vector compression module, input of which is the input of the behavior reward prediction model, and output of which is connected with the gate attention mechanism module, used for calculating the correlation between the integrated vectors of the input and compressing the integrated vectors to obtain and output a compressed vector;

[0088] a gate attention mechanism module, input of which is the compressed vector output by the vector compression module and the integrated vector corresponding to the unmanned aerial vehicle of the unmanned cluster center in the input of the behavior reward prediction model, used for processing the input vectors by using the gate attention mechanism to obtain and output the average waiting delay.

[0089] In the embodiment of the present application, the vector compression module first calculates the correlation between the integrated vectors and normalizes the correlation into weights, and then performs weighted fusion of the integrated vectors according to the weights to obtain the compressed vector.

[0090] Specifically, the processing process of the vector compression module can be represented as:

[0091]

[0092] wherein, W κ represents the pre-trained parameters, z represents the integrated vector corresponding to the unmanned aerial vehicle of the unmanned cluster center, z1 represents the integrated vector corresponding to the first unmanned aerial vehicle in the adjacent unmanned aerial vehicle set of the unmanned cluster center unmanned aerial vehicle, z2 represents the integrated vector corresponding to the second unmanned aerial vehicle in the adjacent unmanned aerial vehicle set of the unmanned cluster center unmanned aerial vehicle, the superscript T represents the transpose operation, softmax represents the softmax function operation, κ represents the correlation weight vector, z i represents the integrated vector corresponding to the i-th unmanned aerial vehicle in the adjacent unmanned aerial vehicle set of the unmanned cluster center unmanned aerial vehicle, κ i represents the i-th component of κ, represents the adjacent unmanned aerial vehicle set of the unmanned cluster center unmanned aerial vehicle, and q represents the compressed vector.

[0093] In the embodiment of the present application, the gate attention mechanism module first calculates the gate value of the gate attention mechanism by using the correlation weight to deepen the influence of the behavior on the decision, then based on the calculated gate value, the integrated vector of the center unmanned aerial vehicle and the compressed vector obtained by the vector compression module are converted into an intermediate variable by using the gate attention mechanism, and finally the intermediate variable is transformed into a predicted reward by a layer of dimension transformation, and the predicted reward is the predicted value of the average waiting delay.

[0094] Specifically, the processing process of the gate attention mechanism module can be represented as:

[0095]

[0096] wherein, W g , W r and b r represent pre-trained parameters, z represents an integrated vector corresponding to a center unmanned aerial vehicle of the unmanned aerial vehicle cluster, the superscript T represents a transposition operation, sigmoid represents a sigmoid function operation, g represents a gate value, q represents a compressed vector, u represents an intermediate vector, represents a predicted reward, i.e., a predicted value of an average waiting delay.

[0097] It should be noted that the pre-trained parameters are determined by training the model.

[0098] In the embodiment of the present application, the model is divided into an integrated vector generation model and a behavior reward prediction model, the integrated vector generation model is deployed on each unmanned aerial vehicle, and the behavior reward prediction model is deployed on the center unmanned aerial vehicle. The integrated vector generation model is used to realize compression of original large-scale data, so as to realize lightweight knowledge sharing. The center unmanned aerial vehicle receives compressed data of each unmanned aerial vehicle and uses the behavior reward prediction model to predict the reward of behavior. This can not only expand the data field, promote information fusion, and improve the accuracy of the mapping relationship from the fitted state-behavior to the reward, but also fully adapt to the distributed structural characteristics of communication data of the unmanned aerial vehicle cluster, reduce data transmission, and reduce communication cost.

[0099] Further, in the embodiment of the present application, the integrated model is trained in the following manner:

[0100] Step 201, a training data set is obtained, the training data including communication data of the unmanned aerial vehicle cluster at one time and selected behavior, and actual average waiting delay obtained based on the selected behavior;

[0101] Step 202, the communication behavior data of each unmanned aerial vehicle in the training data is taken as the input of the integrated vector generation model on each unmanned aerial vehicle, the communication behavior data of the center unmanned aerial vehicle of the unmanned aerial vehicle cluster and the selected behavior in the training data are taken as the input of the integrated vector generation model on the center unmanned aerial vehicle of the unmanned aerial vehicle cluster, the integrated vectors output by all integrated vector generation models are taken as the input of the behavior reward prediction model on the center unmanned aerial vehicle of the unmanned aerial vehicle cluster, and the actual average waiting delay in the training data is taken as the output of the behavior reward prediction model. The integrated vector generation model and the behavior reward prediction model are trained to obtain model parameters of the integrated vector generation model and model parameters of the behavior reward prediction model.

[0102] Step 203, the model parameters of each integrated vector generation model and the model parameters of the behavior reward prediction model are aggregated by swarm learning to determine final model parameters and obtain an integrated model.

[0103] In this embodiment of the invention, since the communication behavior of the unmanned swarm has a time dependency, the training dataset can be obtained by collecting historical data multiple times.

[0104] Furthermore, in this embodiment of the invention, during the collaborative training of the integrated vector generation model and the behavior reward prediction model in step 202, the gradient descent method is used to train and update the model parameters.

[0105] Specifically, refer to Table 1, which provides the detailed process for model co-training in step 202.

[0106] Table 1 Model Collaborative Training Process

[0107]

[0108]

[0109] In Table 1 above, M1 represents the ensemble vector generation model, z represents the ensemble vector corresponding to the central UAV, None represents the empty set, and η i Represents the set of adjacent drones The i-th drone in the series, z i Let M1 represent the ensemble vector corresponding to the i-th drone in the adjacent drone set, and M2 represent the behavior reward prediction model. r represents the predicted reward output by the behavioral reward prediction model. t This represents the actual reward in the training data; the reward is the average waiting latency. Indicates reward based on prediction and actual reward r t Calculate the loss function, loss, G. c =η.compute_G(M1; M2|loss) means calculating the critical gradient G based on the loss function loss. c The total gradient of the model, G, is used to update the parameters of the model deployed in the central UAV, and the critical gradient G is... c Used for updating parameters of models deployed in adjacent drones.

[0110] During the model co-training process, the ensemble vector generation model of each UAV will use the communication data Y i Compressed into an integrated vector z i And upload it to the temporary central server Γ, and store it in a vector set. The central drone downloads vector sets from the temporary central server Γ. The behavior reward prediction model deployed at the center of the drone is based on the received ensemble vector z and vector set. By fitting the state-behavior relationship to the reward, a predicted reward is obtained; then, based on the predicted reward... and the actual reward r t calculating a loss function loss, calculating a critical gradient G according to the loss function loss c and a model full gradient G, updating the parameters of the deployed model by using the model full gradient G, and sharing the critical gradient G c to each adjacent UAV, so that the adjacent UAV uses the critical gradient G c to update the parameters of the deployed model.

[0111] In the embodiments of the present application, the specific type of the loss function is set according to actual needs, and a conventional loss function can be selected.

[0112] Further, in the embodiments of the present application, considering that the data distributions of different UAVs are different, for example, a certain UAV is always congested, and a certain UAV is always smooth, which will hinder the training progress and reduce the generalization ability of the model. To this end, swarm learning is introduced, the state space of fitting is restricted through the interaction of parameters, and the training process of the model is optimized, and at the same time, in order to avoid centralized training and protect user privacy, each UAV only shares parameters without transmitting data.

[0113] In the embodiments of the present application, in step 203, an integrated model is obtained by swarm learning fusion of the updated parameters trained in step 202.

[0114] Specifically, referring to Table 2, Table 2 shows the specific process of aggregating model parameters by using swarm learning in step 203.

[0115] Table 2 Parameter integration process based on swarm learning

[0116]

[0117]

[0118] In the above Table 2, None represents an empty set, represents a set of UAVs , and the i-th UAV in the set.

[0119] In the process of aggregating model parameters by using swarm learning, the parameters in the parameter set Ψ and the losses loss in the loss function set are derived from the aforementioned model collaborative training process, and each UAV transmits its own parameters to the temporary center server Γ, and after receiving the parameters of all UAVs , the temporary center server Γ aggregates the parameters by the following formula:

[0120]

[0121] The above aggregation formula calculates the integrated model parameters Ψ according to the lossagg , the local parameters with high accuracy are given greater weights, which helps to improve the fitting effect; then, the temporary central server Γ integrates the model parameters Ψ agg Through the wired link, the module is deployed to each UAV in the domain. Subsequently, when signal switching is required, the UAV evaluates each behavior under the current state based on the integrated model, predicts the cumulative reward, and determines the forwarding time with the minimum communication delay.

[0122] In the embodiments of the present application, the fusion of models of different nodes is performed by group learning to obtain the final integrated model, which can fully adapt to the distributed structural characteristics of the communication data of the unmanned cluster, while avoiding centralized training, optimizing the training process of the model, and improving information security.

[0123] It should be noted that, in this document, relational terms such as“first” and“second”, and the like, are used solely to distinguish one entity or action from another entity or action, without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms“comprises”,“comprising”, or any other variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can include other elements not expressly listed or inherent to such process, method, article, or apparatus.

[0124] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacements for part of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for controlling the message forwarding time of an unmanned cluster based on communication data security modeling, characterized in that, The method includes: The system acquires the communication data and optional behaviors of the unmanned swarm at the current moment. The unmanned swarm consists of multiple drones. The communication data includes a standardized position matrix and a communication data matrix. The behaviors represent forwarding times. The position matrix is ​​determined based on the set topology of the unmanned swarm center drone and the unmanned swarm. The communication data matrix is ​​obtained by filling the corresponding drone's communication behavior data into the corresponding positions of the position matrix. Using the communication data and behavior of an unmanned swarm as input to an ensemble model pre-built and trained in the swarm's central UAV, the average waiting delay output by the ensemble model for each behavior under the current communication data is obtained. The behavior with the smallest average waiting delay is selected as the forwarding time under the current communication data. The ensemble model includes an ensemble vector generation model deployed on each UAV and a behavior reward prediction model deployed on the central UAV. The input to the ensemble vector generation models on other UAVs besides the central UAV is the corresponding UAV's communication behavior data. The input to the ensemble vector generation model on the central UAV is the communication behavior data of the central UAV and the selected behavior. The output of the ensemble vector generation model is an ensemble vector. The input to the behavior reward prediction model is the ensemble vector output by all ensemble vector generation models. The output of the behavior reward prediction model is the average waiting delay.

2. The unmanned cluster message forwarding time control method based on communication data security modeling according to claim 1, characterized in that, The communication behavior data of drones includes, but is not limited to: message type, sending and receiving frequency, and data size.

3. The unmanned cluster message forwarding time control method based on communication data security modeling according to claim 1, characterized in that, The ensemble vector generation model processes the input data using a convolutional neural network and a self-attention mechanism to obtain and output an ensemble vector.

4. The unmanned cluster message forwarding time control method based on communication data security modeling according to claim 3, characterized in that, The ensemble vector generation model includes: The first convolutional neural network takes the input of the ensemble vector generation model as input and connects its output to a max pooling layer to compress data through convolution operations and extract local features of the input data. The output of the max pooling layer is connected to the first activation function layer and is used to perform max pooling processing on the input features. The output of the first activation function layer is connected to the second convolutional neural network and is used to perform nonlinear transformation on the input features using a preset activation function. The second convolutional neural network, whose output is connected to an average pooling layer, is used to encode deep communication behavior features and extract more abstract features; The average pooling layer output is connected to the second activation function layer and is used to perform average pooling processing on the input features; The second activation function layer, whose output is connected to the flattening layer, is used to perform nonlinear transformation on the input features using a preset activation function; The output of the flattened layer is connected to the first fully connected layer and is used to map the input multidimensional features into a one-dimensional vector. The first fully connected layer outputs to the second fully connected layer, which is used to compress the input features and enhance the interaction between features; The second fully connected layer outputs to the self-attention mechanism module, which is used to further compress the input features and complete the dimensionality transformation. The self-attention mechanism module is used to fuse input features, obtain and output an integrated vector.

5. The unmanned cluster message forwarding time control method based on communication data security modeling according to claim 4, characterized in that, The self-attention mechanism module performs feature fusion to obtain the ensemble vector in the following manner: Where W1, W2, b1, and b2 represent the pre-trained parameters, F represents the input of the self-attention mechanism module, tanh represents the tanh function operation, the superscript T represents the transpose operation, softmax represents the softmax function operation, θ1 and μ1 represent the intermediate vectors, and μ i f represents the i-th component of μ1. i Let F represent the i-th column, and z′ represent the integrated vector.

6. The unmanned cluster message forwarding time control method based on communication data security modeling according to claim 1 or 5, characterized in that, The behavioral reward prediction model uses a gate attention mechanism for information interaction and deeply encodes the input ensemble vector to obtain and output the average waiting delay.

7. The unmanned cluster message forwarding time control method based on communication data security modeling according to claim 6, characterized in that, The behavioral reward prediction model includes: The vector compression module takes the input of the behavior reward prediction model as input and connects to the gate attention mechanism module at output. It is used to calculate the correlation between the input ensemble vectors and compress the ensemble vectors to obtain and output the compressed vector. The gate attention mechanism module takes as input the compressed vector output by the vector compression module and the integrated vector corresponding to the unmanned cluster center UAV in the input of the behavior reward prediction model. It is used to process the input vector by utilizing the gate attention mechanism to obtain and output the average waiting delay.

8. The unmanned cluster message forwarding time control method based on communication data security modeling according to claim 7, characterized in that, The vector compression module calculates the correlation between the input ensemble vectors and compresses the ensemble vectors to obtain the compressed vectors in the following manner: Among them, W κ Let z represent the pre-trained parameters, z represent the ensemble vector corresponding to the center drone of the swarm, z1 represent the ensemble vector corresponding to the first drone in the neighboring drone set of the center drone, z2 represent the ensemble vector corresponding to the second drone in the neighboring drone set of the center drone, the superscript T indicates transpose operation, softmax indicates softmax function operation, and κ represent the relevance weight vector. i κ represents the ensemble vector corresponding to the i-th drone in the set of adjacent drones of the drone cluster center. i Represents the i-th component of κ. Let q represent the set of adjacent drones of the drone at the center of the unmanned swarm, and let q represent the compressed vector.

9. The unmanned cluster message forwarding time control method based on communication data security modeling according to claim 8, characterized in that, The gate attention mechanism module processes the input vector in the following way to obtain the average waiting delay: Among them, W g W r and b r The parameters represent the pre-training parameters, sigmoid represents the sigmoid function operation, g represents the gate value, and u represents the intermediate vector. This represents the predicted value of the average waiting delay.

10. The unmanned cluster message forwarding time control method based on communication data security modeling according to claim 1, characterized in that, The ensemble model is trained in the following manner: Obtain a training dataset, which includes communication data and selected behaviors of the unmanned swarm at a given time, as well as the actual average waiting latency obtained based on the selected behaviors; The communication behavior data of each UAV in the training data is used as the input to the ensemble vector generation model on each UAV. The communication behavior data and selected behavior of the UAV in the UAV swarm center in the training data are used as the input to the ensemble vector generation model on the UAV swarm center. The ensemble vector output by all ensemble vector generation models is used as the input to the behavior reward prediction model on the UAV swarm center. The actual average waiting delay in the training data is used as the output of the behavior reward prediction model. The ensemble vector generation model and the behavior reward prediction model are trained to obtain the model parameters of the ensemble vector generation model and the model parameters of the behavior reward prediction model. By using group learning to aggregate the model parameters of each ensemble vector generation model and the model parameters of the behavior reward prediction model, the final model parameters are determined, and the ensemble model is obtained.

Images