Battlefield dynamic data collection timeliness optimization method and device and computer equipment

Abstract

Embodiments of the present application disclose a battlefield dynamic data collection timeliness optimization method and device and a computer device. The method comprises: acquiring environment data dynamic collection signals; converting the average information age of mobile terminal data into a mathematical problem; inputting the mathematical problem into an Actor-Critic network model with a fusion attention mechanism to calculate a bandwidth allocation ratio set to obtain a calculation result; and using the calculation result to allocate edge servers to perform offloading calculation of the environment data. The Actor-Critic network model with the fusion attention mechanism is deployed on the mobile terminal and the edge server. The method of the embodiments of the present application can improve the timeliness of dynamic data collection.

Description

Technical Field

[0001] This invention relates to computers, and more specifically to methods, apparatus, and computer equipment for optimizing the timeliness of battlefield dynamic data acquisition. Background Technology

[0002] During missions, combat personnel can use mobile terminals, such as drones and robots, to collect environmental data in order to update and understand the environmental security situation. The collected data needs to be uploaded to the cloud center in a timely manner for data updates, which places high demands on the system's timeliness and data freshness. However, due to the limited computing power of mobile terminals and the potential bandwidth pressure on data transmission, problems such as large data processing latency and transmission interruptions may occur.

[0003] The dynamic data acquisition process is highly sensitive to timeliness. Traditional delay calculation is generally the difference between the receiving time and the transmission time. This method has a drawback: it cannot reflect the freshness of the data. Due to factors such as channel quality, data generated earlier may be sent later, which is unsuitable for systems that need to update the status in real time.

[0004] Therefore, it is necessary to design a new method to improve the timeliness of dynamic data acquisition. Summary of the Invention

[0005] The purpose of this invention is to overcome the shortcomings of the prior art and provide a method, device and computer equipment for optimizing the timeliness of battlefield dynamic data acquisition.

[0006] To achieve the above objectives, the present invention adopts the following technical solution: a method for optimizing the timeliness of battlefield dynamic data acquisition, comprising:

[0007] Dynamically acquire environmental data and signals;

[0008] Convert the average information age of mobile terminal data into a mathematical problem;

[0009] The mathematical problem is input into the Actor-Critic network model that incorporates the attention mechanism to calculate the bandwidth allocation ratio set, so as to obtain the calculation result;

[0010] The calculation results are used to allocate edge servers for offloading calculations of the environmental data;

[0011] Among them, the Actor-Critic network model that incorporates the attention mechanism is deployed on mobile terminals and edge servers.

[0012] The further technical solution is as follows: converting the average information age of mobile terminal data into a mathematical problem includes:

[0013] The optimization problem of the average information age of mobile terminal data is modeled as a Markov process to obtain the mathematical problem.

[0014] The further technical solution is as follows: During the training process of the Actor-Critic network model that integrates the attention mechanism, after each Ef round of learning, all agents share their respective model parameters and update them; the Actor and Critic networks of each mobile terminal agent retain parameters with weight w, and update them in combination with the parameters of other agents.

[0015] The further technical solution is as follows: A federated learning algorithm is used to train the Actor-Critic network model with fused attention mechanism. The process of the agent interacting with the environment is used as an experience cache set and as training samples to train the Actor-Critic network model with fused attention mechanism.

[0016] The further technical solution is as follows: The Actor-Critic network model with integrated attention mechanism uses the set of bandwidth allocation ratios output by the Actor-Critic network model, which is then fused through the attention mechanism, and combined with the data sampled from the experience cache set as the sample set to train the Actor-Critic network model.

[0017] This invention also provides a device for optimizing the timeliness of battlefield dynamic data acquisition, comprising:

[0018] The signal acquisition unit is used to acquire dynamically collected environmental data signals.

[0019] The conversion unit is used to convert the average information age of mobile terminal data into a mathematical problem.

[0020] The computing unit is used to input the mathematical problem into the Actor-Critic network model with fused attention mechanism to perform bandwidth allocation ratio set calculation in order to obtain the calculation result;

[0021] The allocation unit is used to allocate edge servers using the calculation results to perform the offloading calculation of the environmental data.

[0022] The further technical solution is as follows: the conversion unit is used to model the optimization problem of the average information age of mobile terminal data as a Markov process to obtain a mathematical problem.

[0023] The present invention also provides a computer device, the computer device including a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the above-described method.

[0024] The present invention also provides a storage medium storing a computer program that, when executed by a processor, can implement the above-described method.

[0025] The beneficial effects of this invention compared with the prior art are as follows: This invention optimizes the timeliness of dynamic data acquisition system by using an Actor-Critic network model with a fusion attention mechanism deployed on mobile terminals and edge servers, and performs set calculation of bandwidth allocation ratio to improve the timeliness of dynamic data acquisition.

[0026] The present invention will be further described below with reference to the accompanying drawings and specific embodiments. Attached Figure Description

[0027] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0028] Figure 1 A flowchart illustrating the battlefield dynamic data acquisition timeliness optimization method provided in this embodiment of the invention;

[0029] Figure 2 This is a schematic diagram illustrating the change of AoI over time, provided in an embodiment of the present invention.

[0030] Figure 3 A schematic diagram illustrating the training process of the Actor-Critic network model with an integrated attention mechanism provided in this embodiment of the invention;

[0031] Figure 4 This is a schematic diagram illustrating the convergence performance at different learning rates provided in embodiments of the present invention.

[0032] Figure 5 This is a schematic diagram comparing different data generation rates provided in an embodiment of the present invention;

[0033] Figure 6 This is a schematic diagram showing the comparison of average AoI provided in an embodiment of the present invention;

[0034] Figure 7 This is a schematic diagram of the worst AoI comparison provided in an embodiment of the present invention;

[0035] Figure 8 This is a schematic diagram comparing the amount of unloaded data provided in an embodiment of the present invention;

[0036] Figure 9 This is a schematic diagram comparing the total length of unloading data provided in an embodiment of the present invention;

[0037] Figure 10 This is a comparative diagram of different numbers of servers and mobile terminals provided in an embodiment of the present invention;

[0038] Figure 11 Box layout diagrams for different numbers of servers and mobile terminals provided in embodiments of the present invention;

[0039] Figure 12 This is a schematic diagram comparing different environments provided in the embodiments of the present invention;

[0040] Figure 13 This is a schematic diagram comparing different federal factors provided in an embodiment of the present invention;

[0041] Figure 14 Box plots of different federation factors provided in embodiments of the present invention;

[0042] Figure 15 A schematic block diagram of a battlefield dynamic data acquisition timeliness optimization device provided in an embodiment of the present invention;

[0043] Figure 16 A schematic block diagram of a computer device provided for an embodiment of the present invention. Detailed Implementation

[0044] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0045] It should be understood that, when used in this specification and the appended claims, the terms "comprising" and "including" indicate the presence of the described features, integrals, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or collections thereof.

[0046] It should also be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the invention. As used in this specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.

[0047] It should also be further understood that the term "and / or" as used in this specification and the appended claims refers to any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0048] Please see Figure 1, Figure 1 This is a schematic flowchart illustrating the battlefield dynamic data acquisition timeliness optimization method provided in this embodiment of the invention. The dynamic data acquisition system transmits real-time data from the field, including text, voice, images, and video, back to the command center. This places high demands on the timeliness and freshness of the dynamic data acquisition system. Traditional metrics such as latency are insufficiently comprehensive. Furthermore, the limited computing power of mobile terminals and potential bandwidth constraints during data transmission can lead to significant data processing latency and transmission interruptions, necessitating the use of edge servers for auxiliary computation. This battlefield dynamic data acquisition timeliness optimization method significantly improves the timeliness of the dynamic data acquisition system.

[0049] This embodiment addresses the timeliness optimization problem of dynamic data acquisition systems applied on the battlefield by minimizing the sum of AoI (Aspect-Oriented Intelligence) of all mobile terminals in the system. Data is generated randomly and collected by mobile terminals, which possess computing power to preprocess the data. The processed data is then offloaded to edge base stations via an attenuation channel. The algorithm divides time into time slices, each executing two decisions: preprocessing the raw data by the mobile terminal and offloading the preprocessed data. The goal is to ensure good system timeliness.

[0050] This paper introduces AoI (Age of Information) to analyze the timeliness of data in battlefield dynamic data acquisition systems, which is of great significance for data systems sensitive to timeliness and data freshness. Considering the random generation of data, the limited computing power of mobile terminals, and the urban environment with attenuating transmission channels, an optimization problem is proposed with the objective of minimizing the average AoI of all mobile terminals. This problem is modeled as a Markov process. The solution to the optimization problem adopts a deep reinforcement learning multi-agent cooperative algorithm with an attention mechanism. Through continuous interaction between agents and the environment and the use of federated training methods, computational offloading and resource allocation are performed.

[0051] Consider N mobile terminals and M edge servers. The edge servers are vehicle-mounted and move within a certain area. Personnel can use the mobile terminals to collect environmental data. This data is then offloaded to appropriate edge servers for processing. The data is generated randomly and collected by the mobile terminals, which possess some computing power for preprocessing. The processed data is then offloaded to the edge servers via an attenuation channel. The algorithm divides time into time slices, each executing two decisions: preprocessing the raw data by the mobile terminals and offloading the preprocessed data. The goal is to ensure good system timeliness.

[0052] Figure 1This is a flowchart illustrating the method for optimizing the timeliness of battlefield dynamic data acquisition provided in this embodiment of the invention. Figure 1 As shown, the method includes the following steps S110 to S140.

[0053] S110, Acquire environmental data by dynamically collecting signals.

[0054] In this embodiment, when a signal for dynamically acquiring environmental data occurs, timeliness optimization is required to improve the timeliness of dynamic data acquisition.

[0055] S120. Convert the average information age of mobile terminal data into a mathematical problem.

[0056] In this embodiment, the optimization problem of the average information age of mobile terminal data is modeled as a Markov process to obtain a mathematical problem.

[0057] Specifically, in real-world scenarios, the transmission link between the mobile terminal and the edge server is likely to be obstructed by buildings, resulting in a coexistence of LoS (Line of Sight) and NLoS (Non-Line of Sight) communication. In this situation, a simple LoS channel model cannot accurately describe the transmission channel between the terminal and the edge server; therefore, a probabilistic LoS channel model is used. Line of sight and non-line of sight communication each have a specific probability of occurrence, which is a function of the environment, building density and height, and the elevation angle between the terminal and the edge server. The probability of a LoS link existing in the transmission channel can be expressed as: Where C and B are constant values ​​determined by the environment, such as plains, mountains, jungles, or other scenarios, and θ is the elevation angle of the edge server relative to the mobile terminal, thus yielding the probability P of the NLoS link. NLos =1-P Los Currently, the channel gain for probabilistic Loss of Speed ​​(LoS) is... in,

[0058] in, Represents small-scale fading and α is the path loss exponent, d(t) represents the distance between the mobile terminal and the edge server, and β represents the channel power gain at a reference distance of 1m. The total channel bandwidth is W, and in time slice t, the bandwidth allocation rate between the mobile terminal and the edge server is... Therefore, the transmission rate is Where N0 represents the noise power spectral density, P n (t) represents the transmission power, and satisfies: 0 ≤ P n (t)≤P n max(t).

[0059] The continuous time is divided into multiple time slices, denoted by integers t. A list is used to represent the data D collected by each terminal in time slice t. n (t) = [b(t), l(t), x(t)], where b(t) represents the size of the data, l(t) represents the time elapsed, and x(t) represents the mobile terminal's ID. Assume the collected data in each time slice t has a probability... Randomly generated, its size follows the arrival rate The Poisson distribution of can be represented as follows:

[0060] Mobile terminal n has two capacities respectively and The data queues are divided into two parts: a cache queue, which primarily stores newly collected data awaiting local preprocessing on the mobile terminal, and an offload queue, which stores preprocessed data awaiting offload to the edge server for further processing. Both data queues employ a zero-wait strategy, meaning that within each time slice, the mobile terminal can only process one data packet at a time; only after processing one packet can the next be executed. The cached and offload data status of the mobile terminal in each time slice are represented as follows: Among them, e i (t)∈{0,1},O i (t)∈{0,1}; they satisfy:

[0061] The calculated local preprocessing time for mobile terminal n in time slice t is: Among them, f n This indicates the CPU cycle frequency of a mobile terminal, used to measure computing power.

[0062] Dynamic data acquisition systems are highly sensitive to timeliness. Traditional latency calculations typically use the difference between the reception and transmission times. However, this method has a drawback: it fails to reflect data freshness. Channel quality issues can cause data generated earlier to be transmitted later, which is unsuitable for systems requiring real-time state updates. Therefore, to optimize the timeliness of dynamic data acquisition systems, information age is introduced to describe the freshness of data received by the edge server. The timeliness of a dynamic data acquisition system reflects the frequency with which mobile terminals collect, process, and unload data. Information age is defined as the time interval between the sending end generating a timestamped state update message and transmitting it to the receiving end via the communication system. Assuming that at time slice t, edge server m receives unloaded data from mobile terminal n, and the timestamp of this data generation is δ(t), then the AoI of the data in mobile terminal n is defined as the difference between the two time slices, expressed as: Δn (t)=t-δ(t).

[0063] Based on the definition of zero-wait policy and AoI definition, and assuming Δ n (0) = 0, and at the same time, a graph showing the change over time can be derived, such as Figure 2 As shown, the peak point of the curve is denoted as PAoI.

[0064] To optimize the timeliness of the dynamic data acquisition system, the objective is to minimize the average information age of all mobile terminal data, denoted as: P1: stEq(1)to Eq(10)

[0065] The battlefield environment is complex and ever-changing, so data is randomly generated, and the transmission channel between the mobile terminal and the edge server is also attenuating. This means that the mobile terminal's strategy may differ in each time slice. Moreover, the information age of the mobile terminal is not only related to its current state but also to its past states, and may even affect its subsequent states. Based on this, this optimization problem is modeled as an MDP (Markov decision process).

[0066] S130. Input the mathematical problem into the Actor-Critic network model with fused attention mechanism to calculate the bandwidth allocation ratio set, so as to obtain the calculation result.

[0067] In this embodiment, the Actor-Critic network model that incorporates the attention mechanism is deployed on mobile terminals and edge servers.

[0068] Specifically, during the training of the Actor-Critic network model with an attention mechanism, after each Ef round of learning, all agents share and update their respective model parameters; each mobile terminal's agent's Actor and Critic networks retain parameters with weight w, and update them in conjunction with the parameters of other agents. Here, Ef is the model update cycle during federated training, and w is the federation factor.

[0069] A federated learning algorithm is used to train an Actor-Critic network model with an integrated attention mechanism. The process of the agent's interaction with the environment is used as an experience cache set and as training samples to train the Actor-Critic network model with an integrated attention mechanism.

[0070] The Actor-Critic network model with an attention mechanism is trained by combining the set of bandwidth allocation ratios output by the Actor-Critic network model with the data sampled from the experience cache set after the attention mechanism is applied.

[0071] Please see Figure 3 This paper proposes an improved Actor-Critic network model. DRL (Dynamic Relationship Management) has been widely applied in multi-agent collaboration. Multi-agent collaboration refers to agents cooperating to achieve a common goal and thus obtaining a joint reward. There are two main methods: one direct method treats other agents as part of the environment, training each agent individually to maximize the cumulative reward for each agent. However, this method may lead to difficulties in complete convergence due to environmental instability. The other method treats all agents as a whole, with its action space being the sum of all agents' actions. However, this requires an efficient communication mechanism, which presents implementation challenges. Later, researchers proposed a multi-agent deep deterministic policy gradient algorithm, where each agent is independent, generating actions by observing the local environment, while the Critic network optimizes by observing global information. However, during optimization, each agent indiscriminately receives the states and actions of other agents, leading to reduced optimization performance. To address this issue, this paper proposes a multi-agent Actor-Critic algorithm with an attention mechanism to find the optimal solution to the aforementioned mathematical problem.

[0072] An Actor-Critic network model was deployed on all agents. First, the basic elements of the Actor-Critic network model are: State: The state S of the mobile terminal agent includes the location of data collection, the location and available resources of the edge server, the data information to be unloaded, its own two queue states, allocated bandwidth, channel state, etc., which can be represented as s = [l, I, B, w]; where l represents the location information of the mobile terminal and the edge server, I represents the data information to be unloaded, B represents its own two queue states, w represents the allocated bandwidth, and the state of the edge server agent includes the state of all mobile terminals.

[0073] Actions: The mobile terminal needs to collect data, preprocess the data locally, and offload the data to an edge server. Therefore, its actions include preprocessing strategies and offloading strategies, which can be represented as: a n (t)=[cac n (t),off n [(t)]; At the same time, the edge server allocates bandwidth to each mobile terminal, so the actions of the edge server are a set of bandwidth allocation ratios, which can be represented as:

[0074] Reward: The objective of this embodiment is to minimize the average information age of all data uninstalled from mobile terminals, therefore... The opposite of the value is used as the agent's reward, expressed as:

[0075] To study the global optimization problem of the system, consider a long-term reward, expressed as: Where γ represents the reward discount factor and T represents the time length.

[0076] When learning, agents can focus on information beneficial to themselves, so the previous Actor-Critic algorithm was improved by incorporating an attention mechanism. The attention mechanism is a mapping relationship where the agent can determine the importance of other agents by querying their states and actions, and integrate this information into its own action value estimation function. The value function considering information from other agents can be expressed as: Where f represents a neural network, φ n This represents an integration function. The encoded value representing the sum of the weights of other agents, and the contribution of other agents, are calculated using the following formula: Among them, e i It is obtained by substituting the actions and states of other agents into the ensemble function for encoding. Similarly, substituting the actions and states of agent n can encode e. n Normalization e n and e i After similarity is calculated, a softmax layer is used to generate weight β. i During this process, M k e i Convert to "key", M p e n Convert to "query". First, convert e... i By performing a linear transformation on a common matrix Y and then substituting it into a nonlinear function L, the encoded value V can be obtained. i .

[0077] The Actor-Critic network consists of two parts. The Actor network is an improvement on the policy gradient, which can easily select appropriate actions in a continuous action space. However, since the Actor network updates in rounds, its learning efficiency is relatively slow. Therefore, we add a value-based Critic network using the TD algorithm to achieve single-step updates, thus obtaining the Actor-Critic algorithm. The Actor network is an approximation of the policy function π(A|S;θ), selecting actions based on the current state of the input agent. The Critic network is an approximation of the value function, scoring the actions generated by the Actor network. Then, the Actor network updates the probability of actions based on the scores, making it easier for the Actor network to select "good" actions.

[0078] For updating Actor networks, the common method is policy gradient updates. The policy gradient in the continuous action space is: However, it is difficult to directly calculate this expectation, so we use the Monte Carlo approximation, randomly selecting an action based on the policy function π(A|S;θ). calculate,

[0079] Random selection ensures unbiasedness. By inputting the agent's current state and the randomly selected action into the Critic network, and outputting the action value based on an attention mechanism, the Actor network can be updated using stochastic gradients: θ t+1 =θ t +βq(s t ,a t ;θ t ); where β is the learning rate of the Actor network. The Critic network updates by minimizing the Mean Square Error (MSE) loss function. First, it takes the state and action inputs of the agent and obtains a weighted sum of the encoded values ​​through the attention mechanism. Then, it calculates the action value using the formula: l(w)=E[|q(s)] t ,a t ;w t )-y t | 2 ]; in, These are the parameters of the target Critic network, r t γ is the reward obtained by the agent, and γ is the reward discount factor.

[0080] Both the Actor and Critic networks have corresponding target networks, which have the same structure and initialization method as the original networks. Their main function is to determine the next state s. t+1 Estimate the next action a t+1 And the corresponding "scores" This also improves the stability and convergence of training, allowing it to be performed every T... u The learning rounds are performed by updating the original network parameters and fixed weights ξ.

[0081] Experience replay, a common method for improving data utilization, is also used in the algorithms of this chapter. A complete interaction between the agent and the environment is called a transition, represented by a tuple (s...). t ,a t ,s t+1 ,a t+1 All previous transitions are stored in the experience replay D as training samples. Its capacity is limited; when full, new data replaces older data because it is more valuable. A certain number of samples are drawn from this D each time the model is trained. To avoid the agent lacking exploration of certain actions when interacting with the environment, actions are randomly selected with probability ε.

[0082] In this embodiment, the multi-agent DRL algorithm is deployed on all mobile terminals and edge servers, rather than just on edge servers or remote cloud centers, for several reasons: the data for training the agents mainly comes from mobile terminals. If deployed only on edge servers or cloud centers, a large amount of data needs to be transmitted, increasing the transmission pressure on the channel. Moreover, when transmitting highly confidential data, there is a risk of leakage. The deployment scheme chosen in this embodiment can solve these two problems, but it also has shortcomings, namely the non-independent and identically distributed (i.i.d.) and imbalanced training data problem. This is mainly because the training data of a certain mobile terminal does not represent the training data characteristics of all mobile terminals, and the limited channel bandwidth makes it difficult to ensure that all devices are online. Therefore, a federated learning algorithm is introduced to train the agents. This avoids uploading a large amount of training data, only requiring the sharing of updated model parameters; it can solve the non-independent and identically distributed (i.i.d.) and imbalanced training data problem; and it can reduce the impact of limited communication resources. During each Ef round of updates, according to formula θ... t+1 =θ t ·Ω and w t+1 =w t ·Ω updates the parameters.

[0083] After each EF round of learning, all agents share and update their respective model parameters. Specifically, each terminal agent's Actor and Critic networks retain parameters with weight w, and update them in conjunction with the parameters of other agents, represented as: θ t+1 =θ t ·Ω;w t+1 =w t ·Ω; where θ t and w tLet A and Criterion represent the network parameters of all mobile terminal agents in time slice t, respectively, and Ω represent the federated update matrix.

[0084] S140. Use the calculation results to allocate edge servers to perform the offloading calculation of the environmental data.

[0085] The performance of the algorithm presented in this paper will be analyzed through simulation experiments, and this algorithm will be compared with other baseline algorithms based on deep reinforcement learning.

[0086] The experiment was simulated using the Python programming language and ran on a server with an Intel Core i7-9700H 3.6GHz CPU and 8GB of memory. The virtual environment used the Tensorflow-gpu 2.3 framework. Some experimental parameters are shown in Table 1.

[0087] Table 1. Partial Parameter Values

[0088]

[0089]

[0090] For comparison, the two baseline algorithms are described as follows: Actor-Critic-based unloading algorithm: To evaluate the proposed federated update-based multi-agent Actor-Critic algorithm, a standard Actor-Critic algorithm based on continuous action space is also implemented for the computational unloading problem. Multi-agent Actor-Critic-based unloading algorithm: To compare the performance of federated updates, we set up a standard multi-agent Actor-Critic unloading algorithm that does not use a federated approach to update parameters.

[0091] First, by setting different parameters, the corresponding convergence performance is compared to obtain the optimal value. For example... Figure 3 The figure shows the convergence performance of the proposed algorithm at different learning rates. It is assumed that the learning rates for the Actor and Critic networks are different. It can be seen that when α... Actor =0.1,α Critic When α = 0.2, the proposed algorithm exhibits significant oscillations and is difficult to converge. This is likely because the large learning rate causes both the Actor and Critic networks to adopt large update steps. Actor =0.001,α Critic =0.002 or α Actor =0.00001,α CriticWhen the learning rate is 0.00002, all proposed algorithms converge. However, because the network update speed is very slow when the learning rate is very small, more iterations are needed for convergence. Therefore, in the subsequent experiments, we set the learning rate of the Actor network and the Critic network to α respectively. Actor =0.001,α Critic =0.002.

[0092] exist Figure 5 The impact of different data generation rates on the algorithm's convergence performance was compared. The results show that the computational offloading strategy performed best when the data generation rate was 0.3. This may be due to limitations in environmental resources and the system itself. When the data generation rate exceeds a certain value, the proposed algorithm will struggle to optimize the system's timeliness. Therefore, in subsequent experiments, the data generation rate was set to 0.3.

[0093] Compare the performance of the proposed algorithm with the baseline algorithm under the same environmental settings. Figure 6 The horizontal axis represents the number of iterations, and the vertical axis represents the system's average AoI. The average AoI under the Actor-Critic algorithm is significantly higher than the other two algorithms, and it is difficult to converge. However, the results under the Multi-Agent Actor-Critic and Fed-DRL algorithms are very stable and consistently remain at low values, indicating that these two algorithms have comparable performance and are both superior to the Actor-Critic algorithm.

[0094] Figure 7 The figure represents the worst AoI among the three algorithms. The Fed-DRL algorithm also performs best. In the Actor-Critic algorithm, some resources are consistently ignored, possibly because this algorithm requires a complex neural network to extract the relationship between the global input state and the policy of each agent, leading to training difficulties. Additionally, Figure 8 and Figure 9 These represent the quantity and total length of the offloaded data received by the edge server, respectively. The results show that the Actor-Critic and Fed-DRL algorithms can process more data, thereby improving the data utilization rate of the dynamic data acquisition system.

[0095] like Figure 10 As shown, the proposed algorithm enables the dynamic data acquisition system to maintain a low average AoI regardless of the number of edge servers and mobile terminal devices. Consistent with common sense, the more edge servers or the fewer mobile terminals, the better the timeliness performance of the dynamic data acquisition system.

[0096] like Figures 11 to 12As shown, the performance of the proposed algorithm was tested in plains, mountains and jungle environments. The results showed that the results in the jungle environment were not ideal, possibly because the trees blocked the data transmission channel and caused interference. However, the algorithm performed well in the plains and mountains environments, indicating that the proposed algorithm can adapt to general battlefield environments.

[0097] Finally, the performance was compared under different federation factors w (N=4, M=30). The federation factor represents the weights of each agent's own training parameters during parameter updates. If w is too small, the agent will lose its own parameters; therefore, we mainly study... In contrast, the case of w=0.1 was also studied. Figure 13 It is a line chart. Figure 14 The diagram is a box plot. It's easy to see that when w = 1, i.e., without federated updates, the system performs the worst and fails to converge after 5000 iterations. Furthermore, when w = 0.25, the gradient bias is minimized.

[0098] Consider a counter-terrorism intelligence reconnaissance scenario. Special operations units penetrate deep into the battlefield environment to reconnoiter enemy positions and gather intelligence. A dynamic data acquisition system transmits the situation back to the command center in real time via text, voice, images, and video. This places high demands on the timeliness and freshness of the data from the dynamic data acquisition system. Traditional metrics such as latency are insufficient. Furthermore, the limited computing power of mobile terminals and potential bandwidth constraints during data transmission can lead to significant data processing latency and transmission interruptions, potentially necessitating edge servers for auxiliary computation. The offloading strategy proposed in this chapter optimizes the timeliness of the dynamic data acquisition system, significantly improving its effectiveness.

[0099] The aforementioned method for optimizing the timeliness of battlefield dynamic data acquisition optimizes the timeliness of the dynamic data acquisition system by employing an Actor-Critic network model with a fusion attention mechanism deployed on mobile terminals and edge servers, and by performing set calculations of bandwidth allocation ratios to improve the timeliness of dynamic data acquisition.

[0100] Figure 15 This is a schematic block diagram of a battlefield dynamic data acquisition timeliness optimization device 300 provided in an embodiment of the present invention. Figure 15 As shown, corresponding to the above-described method for optimizing the timeliness of battlefield dynamic data acquisition, the present invention also provides a battlefield dynamic data acquisition timeliness optimization device 300. This battlefield dynamic data acquisition timeliness optimization device 300 includes a unit for executing the above-described battlefield dynamic data acquisition timeliness optimization method, and the device can be configured in a server. Specifically, please refer to... Figure 15The battlefield dynamic data acquisition timeliness optimization device 300 includes a signal acquisition unit 301, a conversion unit 302, a calculation unit 303, and a distribution unit 304.

[0101] The signal acquisition unit 301 is used to acquire dynamic environmental data acquisition signals; the conversion unit 302 is used to convert the average information age of mobile terminal data into a mathematical problem; the calculation unit 303 is used to input the mathematical problem into the Actor-Critic network model with fused attention mechanism to perform bandwidth allocation ratio set calculation to obtain the calculation result; the allocation unit 304 is used to allocate edge servers using the calculation result to perform the offloading calculation of the environmental data.

[0102] In one embodiment, the conversion unit 302 is used to model the optimization problem of the average information age of mobile terminal data as a Markov process to obtain a mathematical problem.

[0103] It should be noted that those skilled in the art can clearly understand that the specific implementation process of the above-mentioned battlefield dynamic data acquisition timeliness optimization device 300 and each unit can be referred to the corresponding description in the foregoing method embodiments. For the sake of convenience and brevity, it will not be repeated here.

[0104] The aforementioned battlefield dynamic data acquisition timeliness optimization device 300 can be implemented as a computer program, which can, for example... Figure 16 It runs on the computer device shown.

[0105] Please see Figure 16 , Figure 16 This is a schematic block diagram of a computer device provided in an embodiment of this application. The computer device 500 can be a server, wherein the server can be a standalone server or a server cluster composed of multiple servers.

[0106] See Figure 16 The computer device 500 includes a processor 502, a memory, and a network interface 505 connected via a system bus 501. The memory may include a non-volatile storage medium 503 and internal memory 504.

[0107] The non-volatile storage medium 503 may store an operating system 5031 and a computer program 5032. The computer program 5032 includes program instructions that, when executed, cause the processor 502 to perform a method for optimizing the timeliness of battlefield dynamic data acquisition.

[0108] The processor 502 provides computing and control capabilities to support the operation of the entire computer device 500.

[0109] The internal memory 504 provides an environment for the operation of the computer program 5032 in the non-volatile storage medium 503. When the computer program 5032 is executed by the processor 502, the processor 502 can execute a method for optimizing the timeliness of battlefield dynamic data acquisition.

[0110] This network interface 505 is used for network communication with other devices. Those skilled in the art will understand that... Figure 16 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device 500 to which the present application is applied. The specific computer device 500 may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0111] The processor 502 is used to run a computer program 5032 stored in the memory to perform the following steps:

[0112] The system acquires dynamic environmental data acquisition signals; converts the average information age of mobile terminal data into a mathematical problem; inputs the mathematical problem into an Actor-Critic network model with a fusion attention mechanism to perform bandwidth allocation ratio set calculations to obtain calculation results; and uses the calculation results to allocate edge servers to perform offloading calculations for the environmental data.

[0113] Among them, the Actor-Critic network model that incorporates the attention mechanism is deployed on mobile terminals and edge servers.

[0114] During the training of the Actor-Critic network model with an attention mechanism, after each Ef round of learning, all agents share and update their respective model parameters; the Actor and Critic networks of each mobile terminal agent retain parameters with weight w, and update them in combination with the parameters of other agents.

[0115] A federated learning algorithm is used to train an Actor-Critic network model with an integrated attention mechanism. The process of the agent's interaction with the environment is used as an experience cache set and as training samples to train the Actor-Critic network model with an integrated attention mechanism.

[0116] The Actor-Critic network model with an attention mechanism is trained by combining the set of bandwidth allocation ratios output by the Actor-Critic network model with the data sampled from the experience cache set after the attention mechanism is applied.

[0117] In one embodiment, when implementing the step of converting the average information age of mobile terminal data into a mathematical problem, the processor 502 specifically implements the following steps:

[0118] The optimization problem of the average information age of mobile terminal data is modeled as a Markov process to obtain the mathematical problem.

[0119] It should be understood that in the embodiments of this application, the processor 502 may be a central processing unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor.

[0120] It will be understood by those skilled in the art that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program includes program instructions and can be stored in a storage medium, which is a computer-readable storage medium. The program instructions are executed by at least one processor in the computer system to implement the process steps of the embodiments of the above methods.

[0121] Therefore, the present invention also provides a storage medium. This storage medium can be a computer-readable storage medium. The storage medium stores a computer program, wherein when executed by a processor, the computer program causes the processor to perform the following steps:

[0122] The system acquires dynamic environmental data acquisition signals; converts the average information age of mobile terminal data into a mathematical problem; inputs the mathematical problem into an Actor-Critic network model with an attention mechanism to perform bandwidth allocation ratio set calculation to obtain the calculation result; and uses the calculation result to allocate edge servers to perform offloading calculation of the environmental data.

[0123] Among them, the Actor-Critic network model that incorporates the attention mechanism is deployed on mobile terminals and edge servers.

[0124] During the training of the Actor-Critic network model with an attention mechanism, after each Ef round of learning, all agents share and update their respective model parameters; the Actor and Critic networks of each mobile terminal agent retain parameters with weight w, and update them in combination with the parameters of other agents.

[0125] A federated learning algorithm is used to train an Actor-Critic network model with an integrated attention mechanism. The process of the agent's interaction with the environment is used as an experience cache set and as training samples to train the Actor-Critic network model with an integrated attention mechanism.

[0126] The Actor-Critic network model with an attention mechanism is trained by combining the set of bandwidth allocation ratios output by the Actor-Critic network model with the data sampled from the experience cache set after the attention mechanism is applied.

[0127] In one embodiment, when the processor executes the computer program to implement the step of converting the average information age of the mobile terminal data into a mathematical problem, it specifically implements the following steps:

[0128] The optimization problem of the average information age of mobile terminal data is modeled as a Markov process to obtain the mathematical problem.

[0129] The storage medium can be any computer-readable storage medium capable of storing program code, such as a USB flash drive, portable hard drive, read-only memory (ROM), magnetic disk, or optical disk.

[0130] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0131] In the several embodiments provided by this invention, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For example, the division of each unit is merely a logical functional division, and there may be other division methods in actual implementation. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed.

[0132] The steps in the method of this invention can be adjusted, merged, or reduced in order according to actual needs. The units in the device of this invention can be merged, divided, or reduced according to actual needs. Furthermore, the functional units in the various embodiments of this invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0133] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, a terminal, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention.

[0134] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and these modifications or substitutions should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for optimizing the timeliness of battlefield dynamic data acquisition, characterized in that, include: Dynamically acquire environmental data and signals; The average information age of mobile terminal data is transformed into a mathematical problem; where information age is defined as the difference between two time slices, including the difference between the data generation timestamp and the current time slice. The mathematical problem is input into an Actor-Critic network model that incorporates an attention mechanism to calculate the bandwidth allocation ratio set, thus obtaining the calculation result. Specifically, the Actor-Critic network model is deployed on all agents. First, the basic elements of the Actor-Critic network model are: State: The state S of the mobile terminal agent includes the location of data collection, the location and available resources of the edge server, the data information to be unloaded, the state of its two queues, the allocated bandwidth and channel state, which can be represented as s = [l, I, B, w]; where l represents the location information of the mobile terminal and the edge server, I represents the data information to be unloaded, B represents the state of its two queues, w represents the allocated bandwidth, and the state of the edge server agent includes the state of all mobile terminals; Actions: including preprocessing strategies and unloading strategies, can be represented as: a n (t)=[cac n (t),off n [(t)]; Simultaneously, the edge server allocates bandwidth to each mobile terminal. The actions of the edge server are the set of bandwidth allocation ratios, which can be represented as: ;in, This represents the bandwidth allocation rate for mobile terminals and edge servers, where m is the edge server and n is the mobile terminal. Reward: The opposite of the value is used as the agent's reward, expressed as: The value function, which takes into account information from other agents, is expressed as: ； in, Represents a neural network. Represents an integration function; The encoded value representing the sum of the weights of other agents, and the contribution of other agents, are calculated using the following formula: ； ； in, It is obtained by substituting the actions and states of other agents into the ensemble function for encoding. Similarly, substituting the actions and states of agent n can yield the encoding. Normalization and After similarity is calculated, weights are generated through a softmax layer. During this process, Will Convert to "key" Will Convert to "query"; first convert... By performing a linear transformation on a common matrix Y and then substituting it into a nonlinear function L, the encoded value can be obtained. ; The calculation results are used to allocate edge servers for offloading calculations of the environmental data; The process of converting the average information age of mobile terminal data into a mathematical problem includes: The optimization problem of the average information age of mobile terminal data is modeled as a Markov process that minimizes the average information age of all mobile terminal data to obtain the mathematical problem. Specifically, AoI is introduced to analyze the timeliness of data in the battlefield dynamic data acquisition system. The optimization problem with the objective of minimizing the average AoI of all mobile terminals is modeled as a Markov process. For the solution of the optimization problem, a deep reinforcement learning multi-agent cooperative algorithm with an attention mechanism is adopted. Through the continuous interaction between the agent and the environment and the training of the agent using a federated method, computational offloading and resource allocation are performed.

2. A battlefield dynamic data acquisition timeliness optimization device, which, when operating, executes the battlefield dynamic data acquisition timeliness optimization method as described in claim 1, characterized in that, include: The signal acquisition unit is used to acquire dynamically collected environmental data signals. The transformation unit is used to convert the average information age of mobile terminal data into a mathematical problem; where information age is defined as the difference between two time slices, including the difference between the data generation timestamp and the current time slice; specifically, the optimization problem of the average information age of mobile terminal data is modeled as a Markov process that minimizes the average information age of all mobile terminal data to obtain the mathematical problem. The computing unit is used to input the mathematical problem into the Actor-Critic network model with an attention mechanism to perform bandwidth allocation ratio calculation to obtain the calculation result; The allocation unit is used to allocate edge servers using the calculation results to perform the offloading calculation of the environmental data.

3. A computer device, characterized in that, The computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the method as described in claim 1.

4. A storage medium, characterized in that, The storage medium stores a computer program that, when executed by a processor, can implement the method as described in claim 1.

Images