Semantic communication framework and optimization method for unmanned aerial vehicle video target detection task

Abstract

The application discloses a semantic communication framework and performance optimization method for unmanned aerial vehicle (UAV) video target detection tasks, which is used for solving the resource bottleneck problem in the UAV communication scene. The application introduces the concept of semantic communication in the video transmission scene of the UAV, and designs a UAV video semantic communication framework different from the existing general framework, which is more suitable for the UAV with limited energy. In order to measure the performance of the studied framework, a semantic metric QoDO is proposed. Taking this index as the optimization goal, the flight trajectory planning and power allocation problems of the UAV are modeled as an optimization problem, and a hierarchical reinforcement learning algorithm is adopted to obtain the optimal strategy. Under the condition that the communication resources and the energy that can be carried by the UAV are limited, the semantic communication framework and the optimization method provided by the application can enable the receiving end to obtain sufficient semantic information, so as to promote the UAV to effectively perform the target detection task.

Description

Technical Field

[0001] This invention belongs to the field of semantic communication technology, specifically designing a semantic communication framework and optimization method for UAV video target detection tasks. Background Technology

[0002] In the era of intelligent interconnection, the exponential growth of heterogeneous data has placed enormous pressure on the capacity of wireless networks. Drones, as intelligent device terminals, appear in various wireless network communication scenarios, capable of transmitting large amounts of video data to further perform various intelligent tasks, such as rescue operations and security monitoring. These intelligent tasks have stringent requirements for low latency and high precision, posing significant challenges to bandwidth-constrained ultra-dense network scenarios. In traditional communication methods, one solution is to explore higher frequencies to utilize the large available spectrum resources. Such solutions periodically face unavoidable bottlenecks, manifested in hardware costs, the complexity of wireless communication, and energy efficiency limitations. When resources across all physical layer dimensions approach saturation, a new paradigm shift is required. For communication aimed at intelligent tasks, the key is to ensure that the receiver receives as much data as possible that is relevant to the intelligent task. Semantic communication is a communication architecture well-suited to such agent-based communication; it focuses on extracting the meaning of data and transmitting only data valuable to the final task, perfectly aligning with goal-oriented communication. Unlike traditional communication methods, it does not focus on the precise transmission of every bit, greatly reducing the redundant transmission of semantically irrelevant data, making it an ideal solution to the challenges mentioned above.

[0003] Many researchers have designed various semantic communication frameworks, implemented the design and application of semantic codecs for text, speech, and image data, and performed semantic encoding and compression before information transmission, which greatly reduces the amount of data transmitted in the channel.

[0004] However, these encoding and decoding processes require extensive computation. For drones, given their limited storage, energy consumption, and computing power, performing such large-scale computations to obtain video semantic features is impractical. Furthermore, due to the difficulty in uniformly defining video semantics, research on the design and optimization of video semantic communication frameworks is scarce. There is a lack of research on how to leverage semantic communication concepts to address the bottleneck of limited communication resources in drone video communication for intelligent tasks. Summary of the Invention

[0005] This invention aims to solve the bottleneck problem of limited communication resources for UAVs when performing intelligent tasks by utilizing the concept of semantic communication, and provides a semantic communication framework and optimization method for UAV video target detection tasks.

[0006] To achieve the above-mentioned objectives, the present invention adopts the following technical solution.

[0007] A semantic communication framework and optimization method for UAV video target detection tasks includes the following steps:

[0008] (1) A video semantic communication framework for a UAV video communication scenario is established, the video semantic communication framework including a UAV, a ground control center and an edge server; the UAV uploads the collected video information and location information to the edge server; the edge server obtains the current channel status information with the UAV and decides whether the UAV should hover; the ground control center, the UAV and the edge server perform semantic communication.

[0009] (2) Define a video semantic metric for target detection tasks, and construct an optimization objective based on the video semantic metric to model the trajectory planning and power allocation problem of UAVs when performing tasks as an optimization problem;

[0010] (3) Design a hierarchical reinforcement learning algorithm to resolve the coupling relationship between semantic performance requirements and UAV trajectory decision-making, and complete the decision-making of UAV trajectory planning and power allocation to obtain the optimal strategy;

[0011] In this step, a hierarchical dual DDPG algorithm is used to make decisions on the power allocation and trajectory planning of the UAV. The hierarchical dual DDPG algorithm includes a trajectory planning layer and a power allocation layer, and the two make independent decisions according to the principle of DDPG, while interacting with the environmental state.

[0012] In the semantic communication framework described in step (1), the specific communication process is as follows:

[0013] (11) When the UAV arrives above the target area, it preprocesses the video it captures, initiates a video transmission request, and uploads the acquired video information and the current shooting distance information between the UAV and the target area to the edge server. At the same time, the edge server obtains the current channel status information between the UAV and the UAV.

[0014] (12) Based on the existing information, the edge server sends control commands to the drone to decide whether the drone should hover for video recording or continue flying; if the drone needs to hover, the edge server should make a decision on the transmission power allocated to the drone for video transmission in the current time slot.

[0015] (13) If the drone is instructed to hover, the video is encoded and compressed according to the allocated power and transmitted to the edge server via a wireless channel at the current transmittable rate.

[0016] (14) The edge server decodes the video, performs target detection, and sends the extracted semantic information to the ground control center. The edge server also stores the video for analysis of other intelligent tasks.

[0017] In step (2), the meaning of the video semantic performance metric is the number of targets that can be correctly identified on the edge server after the video has been captured and transmitted by the UAV in each time slot t. The corresponding formula is as follows:

[0018]

[0019] in, M represents the accuracy of object detection in the video. i [t] represents the actual number of targets existing in target region i, T capturing [t] represents the acquisition accuracy determined by the distance from the target area when the drone acquires video;

[0020] The mathematical expression is as follows:

[0021]

[0022] Where α1, α2, α3, β1, and β2 represent accuracy model parameters, which are determined by the specific communication scenario and are obtained by training and fitting through the dataset of the corresponding scenario. R[t] represents the video coding bit rate that can be achieved in time slot t.

[0023] R capturing The mathematical expression for [t] is as follows:

[0024]

[0025] Where r represents the maximum field of view of the camera on the drone. This represents the two-dimensional distance between the UAV and the current target area i in time slot t.

[0026] Furthermore, the modeling process for the optimization problem of UAV path planning and power allocation is as follows:

[0027] In the defined video semantic performance metric, the value of R[t] is limited by the current channel conditions between the drone and the edge server, so its maximum value is expressed as follows:

[0028]

[0029] Where B represents the channel bandwidth, P[t] represents the transmission power, h[t] represents the channel gain, and σ 2 Represents channel noise;

[0030] And R capturing The value of [t] is determined by the distance between the drone in time slot t and the edge server, and further by the hovering position of the drone;

[0031] Based on the conditions of video semantic performance metrics, an optimization model is established between the decisions on the drone's flight trajectory and transmission power and the quality of the final semantic information. Its mathematical expression is as follows:

[0032]

[0033] In addition, the model's constraints include setting flight dynamics constraints for the UAV and numerical constraints on the total transmission power.

[0034] In the hierarchical dual DDPG algorithm described in step (3), the decision process for the power allocation layer is expressed as follows:

[0035]

[0036] For the trajectory planning layer, the decision-making process is represented as follows:

[0037]

[0038] The trajectory planning layer interacts with the environment at high frequency and determines the UAV's flight maneuvers in each time slot t;

[0039] When making a hovering decision, the trajectory planning layer signals the power allocation layer to begin making a power allocation decision. After the video transmission ends, the power allocation layer receives a semantic performance reward and provides feedback to the trajectory planning layer. The trajectory planning layer acts as a goal setter, determining the capture position before the power allocation layer makes its decision, while the power allocation layer acts as a commentator, evaluating the trajectory planning layer's actions based on the received reward.

[0040] Furthermore, in this method, the UAV is used for video acquisition and encoding preprocessing, and then sends a video transmission request to the edge server; the edge server is used for UAV trajectory and power decision-making, including video semantic extraction, and the edge server transmits the extracted video semantic information to the ground control center, where the ground control center performs semantic recognition and processing.

[0041] Beneficial effects: Compared with existing technologies, the semantic communication framework and optimization method provided by this invention can enable the receiving end to obtain sufficient semantic information under limited communication resources, thus enabling UAVs to effectively perform target detection tasks. The substantial features and significant advancements of this method include:

[0042] 1) Design of a video semantic communication framework. Based on the characteristics of the specific intelligent tasks in UAV video communication scenarios (video target detection in this invention) and the characteristics of the UAV itself (such as limited energy), this invention designs a semantic communication framework suitable for UAV video communication scenarios, breaking through the bottleneck problem of communication resource limitations.

[0043] 2) This invention proposes a video semantic metric for target detection tasks. This metric can measure the amount of effective semantic information that can be extracted from the transmitted data, that is, it can measure the performance of the proposed semantic communication framework. For the first time, the metric considers the impact of the UAV's hovering position on the accuracy of video target detection from the perspective of video acquisition.

[0044] 3) For the modeling optimization problem, this invention uses the proposed indicators as guidance to construct optimization objectives, models the trajectory planning and power allocation problem of UAVs when performing tasks as an optimization problem, and adopts two-level decision-making, making independent decisions according to the principle of DDPG, while also interacting with the environmental state.

[0045] 4) In the hierarchical reinforcement learning algorithm, by decoupling the two decision-making processes of UAV trajectory planning and power allocation into independent solution processes, the environment can be explored more completely, the optimal strategy can be obtained, the performance of the semantic communication system can be optimized, and a good intelligent task completion effect can be achieved. Attached Figure Description

[0046] Figure 1 This is a block diagram illustrating the design principle of the method described in this invention;

[0047] Figure 2 A diagram illustrating the video semantic communication framework for UAV target detection tasks;

[0048] Figure 3 A system model diagram for UAV path planning and power allocation driven by video semantics;

[0049] Figure 4 This is a framework diagram of the H2DDPG algorithm in the embodiment;

[0050] Figure 5 The distribution of edge servers, detection areas, and UAV flight trajectories is obtained after solving the optimization problem;

[0051] Figure 6 A comparison chart showing the reward values ​​obtained from training using the H2DDPG algorithm and the ordinary DDPG algorithm, respectively;

[0052] Figure 7 A performance comparison of the method of this invention and the method for solving optimization problems based on QoE and QoS indicators under different power constraints;

[0053] Figure 8 This paper compares the performance of the method of this invention and the method for solving optimization problems based on QoE and QoS indicators under different numbers of target regions. Detailed Implementation

[0054] To illustrate the technical solutions disclosed in this invention in detail, the invention will be further described below with reference to the accompanying drawings and specific examples.

[0055] This invention addresses the bottleneck of limited communication resources for unmanned aerial vehicles (UAVs) performing intelligent tasks by utilizing semantic communication methods. Considering the limitations of UAVs' computing power and storage resources, a suitable video semantic communication framework for target detection tasks is designed for UAVs, such as... Figure 1 and Figure 2 As shown. This invention also proposes a new video semantic performance metric, which can be used to measure the performance of the proposed semantic communication framework. Based on the proposed communication framework, the semantic-driven trajectory design and power allocation problem is studied. Since the decision frequencies of the two decision tasks are different, an algorithm based on the hierarchical idea, H2DDPG, is proposed to solve this problem. The obtained strategy is used to make flight action decisions and allocate power for the UAV, thus completing the target detection task. The specific implementation process is as follows:

[0056] Step 1: Design a video semantic communication framework

[0057] like Figures 1-2 As shown, this invention incorporates an edge server into the communication framework, meaning the scenario includes three devices: a drone, a ground control center, and an edge server. Since the drone's computing power and energy are limited, complex computational tasks are offloaded to the edge server. The specific communication process is as follows:

[0058] (11) When the UAV arrives above the target area, it preprocesses the video it captures, initiates a video transmission request, and uploads the acquired video information and the current shooting distance from the target area to the edge server. At the same time, the edge server acquires the current channel status information (CSI) between itself and the UAV.

[0059] (12) Based on the available information, the edge server sends control commands to the drone to determine whether the drone should hover for video recording or continue flying. If the drone needs to hover, the edge server should decide on the transmission power allocated to the drone for video transmission in the current time slot.

[0060] (13) If the drone is instructed to hover, the video is encoded and compressed according to the allocated power and transmitted to the edge server via a wireless channel at the current transmittable rate.

[0061] (14) The edge server decodes the video, performs object detection, and sends the extracted semantic information (such as object location and category) to the control center. The server can also store the video for analysis of other intelligent tasks.

[0062] The semantic communication framework proposed in this step adds the concept of semantics to the communication process of UAVs for video target detection tasks. Based on this framework, the communication process can be optimized from a semantic level, which is more in line with the communication vision of the Internet of Everything under 6G communication.

[0063] Step 2: Propose video semantic metrics for object detection tasks

[0064] The video semantic performance metric is defined as the number of targets that can be correctly identified on the edge server in each time slot t after the video has been captured and transmitted by the drone. Its formula can be expressed as follows:

[0065]

[0066] in, M represents the accuracy of object detection in the video. i [t] represents the actual number of targets existing in target region i, R capturing [t] represents the acquisition accuracy determined by the distance from the target area when the drone acquires video.

[0067] The mathematical expression is as follows:

[0068]

[0069] Here, α1, α2, α3, β1, and β2 represent accuracy model parameters, which are determined by the specific communication scenario and need to be obtained through training and fitting using the dataset of the corresponding scenario. R[t] represents the video coding bit rate that can be achieved in time slot t.

[0070] R capturing The mathematical expression for [t] is as follows:

[0071]

[0072] Where r represents the maximum field of view of the camera on the drone. This represents the two-dimensional distance between the UAV and the current target area i in time slot t.

[0073] The semantic performance metrics take into account the impact of the drone's hovering position on the quality of the captured video, making them more practically meaningful.

[0074] Step 3: Modeling and Optimizing the Problem

[0075] The modeling process for the optimization problem of UAV path planning and power allocation is as follows:

[0076] Figure 3 This paper illustrates a scenario where a drone performs a target detection task, including the target area to be detected, the drone, and an edge server. The drone's flight maneuver decisions and the power decisions for transmitting video to the edge server affect the quality of the video received by the server, which in turn affects the quality of the semantic information that the receiving end can obtain. Therefore, this invention models this process as an optimization problem with semantic performance as the optimization objective.

[0077] In the proposed video semantic performance metric, the value of R[t] is limited by the current channel conditions between the drone and the edge server, so its maximum value is expressed as follows:

[0078]

[0079] Where B represents the channel bandwidth, P[t] represents the transmission power, h[t] represents the channel gain, and σ 2 This represents channel noise.

[0080] And R capturing The value [t] is determined by the distance between the drone in time slot t and the edge server, and further by the drone's hovering position.

[0081] Based on the above conditions, an optimization model can be established between the decision on the UAV's flight trajectory and transmission power and the quality of the final semantic information, and its mathematical expression is as follows:

[0082]

[0083] In addition, the model's constraints include flight dynamics constraints for the UAV and numerical constraints on the total transmitted power. θ, v, hover, and p represent the UAV's flight angle, speed, hovering status, and power, respectively. Indicates the service period. This indicates the quality of the semantic information acquired by the receiver of the semantic communication framework at time t.

[0084] Step 4: Design a hierarchical reinforcement learning algorithm

[0085] The hierarchical reinforcement learning algorithm is designed as follows:

[0086] Solving the optimization problem is challenging due to the coupling between semantic performance requirements and UAV trajectory decision-making. Specifically, UAVs can improve channel quality by moving closer to edge servers, while simultaneously achieving higher acquisition accuracy by moving closer to the detection area. These conflicting optimization directions require a reasonable decision to maximize video semantic performance. Furthermore, the decision frequencies for UAV power allocation and trajectory planning differ. To address this issue, this invention proposes a hierarchical dual DDPG algorithm, such as... Figure 4 As shown, the algorithm divides the decision-making process into two layers: a trajectory designing optimization (TD-DDPG) layer and a power allocating optimization (PA-DDPG) layer. The two layers make independent decisions according to the principles of DDPG, while also interacting with the environmental state.

[0087] For the PA-DDPG layer, the decision process is represented as follows:

[0088]

[0089] For the TD-DDPG layer, the decision-making process is represented as follows:

[0090]

[0091] The TD-DDPG interacts with the environment at high frequency and determines the UAV's flight maneuvers in each time slot t. When making hovering decisions, the TD-DDPG signals the PA-DDPG to initiate power allocation decisions. After video transmission ends, the PA-DDPG receives a semantic performance reward and provides feedback to the TD-DDPG. The TD-DDPG acts as a target setter, determining the capture position before the PA-DDPG makes its decisions, while the PA-DDPG acts as a commentator, evaluating the TD-DDPG's actions based on the received rewards.

[0092] This step effectively applies the hierarchical dual DDPG algorithm to solving the optimization problem in this scenario, enabling more effective exploration of the environment and obtaining the optimal decision solution.

[0093] Depend on Figure 5 It can be seen that the drones are able to successfully fly over each target area to complete video acquisition and transmission tasks. Figure 6 It can be seen that the H2DDPG algorithm proposed in this invention can converge after about 800 training rounds and obtain a higher reward value than the ordinary DDPG algorithm. Figure 7 and Figure 8It can be seen that, under different environmental conditions, the semantic-oriented performance optimization method proposed in this invention achieves better performance than the QoE and QoS-oriented methods. In summary, the semantic communication framework for UAV video target detection tasks proposed in this invention achieves better semantic performance in UAV video semantic communication scenarios for intelligent tasks, demonstrating the feasibility and effectiveness of semantic communication under the new vision of 6G, and providing a new solution for performance optimization of such intelligent tasks.

Claims

1. A semantic communication framework and optimization method for unmanned aerial vehicle (UAV) video target detection task, characterized in that: Includes the following steps: (1) A video semantic communication framework for a UAV video communication scenario is established, the video semantic communication framework includes a UAV, a ground control center and an edge server; the UAV uploads the collected video information and location information to the edge server; the edge server obtains the current channel status information with the UAV and decides whether the UAV should hover; the ground control center, the UAV and the edge server perform semantic communication. (2) Define a video semantic metric for target detection tasks, and construct an optimization target based on the video semantic metric to model the trajectory planning and power allocation problem of UAVs when performing tasks as an optimization problem; The meaning of the video semantic performance metric is the number of targets that can be correctly identified at the edge server end after the video is collected and transmitted by the unmanned aerial vehicle in each time slot The corresponding formula is as follows: in, This represents the accuracy of object detection in the video. Represents the target area The actual number of targets in the middle, The accuracy of the video capture is determined by the distance from the drone to the target area when it captures the video. The mathematical expression is as follows: in, The parameters representing the accuracy model are determined by the specific communication scenario and are obtained through training and fitting using the dataset corresponding to that scenario. Represents time slot The achievable video encoding bit rate; The mathematical expression is as follows: in, This represents the maximum field of view of the camera mounted on the drone. This represents the distance of the drone from the current target area in time slot t. Two-dimensional distance; The modeling process for the optimization problem of UAV path planning and power allocation is as follows: In the defined video semantic performance metrics This value is limited by the current channel conditions between the drone and the edge server, so its maximum value is expressed as follows: in, Represents channel bandwidth. Represents transmission power. Represents channel gain. Represents channel noise. This represents the maximum value of the video semantic performance metric. and This value was obtained by the drone during the time slot. The distance to the edge server is determined, and further, by the hovering position of the drone; Based on the conditions of video semantic performance metrics, an optimization model is established between the decisions on the drone's flight trajectory and transmission power and the quality of the final semantic information. Its mathematical expression is as follows: In addition, the model's constraints include setting the UAV to satisfy flight dynamics constraints and imposing numerical constraints on the total transmission power. (3) Design a hierarchical reinforcement learning algorithm to solve the coupling relationship between semantic performance requirements and UAV trajectory decision-making, and complete the decision-making of UAV trajectory planning and power allocation to obtain the optimal strategy; In this step, a hierarchical dual DDPG algorithm is used to make decisions on the power allocation and trajectory planning of the UAV. The hierarchical dual DDPG algorithm includes a trajectory planning layer and a power allocation layer, and the two make independent decisions according to the principle of DDPG, while interacting with the environmental state.

2. The semantic communication framework and optimization method for UAV video target detection tasks according to claim 1, characterized in that: In the semantic communication framework described in step (1), the specific communication process is as follows: (11) When the UAV arrives above the target area, it preprocesses the video it captures, initiates a video transmission request, and uploads the acquired video information and the current shooting distance information between the UAV and the target area to the edge server. At the same time, the edge server obtains the current channel status information between itself and the UAV. (12) Based on the existing information, the edge server sends control commands to the drone to decide whether the drone should hover for video recording or continue flying; if the drone needs to hover, the edge server should make a decision on the transmission power allocated to the drone for video transmission in the current time slot. (13) If the UAV is instructed to hover, the video is encoded and compressed according to the allocated power and transmitted to the edge server via a wireless channel at the current transmittable rate. (14) The edge server decodes the video, performs target detection, and sends the extracted semantic information to the ground control center. The edge server also stores the video for analysis of other intelligent tasks.

3. The semantic communication framework and optimization method for UAV video target detection tasks according to claim 1, characterized in that: In the hierarchical dual DDPG algorithm described in step (3), the decision process for the power allocation layer is expressed as follows: For the trajectory planning layer, the decision-making process is represented as follows: The trajectory planning layer interacts with the environment frequently and determines the drone's trajectory in each time slot. Flight maneuvers; When making a hovering decision, the trajectory planning layer signals the power allocation layer to begin making a power allocation decision. After the video transmission ends, the power allocation layer receives a semantic performance reward and provides feedback to the trajectory planning layer. The trajectory planning layer acts as a goal setter, determining the capture position before the power allocation layer makes its decision, while the power allocation layer acts as a commentator, evaluating the trajectory planning layer's actions based on the received reward.

4. The semantic communication framework and optimization method for UAV video target detection tasks according to claim 1, characterized in that: In this method, the UAV is used for video acquisition and encoding preprocessing, and then sends a video transmission request to the edge server; the edge server is used for UAV trajectory and power decision-making, including video semantic extraction, and the edge server transmits the extracted video semantic information to the ground control center, where the ground control center uses and stores the semantic information.

Images