Unmanned aerial vehicle network slice resource optimization method based on hybrid quantum neural network

Abstract

The invention discloses an unmanned aerial vehicle network slice resource optimization method based on a hybrid quantum neural network, which solves the optimization bottleneck problem under complex constraints and high-dimensional space in the prior art, reduces the service time delay and improves the resource utilization rate. The method comprises the following steps: constructing an unmanned aerial vehicle network system model, establishing an optimization problem based on the unmanned aerial vehicle network system model by taking minimization of service delay as a target, and decoupling the optimization problem into a communication delay quantile minimization sub-problem and a processing delay minimization sub-problem; sequentially solving the communication delay quantile minimization sub-problem and the processing delay minimization sub-problem by adopting a multi-layer intelligent agent architecture to obtain an association indication variable set, a power distribution variable set and a function segmentation decision variable set; and realizing joint optimization of unmanned aerial vehicle network slice resources according to the association indication variable set, the power distribution variable set and the function segmentation decision variable set.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of artificial intelligence and multi-agent technology, and particularly relates to a method for optimizing UAV network slice resources based on a hybrid quantum neural network. BACKGROUND

[0002] With the wide application of unmanned aerial vehicle (UAV) networks in high mobility, flexible deployment, and low-cost communication, its complementary ability to ground base stations gradually emerges. In addition, the introduction of network slicing technology further enhances the flexibility of UAV networks, meeting the diverse needs of users through customized virtual logical subnets, thereby improving business processing capacity. However, the computing and communication resources of a single UAV are limited, making it difficult to independently complete the user-related functions (URFs) of users, and function splitting (FS) among multiple UAVs is needed to improve resource utilization. However, the problem of UAV network slicing collaborative resource optimization is highly challenging because it involves mixed integer variables and lacks a closed expression. To solve the problem of multi-UAV collaborative resource optimization, traditional methods such as Bernstein-type inequality (BTI) method, step-by-step iteration method based on conditional relaxation, and Lyapunov optimization often only solve part of the problem (channel uncertainty or resource optimization), which increases the computational complexity and algorithm delay. In complex scenarios involving multiple users and multiple network slices, multi-agent deep reinforcement learning (MADRL) has gradually become a key technology for optimizing the collective behavior of multiple elements, showing great potential for improving UAV network slicing resource management strategies. This method approximates the mapping relationship between the input and the optimal solution in traditional optimization algorithms through multiple different deep neural networks (DNNs), thereby bypassing the cumbersome iterative solution process and more efficiently handling non-convex optimization problems. Although some classic MADRL algorithms have been proposed, there are still shortcomings. For example, during the training process, once the action space is too large or the computational complexity is too high, gradient explosion may occur, making it difficult to converge to the optimal solution. Quantum neural networks (QNNs) can precisely compensate for the shortcomings of current MADRL algorithms due to their strong parallel computing capabilities and feature extraction capabilities. However, pure QNNs are limited by current immature quantum hardware and software devices. Therefore, some research focuses on the combination of quantum learning and machine learning, such as hybrid quantum-classical neural networks (HQCNNs) that fuse classical deep learning with QNNs, and quantum-empowered deep reinforcement learning (QeDRL) that balances action exploration and utilization through quantum parallelism.However, to our knowledge, no research has yet integrated the classic DNN and hybrid QNN (HQNN) structures into MADRL. Such integration is expected to perfectly combine MADRL's mature technology for processing multiple elements with the advanced computing power of QNN.

[0003] The existing technology is a multi-agent dual-deep Q-network method based on classical deep neural networks. This scheme optimizes the allocation of network slice resources for UAVs through a multi-agent reinforcement learning framework and uses a dual-deep Q-network to avoid overestimating action values. However, it suffers from gradient explosion and convergence difficulties when dealing with high-dimensional state spaces. Summary of the Invention

[0004] This invention provides a UAV network slicing resource optimization method based on a hybrid quantum neural network, which solves the optimization bottleneck problem of existing technologies under complex constraints and high-dimensional spaces. Through hierarchical decision-making and action compression, the algorithm is able to process large-scale networks, reducing service latency and improving resource utilization.

[0005] This invention provides a method for optimizing UAV network slicing resources based on a hybrid quantum neural network, the method comprising: A drone network system model is constructed, which includes: multiple users, multiple physical drones, and multiple network slice configurations; wherein each physical drone is virtualized into multiple virtual drones, and each virtual drone is used to serve a type of network slice; Based on the aforementioned UAV network system model, an optimization problem is established with the goal of minimizing service latency, and the optimization problem is decoupled into a subproblem of minimizing communication latency quantiles and a subproblem of minimizing processing latency. A multi-layer intelligent agent architecture is used to sequentially solve the communication delay quantile minimization subproblem and the processing delay minimization subproblem to obtain a set of associated indicator variables, a set of power allocation variables, and a set of function partitioning decision variables; wherein, each of the multi-layer intelligent agents uses a hybrid quantum duel dual-deep Q network to realize its decision-making function; Based on the set of associated indicator variables, the set of power allocation variables, and the set of function segmentation decision variables, the joint optimization of UAV network slice resources is achieved.

[0006] One or more technical solutions provided in this invention have at least the following technical effects or advantages: This invention constructs a drone network system model, which includes multiple users, multiple physical drones, and multiple network slice configurations. Each physical drone is virtualized into multiple virtual drones, and each virtual drone serves a specific type of network slice. This step achieves fine-grained resource management and isolation through virtualization technology, enabling a single physical platform to simultaneously and efficiently support differentiated business needs, significantly improving resource utilization and service diversity. Based on the drone network system model, an optimization problem is established with the goal of minimizing service latency, and this optimization problem is decoupled into a communication latency quantile minimization subproblem and a processing latency minimization subproblem. This step precisely targets the core requirement of low latency and rationally decomposes the complex joint problem through a "divide and conquer" strategy, greatly reducing the difficulty of solving the problem. In particular, optimizing the communication latency quantile rather than the mean effectively improves the "long tail" phenomenon of latency distribution, ensuring fairness in experience for all users and system reliability. A multi-layered intelligent agent architecture is employed to sequentially solve the communication delay quantile minimization subproblem and the processing delay minimization subproblem, resulting in a set of association indicator variables, a set of power allocation variables, and a set of function partitioning decision variables. Each of the multi-layered intelligent agents utilizes a hybrid quantum duel dual-deep Q-network to implement its decision-making function. This approach enhances the system's scalability through a distributed intelligent agent architecture and introduces the advanced algorithm of the hybrid quantum duel dual-deep Q-network. Its duel mechanism accurately assesses state value, dual-Q learning avoids overestimation, and quantum computing potential facilitates processing high-dimensional spaces, thereby efficiently and stably achieving joint intelligent decision-making for association, power, and computational offloading. Based on the set of association indicator variables, the set of power allocation variables, and the set of function partitioning decision variables, joint optimization of UAV network slicing resources is achieved. Finally, by executing the above optimization strategy, the system can dynamically and intelligently allocate wireless and computing resources, thereby minimizing end-to-end service latency, achieving deep collaboration between communication and computing resources, and improving overall network efficiency and automation. Attached Figure Description

[0007] Figure 1 A flowchart illustrating the steps of a UAV network slicing resource optimization method based on a hybrid quantum neural network, as provided in this embodiment of the invention. Figure 2 This is a schematic diagram of the overall model architecture of HQMAD3QN provided in an embodiment of the present invention; Figure 3 This is a schematic diagram of the specific structure of D3QN based on HQNN provided in an embodiment of the present invention. Detailed Implementation

[0008] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0009] This invention provides a method for optimizing UAV network slicing resources based on a hybrid quantum neural network. See [link to relevant documentation]. Figure 1 The method includes the following steps S101 to S104.

[0010] S101, Construct a drone network system model, which includes: multiple users, multiple physical drones, and multiple network slice configurations; wherein each physical drone is virtualized into multiple virtual drones, and each virtual drone is used to serve a type of network slice; For example, the present invention considers a two-layer network architecture: a ground layer and a spatial layer. The ground layer includes... One user and one data center; the airspace layer includes M The system comprises 10 drones, each equipped with dedicated communication and computing capabilities. A three-tiered airspace subnet architecture based on geographical distance was constructed. (1) Subnet 1: d ≤1000m, directly associated with users, supporting high real-time functions; (2) Subnet 2: 1000m d ≤2000m, directly connected to subnet 1, handles low to medium real-time tasks and acts as a relay for subnet 3; (3) Subnet 3: 2000m< d For distances ≤3000m, non-real-time batch processing tasks are executed via subnet 2 relay.

[0011] Inter-subnet communication follows hierarchical constraints: Subnet 1 and subnet 2 can communicate point-to-point, while subnet 3 can only interact via relay from subnet 2. The total system bandwidth is set to... B Hz, divided into W There are 1 sub-channels, each with a bandwidth of 1. B / W Hz, using orthogonal frequency division multiple access technology to avoid interference within the cell, drones m The total transmit power is denoted as P m .

[0012] The system supports two types of network slices (S=2): high priority ( Delay-sensitive slices (s=1) and low-priority slices ( The network uses a latency-tolerant slice (s=2). Utilizing network slicing technology, each physical UAV is virtualized into two virtual UAVs (vUAVs), each vUAV serving a specific slice type. The two vUAVs of the same physical UAV have the same location (still denoted as vUAVm in slice s). Network resources support real-time sharing between slices. In time slot t, the number of users requesting slice s is K. s (t), whose set K s (t)={1,...,K} s {(t)} is a subset of the total user set K(t). The priority relation is... .

[0013] User k (∀k∈K) in slice s within time slot t s Data arrivals (t) A k，s (t) Follow the rate of Lambda s A Poisson model with speeds of (kbps) is used, and the network satisfies independent and identically distributed characteristics across different time slots. Meanwhile, in subnetwork 1... s Class slice intra-association vUAV m ( ) users k Its channel coefficient They also satisfy the independent and identically distributed property across different time slots, as modeled below: ; in, Indicates the first In the slice class, the first Individuals and users The distance; This represents the channel gain at a reference distance of 1 meter. Indicates the path loss index; Indicates the line-of-sight component; Represents the Rice decay factor; This represents the non-line-of-sight component, which follows a complex Gaussian distribution.

[0014] S102, based on the UAV network system model, establishes an optimization problem with the goal of minimizing service latency, and decouples the optimization problem into a subproblem of minimizing communication latency quantiles and a subproblem of minimizing processing latency; Specifically, in step S102, based on the UAV network system model, an optimization is established with the goal of minimizing service latency, and the optimization problem is decoupled into a subproblem of minimizing communication latency quantiles and a subproblem of minimizing processing latency, including: The constraints of the optimization problem include: delay interruption probability constraints, processing capacity constraints, power constraints, and function chain break constraints. Here, the optimization problem is expressed as: ; Total service delay for users As the optimization objective, the optimization objective is decomposed into communication delay components. and processing delayed components Two relatively independent parts; Based on processing delay components The characteristic of being independent of channel state information decouples the optimization problem into one concerning the communication delay component. The communication delay quantile minimization subproblem and the handling of delay components The processing delay minimization subproblem is defined as follows: the constraints of the communication delay quantile minimization subproblem are: power constraint and even delay interruption probability constraint; the constraints of the processing delay minimization subproblem are: processing capacity constraint and function chain break constraint.

[0015] Here, the subproblem of minimizing the communication delay quantiles is expressed as: ; The subproblem of minimizing delay is represented as: , in, This represents the sum of time slots; Indicates the first Slices in time slots The maximum delay quantile; Represents the set of power allocation variables; Indicates the set of associated indicator variables; Indicates the first slice-like; Indicates the first One time slot; Indicates the maximum processing delay; This represents the set of decision variables for functional segmentation; This indicates the communication delay quantile.

[0016] For example, the channel error model is defined as follows:

[0017] in, Indicates user k In the time slot t The estimated Channel State Information (CSI) value in vUAV. m The location is known; and It is a Gaussian channel error, which follows a complex Gaussian distribution of independent and identically distributed (i.i.d.) CN (0, Please note the subscripts of all variables in this invention.k, s, m All represent association with vUAV m The s Users on slices k .

[0018] Therefore, data rate The following is given: ; in, ; Among them, the definition For time slots t In-system virtual drone m To the s User on slice k The set of their transmission powers is denoted as . . N 0 represents the power spectral density of additive white Gaussian noise. Indicates user k Inter-cell interference, among which and These represent virtual drones that are the sources of interference. \{m} The channel coefficients and transmit power are then used to define the associated indicator set. Its specific expression is as follows: ; Therefore, with time slots t The first s Users associated with vUAV on class slices k The downlink data rate is expressed as: .

[0019] For the two types of slices, the core objective of this invention is to ensure user service quality by reducing service latency. Considering the differences in slice characteristics, different latency thresholds are set for them. User service latency consists of two components: (1) communication latency: the latency generated by transmitting user data; (2) processing latency: the latency of completing user baseband processing functions between different UAV subnetworks.

[0020] slice s user k Data arrives Follows a Poisson distribution, time slot t The communication process is modeled as an M / D / 1 queuing model. (User) k The communication delay is denoted as Composed of transmission delay and queuing delay, it can be expressed as: ; The first item is the user. k The first term represents the transmission delay, while the second term represents the queuing delay.

[0021] Define the decision variables for function segmentation: This represents the decision of FS, i.e., if the first... s User in slice k URF Decide on vUAV m If it runs on, then there is ;otherwise 0 URFs placed in subnet 1 must be fully executed on the same associated vUAV and must not be shared by other vUAVs.

[0022] The processing capacity of each drone m in subnetwork 1 is set to The overall processing capacity of subnetwork 2 is In this model, the processing capacity of subnetwork 3 is assumed to be infinite; meanwhile, the unit processing latency of URFs within each subnetwork is defined as follows: , and The unit transmission delay of the line-of-sight (LOS) links between sub-networks is respectively that of sub-networks 1 to 2. Sub-networks 2 to 3 .

[0023] Therefore, users k The processing delay is expressed as:

[0024] Therefore, in time slots t Time s Users in slices k The one-way service latency is: ; This invention aims to minimize long-term user service latency for two types of network slices: ; Due to channel error The existence of this condition requires simultaneously satisfying the delay interruption probability constraint. Therefore, the original delay minimization problem is transformed into a constrained maximum delay quantile minimization problem, i.e. ; definition , Indicates user k The delay quantile, and the specific optimization problem is as follows: As shown: ; ; ; ; ; ; ; ; ; ; ; ; ; Among them, decision variables P and Y, These involve a transmission power set, a binary virtual unmanned aerial vehicle (vUAV) association indicator set, and a binary function partitioning (FS) decision set, respectively. Specifically: C 1 represents user k The delay interruption probability constraint, where It is the set low probability of target interruption (e.g., 5%). C 2. This is designed to meet the service latency requirements requested by users. C 3. Ensure the drones in subnetwork 1 m The processing capacity does not exceed its upper limit. . C 4. Ensure that the maximum User Related Functions (URFs) processing capacity of subnet 2 is [value missing]. .

[0025] C 5 represents the user-related functions for each user. U i Only one virtual drone can be used at most. C 6 respectively stipulate , and The range of values ​​for . C 7. Ensure that the user-related functional chain is broken only once between subnetwork 1 and other subnetworks. That is, once the user... k User-related functions U i No virtual drones in subnet 1 m Execute above (i.e.) Therefore, all remaining upper-layer user-related functions cannot be executed on any virtual drone in subnetwork 1 (i.e., they must also satisfy...). Similarly, C8. Ensure that the user-related function chain breaks only once between subnetwork 2 and subnetwork 3. C 9 indicates that a drone's transmit power cannot exceed its maximum power. P m .

[0026] The above optimization problem Essentially, this is a mixed-integer non-convex problem involving three sets of decision variables and multiple constraints, belonging to the non-deterministic polynomial (NP) problem category. Due to its high computational complexity and the difficulty of convex approximation, traditional methods struggle to solve this problem. Furthermore, due to the large action space and high computational complexity, using existing multi-agent deep reinforcement learning (MADRL) algorithms in a "plug-and-play" manner is inefficient. Therefore, this invention decouples and reformulates the optimization problem to make it easier to solve. It can be seen that the problem... The communication and computational processing parts are independent of each other. Therefore, the proposed problem can be decoupled into two sub-problems. First, extract the sub-problems for the first... s The subproblem of minimizing the maximum communication delay quantile of slice-like structures is as follows: P1: ; ; ; ; In this problem, decision variables P and Y are optimized in subproblem P1. It is important to note the constraints... C 1. Processing delayed components This will not affect the result of the delay interruption probability constraint because it is independent of channel state information (CSI). Therefore, this invention achieves this by considering the total service delay. Remove irrelevant components , will constrain C The expression for 1 simplifies to C 1'. Furthermore, It is the first s The maximum communication delay quantile for slice-like data.

[0027] Next, in order to optimize the set of decision variables , and proposed measures targeting the s The subproblem of minimizing the maximum processing latency of slices is expressed as follows: P2: ; ; .

[0028] S103 employs a multi-layered intelligent agent architecture to sequentially solve the communication delay quantile minimization subproblem and the processing delay minimization subproblem, obtaining a set of associated indicator variables, a set of power allocation variables, and a set of function partitioning decision variables; among them, each multi-layered intelligent agent uses a hybrid quantum duel dual-deep Q-network to realize its decision-making function; Specifically, in step S103, the hybrid quantum duel dual-deep Q network includes: an evaluation network module and a target network module.

[0029] The evaluation network module receives state observation information from each agent, processes the information using a hybrid quantum neural network module, and outputs a Q-value function for the agent to select actions. The target network module calculates the target Q-value. Both the evaluation network module and the target network module include a hybrid quantum neural network module for quantum encoding and processing of the state observation information.

[0030] Specifically, in step S103, the agent's decision-making function is implemented through the following process: (1) Obtain the current state observation information detected by each agent; (2) Encode the state observation information into a quantum state using an amplitude coding method; (3) Through CNOT gate and The parameterized quantum circuit of the revolving door transforms the quantum state to obtain the transformed quantum state; (4) The transformed quantum state is measured using the Pauli Z matrix to obtain the classical measurement result; (5) Input the classical measurement results into the classical neural network layer to obtain the decision variable output of the agent.

[0031] Specifically, in step S103, a multi-layer intelligent agent architecture is used to solve the communication delay quantile minimization subproblem and the processing delay minimization subproblem in sequence to obtain the set of associated indicator variables, the set of power allocation variables and the set of function partitioning decision variables, including the following steps S1031 to S1033.

[0032] (1) Network operators construct a three-layer intelligent agent for each user; (2) Using the first-layer intelligent agent, determine the relationship between each user and the virtual drone, and output a set of association indicator variables. ; Here, the first-layer agent is used to determine the relationship between each user and the virtual drone, and outputs a set of relationship indicator variables. ,include: (2.1) Perform the following power association decisions for each user in sequence: (2.2) The first-layer agent receives the first state observation information of the user to be decided. Among them, the first state observation information This includes: the distance between the user currently making a decision and all available virtual drones. The number of remaining sub-channels for each virtual drone and the current channel state estimate ; (2.3) Utilizing the hybrid quantum duel dual-deep Q-network in the first-layer agent, the first-state observation information is analyzed. Process the data and output the associated actions between the user to be decided and the specific virtual drone. And based on the associated actions Update the remaining sub-channel count of the associated virtual drone; (2.4) Based on associated actions This generates a binary association indicator variable that identifies the relationship between the current user to be decided and the selected virtual drone. (2.5) After all user association decisions are completed, the association indicator variables corresponding to all users are combined to obtain the association indicator variable set. and associate the set of indicator variables It is passed to the second-layer agent as input for its decision-making.

[0033] (3) Utilizing a second-layer intelligent agent, based on the set of associated indicator variables Decision power allocation, output power allocation variable set ; Here, a second-layer agent is used, based on the set of associated indicator variables. Decision power allocation, output power allocation variable set ,include: (3.1) Perform the following power allocation decisions for each user in sequence: (3.2) The second-layer agent receives the second state observation information of the user to be decided. Second-state observation information This includes: the amount of data arriving for users currently awaiting decision-making. The current remaining power of the virtual drone belonging to the user currently awaiting decision. Current channel state estimate ; (3.3) Utilizing a hybrid quantum duel dual-deep Q-network in the second-layer agent to observe information from the second state. The process is performed to obtain discretized power level actions. ; (3.4) Discretize the power level operation Mapping is performed to obtain the power allocation variables for the current user to be decided. And based on the power allocation variables of the user currently making a decision. Update the remaining power of the virtual drone belonging to the user currently awaiting decision; (3.5) After all users have made their power allocation decisions, the power allocation variables of all users are summarized to obtain the power allocation variable set. and the power allocation variable set It is passed to the third-layer intelligent agent.

[0034] (4) Utilizing a third-layer intelligent agent, based on the set of associated indicator variables and power allocation variable set Decision function segmentation strategy, output function segmentation decision variable set .

[0035] Here, using a third-layer intelligent agent, a function segmentation strategy is decided based on the set of associated indicator variables and the set of power allocation variables, outputting a set of function segmentation decision variables, including: (4.1) Perform the following feature splitting decisions sequentially for each user or slice: (4.2) The third-layer agent receives the third state observation information of the user to be decided. Third-state observation information Includes: average user data arrivals for both types of slices 、 The first remaining processing capacity of the virtual drone in subnetwork 1 and the second remaining processing capacity of subnetwork 2; (4.3) The third state observation information is obtained through a hybrid quantum duel dual-deep Q-network in the third-layer agent. The process is performed to obtain the compressed and encoded functional segmentation action. Functional segmentation action This indicates the allocation scheme of the processing functions of the user currently awaiting decision-making in each level of the subnetwork; (4.4) Action segmentation based on function Decode and generate the function segmentation decision variables for the current user to be decided, and update the remaining processing capacity status of each level of the processing sub-network based on the function segmentation action and the function segmentation decision variables; (4.5) After all users have made their function partitioning decisions, the function partitioning decision variables for all users are summarized to obtain the function partitioning decision variable set. .

[0036] For example, this invention proposes a joint function partitioning (FS), user association, and UAV power allocation scheme to minimize user service latency. This scheme can be broken down into three steps: Step 1: Determine the user-virtual drone (vUAV) association set Y in drone subnetwork 1.

[0037] Step 2: Determine the transmission power allocation set P for the virtual UAV (vUAV) to provide services to its associated users. The first two steps are used together to solve the first subproblem P1 (minimizing the communication delay quantiles).

[0038] Step 3: For the two types of slicing requirements, perform functional segmentation (FS) and functional segmentation decision variable set between different UAV subnetworks. This step aims to solve the second subproblem P2 (minimizing processing delay).

[0039] This three-step iterative resource optimization scheme is executed sequentially, meaning that the next step decision is made based on the decision results of the previous step or the previous two steps, ultimately completing the solution to the original optimization problem P∗.

[0040] In this way, the original optimization problem is transformed into a form that is easier to solve using multi-agent deep reinforcement learning (MADRL) methods. Next, this invention will customize a multi-agent architecture (HQMAD3QN) algorithm specifically to address the aforementioned problem.

[0041] First, the overall model structure of HQMAD3QN is introduced. This invention innovatively improves the traditional Dual-Depth Q-Network (D3QN) structure by introducing Hybrid Quantum Neural Network (HQNN) units to enhance performance. Furthermore, the action space is reduced, and problem constraints are innovatively addressed by introducing residual variables. Finally, the specific algorithm flow is described, and complexity analysis is performed.

[0042] See Figure 2 , Figure 2 As shown in the left half, the HQMAD3QN model consists of an environment module and a three-layer intelligent algorithm, corresponding to the three decision steps of iterative resource optimization. The mobile network operator (MNO) constructs three agents for each user, each responsible for executing a single-step resource decision. Specifically, in the... w In layer (w∈{1,2,3}), system deployment K A user-level intelligent agent (the third layer is...) S Each slice-level agent is labeled as an agent. l w，j ( j e {k, s}These agents collaborate through a "centralized training-distributed execution" framework: training at each layer is centrally processed, while inference execution by each agent is distributed and independent. This design ensures complete collection of all user information while avoiding prolonged training time due to an increase in the number of users. To achieve dynamic optimization decisions for each time slot, the algorithm first undergoes centralized offline training in a ground data center, followed by online learning and inference on distributed UAV nodes. This mechanism ensures that the system can obtain the globally optimal solution while maintaining real-time responsiveness.

[0043] intelligent agent l w，j Detailed structure as follows Figure 2 The right half is shown. Since this optimization problem mainly consists of discrete decision variables (such as Y and...), The system is dominated by a dual deep duel Q network (D3QN) and, in order to ensure training accuracy in a high-dimensional state-action space, two additional modules, "evaluation D3QN" and "target D3QN", are introduced within each agent.

[0044] See Figure 3 All agent tasks satisfy the Markov property, and each task can be represented as a quadruple: state observation, action space, reward function, and next-time state observation. Specifically, residual variables are introduced to handle difficult-to-solve constraints, and the action space is reduced in dimensionality. Furthermore, a penalty mechanism is introduced into the reward function to avoid constraint violations.

[0045] Each agent l w，j Modeled as a Markov decision process, the task is determined by a quadruple. This indicates that, specifically, intelligent agents... l w，j First, observe the current state of the environment. Then, the optimal action is selected based on the action value function. After performing this action, you will receive a reward from the environment. Then proceed to the next state. .

[0046] Current state and next state: The three levels have different state spaces: (1) First layer (user-vUAV association layer): State observation includes the distance between the user and the virtual drone (vUAV). And the number of remaining sub-channels for the virtual drone when user k arrives. and estimated channel state The centralized training and distributed execution (CTDE) architecture adds index elements to each individual state space. k To distinguish the state of each agent: ; (2) Second layer (transmit power allocation layer): The state includes: the amount of data arriving for user k. When user k arrives, the remaining power of the virtual drone m... Estimated channel state .

[0047] ; (3) Third layer (functional segmentation layer): targeting the first s The status of slices includes: the average amount of user data arriving in both slice types. When the first s When a slice request arrives, the virtual drone in subnetwork 1 m Remaining user-related function processing capacity The remaining user-related function processing capacity of subnet 2 .

[0048] ; The state observations for the next time slot are respectively expressed as: .

[0049] Action: Reviewing the three-step iterative resource optimization, the action space of each layer is defined as follows: ; ; ; Some action spaces containing binary variables, such as the first-level action space. and the third layer of action The motion space contains a large number of redundant actions, therefore, the motion space is compressed. The first layer of compression involves associating the user's motion space with other actions. Size from the original (representing all possible combinations of associations) compressed to .

[0050] Right now ,in .For example, Indicates user k In the time slot t Associated with virtual drone 1. Third layer compression: Remove function segmentation motion space. Violation of constraints C 5,C 7 and C The invalid action is 8.

[0051] Ultimately Size from 8 M Compressed to 10 effective actions, i.e. .

[0052] Encoding rule: the third digit in a three-digit number i The number indicates where it is placed in the subnet. i The number of user-related functions (URFs) processed in the process.

[0053] For example: =012 indicates: URF U 1. Processed in subnet 2, URF U 2 and URF U 3. Processed within subnet 3. Second layer discretization: for the power allocation action space. , will continuous power value Discretize into E Each power allocation level is represented as .

[0054] To better learn the original optimization problem Tricky constraints C 3, C 4, C 6 and C 9. Introduce residual variables into the state space, such as the number of remaining sub-channels. Remaining power Subnet 1's remaining URF processing capacity Subnet 2 remaining URF processing capacity .like Figure 2 As shown, the agent modifies the remaining variable values ​​in the next user's state set based on the action decision of the previous user. For example, for a time slot... t For user 1, the initial number of remaining sub-channels for each associated vUAV is: and (Assume there are 2 vUAVs). Assume an intelligent agent. l 1，1 Output Action (Associated with vUAV1), then the agent l 1，2 The remaining number of sub-channels in user 2's state will be set to (vUAV1 has used 1 sub-channel) and (vUAV2 is unused and remains unchanged), and subsequent users will follow the same pattern. This takes into account slice priority. The decision-making process proceeds in the following order: first, for all slices in the first category... K One user makes the decision; then all users in the second slice... K Two users make decisions. The order of users' decisions within the same slice is randomized and is shuffled again in each time slot.

[0055] Reward: To ensure that constraints are not violated, this invention introduces a penalty mechanism into the reward function. When the remaining variables in the state space become negative, it indicates that the constraint has been violated, and a portion of the reward needs to be deducted as a penalty. Considering that the first two layers are used together to solve P1, this invention sets the same reward function for them, namely: ; Then, the third layer is applied to solve for P2, so the reward function is defined as: ; in, I is the weight coefficient of the penalty function, and I(·) is the indicator function (1 when the condition is met, 0 otherwise).

[0056] In addition, a data-augmented quantile estimation module is used to handle the interruption probability constraint. C 1', thus obtaining during the training phase and Maximum delay quantile This data augmentation method uses Monte Carlo sampling to compute... The value of .

[0057] To improve the stability of nonlinear Q-network training, this invention simultaneously constructs an evaluation Q-network and a target Q-network in each D3QN. Specifically, a hybrid quantum neural network (HQNN) is introduced to redesign the structures of the evaluation Q-network and the target Q-network, such as... Figure 3 As shown below. It will be described in detail below.

[0058] (1) Introduction to HQNN: Quantum Neural Networks (QNNs) are mainly composed of qubits and quantum gates. Qubits are the basic units of quantum information; they can take any value between 0 and 1 because they are represented as a combination of two bases. Since they are represented as two ground states in Hilbert space... and The combination , This represents Dirac's bracket notation; a and b are those that satisfy... The probability amplitude of a qubit. Therefore, unlike classical bits, qubits can exist in superposition states simultaneously. This greatly increases the correlation between individual qubits, enabling QNNs to efficiently represent and process complex data. A quantum gate is a unitary operator that changes the state of a qubit. Commonly used quantum gates include rotation gates, Pauli-X / Y / Z gates, and controlled-NOT (CNOT) gates. Among them, rotation gates act on a single qubit, performing a rotation operation on its state. Rotation gates include... 、 and The gates represent the X, Y, and Z axes along the Bloch sphere at a given angle. Theta Parameterized quantum gates for rotating qubits. For example, The gate is represented as: ; Pauli gates are used to flip the amplitude or phase of a single qubit. For example, a Pauli Z-gate performs a rotation of π around the Z-axis of a Bloch sphere, i.e.: ; In addition, there are quantum gates that operate on multiple qubits, such as the controlled-NOT gate. This is a two-qubit gate that flips the target qubit (the second qubit) according to the state of the control qubit (the first qubit) to achieve quantum entanglement between these qubits. The CNOT gate can be represented as: ; Convolutional neural networks (CNNs) possess exceptional feature extraction capabilities. They can capture spatial and temporal dependencies in input data and apply convolutional filters to local regions of the input to generate useful features. Furthermore, CNNs require fewer parameters compared to fully connected networks.

[0059] The proposed Hybrid Quantum Neural Network (HQNN) combines quantum neural networks and convolutional neural networks, transforming local data input into variable quantum circuits. VQC (Variable Quantum Circuits) can extract useful features using specially designed parameterized quantum circuits through quantum filters. In this way, HQNN leverages the powerful quantum computing capabilities of controllable quantum entanglement and quantum interference to achieve performance improvements. Figure 3 As shown, the design of VQC consists of three stages: (i) state encoding; (ii) parameterized quantum circuit (PQC); and (iii) measurement.

[0060] (i) State Encoding: Based on this, classical input information can be encoded into corresponding qubit states and processed in quantum circuits. Specifically, a 2×2 input channel vector is used, for example... s 1 ，s 2 ，s n ，s n-1 ,in s 1 ，s 2. Extract the original state-space data. s n ，s n-1 Extract the mirror state space data (i.e., the reverse of the original order). This invention uses an amplitude encoding method to encode classical data into quantum states, requiring only 2 qubits (Q=log2(4)): ,in The computational basis for two qubits is represented by |00>, |01>, |10>, and |11>.

[0061] (ii) Parameterized Quantum Circuit (PQC): The encoded quantum state is transformed through a C-layer parameterized circuit. This circuit uses CNOT gates and A rotating door generates entangled qubits, in which rotation angle and These are the trainable parameters to be optimized. Therefore, this module contains a total of 2C trainable parameters.

[0062] (iii) Measurement: In this stage, the eigenstates are obtained using the Pauli Z-matrix, and the expected values ​​of multiple measurements are collected under the Z-basis. This operation projects the quantum state to one of the |0> or |1> states. The output dimension of the measurement layer remains Q, and the measurement results are input into subsequent neural network layers, such as... Figure 3 As shown.

[0063] (2) The specific structure of D3QN: The complete structure of D3QN is as follows: Figure 3 As shown, the input state set is first processed and flattened by the HQNN module, and then fed into the FC layer. After passing through the ReLU activation function, the output data is further fed into the D3QN's duel module. Through data processing of the value function and advantage function in the duel module, the Q value corresponding to each action is output. Finally, the action corresponding to the maximum Q value is the output action selected by D3QN.

[0064] No. w The specific training process of D3QN in the layer is summarized as Algorithm 1. To solve the overall optimization problem P... ∗The three-layer algorithm needs to be executed sequentially and collaboratively. Specifically, the HQMAD3QN algorithm uses the first layer of D3QNs to handle user association decisions Y, the second layer of D3QNs to be specifically responsible for user transmit power allocation P, and the third layer of D3QNs to decide the FS strategy between the two types of slices. The entire system is in T Iterative training is performed within each time slot, and the training results are fed back until convergence is achieved. The overall process of this collaborative training is summarized as Algorithm 2.

[0065] Here, we present Algorithm 1: the centralized training process for the w-th layer of D3QNs.

[0066]

[0067]

[0068] S104. Based on the set of associated indicator variables, the set of power allocation variables, and the set of functional partitioning decision variables, the joint optimization of UAV network slice resources is achieved.

[0069] The key point of this invention lies in the innovative proposal of a multi-agent deep reinforcement learning framework (HQMAD3QN) based on a hybrid quantum neural network, which optimizes UAV network slicing resources through a three-layer agent collaborative decision-making mechanism. This scheme first decouples the complex mixed-integer non-convex optimization problem into two sub-problems: minimizing communication delay quantiles and minimizing processing delay, and introduces delay interruption probability constraints to improve robustness. Secondly, it designs a three-layer agent architecture including user association, power allocation, and function partitioning, adopting a centralized training-distributed execution mode. Simultaneously, it innovatively combines quantum neural networks with traditional CNNs, leveraging the advantages of quantum parallel computing to solve the high-dimensional state space processing problem. Finally, through optimizations such as action space compression and dynamic constraint mechanisms, the practicality and efficiency of the algorithm are significantly improved.

[0070] Compared with existing technologies, the advantages of this invention lie in its first-ever embedding of quantum neural networks into a multi-agent reinforcement learning framework. This solves the optimization bottleneck of traditional methods under complex constraints and high-dimensional spaces. Through hierarchical decision-making and action compression, the algorithm is equipped to handle large-scale networks, reducing service latency and improving resource utilization. Compared with traditional optimization methods, the classical MADRL algorithm, and existing quantum hybrid schemes, this invention demonstrates significant advantages in computational efficiency and scalability.

[0071] The various embodiments described in this specification are presented in a progressive manner. Similar or identical parts between embodiments can be referred to interchangeably. Each embodiment focuses on its differences from other embodiments. All or part of this invention can be used in numerous general-purpose or special-purpose computer system environments or configurations. Examples include: personal computers, server computers, handheld or portable devices, tablet devices, mobile communication terminals, multiprocessor systems, microprocessor-based systems, programmable electronic devices, network PCs, minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices, etc.

[0072] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the present invention.

Claims

1. A method for optimizing UAV network slicing resources based on a hybrid quantum neural network, characterized in that, include: A drone network system model is constructed, which includes: multiple users, multiple physical drones, and multiple network slice configurations; wherein each physical drone is virtualized into multiple virtual drones, and each virtual drone is used to serve a type of network slice; Based on the aforementioned UAV network system model, an optimization problem is established with the goal of minimizing service latency, and the optimization problem is decoupled into a subproblem of minimizing communication latency quantiles and a subproblem of minimizing processing latency. A multi-layer intelligent agent architecture is used to sequentially solve the communication delay quantile minimization subproblem and the processing delay minimization subproblem to obtain a set of associated indicator variables, a set of power allocation variables, and a set of function partitioning decision variables; wherein, each of the multi-layer intelligent agents uses a hybrid quantum duel dual-deep Q network to realize its decision-making function; Based on the set of associated indicator variables, the set of power allocation variables, and the set of function segmentation decision variables, the joint optimization of UAV network slice resources is achieved.

2. The UAV network slicing resource optimization method based on hybrid quantum neural networks according to claim 1, characterized in that, Based on the UAV network system model, an optimization is established with the goal of minimizing service latency, and the optimization problem is decoupled into a communication latency quantile minimization subproblem and a processing latency minimization subproblem, including: The constraints of the optimization problem include: delay interruption probability constraint, processing capacity constraint, power constraint, and function chain break constraint. Total service delay for users As an optimization objective, the optimization objective is decomposed into communication delay components. and processing delayed components Two relatively independent parts; Based on the processing delay component The characteristic of being independent of channel state information decouples the optimization problem into one concerning the communication delay component. The communication delay quantile minimization subproblem and the processing delay components The processing delay minimization subproblem; wherein, the constraints of the communication delay quantile minimization subproblem are: the power constraint and the even delay interruption probability constraint; the constraints of the processing delay minimization subproblem are: the processing capacity constraint and the function chain break constraint.

3. The UAV network slicing resource optimization method based on hybrid quantum neural networks according to claim 2, characterized in that, The optimization problem is expressed as: ； The communication delay quantile minimization subproblem is expressed as: ； The subproblem of minimizing processing delay is expressed as: ； in, This represents the sum of time slots; Indicates the first Slices in time slots The maximum delay quantile; Represents the set of power allocation variables; Indicates the set of associated indicator variables; Indicates the first slice-like; Indicates the first One time slot; Indicates the maximum processing delay; This represents the set of decision variables for functional segmentation; This indicates the communication delay quantile.

4. The UAV network slicing resource optimization method based on hybrid quantum neural networks according to claim 1, characterized in that, The hybrid quantum duel dual-deep Q network includes: an evaluation network module and a target network module; The evaluation network module is used to receive state observation information from each agent, process the information using a hybrid quantum neural network module, and output a Q-value function for the agent to select actions. The target network module is used to calculate the target Q value; Both the evaluation network module and the target network module include a hybrid quantum neural network module for quantum encoding and processing of state observation information.

5. The UAV network slicing resource optimization method based on hybrid quantum neural networks according to claim 4, characterized in that, The decision-making function of the intelligent agent is achieved through the following process: Obtain the current state observation information detected by each agent; The state observation information is encoded into a quantum state using an amplitude encoding method; Through including CNOT gate and The parameterized quantum circuit of the revolving door transforms the quantum state to obtain the transformed quantum state; The transformed quantum state is measured using the Pauli Z-matrix to obtain classical measurement results; The classical measurement results are input into a classical neural network layer to obtain the decision variables output of the agent.

6. The UAV network slicing resource optimization method based on hybrid quantum neural networks according to claim 1, characterized in that, The method employs a multi-layered intelligent agent architecture to sequentially solve the communication delay quantile minimization subproblem and the processing delay minimization subproblem, obtaining a set of associated indicator variables, a set of power allocation variables, and a set of function partitioning decision variables, including: Network operators build a three-layer intelligent agent for each user; Using the first-layer intelligent agent, the association relationship between each user and the virtual drone is determined, and a set of association indicator variables is output. ; Using a second-layer agent, based on the set of associated indicator variables Decision power allocation, output power allocation variable set ; Using a third-layer intelligent agent, based on the set of associated indicator variables and power allocation variable set Decision function segmentation strategy, output function segmentation decision variable set .

7. The UAV network slicing resource optimization method based on hybrid quantum neural networks according to claim 6, characterized in that, The first-layer intelligent agent determines the association between each user and the virtual drone, and outputs a set of association indicator variables. ,include: Perform the following power association decisions for each user in sequence: The first-layer agent receives the first state observation information of the user currently making a decision. Among them, the first state observation information This includes: the distance between the user currently making a decision and all available virtual drones. The number of remaining sub-channels for each virtual drone and the current channel state estimate ; Using a hybrid quantum duel dual-deep Q-network in the first-layer agent, the first state observation information is... Process the data and output the associated actions between the user to be decided and the specific virtual drone. And according to the associated action Update the remaining sub-channel count of the associated virtual drone; According to the associated action A binary association indicator variable is generated, which identifies the association between the current user to be decided and the selected virtual drone; After all user association decisions are completed, the association indicator variables corresponding to all users are combined to obtain the association indicator variable set. and the set of associated indicator variables It is passed to the second-layer agent as input for its decision-making.

8. The UAV network slicing resource optimization method based on hybrid quantum neural networks according to claim 6, characterized in that, The second-layer agent is used based on the set of associated indicator variables. Decision power allocation, output power allocation variable set ,include: Perform the following power allocation decisions for each user in sequence: The second-layer agent receives the second state observation information of the user currently making a decision. The second state observation information This includes: the amount of data arriving for users currently awaiting decision-making. The current remaining power of the virtual drone belonging to the user currently awaiting decision. Current channel state estimate ; The second state observation information is obtained through a hybrid quantum duel dual-deep Q-network in the second-layer agent. The process is performed to obtain discretized power level actions. ; The discretized power level is activated. Mapping is performed to obtain the power allocation variables for the current user to be decided. And based on the power allocation variables of the user currently making a decision. Update the remaining power of the virtual drone belonging to the user currently awaiting decision; After all users' power allocation decisions are completed, the power allocation variables of all users are summarized to obtain the power allocation variable set. and the power allocation variable set It is passed to the third-layer intelligent agent.

9. The UAV network slicing resource optimization method based on hybrid quantum neural networks according to claim 6, characterized in that, The third-layer intelligent agent is used based on the set of associated indicator variables. and power allocation variable set Decision function segmentation strategy, output function segmentation decision variable set ,include: Perform the following feature splitting decisions sequentially for each user or slice: The third-layer agent receives the third-state observation information of the user currently making a decision. The third state observation information Includes: average user data arrivals for both types of slices 、 The first remaining processing capacity of the virtual drone in subnetwork 1 and the second remaining processing capacity of subnetwork 2; The third state observation information is obtained through a hybrid quantum duel dual-depth Q-network in the third-layer agent. The process is performed to obtain the compressed and encoded functional segmentation action. The functional segmentation action This indicates the allocation scheme of the processing functions of the user currently awaiting decision-making in each level of the subnetwork; Based on the functional segmentation action Decode and generate the function segmentation decision variables for the current user to be decided, and update the remaining processing capacity status of each level of the processing sub-network according to the function segmentation action and the function segmentation decision variables; After all users have made their function partitioning decisions, the function partitioning decision variables for all users are summarized to obtain the function partitioning decision variable set. .

Images