A DNN-based split learning method in UAV synaesthesia and computing fusion network

Abstract

The invention discloses a split learning method based on DNN in a UAV interaural computing fusion network, which belongs to the field of mobile edge computing and wireless communication. The UAV interaural computing fusion network comprises multiple UAVs, a central controller equipped with an edge server and ground terminal users; the UAV-assisted interaural computing fusion network environment comprises a communication user set and a computing server set; the DNN model is adaptively split and collaboratively executed on the UAV and the edge server, the allocation of computing resources and communication resources is optimized, and the performance of the ISCC system is improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of mobile edge computing and wireless communication, and in particular relates to a splitting learning method based on DNN in a UAV synaesthesia computing fusion network. Background Art

[0002] Due to their high mobility and flexibility, drones have attracted much attention in applications such as autonomous driving, disaster relief, emergency communications, and aerial 3D reconstruction, and have become an important part of future wireless systems. As an emerging technology, Integrated Sensing, Computation and Communication (ISCC) networks greatly improve spectrum efficiency and reduce costs by using the same signal and wireless infrastructure to support perception, communication, and computing services.

[0003] In the existing technology, the distributed computing capabilities of multiple drones can effectively alleviate the problem of limited computing resources of a single drone. However, the system still faces the challenge of managing computing resources and communication resources, especially in a dynamic wireless network environment. How to adaptively allocate computing and communication resources to meet the requirements of timeliness and energy efficiency has become an urgent problem to be solved.

[0004] Split Learning achieves collaborative processing by splitting the Deep Neural Network (DNN) between different mobile devices and edge servers to fully utilize the computing resources at the edge of the network. It also allows data to be initially processed locally and then only transmits intermediate activation values ​​or gradients to the server, reducing the security risk of raw data transmission and protecting user privacy. This approach helps balance communication costs and computing efficiency, especially in environments with high dynamics of wireless networks.

[0005] Therefore, the present invention proposes a synaesthesia-computing fusion network based on drones, which improves the overall performance of the system while meeting different task delay requirements and dynamic network conditions through adaptive segmentation DNN model calculation. Summary of the invention

[0006] The technical problem to be solved by the present invention is to provide a UAV-based DNN splitting and learning method for the synaesthesia-computing fusion network in response to the shortcomings of the background technology. By adaptively splitting the DNN model and collaboratively executing it on the UAV and edge server, the allocation of computing resources and communication resources is optimized, the computing efficiency and energy efficiency of the system are improved, and the needs of delay-sensitive tasks are met.

[0007] The present invention adopts the following technical solutions to solve the above technical problems:

[0008] A DNN split learning method based on UAV-based synaesthesia-computing fusion network, which includes multiple UAVs, a central controller equipped with an edge server, and ground terminal users;

[0009] Among them, each drone is equipped with a computing module and a communication module for data interaction with the edge server;

[0010] Edge servers, which have computing power and storage resources and are responsible for centrally processing complex computing tasks;

[0011] Ground end users, which generate computing tasks and offload them to drones and edge servers via wireless links;

[0012] The UAV-assisted synaesthesia-computing fusion network environment includes a communication user set and a computing server set:

[0013] Among them, the communication user set includes users who need communication services and are perceived by the target, including smart phones, mobile tablets, and unmanned vehicle terminal devices;

[0014] The computing server set includes drones equipped with DNN processing capabilities and edge servers; each drone can interact with multiple communication users, and each communication user can only connect to one drone; communication and perception data can be processed by the drone or offloaded to the edge server;

[0015] The specific steps include:

[0016] Step 1, initialization and perception: The UAV flies over the end user, the end user generates a computing task and requests a service, the UAV uses the perception function to select the perception target and collect environmental data; after the perception data collection is completed, the perception data is transmitted to the central controller via a wireless link; then, it enters the task allocation and scheduling stage;

[0017] Step 2, task allocation and scheduling: Assign tasks according to the optimization strategy algorithm, use the adaptive split learning method to determine the DNN split strategy, the computing tasks of each drone, the transmission tasks and the trajectory optimization plan; after the task allocation is completed, enter the DNN splitting and computing stage;

[0018] Step 3, DNN splitting and calculation: The drone performs the early layer calculation of the DNN, unloads the intermediate results to the central controller and further performs the subsequent layer calculation of the DNN to complete the task; after the calculation is completed, it enters the result return stage;

[0019] Step 4, result return: The central controller returns the calculation result to the end user through the wireless link and the drone; the result return is completed and the entire operation cycle ends;

[0020] In step 2, the optimization strategy algorithm includes drone trajectory optimization, adaptive adjustment of DNN splitting, and allocation of computing resources to optimize the offloading of multi-user tasks and the allocation of computing resources to ensure low latency and low energy consumption for task processing;

[0021] In step 2, the adaptive split learning method divides the deep neural network model into multiple parts, which are executed on different drones and edge servers respectively; the specific steps are as follows:

[0022] Model segmentation: Adaptively segment the DNN model into several sub-models based on the task’s computing requirements and network status; each sub-model can be executed independently on different computing nodes, i.e., drones or edge servers;

[0023] Task offloading: Based on the current network status and computing resources, the Actor-Critic learning algorithm is used to dynamically select the optimal communication and computing resource allocation and sub-model allocation to the appropriate computing node;

[0024] Collaborative computing: Each computing node collaboratively executes the sub-model and transmits the intermediate results through wireless links to achieve forward and backward propagation of the DNN model;

[0025] Result return: After the calculation is completed, the result is returned to the ground terminal user via a wireless link;

[0026] To achieve the above-mentioned adaptive split learning, the Actor-Critic learning algorithm is used for training, which specifically includes the following steps:

[0027] State representation: The system’s computing resources, communication resources, and task requirements are represented as a state vector, including the drone’s computing power, battery power, network bandwidth, and task size;

[0028] Actor network action selection: At each decision moment, the optimal strategy action is selected based on the current state, including model split points and resource allocation schemes;

[0029] Reward function: Define a reward function that takes into account the computational latency and energy consumption of the task to optimize the long-term performance of the system;

[0030] Critic network strategy update: Update the Actor network, optimize the strategy scheme, and improve system performance;

[0031] In step 2, split learning is used to train the DNN model; the drone and edge server cooperate to train the DNN model, which is divided into N layers, and the layer set is represented as

[0032] N={1,2,...,N}

[0033] , the training rounds are T times, and the round set is represented by T = {1,2,...,T}; modeling the model computing delay, energy consumption and total cost; specifically including the following steps;

[0034] Step 2.1, forward propagation calculation:

[0035] Calculation on the drone side:

[0036] in, is the forward propagation calculation delay of the UAV side executing the i-th layer task of the DNN network at the t-th iteration, L i is the number of floating-point operations required for layer i calculation, C0 is the number of floating-point operations processed in one CPU cycle, is the CPU calculation frequency of the drone end; the calculation energy consumption of the drone end is set to E MD (t);

[0037] Edge server computing:

[0038] in, is the forward propagation computation delay of the edge server executing the i-th layer task of the DNN network at the t-th iteration, L i The number of floating point operations required to calculate the i-th layer, The CPU calculation frequency allocated by the edge server to drone m;

[0039] Step 2.2, back propagation calculation:

[0040] Calculation on the drone side:

[0041] in, is the back propagation calculation delay of the UAV side executing the i-th layer task of the DNN network at the t-th iteration, L i,B The number of floating point operations required to update the gradient calculation for the i-th layer; the energy consumption of the drone end is set to E MB (t);

[0042] Edge server computing:

[0043] in, is the back propagation calculation delay of the edge server executing the i-th layer task of the DNN network at the t-th iteration, L i,B Represents the number of floating-point operations required to calculate the gradient of the i-th layer update;

[0044] Step 2.3, communication delay and energy consumption:

[0045] Uplink transmission delay from MD to ES:

[0046] in, is the transmission delay of the output data of the i-th layer task at the t-th iteration; I i is the output data volume of the i-th layer, in bits, R U (t) is the uplink transmission rate; Indicates the energy consumption of the drone during uplink transmission;

[0047] Downlink transmission delay from ES to MD:

[0048] in, is the transmission delay of the output data of the i-th layer task at the t-th iteration; I i is the output data volume of the i-th layer, R D (t) is the downlink transmission rate; Indicates the energy consumption of the drone during downlink transmission;

[0049] Step 2.4, cost calculation: Let x i (t)∈{0,1} is a binary DNN split indicator. If the i-th layer of DNN is calculated on the drone side, x i (t)=1, otherwise x i (t) = 0;

[0050] The computational cost is composed of the execution delay of all computational tasks and the computational energy consumption on the drone side:

[0051]

[0052] Among them, w is the weight factor between delay and energy consumption;

[0053] Communication cost: Communication cost includes the delay and energy consumption generated when data is transmitted between the drone and the edge server during forward propagation and backpropagation.

[0054]

[0055] Forward propagation communication cost:

[0056]

[0057] Back-propagation communication cost:

[0058]

[0059] Total system cost:

[0060] ETC all =ETC cmp (t)+ETC com (t)+ETCres (t)+ETC sc (t)

[0061] Among them, ETC res (t) represents the DNN re-segmentation cost, ETC sc (t) represents the perceived communication cost;

[0062] The model is trained using the Actor-Critic learning algorithm to adjust resource allocation to minimize the long-term average cost; the optimization algorithm is implemented as follows:

[0063] 1) Obtain sufficient environmental information: including the maximum computing frequency of the drone CPU, the maximum computing frequency of the edge server CPU, the maximum tolerable total delay of the computing task, the energy limit of the drone, and the current system status information;

[0064] 2) Initialize experience replay storage D and average reward Initialize the network with random weights; for each training round t, initialize the state s(t), and the Actor network selects an action a(t) according to the selection strategy; execute the selected action a(t), obtain the reward U, and observe the new state s(t+1), store (s(t), a(t), U, s(t+1)) in D, uniformly extract a small batch from the experience replay storage D for training, and update the target state-action value R; the Critic network performs gradient descent to minimize the loss function L, updates the Actor network weights, and regularly updates the network weights of the target network to stabilize the training process; if a non-exploratory action is selected, update the average reward Output action a(t) until the maximum number of training rounds is reached;

[0065] 3) Complete DNN segmentation and resource allocation, start calculation, and return the calculation results.

[0066] As a further preferred solution of the UAV-based synaesthesia fusion network DNN splitting learning method of the present invention, in step 1, the environmental data adopts a Cartesian coordinate system, all UAVs fly at the same altitude, and ground terminal users are randomly distributed; in each time slot, the terminal users are considered to be static.

[0067] Compared with the prior art, the present invention adopts the above technical solution and has the following technical effects:

[0068] 1. The present invention provides a DNN-based split learning method in a UAV interawareness computing fusion network, which optimizes the allocation of computing resources and communication resources by adaptively splitting the DNN model and collaboratively executing it on the UAV and edge server, thereby improving the performance of the ISCC system;

[0069] 2. The present invention introduces the latest split learning method and Actor-Critic learning algorithm to improve the computing efficiency and energy efficiency of the system, meet the needs of delay-sensitive tasks, make the method of the present invention more effective and superior, and have broad prospects in practical applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0070] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without paying any creative work.

[0071] Figure 1 It is an application scenario of the present invention, including an ISCC network environment based on drones, a computing environment of drones and edge servers, and a communication and computing environment of drones and user terminals;

[0072] Figure 2 It is a flow chart of a splitting learning method based on DNN in a UAV synaesthesia computing fusion network of the present invention;

[0073] Figure 3 It is an optimization strategy algorithm framework based on Actor-Critic learning in the present invention. DETAILED DESCRIPTION

[0074] The technical solution of the present invention is further described in detail below in conjunction with the accompanying drawings:

[0075] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention. The present invention is described in detail below according to the drawings and preferred embodiments, and the purpose and effect of the present invention will become clearer. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0076] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present invention.

[0077] The drone-based ISCC network described in the present invention includes multiple drones, a central controller equipped with an edge server, and ground terminal users. Each drone is equipped with a computing module and a communication module for data interaction with the edge server. The edge server has powerful computing power and storage resources, and is responsible for centralized processing of complex computing tasks. The ground terminal user generates computing tasks and offloads the tasks to the drone and the edge server through a wireless link. The drone-assisted ISCC environment includes a communication user set and a computing server set: the communication user set includes users who need communication services and are perceived by the target, including terminal devices such as smart phones, mobile tablets, and unmanned vehicles; the computing server set includes drones and edge servers equipped with DNN processing functions; at the same time, each drone can interact with multiple communication users, and each communication user can only be connected to one drone; communication and perception data can be processed by the drone or offloaded to the edge server.

[0078] The computing environment of ISCC involves drones and edge servers. In this environment, a split learning method is proposed to achieve adaptive DNN segmentation and resource allocation, thereby optimizing the user's service quality and system efficiency. The communication and computing environment between drones and users includes: the uplink where users offload data processing tasks through drones, the task offloading transmission link between drones and edge servers, and the downlink where drones or edge servers return processing results to users. In this computing environment, user tasks can be processed on the associated drones or further offloaded to edge servers.

[0079] The invention provides a DNN splitting and learning method based on unmanned aerial vehicle (UAV) and is applied to Figure 1 In the environment of the UAV interawareness integrated communication, perception and computing system shown in FIG, the UAV interawareness integrated communication and perception system environment includes at least K users, an edge server and M UAVs flying above the users. The set of UAVs is represented by M = {1, 2, 3, ..., M}, and the set of users is represented by K = {1, 2, 3, ..., K}. Using the Cartesian coordinate system, the position of UAV m at time t is recorded as (x m,t ,y m,t ,Z0), the altitude of the drone is Z0, where x m,t and m,t and represent the x-axis and y-axis coordinates of drone m at time t; the position of user k at time t is (x k,t ,y k,t ,0); the position of the central controller cs is recorded as (x c ,y c ,0).

[0080] Among them, edge servers and drones are both used as computing platforms in the ISCC system environment; users in the user set can be smartphones, tablets, smart wearable devices, etc. At the same time, each drone can provide services to multiple target users, and each target user is only served by one drone; drones serve as aerial platforms to communicate with target users.

[0081] Among them, at the same time, the UAV can communicate with multiple users on the ground, and each user can only communicate with one UAV at most, so the association variable index α is given k,m ∈{0,1}, represents the association between drone m and user k. k,m =1, it means that user k is served by drone m; when α k,m = 0 means that UAV m is not providing service to user k, that is, user k and UAV m have not established a communication connection. Similarly, for simplicity, assume that a UAV senses a target. In this regard, the target perception index β is given j,m ∈{0,1},j∈{1,2,…,m}, when β j,m =1, it means that target j is sensed by drone m, otherwise β j,m =0.

[0082] Among them, all drones fly at the same altitude, and all drones keep hovering or flying horizontally in the target area; terminal users are randomly distributed on the ground. Since each decision time slot is very short, it is assumed that the position of the target user remains unchanged in each time slot. The drone perceives the environmental state through the sensing signal to obtain the echo signal, and then optimizes the trajectory planning, optimal hovering position, DNN segmentation strategy and resource allocation strategy of the drone itself, and adopts the method of drone communication and edge server auxiliary calculation to provide communication and computing services for the target user. In this environment, a two-layer optimization strategy algorithm is adopted. The k-means++ clustering algorithm is used in the upper optimization to group the ground users, and the centroid of each user group is found as the optimal hovering position of the drone according to the nearest center strategy. The flight route of the drone is converted into a traveling salesman problem, and the simulated annealing algorithm is used to solve the shortest flight path. Split learning and Actor-Critic learning algorithms are used in the lower optimization to optimize the DNN segmentation strategy and resource allocation strategy. The present invention is mainly aimed at optimizing DNN segmentation and resource allocation.

[0083] The specific steps include:

[0084] Step 1, initialization and perception: The UAV flies over the end user, the end user generates a computing task and requests a service, the UAV uses the perception function to select the perception target and collect environmental data; after the perception data collection is completed, the perception data is transmitted to the central controller via a wireless link; then, it enters the task allocation and scheduling stage;

[0085] Step 2, task allocation and scheduling: Assign tasks according to the optimization strategy algorithm, use the adaptive split learning method to determine the DNN split strategy, the computing tasks of each drone, the transmission tasks and the trajectory optimization plan; after the task allocation is completed, enter the DNN splitting and computing stage;

[0086] Step 3, DNN splitting and calculation: The drone performs the early layer calculation of the DNN, unloads the intermediate results to the central controller and further performs the subsequent layer calculation of the DNN to complete the task; after the calculation is completed, it enters the result return stage;

[0087] Step 4, result return: The central controller returns the calculation results to the end user through the wireless link and the drone; the result return is completed and the entire operation cycle ends.

[0088] The environmental data adopts Cartesian coordinate system, all drones fly at the same altitude, and ground terminal users are randomly distributed; in each time slot, the terminal users are considered to be static.

[0089] The optimization strategy algorithm includes drone trajectory optimization, adaptive adjustment of DNN splitting and allocation of computing resources to optimize the offloading of multi-user tasks and the allocation of computing resources to ensure low latency and low energy consumption for task processing.

[0090] The adaptive split learning method divides the deep neural network model into multiple parts and executes them on different drones and edge servers respectively; the specific steps are as follows:

[0091] Model segmentation: Adaptively segment the DNN model into several sub-models based on the task’s computing requirements and network status; each sub-model can be executed independently on different computing nodes, i.e., drones or edge servers;

[0092] Task offloading: Based on the current network status and computing resources, the Actor-Critic learning algorithm is used to dynamically select the optimal communication and computing resource allocation and sub-model allocation to the appropriate computing node;

[0093] Collaborative computing: Each computing node collaboratively executes the sub-model and transmits the intermediate results through wireless links to achieve forward and backward propagation of the DNN model;

[0094] Result return: After the calculation is completed, the result is returned to the ground terminal user via a wireless link;

[0095] To achieve the above-mentioned adaptive split learning, the Actor-Critic learning algorithm is used for training, which specifically includes the following steps:

[0096] State representation: The system’s computing resources, communication resources, and task requirements are represented as a state vector, including the drone’s computing power, battery power, network bandwidth, and task size;

[0097] Actor network action selection: At each decision moment, the optimal strategy action is selected based on the current state, including model split points and resource allocation schemes;

[0098] Reward function: Define a reward function that takes into account the computational latency and energy consumption of the task to optimize the long-term performance of the system;

[0099] Critic network strategy update: Update the Actor network, optimize the strategy scheme, and improve system performance;

[0100] In one example, if Figure 2 As shown, the adaptive computing and communication resource management method using split learning in the ISCC system based on drones includes the following steps:

[0101] Initialization and perception: The drone flies over the end user, and the end user generates a computing task and requests a service. The drone starts the perception task, sends out communication signals and perception signals, perceives the environmental state to obtain echo signals, collects environmental data or monitors targets. The perception data is sent to the central controller via a wireless link.

[0102] Task allocation and scheduling: The model uses split learning to train the DNN model. The drone and edge server cooperate to train the DNN model. The model is divided into N layers, and the layer set is represented by N = {1,2,...,N}. The training rounds are T times, and the round set is represented by T = {1,2,...,T}. The model calculation delay, energy consumption and total cost are modeled.

[0103] 1) Forward propagation calculation:

[0104] Calculation on the drone side:

[0105] in, is the forward propagation calculation delay of the UAV side executing the i-th layer task of the DNN network at the t-th iteration, L i is the number of floating-point operations required for layer i calculation, C0 is the number of floating-point operations processed in one CPU cycle, is the CPU calculation frequency of the drone. The calculation energy consumption of the drone is set to E MD (t);

[0106] Edge server computing:

[0107] in, is the forward propagation computation delay of the edge server executing the i-th layer task of the DNN network at the t-th iteration, L i The number of floating point operations required to calculate the i-th layer, The CPU calculation frequency allocated by the edge server to drone m;

[0108] 2) Back propagation calculation:

[0109] Calculation on the drone side:

[0110] in, is the back propagation calculation delay of the UAV side executing the i-th layer task of the DNN network at the t-th iteration, L i,B The number of floating point operations required to update the gradient calculation for the i-th layer; the energy consumption of the drone end is set to E MB (t);

[0111] Edge server computing:

[0112] in, is the back propagation calculation delay of the edge server executing the i-th layer task of the DNN network at the t-th iteration, L i,B Represents the number of floating-point operations required to calculate the gradient of the i-th layer update;

[0113] 3) Communication delay and energy consumption:

[0114] Uplink transmission delay (from MD to ES):

[0115] in, is the transmission delay of the output data of the i-th layer task at the t-th iteration. i is the output data volume of the i-th layer (in bits), R U (t) is the uplink transmission rate; Indicates the energy consumption of the drone during uplink transmission;

[0116] Downlink transmission delay (from ES to MD):

[0117] in, is the transmission delay of the output data of the i-th layer task at the t-th iteration. i is the amount of output data (in bits) at the i-th layer, R D (t) is the downlink transmission rate. Indicates the energy consumption of the drone during downlink transmission.

[0118] 4) Cost calculation:

[0119] Let x i (t)∈{0,1} is a binary DNN split indicator. If the i-th layer of DNN is calculated on the drone side, x i (t)=1, otherwise x i (t)=0.

[0120] Computational cost: The computational cost is composed of the execution delay of all tasks and the computational energy consumption on the drone side.

[0121]

[0122] Among them, w is the weight factor between delay and energy consumption.

[0123] Communication cost: The communication cost includes the latency and energy consumption when data is transmitted between the drone and the edge server during forward and backward propagation, assuming that segmentation is performed after some layers.

[0124] Forward propagation communication cost:

[0125]

[0126] Back-propagation communication cost:

[0127]

[0128] Total communication cost:

[0129] If the i-th layer task is executed on the MD and the i+1-th layer task is executed on the ES, uplink transmission delay and uplink transmission energy consumption will be generated; if the i-th layer task is executed on the ES and the i+1-th layer task is executed on the MD, downlink transmission delay and downlink transmission energy consumption will be generated.

[0130] Total system cost:

[0131] ETC all =ETC cmp (t)+ETC com (t)+ETC res (t)+ETC sc (t)

[0132] Among them, ETC res (t) represents the DNN re-segmentation cost, ETC sc (t) represents the perceived communication cost.

[0133] like Figure 3As shown, the Actor-Critic learning algorithm is used to train the model and adjust resource allocation to minimize the long-term average cost. The optimization algorithm is implemented as follows:

[0134] 1) Obtain sufficient environmental information: including the maximum computing frequency of the drone CPU, the maximum computing frequency of the edge server CPU, the maximum tolerable total delay of the computing task, the drone’s energy limit, and the current system status.

[0135] 2) Initialize experience replay storage D and average reward Initialize the network with random weights. For each training round t, initialize the state s(t), and the Actor network selects an action a(t) according to the selection strategy; execute the selected action a(t), obtain the reward U, and observe the new state s(t+1), store (s(t), a(t), U, s(t+1)) in D, uniformly extract a small batch from the experience replay storage D for training, and update the target state-action value R. The Critic network performs gradient descent to minimize the loss function L, updates the Actor network weights, and periodically updates the network weights of the target network to stabilize the training process. If a non-exploratory action is selected, update the average reward Output action a(t) until the maximum number of training epochs is reached.

[0136] 3) Complete DNN segmentation and resource allocation, start calculation, and return the calculation results.

[0137] According to the strategy determined by the above algorithm, the drone and edge server allocate corresponding resources and complete the computing tasks, and return the computing results to the user terminal through the downlink. By adaptively segmenting the DNN model and collaboratively executing it on the drone and edge server, it can better adapt to the highly dynamic network environment and be more in line with actual application scenarios, so as to achieve efficient resource utilization, meet users' requirements for low latency and low energy consumption, and improve the performance of edge intelligent networks.

[0138] Those skilled in the art can understand that the above are only preferred examples of the invention and are not intended to limit the invention. Although the invention is described in detail with reference to the aforementioned embodiments, those skilled in the art can still modify the technical solutions recorded in the aforementioned embodiments or replace some of the technical features therein with equivalents. Any modifications, equivalent replacements, etc. made within the spirit and principles of the invention should be included in the protection scope of the invention. All technical features in this embodiment can be freely combined according to actual needs.

[0139] Finally, it should be noted that the above is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, it is still possible for those skilled in the art to modify the technical solutions described in the aforementioned embodiments or to make equivalent substitutions for some of the technical features therein. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A DNN splitting learning method based on drone-based synaesthesia fusion network, characterized by: The UAV inter-sensory computing fusion network includes multiple UAVs, a central controller equipped with edge servers, and ground terminal users; Among them, each drone is equipped with a computing module and a communication module for data interaction with the edge server; Edge servers, which have computing power and storage resources and are responsible for centrally processing complex computing tasks; Ground end users, which generate computing tasks and offload them to drones and edge servers via wireless links; The UAV-assisted synaesthesia-computing fusion network environment includes a communication user set and a computing server set: Among them, the communication user set includes users who need communication services and are perceived by the target, including smart phones, mobile tablets, and unmanned vehicle terminal devices; The computing server set includes drones equipped with DNN processing capabilities and edge servers; each drone can interact with multiple communication users, and each communication user can only connect to one drone; communication and perception data can be processed by the drone or offloaded to the edge server; The specific steps include: Step 1, initialization and perception: The UAV flies over the end user, the end user generates a computing task and requests a service, the UAV uses the perception function to select the perception target and collect environmental data; after the perception data collection is completed, the perception data is transmitted to the central controller via a wireless link; then, it enters the task allocation and scheduling stage; Step 2, task allocation and scheduling: Assign tasks according to the optimization strategy algorithm, use the adaptive split learning method to determine the DNN split strategy, the computing tasks of each drone, the transmission tasks and the trajectory optimization plan; after the task allocation is completed, enter the DNN splitting and computing stage; Step 3, DNN splitting and calculation: The drone performs the early layer calculation of the DNN, unloads the intermediate results to the central controller and further performs the subsequent layer calculation of the DNN to complete the task; after the calculation is completed, it enters the result return stage; Step 4, result return: The central controller returns the calculation result to the end user through the wireless link and the drone; the result return is completed and the entire operation cycle ends; In step 2, the optimization strategy algorithm includes drone trajectory optimization, adaptive adjustment of DNN splitting, and allocation of computing resources to optimize the offloading of multi-user tasks and the allocation of computing resources to ensure low latency and low energy consumption for task processing; In step 2, the adaptive split learning method divides the deep neural network model into multiple parts, which are executed on different drones and edge servers respectively; the specific steps are as follows: Model segmentation: Adaptively segment the DNN model into several sub-models based on the task’s computing requirements and network status; each sub-model can be executed independently on different computing nodes, i.e., drones or edge servers; Task offloading: Based on the current network status and computing resources, the Actor-Critic learning algorithm is used to dynamically select the optimal communication and computing resource allocation and sub-model allocation to the appropriate computing node; Collaborative computing: Each computing node collaboratively executes the sub-model and transmits the intermediate results through wireless links to achieve forward and backward propagation of the DNN model; Result return: After the calculation is completed, the result is returned to the ground terminal user via a wireless link; To achieve the above-mentioned adaptive split learning, the Actor-Critic learning algorithm is used for training, which specifically includes the following steps: State representation: The system’s computing resources, communication resources, and task requirements are represented as a state vector, including the drone’s computing power, battery power, network bandwidth, and task size; Actor network action selection: At each decision moment, the optimal strategy action is selected based on the current state, including model split points and resource allocation schemes; Reward function: Define a reward function that takes into account the computational latency and energy consumption of the task to optimize the long-term performance of the system; Critic network strategy update: Update the Actor network, optimize the strategy scheme, and improve system performance; In step 2, split learning is used to train the DNN model; the drone and the edge server cooperate to train the DNN model, the model is divided into N layers, the layer set is represented by N = {1,2,...,N}, the training rounds are T times, and the round set is represented by T = {1,2,...,T}; the model calculation delay, energy consumption and total cost are modeled; specifically, the following steps are included; Step 2.1, forward propagation calculation: Calculation on the drone side: in, is the forward propagation calculation delay of the UAV side executing the i-th layer task of the DNN network at the t-th iteration, L i is the number of floating-point operations required for layer i calculation, C0 is the number of floating-point operations processed in one CPU cycle, is the CPU calculation frequency of the drone end; the calculation energy consumption of the drone end is set to E MD (t); Edge server computing: in, is the forward propagation computation delay of the edge server executing the i-th layer task of the DNN network at the t-th iteration, L i The number of floating point operations required to calculate the i-th layer, The CPU calculation frequency allocated by the edge server to drone m; Step 2.2, back propagation calculation: Calculation on the drone side: in, is the back propagation calculation delay of the UAV side executing the i-th layer task of the DNN network at the t-th iteration, L i,B The number of floating point operations required to update the gradient calculation for the i-th layer; the energy consumption of the drone end is set to E MB (t); Edge server computing: in, is the back propagation calculation delay of the edge server executing the i-th layer task of the DNN network at the t-th iteration, L i,B Represents the number of floating-point operations required to calculate the gradient of the i-th layer update; Step 2.3, communication delay and energy consumption: Uplink transmission delay from MD to ES: in, is the transmission delay of the output data of the i-th layer task at the t-th iteration; I i is the output data volume of the i-th layer, in bits, P U (t) is the uplink transmission rate; Indicates the energy consumption of the drone during uplink transmission; Downlink transmission delay from ES to MD: in, is the transmission delay of the output data of the i-th layer task at the t-th iteration; I i is the output data volume of the i-th layer, R D (t) is the downlink transmission rate; Indicates the energy consumption of the drone during downlink transmission; Step 2.4, cost calculation: Let x i (t)∈{0,1} is a binary DNN split indicator. If the i-th layer of DNN is calculated on the drone side, x i (t)=1, otherwise x i (t) = 0; The computational cost is composed of the execution delay of all computational tasks and the computational energy consumption on the drone side: Among them, w is the weight factor between delay and energy consumption; Communication cost: Communication cost includes the delay and energy consumption generated when data is transmitted between the drone and the edge server during forward propagation and backpropagation. Forward propagation communication cost: Back-propagation communication cost: Total system cost: ETC all =ETC cmp (t)+ETC com (t)+ETC res (t)+ETC sc (t) Among them, ETC res (t) represents the DNN re-segmentation cost, ETC sc (t) represents the perceived communication cost; The model is trained using the Actor-Critic learning algorithm to adjust resource allocation to minimize the long-term average cost; the optimization algorithm is implemented as follows: 1) Obtain sufficient environmental information: including the maximum computing frequency of the drone CPU, the maximum computing frequency of the edge server CPU, the maximum tolerable total delay of the computing task, the energy limit of the drone, and the current system status information; 2) Initialize experience replay storage D and average reward Initialize the network with random weights; for each training round t, initialize the state s(t), and the Actor network selects an action a(t) according to the selection strategy; execute the selected action a(t), obtain the reward U, and observe the new state s(t+1), store (s(t), a(t), U, s(t+1)) in D, uniformly extract a small batch from the experience replay storage D for training, and update the target state-action value R; the Critic network performs gradient descent to minimize the loss function L, updates the Actor network weights, and regularly updates the network weights of the target network to stabilize the training process; if a non-exploratory action is selected, update the average reward Output action a(t) until the maximum number of training rounds is reached; 3) Complete DNN segmentation and resource allocation, start calculation, and return the calculation results.

2. The method for splitting and learning a DNN based on a drone according to claim 1 is characterized in that: In step 1, the environmental data adopts Cartesian coordinate system, all UAVs fly at the same altitude, and the ground terminal users are randomly distributed; in each time slot, the terminal users are considered to be static.

Images