Distributed large language model collaborative reasoning method and system based on unmanned aerial vehicle cluster

Abstract

The application provides a kind of based on unmanned aerial vehicle cluster distributed large language model collaborative inference method and system, method includes: obtaining each node hardware state and inter-node link feature, constructs cluster topology graph;Large language model is vertically divided into multiple pipeline stages and deployed in each node;The boundary intermediate activation tensor generated by forward calculation is backed up to local ring buffer;When node resource or link quality is lower than threshold, freeze collaborative inference, generate candidate configuration pool based on hardware state, link feature and last time topology configuration;Determine target topology configuration based on cluster topology graph and last time topology configuration;Identify migration stage and new node, keep the history key-value cache of un-migrated node, read boundary intermediate activation tensor from previous node and transmit to new node, make it locally pre-filled to reconstruct local key-value cache, and recover collaborative decoding calculation.The application can reduce topology reconstruction overhead and ensure the continuity of unmanned aerial vehicle cluster distributed large model collaborative inference.

Description

Technical Field

[0001] This invention belongs to the field of unmanned aerial vehicle (UAV) communication technology, specifically relating to a distributed large language model collaborative reasoning method and system based on UAV swarms. Background Technology

[0002] With the increasing computing power of edge computing platforms, drone swarms are gradually being applied to complex tactical reconnaissance, environmental monitoring, and autonomous inspection missions. Deploying large language models in the edge network of drones can reduce the reliance of drone swarms on ground base stations in environments without or with weak network coverage, and enable them to perform autonomous analysis, collaborative decision-making, and command generation based on on-site mission information.

[0003] Due to limitations in the GPU memory, computing power, and energy consumption of a single drone, large language models are typically difficult to deploy entirely on a single drone node. Existing solutions often employ a pipelined parallel approach, dividing the large language model into multiple pipeline stages according to the network layer sequence, and distributing these different pipeline stages across multiple drones for collaborative inference. During the regular inference process, each drone node sequentially transmits intermediate activation results, thereby completing the computation of the prefill and autoregressive decoding stages.

[0004] However, drone swarms are characterized by high-speed movement, significant link fluctuations, and frequent changes in drone node states. When drone movement causes topology changes, drone node failures, or link disconnections, existing solutions typically require discarding existing states and recalculating from historical inputs, or transmitting a large dynamic key-value cache (KVCache) over the airborne wireless network. The former results in high recalculation latency and wasted computing power, while the latter easily consumes airborne wireless link bandwidth. Furthermore, if the scheduling system lacks awareness of the previous topology configuration, short-term fluctuations in link quality can trigger frequent migrations, causing the pipeline to remain in a refactoring and blocked state for extended periods. Summary of the Invention

[0005] To address the aforementioned problems in existing technologies, this invention provides a distributed large-scale language model collaborative reasoning method and system based on unmanned aerial vehicle (UAV) swarms. The technical problem to be solved by this invention is achieved through the following technical solution: In a first aspect, the present invention provides a distributed large language model collaborative reasoning method based on unmanned aerial vehicle (UAV) swarms, comprising: The hardware status information of each drone node in the drone cluster and the wireless link characteristics between each drone node are acquired in real time, and a real-time cluster topology map is constructed based on the hardware status information and the wireless link characteristics. The large language model is vertically divided into multiple pipeline stages, and each pipeline stage is deployed on each UAV node based on the real-time cluster topology graph. By executing the drone nodes at each pipeline stage, the boundary intermediate activation tensor generated by the forward computation is serialized and backed up to the local circular buffer. If the remaining available resources of a drone node are lower than a preset resource threshold, or the link quality between adjacent drone nodes is lower than a preset connectivity threshold, the collaborative inference state of the current pipeline is frozen, and a candidate configuration pool is generated based on the hardware status information, the wireless link characteristics, and the topology configuration executed in the previous moment. Based on the real-time cluster topology graph and the topology configuration executed in the previous moment, the target topology configuration is determined from the candidate configuration pool; Based on the target topology configuration, identify the pipeline stages that have migrated and the new UAV nodes, maintain the historical key-value cache on the UAV nodes that have not migrated, and read the boundary intermediate activation tensor from the local circular buffer of the preceding UAV node of the pipeline stage that has migrated and transmit it to the new UAV node, so that the new UAV node performs a local pre-filling calculation and rebuilds the local key-value cache, and resumes the collaborative decoding calculation.

[0006] Secondly, the present invention provides a distributed large language model collaborative reasoning system based on unmanned aerial vehicle (UAV) swarms, comprising: The data acquisition module is used to acquire the hardware status information of each drone node in the drone cluster and the wireless link characteristics between each drone node in real time, and to construct a real-time cluster topology map based on the hardware status information and the wireless link characteristics. The distributed inference module is used to vertically divide the large language model into multiple pipeline stages, and deploy each pipeline stage to each UAV node based on the real-time cluster topology graph. The tensor caching module is used to serialize and back up the boundary intermediate activation tensors generated by forward computation to the local circular buffer by executing the drone nodes at each pipeline stage. The reconstruction trigger module is used to freeze the collaborative inference state of the current pipeline when the remaining available resources of the UAV node are lower than a preset resource threshold, or the link quality between adjacent UAV nodes is lower than a preset connectivity threshold, and generate a candidate configuration pool based on the hardware status information, the wireless link characteristics and the topology configuration executed in the previous moment. The target decision determination module is used to determine the target topology configuration from the candidate configuration pool based on the real-time cluster topology graph and the topology configuration executed in the previous moment; The fault-tolerant recovery module is used to identify the pipeline stage that has migrated and the new UAV node based on the target topology configuration, maintain the historical key-value cache on the UAV node that has not migrated, and read the boundary intermediate activation tensor from the local circular buffer of the preceding UAV node of the pipeline stage that has migrated and transmit it to the new UAV node, so that the new UAV node performs a local pre-filling calculation and rebuilds the local key-value cache, and resumes the collaborative decoding calculation.

[0007] Thirdly, the present invention provides an electronic device, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; The memory is used to store computer programs; When the processor executes the computer program stored in the memory, it implements the steps of the distributed large language model collaborative reasoning method based on UAV swarm provided in the first aspect.

[0008] Compared with the prior art, the beneficial effects of the present invention are as follows: The distributed large language model collaborative inference method based on UAV swarms provided by this invention acquires the hardware status information and wireless link characteristics of each UAV node in the UAV swarm in real time and constructs a real-time swarm topology map, providing a state-aware foundation for the distributed pipeline deployment and dynamic reconstruction of large language models on UAV swarms. By serializing and backing up the boundary intermediate activation tensor generated by forward computation to the local circular buffer of each UAV node, the boundary intermediate activation tensor can be transmitted by the preceding UAV node in the pipeline stage of migration during topology reconstruction. This drives newly connected UAV nodes to perform local pre-filling computation and rebuild the local key-value cache, thereby avoiding the need for all UAV nodes in the network to discard historical key-value caches and recalculate, and also avoiding the transmission of large dynamic key-value caches in the wireless network, reducing reconstruction latency and Communication overhead is reduced by freezing the current pipeline's collaborative inference state when the remaining available resources of a UAV node fall below a threshold or the link quality between adjacent nodes falls below a threshold. A candidate configuration pool is generated based on hardware state information, wireless link characteristics, and the topology configuration executed in the previous time step. This limits the search scope of topology decision-making to candidate configurations that meet the node resource and link conditions, reducing the computational burden of onboard scheduling. The target topology configuration is determined from the candidate configuration pool based on the real-time cluster topology graph and the topology configuration executed in the previous time step. This allows topology decision-making to reference historical execution states, reducing the probability of unnecessary migrations during short-term fluctuations in link quality and suppressing the ping-pong effect in topology reconstruction. Furthermore, by maintaining historical key-value caches on UAV nodes that have not migrated and performing only local pre-filling calculations during the migration phase, the continuity of collaborative decoding computation is ensured. Ultimately, this invention reduces the reconstruction overhead caused by global recalculation and dynamic key-value cache transmission, ensuring the continuity of collaborative inference in a distributed large-scale UAV cluster model.

[0009] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0010] Figure 1 This is a schematic diagram illustrating the implementation process of a distributed large language model collaborative reasoning method based on UAV swarms provided in an embodiment of the present invention; Figure 2 This is a schematic diagram illustrating the implementation process of another distributed large language model collaborative reasoning method based on UAV clusters provided in this embodiment of the invention; Figure 3 This is a schematic diagram of the structure of a distributed large language model collaborative reasoning system based on a drone swarm provided in an embodiment of the present invention; Figure 4 This is a schematic diagram of the hardware entity of an electronic device to which embodiments of the present invention are applied. Detailed Implementation

[0011] The present invention will be further described in detail below with reference to specific embodiments, but the implementation of the present invention is not limited thereto.

[0012] Please see Figure 1 , Figure 1 This is a schematic diagram illustrating the implementation process of a distributed large language model collaborative reasoning method based on a drone swarm, provided in an embodiment of the present invention. Figure 1 As shown in the embodiments of the present invention, the distributed large language model collaborative reasoning method based on UAV swarms may include the following steps: Step S101: Obtain the hardware status information of each drone node in the drone cluster and the wireless link characteristics between each drone node in real time, and construct a real-time cluster topology map based on the hardware status information and wireless link characteristics.

[0013] In this context, a drone node refers to a drone participating in distributed inference of a large language model within a drone swarm. Hardware status information may include at least one of the following for each drone node: available computing power, remaining battery power, and idle video memory. Wireless link characteristics may include at least one of the following for each drone node: signal-to-noise ratio, link bandwidth, transmission latency, and packet loss status.

[0014] In one possible implementation, UAV node resource characteristics can be generated based on hardware status information, UAV node inter-connectivity characteristics can be generated based on wireless link characteristics, and a real-time cluster topology graph can be constructed based on the UAV node resource characteristics and UAV node inter-connectivity characteristics.

[0015] Step S102: Vertically divide the large language model into multiple pipeline stages, and deploy each pipeline stage to each UAV node based on the real-time cluster topology diagram.

[0016] In one possible implementation, the continuous network layers can be divided into multiple pipeline stages according to the network layer order of the large language model; the drone nodes that meet the operational requirements of each pipeline stage can be determined based on the real-time cluster topology graph; and each pipeline stage can be deployed to the corresponding drone nodes in the order of the network layers.

[0017] Understandably, each pipeline stage handles different numbers of model layers, parameters, and computational characteristics, thus placing varying demands on the hardware resources and network conditions of UAV nodes. Based on the hardware status information and wireless link characteristics of each UAV node in the real-time cluster topology diagram, it can be determined whether each UAV node meets the operational requirements of the pipeline stage. For example, it can be determined whether the available computing power and idle GPU memory of each UAV node meet the computing power and GPU memory requirements of the pipeline stage, and whether the signal-to-noise ratio, link bandwidth, and transmission latency between each UAV node and its preceding and following nodes meet the communication requirements of the intermediate activation tensor at the transmission boundary between pipeline stages.

[0018] In one possible implementation, drone nodes corresponding to adjacent pipeline stages can transmit the boundary intermediate activation tensors output by the previous pipeline stage through an asynchronous message queue, and collaboratively perform the computation of the prefill and decoding stages of the large language model in an autoregressive manner.

[0019] Step S103: By executing the UAV nodes of each pipeline stage, the boundary intermediate activation tensor generated by the forward computation is serialized and backed up to the local circular buffer.

[0020] The boundary intermediate activation tensor is the intermediate result output to the next pipeline stage after the current pipeline stage completes its forward computation. The local circular buffer is a cache space allocated in the local memory of the UAV node to store the boundary intermediate activation tensor generated during the pre-filling stage.

[0021] In one possible implementation, when the drone node executing the current pipeline stage sends the boundary intermediate activation tensor to the drone node corresponding to the next pipeline stage, the boundary intermediate activation tensor is copied; the copied boundary intermediate activation tensor is locally serialized to obtain the serialized boundary intermediate activation tensor; and the serialized boundary intermediate activation tensor is written into the local circular buffer of the current drone node.

[0022] In one possible implementation, the boundary intermediate activation tensor in the local circular buffer can also be kept in a cached read-only state during the decoding phase to prevent the boundary intermediate activation tensor from being overwritten.

[0023] Step S104: When the remaining available resources of the UAV node are lower than the preset resource threshold, or the link quality between adjacent UAV nodes is lower than the preset connectivity threshold, freeze the collaborative inference state of the current pipeline and generate a candidate configuration pool based on hardware status information, wireless link characteristics and the topology configuration executed in the previous moment.

[0024] In one possible implementation, the remaining available resources of a drone node can include its current available computing power, remaining battery power, or idle GPU memory. Preset resource thresholds could be 4GB of idle GPU memory, 25% of remaining battery power, or 20% of remaining available computing power, or other values ​​set according to actual needs. The link quality between adjacent drone nodes can be determined using wireless link characteristics such as signal-to-noise ratio (SNR). A preset connectivity threshold could be a SNR of 15dB, or other values ​​set according to actual needs.

[0025] Understandably, when the remaining available resources of a drone node fall below a preset resource threshold, it indicates that the drone node may be unable to stably support the corresponding pipeline stage. Similarly, when the link quality between adjacent drone nodes falls below a preset connectivity threshold, it indicates that data transmission between adjacent pipeline stages may not meet the requirements for collaborative inference. Therefore, when the remaining available resources of a drone node fall below the preset resource threshold, or the link quality between adjacent drone nodes falls below the preset connectivity threshold, a suspension command can be sent to pause the decoding phase computation of the current pipeline.

[0026] In one possible implementation, a candidate configuration pool is generated based on hardware status information, wireless link characteristics, and the topology configuration executed in the previous time step. This includes: determining the physical constraints of each UAV node based on its hardware status information and wireless link characteristics; selecting pipeline stage and UAV node mapping combinations that meet the requirements of running a large language model based on the physical constraints; writing the selected pipeline stage and UAV node mapping combinations as candidate configurations into the candidate configuration pool; and writing the topology configuration executed in the previous time step as a historical candidate configuration into the candidate configuration pool.

[0027] Physical constraints include free video memory, remaining power, available computing power, CPU load, signal-to-interference-plus-noise ratio, and transmission latency.

[0028] For example, the process of generating a candidate configuration pool can be divided into two stages: coarse filtering and fine filtering. First, physical constraints can be constructed based on the available computing power, remaining power, idle GPU memory, and link quality between each UAV node. Invalid combinations that would lead to GPU memory overflow or link disconnection are filtered out according to the physical constraints, resulting in an initial candidate set that meets the requirements of running a large language model (i.e., the coarse filtering stage). Subsequently, using a preset proxy overhead function, the prediction overhead of the pipeline stage and the mapping combination of UAV nodes in the initial candidate set is calculated. The above prediction overhead can include inference latency overhead and node migration overhead, which can be used to comprehensively evaluate the estimated latency of single-step forward computation, the transmission time of wireless transmission, and the time cost of state recalculation due to topology node changes. The initial candidate set is sorted from low to high predicted cost. A predetermined number (e.g., 24, or other values ​​set according to actual needs) of pipeline stage and UAV node mapping combinations are selected as candidate configurations and written into the candidate configuration pool (i.e., fine-tuning stage). To avoid frequent topology configuration changes due to minor link fluctuations, the topology configuration executed at the previous time step can be used as a historical candidate configuration and forcibly written into the candidate configuration pool. This ensures that the candidate configuration pool retains the configuration option to maintain the original topology, thus ensuring that subsequent decision models always have the option to "keep the status quo unchanged." The preset proxy cost function can be a weighted cost calculation function that integrates inference latency cost, communication energy consumption cost, and node migration cost. For any given pipeline stage and UAV node mapping combination, its predicted cost... The calculation process is as follows: ; in, These are the weighting coefficients. The normalized inference latency overhead is determined by the sum of the computation and transmission times of the pre-filling stage in each pipeline stage, and the generation time of the bottleneck node in the decoding stage. The normalized communication energy consumption can be obtained by summing the power consumption corresponding to the active computing time allocated to each UAV node and the transmission and reception power consumption corresponding to tensor data transmission between adjacent UAV nodes. To account for the normalized node migration cost (i.e., local refactoring penalty), the system introduces a refactoring indicator variable. When the first When the deployment node of each pipeline stage changes compared to the previous moment... Otherwise, it is 0, and the node migration cost is only accumulated. The local pre-filling computation time and intermediate activation value transfer time required for the pipeline stage.

[0029] The aforementioned proxy overhead function enables the system to quickly quantify and evaluate the overall performance loss of any candidate topology combination within milliseconds without actually performing model reconstruction, thereby achieving precise fine filtering.

[0030] Step S105: Based on the real-time cluster topology graph and the topology configuration executed in the previous moment, determine the target topology configuration from the candidate configuration pool.

[0031] In one possible implementation, global topology features can be determined based on the drone node resource features and inter-drone node connectivity features in the real-time cluster topology graph; an embedding vector of the topology configuration executed in the previous time step can be generated; the global topology features and the embedding vector can be fused to obtain state context features; and each candidate configuration in the candidate configuration pool can be scored based on the state context features, and the candidate configuration that meets the conditions can be determined as the target topology configuration.

[0032] For example, a real-time cluster topology graph can be input into a Graph Attention Network (GAT). The GAT extracts global topology features based on the resource features of UAV nodes and the connectivity features between UAV nodes in the real-time cluster topology graph. These global topology features are then fused with the embedding vector of the topology configuration executed in the previous time step to obtain historically perceived state context features. Based on these state context features, each configuration in the candidate configuration pool is scored and sampled, and the candidate configuration that meets the scoring criteria is determined as the target topology configuration. For example, the communication efficiency, resource adaptability, and similarity to the previous time step of each configuration in the candidate configuration pool can be scored, and the scores of these three dimensions can be weighted and fused to obtain a comprehensive score. The scoring criteria can be a comprehensive score greater than 80 points, a communication efficiency score greater than 70 points, and a similarity score greater than 60 points. These scoring criteria can also be set according to actual conditions, and this invention does not impose any particular limitations on them. The target topology configuration can be used to indicate the correspondence between the migrated pipeline stages and the UAV nodes.

[0033] Understandably, a real-time cluster topology graph can reflect the current drone node resources and the wireless link status between drone nodes. The topology configuration executed at the previous moment can reflect the model layer segmentation and drone node mapping relationship actually used by the system before the reconstruction was triggered. Using the current topology status and historical topology configuration together for decision-making can make the scheduling process consider both the current link benefits and migration costs.

[0034] Step S106: Identify the pipeline stage that has migrated and the new UAV node based on the target topology configuration, maintain the historical key-value cache on the UAV node that has not migrated, and read the boundary intermediate activation tensor from the local circular buffer of the preceding UAV node of the pipeline stage that has migrated and transmit it to the new UAV node, so that the new UAV node performs a local pre-filling calculation and rebuilds the local key-value cache, and resumes the collaborative decoding calculation.

[0035] The pipeline stage where migration occurs is the model stage where the drone nodes undergo changes in both the current pipeline topology configuration and the target topology configuration. The new drone node is the drone node in the target topology configuration used to take over the pipeline stage where migration has occurred. The historical key-value cache is the key-value cache state accumulated on the unmigrated drone nodes during the decoding phase.

[0036] In one possible implementation, identifying the pipeline stage undergoing migration and the new drone node based on the target topology configuration includes: determining the pipeline stage undergoing migration based on the topology configuration corresponding to the current pipeline and the target topology configuration; and identifying the drone node in the target topology configuration used to execute the pipeline stage undergoing migration as the new drone node.

[0037] For example, the current pipeline topology configuration and the target topology configuration can be compared to determine the pipeline stage where the migration occurred.

[0038] In one possible implementation, a state lock command can be sent to drone nodes that have not migrated to prevent the historical key-value cache from being cleared. Subsequently, the preceding drone node in the pipeline stage that has migrated can be controlled to read the boundary intermediate activation tensor from its local circular buffer and send the read boundary intermediate activation tensor to the new drone node.

[0039] In one possible implementation, the new UAV node performs a local pre-filling computation and rebuilds the local key-value cache to resume the collaborative decoding computation. This includes: enabling the new UAV node to perform a local pre-filling computation on its pipeline stage based on the received boundary intermediate activation tensor, generating a new boundary intermediate activation tensor and a new local key-value cache corresponding to the pipeline stage executed by the new UAV node; using the new local key-value cache as the local key-value cache; transmitting the new boundary intermediate activation tensor to the subsequent UAV node; and resuming the collaborative decoding computation according to the target topology configuration after aligning the new boundary intermediate activation tensor with the input of the subsequent UAV node.

[0040] For example, after receiving the boundary intermediate activation tensor, the new UAV node uses the received boundary intermediate activation tensor as input to perform a forward computation on the pipeline stage executed by the new UAV node, generating a new boundary intermediate activation tensor and a new local key-value cache corresponding to the pipeline stage undertaken by the new UAV node.

[0041] like Figure 2 As shown, in one possible implementation, the drone swarm 200 includes a head node 201, original intermediate nodes 202, and a tail node 203, with a reserved backup node 205. The large language model is divided into three pipeline stages: stage 1, stage 2, and stage 3. The head node 201 executes stage 1, the original intermediate nodes 202 execute stage 2, and the tail node 203 executes stage 3. Here, the head node 201, the original intermediate nodes 202, the tail node 203, and the backup node 205 are all drone nodes.

[0042] During the pre-filling phase, the first node 201 receives the full prompt text or image input and performs forward computation. The first node 201 sends the output boundary intermediate activation tensor to the original intermediate node 202, and simultaneously serializes and backs up this boundary intermediate activation tensor to its local circular buffer 204. In the subsequent decoding phase, the first node 201, the original intermediate node 202, and the tail node 203 progressively generate tokens and accumulate historical key-value caches (KV Caches) in their respective memory. In one specific embodiment, for a given input length, the data size of this boundary intermediate activation tensor is approximately 71.69 MB, and the amount of data transmitted between nodes is relatively small (approximately tens of KB per transmission).

[0043] When the wireless link quality between the original intermediate node 202 and the tail node 203 falls below a preset connectivity threshold (e.g., when SINR is below 15dB), the status monitoring module 201 triggers a reconstruction. The reconstruction triggering and filtering module 204 generates a candidate configuration pool based on the current resources of each UAV node and the link quality between each UAV node, and injects the topology configuration executed in the previous moment into the candidate configuration pool. The target decision selection module 205 selects the target topology configuration based on the real-time cluster topology map and the topology configuration executed in the previous moment, determining that the backup node 205 will take over the responsibility of carrying out phase 2 from the original intermediate node 202.

[0044] The fault-tolerant recovery module sends state lock commands to the first node 201 and the last node 203 to prevent the historical key-value cache on the drone nodes that have not migrated from being cleared. The first node 201 reads the boundary intermediate activation tensor backed up in the pre-filling stage from the local circular buffer 204 and sends it to the backup node 205 via the asynchronous communication interface. The backup node 205 uses this boundary intermediate activation tensor as input and performs a local pre-filling forward calculation only once on the stage 2 it is responsible for, rebuilding the local key-value cache corresponding to stage 2. Subsequently, the backup node 205 sends the calculation output to the last node 203, and the entire pipeline resumes autoregressive decoding generation according to the target topology configuration.

[0045] Through the above processing, the system does not need to transmit a large dynamic key-value cache in the air wireless network, nor does it need all UAV nodes to recalculate from the historical input. Instead, it uses the boundary intermediate activation tensor in the local circular buffer to complete the local state reset, thereby improving the continuity of the UAV swarm distributed large language model inference process and the anti-link degradation capability.

[0046] The distributed large language model collaborative inference method based on UAV swarms provided by this invention acquires the hardware status information and wireless link characteristics of each UAV node in the UAV swarm in real time and constructs a real-time swarm topology map, providing a state-aware foundation for the distributed pipeline deployment and dynamic reconstruction of large language models on UAV swarms. By serializing and backing up the boundary intermediate activation tensor generated by forward computation to the local circular buffer of each UAV node, the boundary intermediate activation tensor can be transmitted by the preceding UAV node in the pipeline stage of migration during topology reconstruction. This drives newly connected UAV nodes to perform local pre-filling computation and rebuild the local key-value cache, thereby avoiding the need for all UAV nodes in the network to discard historical key-value caches and recalculate, and also avoiding the transmission of large dynamic key-value caches in the wireless network, reducing reconstruction latency and Communication overhead is reduced by freezing the current pipeline's collaborative inference state when the remaining available resources of a UAV node fall below a threshold or the link quality between adjacent nodes falls below a threshold. A candidate configuration pool is generated based on hardware state information, wireless link characteristics, and the topology configuration executed in the previous time step. This limits the search scope of topology decision-making to candidate configurations that meet the node resource and link conditions, reducing the computational burden of onboard scheduling. The target topology configuration is determined from the candidate configuration pool based on the real-time cluster topology graph and the topology configuration executed in the previous time step. This allows topology decision-making to reference historical execution states, reducing the probability of unnecessary migrations during short-term fluctuations in link quality and suppressing the ping-pong effect in topology reconstruction. Furthermore, by maintaining historical key-value caches on UAV nodes that have not migrated and performing only local pre-filling calculations during the migration phase, the continuity of collaborative decoding computation is ensured. Ultimately, this invention reduces the reconstruction overhead caused by global recalculation and dynamic key-value cache transmission, ensuring the continuity of collaborative inference in a distributed large-scale UAV cluster model.

[0047] Please see Figure 3 , Figure 3 This is a schematic diagram of the structure of a distributed large language model collaborative reasoning system based on UAV swarms provided by an embodiment of the present invention. The distributed large language model collaborative reasoning system based on UAV swarms provided by the present invention includes a data acquisition module 301, a distributed reasoning module 302, a tensor caching module 303, a reconstruction triggering module 304, a target decision determination module 305, and a fault tolerance and recovery module 306.

[0048] Among them, the data acquisition module 301 is used to acquire the hardware status information of each UAV node in the UAV cluster and the wireless link characteristics between each UAV node in real time, and construct a real-time cluster topology map based on the hardware status information and wireless link characteristics. The distributed inference module 302 is used to vertically divide the large language model into multiple pipeline stages and deploy each pipeline stage to each UAV node based on the real-time cluster topology graph. Tensor caching module 303 is used to serialize and back up the boundary intermediate activation tensor generated by forward computation to a local circular buffer by executing the UAV nodes of each pipeline stage. The reconstruction trigger module 304 is used to freeze the collaborative reasoning state of the current pipeline when the remaining available resources of the UAV node are lower than the preset resource threshold, or the link quality between adjacent UAV nodes is lower than the preset connectivity threshold, and generate a candidate configuration pool based on hardware status information, wireless link characteristics and the topology configuration executed in the previous moment. The target decision determination module 305 is used to determine the target topology configuration from the candidate configuration pool based on the real-time cluster topology graph and the topology configuration executed in the previous moment. The fault-tolerant recovery module 306 is used to identify the pipeline stage that has migrated and the new UAV node based on the target topology configuration, maintain the historical key-value cache on the UAV node that has not migrated, and read the boundary intermediate activation tensor from the local circular buffer of the preceding UAV node of the pipeline stage that has migrated and transmit it to the new UAV node, so that the new UAV node performs a local pre-filling calculation and rebuilds the local key-value cache, and resumes the collaborative decoding calculation.

[0049] Please see Figure 4 , Figure 4 This is a schematic diagram of the hardware entity of an electronic device to which an embodiment of the present invention applies. The electronic device provided in this embodiment includes a processor 401, a communication interface 402, a memory 403, and a communication bus 404, wherein the processor 401, the communication interface 402, and the memory 403 communicate with each other via the communication bus 404. Memory 403 is used to store computer programs; When the processor 401 executes the program stored in the memory 403, it implements the steps in the above method embodiments.

[0050] The communication bus 404 mentioned in the above electronic device can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus 404 can be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, it is represented by only one thick line in the figure, but this does not indicate that there is only one bus or one type of bus.

[0051] Communication interface 402 is used for communication between the above-mentioned electronic device and other devices.

[0052] The memory 403 may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage device. Optionally, the memory 403 may also be at least one storage device located remotely from the aforementioned processor.

[0053] The processor 401 mentioned above can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0054] The method provided in this invention can be applied to electronic devices. Specifically, the electronic device can be a desktop computer, a portable computer, a smart mobile terminal, a server, etc. No limitation is made herein; any electronic device that can implement this invention falls within the protection scope of this invention.

[0055] The present invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps provided in the above-described method embodiments.

[0056] For system / electronic device / storage medium embodiments, since they are basically similar to method embodiments, the description is relatively simple. For specific content and beneficial effects, please refer to the description of the method embodiments.

[0057] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. In addition, those skilled in the art can combine and integrate the different embodiments or examples described in this specification.

[0058] Those skilled in the art will understand that embodiments of this application can be provided as methods, apparatus (devices), or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects, all of which are collectively referred to herein as "modules" or "systems." Furthermore, this application can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code. The computer program may be stored / distributed in a suitable medium, provided with or as part of other hardware, or may take other distribution forms, such as via the Internet or other wired or wireless telecommunications systems.

[0059] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (devices), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0060] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0061] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0062] The above description, in conjunction with specific preferred embodiments, provides a further detailed explanation of the present invention. It should not be construed that the specific implementation of the present invention is limited to these descriptions. For those skilled in the art, various simple deductions or substitutions can be made without departing from the concept of the present invention, and all such modifications and substitutions should be considered within the scope of protection of the present invention.

Claims

1. A distributed large language model collaborative reasoning method based on UAV swarms, characterized in that, include: The hardware status information of each drone node in the drone cluster and the wireless link characteristics between each drone node are acquired in real time, and a real-time cluster topology map is constructed based on the hardware status information and the wireless link characteristics. The large language model is vertically divided into multiple pipeline stages, and each pipeline stage is deployed on each UAV node based on the real-time cluster topology graph. By executing the drone nodes at each pipeline stage, the boundary intermediate activation tensor generated by the forward computation is serialized and backed up to the local circular buffer. If the remaining available resources of a drone node are lower than a preset resource threshold, or the link quality between adjacent drone nodes is lower than a preset connectivity threshold, the collaborative inference state of the current pipeline is frozen, and a candidate configuration pool is generated based on the hardware status information, the wireless link characteristics, and the topology configuration executed in the previous moment. Based on the real-time cluster topology graph and the topology configuration executed in the previous moment, the target topology configuration is determined from the candidate configuration pool; Based on the target topology configuration, identify the pipeline stages that have migrated and the new UAV nodes, maintain the historical key-value cache on the UAV nodes that have not migrated, and read the boundary intermediate activation tensor from the local circular buffer of the preceding UAV node of the pipeline stage that has migrated and transmit it to the new UAV node, so that the new UAV node performs a local pre-filling calculation and rebuilds the local key-value cache, and resumes the collaborative decoding calculation.

2. The method according to claim 1, characterized in that, The hardware status information includes at least one of the following: available computing power, remaining power, and idle video memory for each drone node. The wireless link characteristics include at least one of the following: signal-to-noise ratio between each UAV node, link bandwidth, transmission delay, and packet loss status. The construction of a real-time cluster topology map based on the hardware status information and the wireless link characteristics includes: Based on the hardware status information, drone node resource characteristics are generated; based on the wireless link characteristics, inter-drone node connectivity characteristics are generated; and based on the drone node resource characteristics and the inter-drone node connectivity characteristics, the real-time cluster topology is constructed.

3. The method according to claim 1, characterized in that, The process of vertically dividing the large language model into multiple pipeline stages, and deploying each pipeline stage on each UAV node based on the real-time cluster topology graph, includes: According to the network layer order of the large language model, the continuous network layer is divided into multiple pipeline stages; Based on the real-time cluster topology map, the drone nodes that meet the operational requirements of each pipeline stage are determined; Each of the pipeline stages is deployed to the corresponding drone node in the order of the network layers.

4. The method according to claim 1, characterized in that, The step of serializing and backing up the boundary intermediate activation tensor generated by forward computation to a local circular buffer by executing the UAV nodes at each pipeline stage includes: When the drone node executing the current pipeline stage sends the boundary intermediate activation tensor to the drone node corresponding to the next pipeline stage, the boundary intermediate activation tensor is copied. The copied boundary intermediate activation tensor is locally serialized to obtain the serialized boundary intermediate activation tensor. Write the serialized boundary intermediate activation tensor into the local circular buffer of the current drone node.

5. The method according to claim 1, characterized in that, The process of generating a candidate configuration pool based on the hardware status information, the wireless link characteristics, and the topology configuration executed in the previous moment includes: Based on the hardware status information and wireless link characteristics of each UAV node, the physical constraints of each UAV node are determined. Based on the physical constraints, the pipeline stage and UAV node mapping combination that meet the running requirements of the large language model are selected to generate an initial candidate set. Using a preset proxy cost function, the predicted cost of each pipeline stage and UAV node mapping combination in the initial candidate set is calculated. According to the order of predicted overhead from low to high, a preset number of pipeline stages and UAV node mapping combinations are extracted from the initial candidate set and written into the candidate configuration pool as candidate configurations. The topology configuration executed at the previous moment is written into the candidate configuration pool as a historical candidate configuration.

6. The method according to claim 1, characterized in that, The step of determining the target topology configuration from the candidate configuration pool based on the real-time cluster topology graph and the topology configuration executed at the previous moment includes: Global topology features are determined based on the drone node resource characteristics and inter-drone node connectivity characteristics in the real-time cluster topology graph. Generate the embedding vector of the topology configuration executed in the previous time step; The global topological features are fused with the embedding vector to obtain the state context features; Based on the state context features, each candidate configuration in the candidate configuration pool is scored, and the candidate configuration that meets the conditions is determined as the target topology configuration.

7. The method according to claim 1, characterized in that, The identification of the pipeline stages and new drone nodes that have migrated based on the target topology configuration includes: The pipeline stage in which the migration occurs is determined based on the current pipeline topology configuration and the target topology configuration. The drone node in the target topology configuration used to perform the pipeline stage where the migration occurs is identified as the new drone node.

8. The method according to claim 1, characterized in that, The step of having the new UAV node perform a local pre-filling calculation and rebuild the local key-value cache to resume the collaborative decoding calculation includes: The new UAV node performs local pre-filling calculations on the pipeline stage it executes based on the received boundary intermediate activation tensor, generating a new boundary intermediate activation tensor and a new local key-value cache corresponding to the pipeline stage executed by the new UAV node. Use the new local key-value cache as the local key-value cache; The new boundary intermediate activation tensor is transmitted to the subsequent UAV node. After the new boundary intermediate activation tensor is aligned with the input of the subsequent UAV node, the collaborative decoding computation is restored according to the target topology configuration.

9. A distributed large language model collaborative reasoning system based on unmanned aerial vehicle (UAV) swarms, characterized in that, include: The data acquisition module is used to acquire the hardware status information of each drone node in the drone cluster and the wireless link characteristics between each drone node in real time, and to construct a real-time cluster topology map based on the hardware status information and the wireless link characteristics. The distributed inference module is used to vertically divide the large language model into multiple pipeline stages, and deploy each pipeline stage to each UAV node based on the real-time cluster topology graph. The tensor caching module is used to serialize and back up the boundary intermediate activation tensors generated by forward computation to the local circular buffer by executing the drone nodes at each pipeline stage. The reconstruction trigger module is used to freeze the collaborative inference state of the current pipeline when the remaining available resources of the UAV node are lower than a preset resource threshold, or the link quality between adjacent UAV nodes is lower than a preset connectivity threshold, and generate a candidate configuration pool based on the hardware status information, the wireless link characteristics and the topology configuration executed in the previous moment. The target decision determination module is used to determine the target topology configuration from the candidate configuration pool based on the real-time cluster topology graph and the topology configuration executed in the previous moment; The fault-tolerant recovery module is used to identify the pipeline stage that has migrated and the new UAV node based on the target topology configuration, maintain the historical key-value cache on the UAV node that has not migrated, and read the boundary intermediate activation tensor from the local circular buffer of the preceding UAV node of the pipeline stage that has migrated and transmit it to the new UAV node, so that the new UAV node performs a local pre-filling calculation and rebuilds the local key-value cache, and resumes the collaborative decoding calculation.

10. An electronic device, characterized in that, It includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; The memory is used to store computer programs; When the processor executes the computer program stored in the memory, it implements the steps of the distributed large language model collaborative reasoning method based on UAV swarm as described in any one of claims 1-8.

Images