Distributed computing system, method, electronic device, storage medium and program product

Abstract

The present invention discloses a distributed computing system, method, electronic device, storage medium and program product, which relate to the field of artificial intelligence technology. The system includes a first device and a second device with a terminal-side diffusion model locally deployed, and an input device. The input device is used to input the reasoning task of the terminal-side diffusion model into the first device. The first device splits the reasoning task into a first task and a second task, inputs the first task into the terminal-side diffusion model locally deployed on the first device, and inputs the second task into the terminal-side diffusion model locally deployed on the second device. The first device and the second device execute their respective tasks in parallel, and the two computing paths are executed in parallel to improve computing efficiency. This solves the technical problem that the related technologies face the defects of large computing resource consumption, weak parallel reasoning capability, and lack of distributed communication support when running the diffusion model on the terminal side, making it difficult to meet efficiency requirements, and achieves the technical effect of improving the computing efficiency of terminal-side diffusion model reasoning.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a distributed computing system, method, electronic device, storage medium, and program product. Background Art

[0002] Diffusion models are a type of generative model that learns data distributions by adding noise through forward diffusion and gradually removing it through backward diffusion. They offer advantages such as stable training, high-quality generation, and strong controllability, and have been widely used in fields such as image, video, and audio generation, as well as in biomedicine. In on-device inference scenarios, diffusion models face numerous challenges: The limited computing resources of on-device devices make it difficult to support large-scale computing; limited batch processing capabilities typically limit inference to sample-by-sample, impacting efficiency; and a lack of support for distributed communication architectures makes efficient collaborative computing difficult, limiting their application in tasks such as real-time video generation. Summary of the Invention

[0003] The present invention provides a distributed computing system, method, electronic device, storage medium and program product to at least solve the technical problems in the related art that when running a diffusion model on the end side, there are defects such as large computing resource consumption, weak parallel reasoning capability, and lack of distributed communication support, which make it difficult to meet efficiency requirements.

[0004] The present invention provides a distributed computing system, including: a first device and a second device locally deployed with an end-side diffusion model; an input device, used to input an inference task of the end-side diffusion model into the first device; the first device splits the inference task into a first task and a second task, inputs the first task into the end-side diffusion model locally deployed on the first device, and inputs the second task into the end-side diffusion model locally deployed on the second device, the first device and the second device execute their respective tasks in parallel, and the first task and the second task are different types of tasks.

[0005] The present invention also provides a distributed computing method, which is applied to the first device in the above-mentioned distributed computing system, and the first device and the second device call their respective locally deployed end-side diffusion models to iteratively calculate inference tasks. The method includes: obtaining the inference task of the end-side diffusion model; splitting the inference task into a first task and a second task; inputting the first task into the end-side diffusion model locally deployed on the first device, and inputting the second task into the end-side diffusion model locally deployed on the second device, and the first device and the second device execute their respective tasks in parallel, and the first task and the second task are tasks of different types; in the current iterative calculation of the inference task process, identifying the status of the first device and the second device in executing the tasks respectively, and if the status of the first device and the second device in executing the tasks respectively is the execution completion status, then starting the next iterative calculation of the inference task.

[0006] The present invention also provides a distributed computing method, which is applied to the second device in the above-mentioned distributed computing system, and the first device and the second device call their respective locally deployed end-side diffusion models to iteratively calculate the inference task. The method includes: obtaining the second task sent by the first device, wherein the first device obtains the inference task of the end-side diffusion model, splits the inference task into a first task and a second task, and the first device calls the locally deployed end-side diffusion model to execute the first task; calls the locally deployed end-side diffusion model of the second device to execute the second task, and the first device and the second device execute their respective tasks in parallel, and the first task and the second task are tasks of different types; in the current iterative calculation of the inference task process, the status of the second device executing the task is sent to the first device. If the status of the first device and the second device executing the task is respectively in the execution completion state, the next iterative calculation of the inference task is started.

[0007] The present invention also provides an electronic device, comprising: a memory for storing a computer program; and a processor for implementing the steps of any one of the above-mentioned distributed computing methods when executing the computer program.

[0008] The present invention also provides a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, the steps of any of the above-mentioned distributed computing methods are implemented.

[0009] The present invention also provides a computer program product, comprising a computer program, which implements the steps of any of the above-mentioned distributed computing methods when executed by a processor.

[0010] Through the present invention, since the input device is used to input the inference task of the end-side diffusion model into the first device, the first device splits the inference task into a first task and a second task, the first task is input into the end-side diffusion model locally deployed on the first device, and the second task is input into the end-side diffusion model locally deployed on the second device. The first device and the second device execute their respective tasks in parallel, and the two calculation paths are executed in parallel to improve the calculation efficiency. Therefore, it can solve the technical problems that the related technologies have in the event of end-side operation of the diffusion model, such as large consumption of computing resources, weak parallel reasoning capability, and lack of distributed communication support, which make it difficult to meet efficiency requirements, and achieve the technical effect of improving the calculation efficiency of end-side diffusion model reasoning. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] In order to more clearly illustrate the embodiments of the present invention, the following is a brief introduction to the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0012] Figure 1 A schematic diagram of the structure of a distributed computing system provided by an embodiment of the present invention;

[0013] Figure 2 A schematic diagram of a distributed computing method for a diffusion model provided by one embodiment of the present invention;

[0014] Figure 3 A schematic diagram of a communication management module provided in one embodiment of the present invention;

[0015] Figure 4 A schematic diagram of device connections provided as an example of an embodiment of the present invention;

[0016] Figure 5 A schematic flow chart of a distributed computing method applied to a first device in a distributed computing system according to an embodiment of the present invention;

[0017] Figure 6 A schematic flow chart of a distributed computing method applied to a second device in a distributed computing system according to an embodiment of the present invention;

[0018] Figure 7 A schematic structural diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0019] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making any creative efforts shall fall within the scope of protection of the present invention.

[0020] It should be noted that, in the description of the present invention, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or apparatus comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus. The terms "first," "second," etc., in the present invention are used to distinguish similar objects, and are not used to describe a particular order or precedence.

[0021] In order to enable those skilled in the art to better understand the solutions of the present invention, the present invention is further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0022] Figure 1 A structural diagram of a distributed computing system provided by an embodiment of the present invention is shown in FIG. Figure 1As shown, the distributed computing system 10 specifically includes: a first device 101 , a second device 102 and an input device 103 .

[0023] Among them, the first device 101 and the second device 102 have end-side diffusion models deployed locally; the input device 103 is used to input the inference task of the end-side diffusion model into the first device 101; the first device 101 splits the inference task into a first task and a second task, inputs the first task into the end-side diffusion model deployed locally on the first device 101, and inputs the second task into the end-side diffusion model deployed locally on the second device 102. The first device 101 and the second device 102 execute their respective tasks in parallel, and the first task and the second task are tasks of different types.

[0024] Among them, the first device 101 and the second device 102 are edge computing devices, which are mainly responsible for conditional prediction tasks; the input device 103 is the source of providing reasoning task requests to the system, such as a user interface or other trigger mechanism; the end-side diffusion model is an AI (Artificial Intelligence) model deployed on the edge computing device, which is used to perform reasoning tasks; the reasoning task is the process of using a trained model to predict or generate input data, such as text prompts; the first task and the second task are different types of tasks, the first task is a conditional prediction task, and the second task is an unconditional prediction task.

[0025] It can be understood that the distributed computing system 10 of the embodiment of the present invention specifically includes: a first device 101, a second device 102 and an input device 103, wherein the first device 101 and the second device 102 each deploy an end-side diffusion model locally; the input device 103 is used to input the inference task of the end-side diffusion model into the first device 101; after the first device 101 receives the task, it splits it into a first task (conditional prediction task) and a second task (unconditional prediction task), and sends the first task to the end-side diffusion model locally deployed on the first device 101 for processing, and sends the second task to the end-side diffusion model locally deployed on the second device 102 for processing. The first device 101 and the second device 102 execute their respective tasks in parallel, thereby improving processing efficiency and speed.

[0026] In the embodiment of the present invention, the first device 101 includes an encoder, wherein the encoder receives text data of the reasoning task input by the input device 103 and encodes the text data of the reasoning task into a first task and a second task.

[0027] It can be understood that the first device 101 of the embodiment of the present invention includes an encoder, the function of which is to convert the input text data into a task form that can be processed by the system. Specifically, the encoder is responsible for parsing the text data of the reasoning task from the input device and allocating it into conditional prediction tasks and unconditional prediction tasks. In this way, the first device 101 can effectively split the complex reasoning task into two parts for subsequent processing and execution.

[0028] In an embodiment of the present invention, the first device 101 and the second device 102 call their respective locally deployed end-side diffusion models to iteratively calculate the inference task of the end-side diffusion model. During each iterative calculation of the inference task, the first device 101 calls the locally deployed end-side diffusion model to perform the first task, and the second device 102 calls the locally deployed end-side diffusion model to perform the second task.

[0029] Among them, iterative computing means that in the entire reasoning process, tasks need to be processed step by step through multiple repeated steps.

[0030] It can be understood that the first device 101 and the second device 102 of the embodiment of the present invention call their respective locally deployed end-side diffusion models to perform iterative calculations to complete the inference tasks of the end-side diffusion models. During each iterative calculation of the inference task, the first device 101 calls its locally deployed end-side diffusion model to perform the first task, that is, the conditional prediction task; and the second device 102 calls its locally deployed end-side diffusion model to perform the second task, that is, the unconditional prediction task. During the entire inference process, the two devices work in parallel, each focusing on processing different types of tasks, thereby improving the overall computing efficiency and speed.

[0031] In an embodiment of the present invention, during each iterative calculation of an inference task, the first device 101 fuses the respective task execution results of the first device 101 and the second device 102, and sends the fusion results to the end-side diffusion models locally deployed on the first device 101 and the second device 102 respectively.

[0032] It can be understood that in the embodiment of the present invention, during each iterative calculation and inference task, the first device 101 fuses the task execution results of the first device 101 and the second device 102 respectively. Specifically, the first device 101 receives the unconditional prediction result from the second device 102, and combines it with its own conditional prediction result to form a fusion result. Subsequently, the fusion result is sent to the end-side diffusion model locally deployed by the first device 101 and the second device 102 respectively, and the next round of iterative calculation is continued based on this, ensuring that in each round of iteration, the two devices can process based on the latest comprehensive information, thereby effectively improving the computing efficiency and accuracy of the entire system.

[0033] In the embodiment of the present invention, the first device 101 and the second device 102 are provided with a synchronization tool, and the synchronization tool is used to synchronize task triggering and task execution between the first device 101 and the second device 102 .

[0034] Synchronization tools refer to software or mechanisms used to ensure coordinated task triggering and execution between two devices. For example, Pyro5 (Python Remote Objects 5) enables cross-device task scheduling, synchronous triggering, task execution, and message passing.

[0035] It can be understood that the first device 101 and the second device 102 of the embodiment of the present invention are provided with a synchronization tool, such as Pyro5, which is used to synchronize task triggering and task execution between the first device 101 and the second device 102. During each iterative denoising process, the synchronization tool can ensure that the two devices can start their respective tasks in a coordinated and consistent manner, and ensure that the task execution processes of the two are carried out synchronously, thereby achieving efficient task scheduling and result fusion. For example, the first device 101 can call the function on the second device 102 through Pyro5 to execute remote tasks, and ensure message transmission and synchronization between the two, making the entire distributed computing process smoother and more efficient.

[0036] In an embodiment of the present invention, the first device 101 and the second device 102 both include multiple processors and a memory architecture. The multiple processors are connected to the memory architecture, and the multiple processors share memory through the memory architecture.

[0037] The memory architecture refers to the UM (Unified Memory) architecture, which allows multiple processors to share the same memory space.

[0038] It can be understood that the first device 101 and the second device 102 of the embodiment of the present invention both include multiple processors, wherein the multiple processors are connected to the memory architecture. Specifically, the first device 101 and the second device 102 can utilize the UM architecture to realize memory sharing between the multiple processors, which means that in the first device 101 and the second device 102, multiple processors can efficiently access and share data through this memory architecture without the need for an additional data copy process, thereby greatly reducing data transmission delay and improving computing efficiency, so that the multiple processors of each device can work together more smoothly to accelerate the reasoning tasks of the end-side diffusion model.

[0039] In the embodiment of the present invention, the multiple processors include multiple processors of different types.

[0040] Among them, different types of processors include CPU (Central Processing Unit) and GPU (Graphics Processing Unit). The CPU is responsible for performing most computing tasks, usually includes several cores, and can process multiple threads simultaneously. The GPU provides powerful graphics processing and parallel computing capabilities.

[0041] In the embodiment of the present invention, the memory architectures of the first device 101 and the second device 102 are connected, and the first device 101 and the second device 102 share memory through the memory architecture.

[0042] It can be understood that the first device 101 and the second device 102 of the embodiment of the present invention are interconnected through the memory architecture. The two can use the memory architecture to share memory and directly read and write memory data on each other's devices without relying on active CPU intervention, which significantly improves the efficiency of cross-device data transmission, reduces system latency, and provides an efficient communication foundation for multiple edge devices to collaborate to complete complex reasoning tasks.

[0043] In the embodiment of the present invention, data transmission is achieved between the first device 101 and the second device 102 through direct memory access technology.

[0044] Among them, direct memory access technology, namely DMA (Direct Memory Access), is a technology that allows direct data exchange between peripheral devices and main memory, which can significantly improve data transmission efficiency and reduce latency.

[0045] It can be understood that data transfer is achieved between the first device 101 and the second device 102 in the embodiment of the present invention through direct memory access technology. Specifically, during each iteration, the unconditional prediction result generated by the second device 102 needs to be transmitted from the second device 102 to the first device 101 for result fusion. Using DMA technology, these data can be transmitted from the CPU of the second device 102 to the CPU of the first device 101. The entire process achieves efficient data transfer without the need for frequent CPU intervention, thereby significantly reducing data transmission delay and improving overall computing efficiency.

[0046] In an embodiment of the present invention, the time it takes for the first device 101 to complete the first task is the first inference time, and the time it takes for the second device 102 to complete the second task is the second inference time. One of the first device 101 and the second device 102 is provided with a computing optimization module, wherein the computing optimization module obtains the first inference time of the first device 101 and the second inference time of the second device 102, calculates the time difference between the first inference time and the second inference time, and adjusts the operating power consumption of at least one of the first device 101 and the second device 102 according to the time difference.

[0047] The difference between the first reasoning duration and the second reasoning duration is calculated by subtracting the second reasoning duration from the first reasoning duration.

[0048] It can be understood that, in an embodiment of the present invention, the time it takes for the first device 101 to complete the first task is set as the first inference time, and the time it takes for the second device 102 to complete the second task is set as the second inference time, and a computing optimization module is set in any one of the first device 101 and the second device 102. The computing optimization module is responsible for obtaining the first inference time of the first device 101 and the second inference time of the second device 102, and calculating the difference between the two inference times. The operating power consumption of at least one of the first device 101 and the second device 102 is dynamically adjusted according to the time difference, so that the computing time of the two devices is closer, thereby achieving the best balance between efficiency and power consumption, and ensuring that during the entire distributed computing process, the system can flexibly adjust the working status of each device according to actual needs to achieve optimal performance. Specifically, the method in which the computing optimization module dynamically adjusts the operating power consumption of at least one of the first device 101 and the second device 102 according to the time difference is as follows:

[0049] In an embodiment of the present invention, the operating parameters of at least one device among the first device 101 and the second device 102 are modulated according to the duration difference, including: if the absolute value of the duration difference is less than the adjustment threshold, the operating power consumption of the first device 101 and the second device 102 is kept unchanged, or the operating power consumption of the first device 101 and the second device 102 is adjusted to the maximum power consumption; if the absolute value of the duration difference is greater than or equal to the adjustment threshold, if the duration difference is positive, the operating power consumption of the second device 102 is reduced, and if the duration difference is negative, the operating power consumption of the first device 101 is reduced.

[0050] The adjustment threshold is set according to actual needs and is not specifically limited here.

[0051] It can be understood that the embodiment of the present invention modulates the operating parameters of at least one of the first device 101 and the second device 102 according to the duration difference between the first inference duration of the first device 101 and the second inference duration of the second device 102. Specifically: if the absolute value of the duration difference is less than the adjustment threshold, the operating power consumption of the first device 101 and the second device 102 is kept unchanged, or the operating power consumption of both is adjusted to the maximum power consumption to ensure the highest computing efficiency, wherein, since the first device 101 and the second device 102 are the same edge computing devices, the maximum power consumption of the two is basically the same; if the absolute value of the duration difference is greater than or equal to the adjustment threshold, if the duration difference is positive, that is, the inference time of the first device 101 If the time difference is negative, that is, the inference time of the second device 102 is longer, the operating power consumption of the second device 102 is reduced, so that the inference time of the second device 102 is prolonged, and the time difference between the first device 101 and the second device 102 is shortened. In this way, the computing optimization module can dynamically adjust the operating power consumption of the first device 101 and the second device 102, so that the computing time of the two devices is more balanced, thereby achieving the best balance between performance and energy consumption, improving the overall efficiency of the system, and ensuring that the power consumption of the devices is in the optimal state under different load conditions.

[0052] According to the distributed computing system proposed in an embodiment of the present invention, an input device is used to input the inference task of the end-side diffusion model into the first device. The first device splits the inference task into a first task and a second task. The first task is input into the end-side diffusion model locally deployed on the first device, and the second task is input into the end-side diffusion model locally deployed on the second device. The first device and the second device execute their respective tasks in parallel, and the two computing paths are executed in parallel, thereby improving computing efficiency and achieving the technical effect of improving the computing efficiency of end-side diffusion model inference.

[0053] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus the necessary general hardware platform, and of course it can also be implemented by hardware, but in many cases the former is a better implementation method.

[0054] The distributed computing system is further described below through a specific embodiment.

[0055] This embodiment mainly designs the model splitting module, remote scheduling module, communication management module, and load optimization module. Figure 2The model splitting module deploys part of the diffusion model reasoning process on the first device and part of the reasoning process on the second device, and the two calculation paths are executed in parallel to improve the calculation efficiency; the remote scheduling module uses the Pyro5 tool to achieve cross-device synchronous triggering and task execution. Pyro5 is responsible for task scheduling, achieving synchronous triggering, task execution, message passing and message synchronization of tasks in the two devices; the communication management module uses a unified memory architecture combined with DMA channels to support CPU CPU, GPU GPU and CPU across devices GPU memory is shared, and after iteration, the result is decoded by VAE (Variational Auto Encoder), which greatly reduces data transmission latency. The load optimization module introduces a dynamic load balancing algorithm based on the difference in inference time between the two devices, achieving optimal power consumption while ensuring the lowest inference latency. The specific solution is as follows:

[0056] 1. Model Splitting Module

[0057] The common design of a diffusion model includes two computational paths: conditional prediction and unconditional prediction. Conditional prediction generates content and prompts by inputting text or video prompts. Unconditional prediction does not have clear conditional inputs and achieves diversity in model output results by introducing randomness. By combining these two modes, the goals of controlling generation quality and diversity are achieved. In the above calculation process, the two independent computational paths of conditional prediction and unconditional prediction can be assigned to two different computing devices for calculation. The conditional prediction is assigned to the first edge device for execution, and the unconditional prediction is assigned to the second device for execution. Processing these two computational paths separately can effectively utilize multi-device resources and reduce the load on a single device.

[0058] 2. Remote Scheduling Module

[0059] When performing on-device diffusion model inference, an iterative denoising cycle is required. During each denoising cycle, two devices each perform a conditional prediction and an unconditional prediction. The results generated by the two devices are combined before the next denoising cycle. This process requires consideration of parallel computing scheduling and communication, specifically the synchronous triggering, execution, message passing, and synchronization of tasks on the two devices. Cross-device synchronous triggering and task execution are achieved using the Pyro5 tool. Pyro5 is responsible for task scheduling, allowing the first device to call functions on the second device to execute tasks.

[0060] 3. Communication Management Module

[0061] During each denoising process, after the two end-side devices execute inference and generate results, the results of the second device need to be sent to the first device for result synthesis, which involves big data communication. The unified memory UM architecture in a single device realizes memory sharing between the CPU and GPU. For the DMA memory sharing results of the two devices in this embodiment, due to its unified memory characteristics, the CPU and GPU memory sharing on the two devices can be realized. Figure 3 As shown, the first device and the second device share memory through DMA, and the first device and the second device respectively implement UM unified memory in the shared memory area, that is, the GPU and CPU in the first device can directly achieve zero copy without going through the memory copy process, and this conclusion also applies to the second device. Based on the above results, the first device and the second device can realize shared memory between CPUs, shared memory between GPUs, and memory sharing between the CPU of the first device and the GPU of the second device, realizing zero copy of GPU memory between the two devices. After each denoising result is generated, the result of the second device realizes data from the GPU through UM+DMA technology. CPU CPU GPU data transmission enables efficient data communication.

[0062] 4. Computation Optimization Module

[0063] During the parallel computation of a diffusion model, different computing devices perform varying amounts of model computation, resulting in different inference times. This results in waiting between devices as they synchronize. Specifically, after the first device completes its computation, it must wait for the second device to complete its computation before it can proceed with the next parallel computation (or vice versa). Since end-side devices are sensitive to power consumption, a dynamic load balancing method has been designed to dynamically balance power consumption and time. Assume that the inference time of the first device is T(A) and the inference time of the second device is T(B). The time difference between the two devices is T, where T = T(A) - T(B). A positive value for T indicates that the second device is taking longer, while a negative value for T indicates that the second device is taking longer. Therefore, a threshold W is set. When the absolute value of T is greater than or equal to W, the time difference exceeds the threshold. The power consumption of the faster device is automatically reduced by one level. This lowers the power consumption, bringing the computation times of the two devices closer together, achieving computational synchronization. When the time difference is less than the threshold, both devices operate at maximum operating mode, achieving a balance between efficiency and power consumption.

[0064] Specifically, this embodiment will be further described in conjunction with the inference process of the generative diffusion model Cosmos-1.0-Diffusion-7B. Cosmos-1.0-Diffusion-7B is an open-source AI model capable of text-to-video and video-to-video generation. It uses an iterative denoising process to progressively generate video frames from random noise. The denoising step involves 35 iterations, each of which involves both conditional and unconditional predictions. The results are then fused to gradually remove noise.

[0065] In this embodiment, the computing platform used is mainly composed of two ARM (Advanced RISC Machine, a processor architecture based on reduced instruction set RISC) architecture edge computing devices, which are connected in two ways: PCIE DMA (Peripheral Component Interconnect Express DMA, peripheral component interconnect high-speed direct memory access) and Ethernet. Figure 4 As shown, the PCIE DMA mode is mainly connected through the PCIE (Peripheral Component Interconnect Express) bus, where the first device is set to EP mode (EndPoint, in this mode, the first device acts as a terminal device on the PCIE bus, usually used to connect to an upstream device, usually as a data initiator or receiver, rather than routing data to other devices), and the second device is set to RP mode (Root Port, in this mode, the second device plays the role of the root port in the PCIE topology, responsible for managing communication with the EndPoint device and connecting these devices to the system's main memory and other resources). The DMA mode is mainly used for data communication; the Ethernet connection works as a remote task call. The specific information of the device is shown in Table 1, where Table 1 is a device specific information table.

[0066] Table 1

[0067]

[0068] The implementation process of this embodiment is as follows:

[0069] 1. Hardware Preparation and Connection: Prepare two edge computing devices, named Device 1 and Device 2, and connect them via Ethernet. The IP address of Device 1 is 192.168.1.1, and the IP address of Device 2 is 192.168.1.2, respectively, to achieve Ethernet intercommunication. At the same time, use PCIE to connect the two devices for PCIE communication. The first device is in EP mode, and the second device is in RP mode.

[0070] 2. Model Splitting: Based on actual business needs, the conditional prediction tasks in each denoising process are executed locally on the first device, while the unconditional prediction tasks are remotely called and executed on the second auxiliary device. A Pyro5 service is launched on the second device, exposing the denoising function interface; this interface is called via Ethernet communication on the first device. During the denoising process, the first device uses a Thread Pool Executor (max_workers=2), a thread pool executor (with a maximum number of worker threads of 2), to trigger local conditional predictions and remote unconditional predictions in parallel, with separate timings.

[0071] III. Communication Management: The model denoising process is repeated 35 times. After each denoising process, both the first and second devices generate intermediate results. The results from the second device need to be sent to the first device for result fusion. Due to the large amount of generated data, an efficient data transmission mechanism is required for inter-device communication. This solution employs an efficient data communication link to transmit the results generated by the second device to the first device via a GPU(B)-CPU(B)-CPU(A)-GPU(A) link. This link involves two key technologies: a unified memory (UM) architecture and DMA memory sharing. The unified memory (UM) architecture enables data communication links between GPU(B) and CPU(B) and between CPU(A) and GPU(A), while DMA memory sharing enables efficient cross-device data transfer between CPU(B) and CPU(A). The data link process is as follows: the results generated by the second device are stored in the unified memory of device B. Then, they are transferred from the unified memory of device B to the unified memory of device A via cross-device DMA memory sharing, achieving efficient data transfer. After data transfer, the contributions of conditional and unconditional predictions are dynamically adjusted using guidance coefficients to achieve data fusion.

[0072] IV. Test Results: The total time for model inference on a single primary device was 130 minutes, with an average of 210 seconds for each conditional and unconditional prediction. The total time for concurrent inference on both the primary and secondary devices was 65 minutes, with an average of 103 seconds for the primary device's conditional prediction and 104 seconds for the secondary device's conditional prediction. Compared to computation on a single edge computing device, the overall generation time was reduced from approximately 130 minutes to approximately 65 minutes, a speed increase of approximately 2x. The latency of a single denoising pass was also halved, achieving a nearly 2x acceleration.

[0073] 5. Output consistency verification: In both parallel and single edge computing device configurations, by setting the same random seed, the final results are exactly the same, ensuring that the parallel computing split does not affect the randomness and output quality of the model.

[0074] Figure 5 A flow chart of a distributed computing method provided by an embodiment of the present invention is provided. The method is applied to the first device in the above-mentioned distributed computing system. The first device and the second device call their respective locally deployed end-side diffusion models to iteratively calculate inference tasks, such as Figure 5 As shown, the method includes:

[0075] In step S201 , an inference task of a device-side diffusion model is obtained.

[0076] It is understandable that the embodiment of the present invention first receives an inference request initiated by a user or a system from an input device or other task scheduling module, and parses it into a specific inference task that can be executed on the end-side diffusion model.

[0077] In step S202 , the reasoning task is split into a first task and a second task.

[0078] The split reasoning task is achieved by parsing through the encoder in the first device and following the standard reasoning process of the diffusion model.

[0079] It can be understood that the embodiment of the present invention sends the input reasoning task to the encoder in the first device for parsing, and splits it into two sub-tasks executed in parallel according to the standard reasoning process of the diffusion model: a first task for generating target features according to the input conditions and a second task for generating diversified potential features. The first task and the second task provide the basis for subsequent iterative calculations and result fusion.

[0080] In step S203 , the first task is input into the end-side diffusion model locally deployed on the first device, and the second task is input into the end-side diffusion model locally deployed on the second device. The first device and the second device execute their respective tasks in parallel. The first task and the second task are different types of tasks.

[0081] Parallel execution means that two devices independently process their assigned tasks simultaneously to improve overall reasoning efficiency.

[0082] It can be understood that the embodiment of the present invention inputs the first task into the end-side diffusion model deployed locally on the first device, and inputs the second task into the end-side diffusion model deployed locally on the second device. Since these two tasks belong to different types of reasoning tasks, they are executed independently and in parallel by the first device and the second device respectively. Through this task splitting and device division of labor mechanism, the advantages of distributed computing are fully utilized, and the reasoning efficiency is improved while ensuring the generation quality.

[0083] In step S204, during the current iterative calculation of the inference task, the status of the first device and the second device in executing the task are identified. If the status of the first device and the second device in executing the task is a completed state, the next iterative calculation of the inference task is started.

[0084] It can be understood that in the current denoising iteration process, the embodiment of the present invention will continuously monitor the execution status of the respective tasks of the first device and the second device. Once it is detected that both devices have completed the tasks assigned in this round, the next iterative calculation of the inference task will be automatically started, and the next denoising and result fusion operations will be continued, thereby advancing the generation process of the entire diffusion model to the final output result.

[0085] According to the distributed computing method applied to the first device in a distributed computing system proposed in an embodiment of the present invention, an input device is used to input the inference task of the end-side diffusion model into the first device. The first device splits the inference task into a first task and a second task. The first task is input into the end-side diffusion model locally deployed on the first device, and the second task is input into the end-side diffusion model locally deployed on the second device. The first device and the second device execute their respective tasks in parallel, and the two computing paths are executed in parallel, thereby improving computing efficiency and achieving the technical effect of improving the computing efficiency of end-side diffusion model inference.

[0086] For descriptions of features in the embodiments corresponding to the distributed computing method applied to the first device in the distributed computing system, reference may be made to the relevant descriptions of the embodiments corresponding to the distributed computing system, which will not be detailed here.

[0087] Figure 6 A flow chart of a distributed computing method provided by an embodiment of the present invention is provided. The method is applied to the second device in the above-mentioned distributed computing system. The first device and the second device call their respective locally deployed end-side diffusion models to iteratively calculate inference tasks, such as Figure 6 As shown, the method includes:

[0088] In step S301 , a second task sent by a first device is obtained. The first device obtains an inference task of a device-side diffusion model, splits the inference task into a first task and a second task, and calls the locally deployed device-side diffusion model to execute the first task.

[0089] It can be understood that the embodiment of the present invention first obtains the second task sent from the first device. This process originates from the first device obtaining the complete inference task from the input source, and then splitting the task into the first task and the second task through its internal encoder. The second task is allocated and sent to the second device for parallel processing. Through task splitting and distributed execution mechanism, efficient collaborative reasoning of the diffusion model among multiple edge devices is achieved.

[0090] In step S302 , the terminal-side diffusion model locally deployed on the second device is called to execute the second task. The first device and the second device execute their respective tasks in parallel. The first task and the second task are different types of tasks.

[0091] It can be understood that the embodiment of the present invention calls the end-side diffusion model locally deployed on the second device to perform the second task. At the same time, the first device also performs the first task on its locally deployed end-side diffusion model. The two are executed in parallel, realizing distributed collaborative reasoning of the diffusion model on multiple edge devices, thereby improving reasoning efficiency while ensuring generation quality.

[0092] In step S303, during the current iterative calculation of the inference task, the status of the second device executing the task is sent to the first device. If the status of the first device and the second device executing the task is respectively in the execution completion state, the next iterative calculation of the inference task is started.

[0093] The status of the second device executing the task is sent to the first device through a method such as PCIe DMA or a network.

[0094] It can be understood that the second device of the embodiment of the present invention feeds back its task execution status to the first device through the communication mechanism, and the system continuously monitors the task completion status of the two devices. Once it is confirmed that the first device and the second device are both in the state of completing their respective task execution, it is determined that all tasks in this round of iterative tasks are completed, and the next iterative calculation of the reasoning task is immediately triggered, and the generation process of the diffusion model is continued until the final result is output.

[0095] According to the distributed computing method applied to the second device in a distributed computing system proposed in an embodiment of the present invention, an input device is used to input the inference task of the end-side diffusion model into the first device. The first device splits the inference task into a first task and a second task. The first task is input into the end-side diffusion model locally deployed on the first device, and the second task is input into the end-side diffusion model locally deployed on the second device. The first device and the second device execute their respective tasks in parallel, and the two computing paths are executed in parallel, thereby improving computing efficiency and achieving the technical effect of improving the computing efficiency of end-side diffusion model inference.

[0096] For descriptions of features in the embodiments corresponding to the distributed computing method applied to the second device in the distributed computing system, reference may be made to the relevant descriptions of the embodiments corresponding to the distributed computing system, which will not be detailed here.

[0097] An embodiment of the present invention further provides an electronic device, such as Figure 7As shown, it includes a memory 401 and a processor 402, wherein the memory 401 stores a computer program, and the processor 402 is configured to run the computer program to execute the steps in any of the above-mentioned distributed computing method embodiments.

[0098] An embodiment of the present invention further provides a computer-readable storage medium, in which a computer program is stored. The computer program is configured to execute the steps of any of the above-mentioned distributed computing method embodiments when running.

[0099] In an exemplary embodiment, the computer-readable storage medium may include, but is not limited to, various media that can store computer programs, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk, or an optical disk.

[0100] An embodiment of the present invention further provides a computer program product, which includes a computer program. When the computer program is executed by a processor, the steps in any one of the above-mentioned distributed computing method embodiments are implemented.

[0101] An embodiment of the present invention also provides another computer program product, including a non-volatile computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the steps in any of the above-mentioned distributed computing method embodiments.

[0102] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the above description has generally described the components and steps of each example according to their functions. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present invention.

[0103] The above is a detailed introduction to a distributed computing system, method, electronic device, storage medium and program product provided by the present invention. Specific examples are used herein to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. It should be pointed out that for ordinary technicians in this technical field, without departing from the principles of the present invention, the present invention can also be improved and modified in a number of ways, and these improvements and modifications also fall within the scope of protection of the claims of the present invention.

Claims

1. A distributed computing system, characterized in that: include: A first device and a second device with a device-side diffusion model locally deployed; an input device, configured to input the inference task of the end-side diffusion model into the first device; The first device splits the inference task into a first task and a second task, inputs the first task into the end-side diffusion model locally deployed on the first device, and inputs the second task into the end-side diffusion model locally deployed on the second device. The first device and the second device execute their respective tasks in parallel. During each iterative calculation of the inference task, the first device fuses the respective task execution results of the first device and the second device, sends the fusion results to the end-side diffusion models locally deployed on the first device and the second device, respectively, and continues the next round of iterative calculation based on the fusion results. The first task and the second task are different types of tasks.

2. The distributed computing system according to claim 1, wherein: The first device includes an encoder, wherein the encoder receives text data of the reasoning task input by the input device and encodes the text data of the reasoning task into the first task and the second task.

3. The distributed computing system according to claim 1, wherein: The first device and the second device call their respective locally deployed end-side diffusion models to iteratively calculate an inference task of the end-side diffusion model. During each iterative calculation of the inference task, the first device calls the locally deployed end-side diffusion model to perform the first task, and the second device calls the locally deployed end-side diffusion model to perform the second task.

4. The distributed computing system according to claim 1, wherein: The first device and the second device are provided with a synchronization tool, and the synchronization tool is used to synchronize task triggering and task execution between the first device and the second device.

5. The distributed computing system according to claim 1, wherein: The first device and the second device both include multiple processors and a memory architecture, the multiple processors are connected to the memory architecture, and the multiple processors share memory through the memory architecture.

6. The distributed computing system according to claim 5, wherein: The memory architecture between the first device and the second device is connected, and the first device and the second device share memory through the memory architecture.

7. The distributed computing system according to claim 6, wherein: Data is transferred between the first device and the second device via direct memory access technology.

8. The distributed computing system according to claim 5, wherein: The plurality of processors include a plurality of processors of different types.

9. The distributed computing system according to claim 1, wherein: The time it takes for the first device to complete the first task is the first inference time, and the time it takes for the second device to complete the second task is the second inference time. One of the first device and the second device is provided with a computing optimization module, wherein: The computing optimization module obtains a first inference duration of the first device and a second inference duration of the second device, calculates a duration difference between the first inference duration and the second inference duration, and adjusts the operating power consumption of at least one of the first device and the second device according to the duration difference.

10. A distributed computing method, characterized in that: The method is applied to a first device in a distributed computing system according to any one of claims 1 to 9, wherein the first device and the second device call the locally deployed end-side diffusion model to iteratively calculate the inference task. The method includes: Obtaining an inference task of the end-side diffusion model; Splitting the reasoning task into the first task and the second task; Inputting the first task into the device-side diffusion model locally deployed on the first device, and inputting the second task into the device-side diffusion model locally deployed on the second device, wherein the first device and the second device execute their respective tasks in parallel, and the first task and the second task are different types of tasks; During the current iterative calculation of the inference task, the status of the task execution of the first device and the second device is identified. If the status of the task execution of the first device and the second device is a completed state, the first device fuses the respective task execution results of the first device and the second device, and sends the fusion results to the end-side diffusion models locally deployed on the first device and the second device respectively. Based on the fusion results, the next iterative calculation of the inference task is started.

11. A distributed computing method, characterized in that: The method is applied to the second device in the distributed computing system according to any one of claims 1 to 9, wherein the first device and the second device call the locally deployed end-side diffusion model to iteratively calculate the inference task. The method includes: Obtaining a second task sent by a first device, wherein the first device obtains an inference task of the device-side diffusion model, splits the inference task into the first task and the second task, and the first device calls the locally deployed device-side diffusion model to execute the first task; Invoking the device-side diffusion model locally deployed on the second device to execute the second task, whereby the first device and the second device execute their respective tasks in parallel, and the first task and the second task are different types of tasks; During the current iterative calculation of the inference task, the status of the task execution by the second device is sent to the first device. If the status of the task execution by the first device and the second device is respectively a completed state, the first device fuses the respective task execution results of the first device and the second device, and sends the fusion results to the end-side diffusion models locally deployed by the first device and the second device respectively. Based on the fusion results, the next iterative calculation of the inference task is started.

12. An electronic device, characterized in that: include: Memory for storing computer programs; A processor, configured to implement the steps of the distributed computing method according to any one of claims 10 to 11 when executing the computer program.

13. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, wherein when the computer program is executed by a processor, the steps of the distributed computing method according to any one of claims 10 to 11 are implemented.

14. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the distributed computing method according to any one of claims 10 to 11 are implemented.

Images