Distributed computing system and method, electronic device, storage medium and program product

Abstract

The invention discloses a distributed computing system and method, an electronic device, a storage medium and a program product, and relates to the technical field of artificial intelligence, the distributed computing system comprises a first device and a second device which are locally deployed with an end side model, and further comprises an input device, and the input device is used for inputting a reasoning task of the end side model into the first device; a first device divides a reasoning task into a first task and a second task, the first task is input into an end side model locally deployed by the first device, the second task is input into an end side model locally deployed by the second device, the first device and the second device execute respective tasks in parallel, and two calculation paths execute in parallel, so that the calculation efficiency is improved; according to the method, the technical problem that the efficiency requirement is difficult to meet due to the defects of high computing resource consumption, weak parallel reasoning capability, lack of distributed communication support and the like when the diffusion model is operated on the end side in the prior art is solved, and the technical effect of improving the reasoning computing efficiency of the end-side diffusion model is achieved.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular, to a distributed computing system, method, electronic device, storage medium, and program product. Background Art

[0002] Diffusion models are a class of generative models that learn data distributions by adding noise in the forward diffusion and gradually denoising in the reverse diffusion. They have advantages such as stable training, high generation quality, and strong controllability, and have been widely used in multiple fields such as image, video, audio generation, and biomedicine. In the end-side inference scenario, diffusion models face many challenges: end-side devices have limited computing resources and are difficult to support their large-scale computing requirements; the batch processing ability is limited, and usually only sample-by-sample inference can be performed, affecting efficiency; at the same time, the lack of support for a distributed communication architecture makes it difficult to achieve efficient collaborative computing, restricting their application effects in tasks such as real-time video generation. Summary of the Invention

[0003] The present invention provides a distributed computing system, method, electronic device, storage medium, and program product to at least solve the technical problems in the related art that when running a diffusion model on the end side, there are defects such as large computing resource consumption, weak parallel inference ability, and lack of distributed communication support, and it is difficult to meet the efficiency requirements.

[0004] The present invention provides a distributed computing system, including: a first device and a second device with an end-side model locally deployed; an input device for inputting the inference task of the end-side model to the first device; the first device splits the inference task into a first task and a second task, inputs the first task to the end-side model locally deployed on the first device, and inputs the second task to the end-side model locally deployed on the second device. The first device and the second device execute their respective tasks in parallel, and the first task and the second task are different types of tasks.

[0005] The present invention also provides a distributed computing method, which is applied to the first device in the above-mentioned distributed computing system. The first device and the second device call the end-side models locally deployed on them to iteratively calculate the inference task. The method includes: obtaining the inference task of the end-side model; splitting the inference task into a first task and a second task; inputting the first task to the end-side model locally deployed on the first device, and inputting the second task to the end-side model locally deployed on the second device. The first device and the second device execute their respective tasks in parallel, and the first task and the second task are different types of tasks; during the current iterative calculation of the inference task, identify the states of the first device and the second device when executing their respective tasks. If the states of the first device and the second device when executing their respective tasks are the execution completion state, then start the next iterative calculation of the inference task.

[0006] The present invention also provides a distributed computing method, which is applied to a second device in the above-mentioned distributed computing system. The first device and the second device call the end-side models deployed locally to iteratively calculate and infer tasks. The method includes: obtaining a second task sent by the first device, where the first device obtains the inference task of the end-side model, splits the inference task into a first task and a second task, and the first device calls the end-side model deployed locally to execute the first task; calling the end-side model deployed locally on the second device to execute the second task, and the first device and the second device execute their respective tasks in parallel, and the first task and the second task are tasks of different types; during the current iterative calculation and inference task, sending the status of the task executed by the second device to the first device, and if the status of the tasks executed by the first device and the second device is the execution completion status, then start the next iterative calculation of the inference task.

[0007] The present invention also provides an electronic device, including: a memory for storing a computer program; a processor for implementing the steps of any of the above-mentioned distributed computing methods when executing the computer program.

[0008] The present invention also provides a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, the steps of any of the above-mentioned distributed computing methods are implemented.

[0009] The present invention also provides a computer program product, including a computer program, and when the computer program is executed by a processor, the steps of any of the above-mentioned distributed computing methods are implemented.

[0010] Through the present invention, since the inference task of the end-side model is input into the first device by using an input device, the first device splits the inference task into a first task and a second task, inputs the first task into the end-side model deployed locally on the first device, and inputs the second task into the end-side model deployed locally on the second device. The first device and the second device execute their respective tasks in parallel, and the two calculation paths execute in parallel, improving the calculation efficiency. Therefore, the technical problems existing in the related art, such as large consumption of computing resources, weak parallel inference ability, and lack of distributed communication support when running a diffusion model on the edge side, and it is difficult to meet the efficiency requirements, can be solved, and the technical effect of improving the calculation efficiency of the edge-side diffusion model inference can be achieved. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] In order to more clearly illustrate the embodiments of the present invention, the drawings required to be used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0012] Figure 1Schematic structural diagram of a distributed computing system provided by an embodiment of the present invention; Figure 2 Schematic diagram of a diffusion model distributed computing method provided by an embodiment of the present invention; Figure 3 Schematic diagram of a communication management module provided by an embodiment of the present invention; Figure 4 Schematic diagram of the connection of example devices provided by an embodiment of the present invention; Figure 5 Schematic flow diagram of a distributed computing method for a first device applied to a distributed computing system provided by an embodiment of the present invention; Figure 6 Schematic flow diagram of a distributed computing method for a second device applied to a distributed computing system provided by an embodiment of the present invention; Figure 7 Schematic structural diagram of an electronic device provided by an embodiment of the present invention. Detailed implementation manners

[0013] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the protection scope of the present invention.

[0014] It should be noted that in the description of the present invention, the terms "include", "comprise" or any other variant thereof are intended to cover a non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements, but also includes other elements not expressly listed, or further includes elements inherent to such process, method, article or device. The terms "first", "second", etc. in the present invention are used to distinguish similar objects, rather than to describe a specific order or sequence.

[0015] In order to enable those skilled in the art of the present technology to better understand the solution of the present invention, the present invention will be further described in detail below in conjunction with the accompanying drawings and specific implementation manners.

[0016] Figure 1 Schematic structural diagram of a distributed computing system provided by an embodiment of the present invention, as Figure 1 shown, the distributed computing system 10 specifically includes: a first device 101, a second device 102, and an input device 103.

[0017] Among them, the first device 101 and the second device 102 deploy edge models locally; the input device 103 is used to input the inference task of the edge model into the first device 101; the first device 101 splits the inference task into a first task and a second task, inputs the first task into the edge model deployed locally on the first device 101, and inputs the second task into the edge model deployed locally on the second device 102. The first device 101 and the second device 102 execute their respective tasks in parallel, and the first task and the second task are tasks of different types.

[0018] Among them, the first device 101 and the second device 102 are edge computing devices, mainly responsible for conditional prediction tasks; the input device 103 is the source of the inference task request provided to the system, such as a user interface or other triggering mechanisms; the edge model is an AI (Artificial Intelligence) model deployed on the edge computing device, used to execute inference tasks; the inference task is the process of using the trained model to predict or generate input data, such as text prompts, etc.; the first task and the second task are tasks of different types, the first task is a conditional prediction task, and the second task is an unconditional prediction task.

[0019] It can be understood that the distributed computing system 10 of the embodiments of the present invention specifically includes: a first device 101, a second device 102, and an input device 103. Among them, the first device 101 and the second device 102 each deploy an edge model locally; the input device 103 is used to input the inference task of the edge model into the first device 101; after receiving the task, the first device 101 splits it into a first task (conditional prediction task) and a second task (unconditional prediction task), and hands the first task to the edge model deployed locally on the first device 101 for processing, and sends the second task to the edge model deployed locally on the second device 102 for processing. The first device 101 and the second device 102 execute their respective tasks in parallel, thereby improving the processing efficiency and speed.

[0020] In the embodiments of the present invention, the first device 101 includes an encoder. Among them, the encoder receives the text data of the inference task input by the input device 103 and encodes the text data of the inference task into a first task and a second task.

[0021] It can be understood that the first device 101 of the embodiments of the present invention includes an encoder. The function of this encoder is to convert the input text data into a task form that the system can process. Specifically, the encoder is responsible for parsing and allocating the text data of the inference task from the input device into a conditional prediction task and an unconditional prediction task. In this way, the first device 101 can effectively split the complex inference task into two parts for subsequent processing and execution.

[0022] In an embodiment of the present invention, the first device 101 and the second device 102 call the locally deployed edge models to iteratively calculate the inference task of the edge model. During each iterative calculation of the inference task, the first device 101 calls the locally deployed edge model to execute the first task, and the second device 102 calls the locally deployed edge model to execute the second task.

[0023] Among them, iterative calculation means that during the entire inference process, the task needs to be gradually processed through multiple repeated steps.

[0024] It can be understood that the first device 101 and the second device 102 in the embodiment of the present invention call the locally deployed edge models respectively for iterative calculation to complete the inference task of the edge model. During each iterative calculation of the inference task, the first device 101 calls the locally deployed edge model to execute the first task, that is, the conditional prediction task; while the second device 102 calls the locally deployed edge model to execute the second task, that is, the unconditional prediction task. During the entire inference process, the two devices work in parallel, each focusing on processing different types of tasks, thereby improving the overall computing efficiency and speed.

[0025] In an embodiment of the present invention, during each iterative calculation of the inference task, the first device 101 fuses the respective task execution results of the first device 101 and the second device 102, and sends the fusion result to the edge models locally deployed on the first device 101 and the second device 102 respectively.

[0026] It can be understood that in the embodiment of the present invention, during each iterative calculation of the inference task, the first device 101 fuses the respective task execution results of the first device 101 and the second device 102. Specifically, the first device 101 receives the unconditional prediction result from the second device 102, combines it with its own conditional prediction result to form a fusion result, and then sends the fusion result to the edge models locally deployed on the first device 101 and the second device 102 respectively, and continues the next round of iterative calculation based on this, ensuring that in each round of iteration, the two devices can process based on the latest comprehensive information, thereby effectively improving the computing efficiency and accuracy of the entire system.

[0027] In an embodiment of the present invention, the first device 101 and the second device 102 are provided with a synchronization tool, and the synchronization tool is used to synchronize the task triggering and task execution between the first device 101 and the second device 102.

[0028] Among them, the synchronization tool refers to software or a mechanism used to ensure that task triggering and execution between two devices can be coordinated, such as the Pyro5 (Python Remote Objects 5) tool, which can achieve cross-device task scheduling, synchronous triggering, task execution, and message passing.

[0029] It can be understood that the first device 101 and the second device 102 in the embodiments of the present invention are provided with a synchronization tool, such as Pyro5, which is used to synchronize task triggering and task execution between the first device 101 and the second device 102. During each iterative denoising process, the synchronization tool can ensure that the two devices can start their respective tasks in a coordinated manner, and ensure that the task execution processes of the two are synchronized, so as to achieve efficient task scheduling and result fusion. For example, the first device 101 can call a function on the second device 102 through Pyro5 to execute a remote task, and ensure message passing and synchronization between the two, making the entire distributed computing process more smooth and efficient.

[0030] In the embodiments of the present invention, both the first device 101 and the second device 102 include multiple processors and a memory architecture. The multiple processors are connected to the memory architecture, and the multiple processors share memory through the memory architecture.

[0031] Among them, the memory architecture refers to the UM (Unified Memory) architecture, and the UM allows multiple processors to share the same memory space.

[0032] It can be understood that both the first device 101 and the second device 102 in the embodiments of the present invention include multiple processors, and among them, the multiple processors are connected to the memory architecture. Specifically, the first device 101 and the second device 102 can utilize the UM architecture to achieve memory sharing between multiple processors, which means that in the first device 101 and the second device 102, multiple processors can efficiently access and share data through this memory architecture without an additional data copy process, thereby greatly reducing data transmission latency and improving computing efficiency, enabling the multiple processors of each device to work together more smoothly to accelerate the inference task of the edge-side model.

[0033] In the embodiments of the present invention, the multiple processors include multiple processors of different types.

[0034] Among them, processors of different types include the CPU (Central Processing Unit) and the GPU (Graphics Processing Unit). The CPU is responsible for executing most computing tasks and usually includes several cores that can process multiple threads simultaneously. The GPU provides powerful graphics processing and parallel computing capabilities.

[0035] In the embodiment of the present invention, the memory architectures of the first device 101 and the second device 102 are connected, and memory sharing is performed between the first device 101 and the second device 102 through the memory architecture.

[0036] It can be understood that the first device 101 and the second device 102 in the embodiment of the present invention are interconnected through the memory architecture, and memory sharing can be performed between them using the memory architecture. Without relying on the active intervention of the CPU, the memory data on the other device can be directly read and written, significantly improving the efficiency of cross-device data transmission, reducing system latency, and providing an efficient communication foundation for multiple edge devices to cooperate to complete complex inference tasks.

[0037] In the embodiment of the present invention, data transfer is achieved between the first device 101 and the second device 102 through the direct memory access technology.

[0038] Among them, the direct memory access technology, namely DMA (Direct Memory Access), is a technology that allows direct data exchange between peripheral devices and the main memory, which can significantly improve data transmission efficiency and reduce latency.

[0039] It can be understood that data transfer is achieved between the first device 101 and the second device 102 in the embodiment of the present invention through the direct memory access technology. Specifically, in each iteration process, the unconditional prediction results generated by the second device 102 need to be transmitted from the second device 102 to the first device 101 for result fusion. Using the DMA technology, these data can be transmitted from the CPU of the second device 102 to the CPU of the first device 101. The whole process realizes efficient data transfer without the frequent intervention of the CPU, thus significantly reducing data transmission latency and improving the overall computing efficiency.

[0040] In an embodiment of the present invention, the duration for the first device 101 to complete the first task is the first inference duration, and the duration for the second device 102 to complete the first task is the second inference duration. One of the first device 101 and the second device 102 is provided with a computing optimization module. The computing optimization module obtains the first inference duration of the first device 101 and the second inference duration of the second device 102, calculates the duration difference between the first inference duration and the second inference duration, and adjusts the operating power consumption of at least one of the first device 101 and the second device 102 according to the duration difference.

[0041] Among them, to calculate the duration difference between the first inference duration and the second inference duration, subtract the second inference duration from the first inference duration.

[0042] It can be understood that in the embodiment of the present invention, the duration for the first device 101 to complete the first task is set as the first inference duration, and the duration for the second device 102 to complete the second task is set as the second inference duration. A computing optimization module is set in any one of the first device 101 and the second device 102. The computing optimization module is responsible for obtaining the first inference duration of the first device 101 and the second inference duration of the second device 102, calculating the difference between these two inference durations, and dynamically adjusting the operating power consumption of at least one of the first device 101 and the second device 102 according to this duration difference, so that the computing times of the two devices are closer, thereby achieving the best balance between efficiency and power consumption, ensuring that during the entire distributed computing process, the system can flexibly adjust the working states of each device according to actual needs to achieve the optimal performance. Specifically, the method for the computing optimization module to dynamically adjust the operating power consumption of at least one of the first device 101 and the second device 102 according to the duration difference is as follows: In an embodiment of the present invention, the operating parameters of at least one of the first device 101 and the second device 102 are modulated according to the duration difference, including: when the absolute value of the duration difference is less than the adjustment threshold, keep the operating power consumption of the first device 101 and the second device 102 unchanged, or adjust the operating power consumption of both the first device 101 and the second device 102 to the maximum power consumption; when the absolute value of the duration difference is greater than or equal to the adjustment threshold, if the duration difference is positive, reduce the operating power consumption of the second device 102, and if the duration difference is negative, reduce the operating power consumption of the first device 101.

[0043] Among them, the adjustment threshold is specifically set according to actual needs and is not specifically limited here.

[0044] It can be understood that, according to the embodiment of the present invention, the operating parameters of at least one of the first device 101 and the second device 102 are modulated according to the duration difference between the first inference duration of the first device 101 and the second inference duration of the second device 102. Specifically: when the absolute value of the duration difference is less than the adjustment threshold, the operating power consumption of the first device 101 and the second device 102 is kept unchanged, or the operating power consumption of both is adjusted to the maximum power consumption to ensure the highest computing efficiency. Since the first device 101 and the second device 102 are the same edge computing devices, their maximum power consumptions are basically the same; when the absolute value of the duration difference is greater than or equal to the adjustment threshold, if the duration difference is positive, that is, the inference time of the first device 101 is longer, the operating power consumption of the second device 102 is reduced, so that the inference time of the second device 102 is lengthened, and the duration difference between the first device 101 and the second device 102 is reduced; if the duration difference is negative, that is, the inference time of the second device 102 is longer, the operating power consumption of the first device 101 is reduced, so that the inference time of the first device 101 is lengthened, and the duration difference between the first device 101 and the second device 102 is reduced. In this way, the computing optimization module can dynamically adjust the operating power consumption of the first device 101 and the second device 102, making the computing times of the two devices more balanced, thereby achieving the best balance between performance and energy consumption, improving the overall efficiency of the system, and ensuring that the power consumption of the device is in the optimal state under different load conditions.

[0045] According to the distributed computing system proposed by the embodiment of the present invention, an input device is used to input the inference task of the edge-side model into the first device. The first device splits the inference task into a first task and a second task, inputs the first task into the edge-side model deployed locally on the first device, and inputs the second task into the edge-side model deployed locally on the second device. The first device and the second device execute their respective tasks in parallel, and the two computing paths are executed in parallel, improving the computing efficiency and achieving the technical effect of enhancing the computing efficiency of the edge-side diffusion model inference.

[0046] Through the description of the above embodiments, those skilled in the art can clearly understand that the method according to the above embodiments can be implemented by means of software plus a necessary general hardware platform. Of course, it can also be implemented by hardware, but in many cases, the former is a better implementation method.

[0047] The distributed computing system will be further described below through a specific embodiment.

[0048] This embodiment mainly designs a model splitting module, a remote scheduling module, a communication management module, and a load optimization module, as Figure 2As shown. The model splitting module improves the computing efficiency by deploying part of the inference process of the diffusion model on the first device and part on the second device, with the two computing paths executed in parallel; the remote scheduling module realizes cross-device synchronous triggering and task execution through the Pyro5 tool. Pyro5 is responsible for task scheduling to achieve synchronous triggering, task execution, message passing, and message synchronization of tasks in the two devices; the communication management module uses the unified memory architecture combined with the DMA channel to support CPU CPU, GPU GPU and cross-device CPU memory sharing of GPU. After the iteration is completed, the result is generated by decoding through VAE (Variational Auto Encoder), greatly reducing the data transmission delay. The load optimization module introduces a dynamic load balancing algorithm according to the inference time difference between the two devices, achieving optimal power consumption allocation while ensuring the lowest inference delay. The specific solution is as follows: I. Model splitting module The common design of the diffusion model includes two computing paths: conditional prediction and unconditional prediction. Conditional prediction realizes the generation of content and prompts by inputting text or video prompts. Unconditional prediction has no explicit conditional input and realizes the diversity of the model output results by introducing randomness. By combining these two modes, the goals of controlling generation quality and diversity are achieved. In the above computing process, the two independent computing paths of conditional prediction and unconditional prediction can be allocated to two different computing devices for calculation. Conditional prediction is allocated to the first edge device for execution, and unconditional prediction is allocated to the second device for execution. Separating these two computing paths can effectively utilize multi-device resources and reduce the load of a single device.

[0049] II. Remote scheduling module When performing end-side diffusion model inference, an iterative denoising loop is required. In each denoising process, conditional prediction and unconditional prediction are each executed once on the two devices, and the generated results of the two devices are synthesized, and then the next denoising process is carried out. In the above process, parallel computing scheduling and communication need to be considered, that is, synchronous triggering, task execution, message passing, and message synchronization of tasks in the two devices need to be realized. Cross-device synchronous triggering and task execution are realized through the Pyro5 tool. Pyro5 is responsible for task scheduling, enabling the first device to call functions on the second device for task execution.

[0050] III. Communication management module During each denoising process, after the two edge devices perform inference to generate results, the results of the second device need to be sent to the first device for result synthesis, so it involves big data communication. The unified memory (UM) architecture in a single device realizes memory sharing between the CPU and the GPU. For the DMA memory sharing results of the two devices in this embodiment, due to its unified memory characteristics, memory sharing between the CPUs and GPUs on the two devices can be achieved. As Figure 3 shown, the first device and the second device share memory through DMA, and the first device and the second device respectively implement UM unified memory in this shared memory area, that is, the GPU and CPU in the first device can directly achieve zero-copy without going through the memory copy process, and the same conclusion applies to the second device. Combining the above results, the first device and the second device can both achieve shared memory between CPUs, and can also achieve shared memory between GPUs, and can also achieve memory sharing between the CPU of the first device and the GPU of the second device, realizing zero-copy of GPU memory between the two devices. After each denoising result is generated, the result of the second device realizes data transfer from the GPU to the CPU of the second device to the GPU, realizing efficient data communication.

[0051] IV. Computational Optimization Module During the parallel computing process of the diffusion model, due to the difference in the model computation amount of different computing devices, there is also a difference in their inference times, resulting in waiting during the synchronization of the two devices, that is, after the first device finishes computing, it needs to wait for the second device to finish computing before synchronously proceeding with the next parallel computing process (or the second device waits for the first device). Since edge devices are sensitive to power consumption, a dynamic load balancing method for dynamically balancing power consumption and time is designed. Assume that the inference time of the first device is T(A), the inference time of the second device is T(B), and the time difference between the two devices is T, that is, T = T(A) - T(B). When T is positive, it means that the second device takes longer, and when T is negative, it means that the second device takes longer. Therefore, a threshold W is set. When the absolute value of T is greater than or equal to W, it means that the time difference exceeds the threshold, and the power consumption of the device with a faster running time is automatically reduced by one level. After the power consumption is reduced, the computing times of the two devices will be closer, achieving the computing synchronization of the two devices. When it is less than this threshold, both devices operate in the maximum working mode, achieving a balance between efficiency and power consumption through this method.

[0052] Specifically, this embodiment will be further described in detail in combination with the inference process of the generative diffusion model Cosmos-1.0-Diffusion-7B. Among them, the generative diffusion model Cosmos-1.0-Diffusion-7B is an open-source AI model that can perform functions such as text-to-video generation and video-to-video generation. It is based on an iterative denoising process to gradually generate video frames from random noise. The denoising step goes through 35 iterations. Each iteration goes through conditional prediction and unconditional prediction, and the results are fused to achieve the goal of gradually removing noise.

[0053] In this embodiment, the computing platform used mainly consists of 2 edge computing devices based on the ARM (Advanced RISC Machine, a processor architecture based on the reduced instruction set RISC) architecture. They are connected in two ways: PCIE DMA (Peripheral Component Interconnect Express DMA, peripheral component interconnect high-speed direct memory access) and Ethernet, as Figure 4 shown. Among them, the PCIE DMA method is mainly connected through the PCIE (Peripheral Component Interconnect Express, peripheral component interconnect high-speed bus) bus. The first device is set to the EP mode (EndPoint. In this mode, the first device acts as a terminal device on the PCIE bus and is usually used to connect to the upstream device. It is usually the initiator or receiver of data, rather than routing data to other devices), and the second device is set to the RP mode (Root Port. In this mode, the second device plays the role of the root port in the PCIE topology, responsible for managing communication with the EndPoint device and connecting these devices to the system's main memory and other resources). The main function of the DMA method is data communication; the Ethernet connection works for remote task invocation. The specific information of the devices is shown in Table 1. Among them, Table 1 is the table of device specific information.

[0054] Table 1

[0055] The specific implementation process of this embodiment is as follows: I. Hardware preparation and connection: Prepare 2 edge computing devices, named the first device and the second device respectively, and connect them through Ethernet. The IP of the first device is 192.168.1.1, and the IP of the second device is 192.168.1.2 to achieve Ethernet intercommunication; at the same time, use PCIE to connect the two devices to achieve PCIE communication, where the first device is in the EP mode and the second device is in the RP mode.

[0056] II. Model Splitting: According to the actual business requirements, the conditional prediction task in each denoising process is left to be executed locally on the first device, and the unconditional prediction task is remotely called to the second device, an auxiliary device, for execution. A Pyro5 service is started on the second device to expose the denoising function interface; the interface is called on the first device through Ethernet communication. The first device uses Thread Pool Executor(max_workers=2), that is, a thread pool executor (with a maximum of 2 worker threads) during the denoising process to trigger local conditional prediction and remote unconditional prediction in parallel and time them separately. III. Communication Management: The model denoising step goes through 35 times in total. After each denoising process is completed, both the first device and the second device generate their respective intermediate results. It is necessary to further send the results generated by the second device to the first device for result fusion. Since the amount of generated result data is large, an efficient data transmission mechanism is required for communication between devices. Using the efficient data communication link designed by this solution, the results generated by the second device are sent to the first device through the GPU(B)-CPU(B)-CPU(A)-GPU(A) link. This link involves two key technologies, the unified memory UM architecture and DMA memory sharing. Among them, the unified memory UM architecture realizes the data communication link between GPU(B)-CPU(B) and CPU(A)-GPU(A), and DMA memory sharing realizes the efficient cross-device data transmission between CPU(B)-CPU(A). The data link process is as follows: The results generated by the second device are stored in the unified memory of B, and then transferred from the unified memory of device B to the unified memory of device A through cross-device DMA memory sharing. Thus, efficient data transfer is achieved. After the data transfer is completed, the contribution of conditional and unconditional prediction is dynamically adjusted through the guiding coefficient to achieve data fusion.

[0057] IV. Test Results: The total duration of model inference on a single first device is 130 minutes, and the total average duration of each conditional prediction and unconditional prediction is 210 seconds; the total duration of parallel inference on the first device and the second device is 65 minutes, with an average conditional prediction duration of 103 seconds on the first device and an average conditional prediction duration of 104 seconds on the second device. Compared with a single edge computing device for calculation, the overall generation duration is shortened from about 130 minutes to about 65 minutes, and the speed is increased by about 2 times; the latency of each denoising is also reduced by half, achieving an acceleration effect of nearly 2 times.

[0058] V. Output Consistency Verification: Under the two configurations of parallel and single edge computing devices, by setting the same random seed, the final results are exactly the same, ensuring that the parallel computing split does not affect the randomness and output quality of the model. Figure 5A flowchart of a distributed computing method provided by an embodiment of the present invention. This method is applied to a first device in the above-mentioned distributed computing system. The first device and the second device call the end-side models locally deployed by them to iteratively calculate and infer tasks, such as Figure 5 As shown, the method includes: In step S201, obtain the inference task of the end-side model.

[0059] It can be understood that in the embodiment of the present invention, first, a reasoning request initiated by a user or a system is received from an input device or other task scheduling modules, and it is parsed into a specific reasoning task that can be executed on the end-side model.

[0060] In step S202, split the inference task into a first task and a second task.

[0061] Among them, through the encoder in the first device and in accordance with the standard inference process of the diffusion model, the inference task is split.

[0062] It can be understood that in the embodiment of the present invention, the input inference task is sent to the encoder in the first device for parsing, and in accordance with the standard inference process of the diffusion model, it is split into two sub-tasks that are executed in parallel: the first task for generating target features according to input conditions and the second task for generating diverse latent features. The first task and the second task provide a basis for subsequent iterative calculation and result fusion.

[0063] In step S203, input the first task into the end-side model locally deployed by the first device, and input the second task into the end-side model locally deployed by the second device. The first device and the second device execute their respective tasks in parallel, and the first task and the second task are different types of tasks.

[0064] Among them, executing in parallel means that the two devices independently process the tasks assigned to them simultaneously to improve the overall inference efficiency.

[0065] It can be understood that in the embodiment of the present invention, the first task is input into the end-side model locally deployed by the first device, and the second task is input into the end-side model locally deployed by the second device. Since these two tasks belong to different types of inference tasks, they are independently and parallelly executed by the first device and the second device respectively. Through this task splitting and device division mechanism, the advantages of distributed computing are fully utilized, and the inference efficiency is improved while ensuring the generation quality.

[0066] In step S204, during the current iterative calculation and inference task process, identify the states of the tasks executed by the first device and the second device respectively. If the states of the tasks executed by the first device and the second device are both in the execution completion state, then initiate the next iterative calculation of the inference task.

[0067] It can be understood that in the current denoising iteration process of the embodiments of the present invention, the execution statuses of the tasks of the first device and the second device are continuously monitored. Once it is detected that both devices have completed the tasks allocated in this round, the next iteration calculation of the inference task is automatically started, and the next denoising and result fusion operations are continued, thereby advancing the generation process of the entire diffusion model closer to the final output result.

[0068] According to the distributed computing method for the first device applied to a distributed computing system proposed by the embodiments of the present invention, an input device is used to input the inference task of the edge model into the first device. The first device splits the inference task into a first task and a second task, inputs the first task into the edge model locally deployed on the first device, and inputs the second task into the edge model locally deployed on the second device. The first device and the second device execute their respective tasks in parallel, and the two computing paths are executed in parallel, improving the computing efficiency and achieving the technical effect of enhancing the computing efficiency of the edge diffusion model inference.

[0069] For the description of the features in the corresponding embodiments of the distributed computing method for the first device applied to a distributed computing system, reference can be made to the relevant descriptions of the corresponding embodiments of the distributed computing system, which will not be elaborated here one by one.

[0070] Figure 6 The flowchart of a distributed computing method provided by the embodiments of the present invention is applied to the second device in the above-mentioned distributed computing system. The first device and the second device call the edge models locally deployed by them to iteratively calculate the inference task, as Figure 6 shown, the method includes: In step S301, obtain the second task sent by the first device. Among them, the first device obtains the inference task of the edge model, splits the inference task into a first task and a second task, and the first device calls the edge model locally deployed to execute the first task.

[0071] It can be understood that the embodiments of the present invention first obtain the second task sent by the first device. This process originates from the first device obtaining a complete inference task from the input source and then splitting the task into a first task and a second task through its internal encoder. The second task is allocated and sent to the second device for parallel processing. Through the task splitting and distributed execution mechanism, efficient collaborative inference of the diffusion model among multiple edge devices is achieved.

[0072] In step S302, call the edge model locally deployed on the second device to execute the second task. The first device and the second device execute their respective tasks in parallel, and the first task and the second task are different types of tasks.

[0073] It can be understood that in the embodiments of the present invention, the second device calls the end - side model deployed locally to execute the second task. At the same time, the first device also executes the first task on the end - side model deployed locally. The two execute in parallel, realizing the distributed collaborative inference of the diffusion model on multiple edge devices, thereby improving the inference efficiency while ensuring the generation quality.

[0074] In step S303, during the current iterative calculation of the inference task, the state of the second device executing the task is sent to the first device. If the states of the first device and the second device executing their respective tasks are both in the execution - completed state, then the next iterative calculation of the inference task is started.

[0075] Among them, the state of the second device executing the task can be sent to the first device through means such as PCIe DMA or network.

[0076] It can be understood that in the embodiments of the present invention, the second device feeds back its task execution state to the first device through the communication mechanism. The system continuously monitors the task completion status of the two devices. Once it is confirmed that both the first device and the second device are in the execution - completed state of their respective tasks, it is determined that the current round of iterative tasks is all completed, and then the next iterative calculation of the inference task is triggered to continue promoting the generation process of the diffusion model until the final result is output.

[0077] According to the distributed computing method for the second device applied to the distributed computing system proposed in the embodiments of the present invention, the input device is used to input the inference task of the end - side model into the first device. The first device splits the inference task into a first task and a second task, inputs the first task into the end - side model deployed locally on the first device, and inputs the second task into the end - side model deployed locally on the second device. The first device and the second device execute their respective tasks in parallel, and the two calculation paths execute in parallel, improving the calculation efficiency and achieving the technical effect of improving the calculation efficiency of the end - side diffusion model inference.

[0078] For the description of the features in the corresponding embodiments of the distributed computing method for the second device applied to the distributed computing system, reference can be made to the relevant description in the corresponding embodiments of the distributed computing system, which will not be elaborated here one by one.

[0079] Embodiments of the present invention also provide an electronic device, as Figure 7 shown, including a memory 401 and a processor 402. A computer program is stored in the memory 401, and the processor 402 is configured to run the computer program to execute the steps in any of the above - mentioned distributed computing method embodiments.

[0080] Embodiments of the present invention also provide a computer-readable storage medium storing a computer program, where the computer program is configured to execute the steps in any of the above-described method embodiments of the distributed computing method when running.

[0081] In an exemplary embodiment, the above computer-readable storage medium may include, but is not limited to: various media such as USB flash drives, read-only memories (ROM for short), random access memories (RAM for short), external hard drives, magnetic disks, or optical discs that can store computer programs.

[0082] Embodiments of the present invention also provide a computer program product, where the computer program product includes a computer program that implements the steps in any of the above-described method embodiments of the distributed computing method when executed by a processor.

[0083] Embodiments of the present invention also provide another computer program product, including a non-volatile computer-readable storage medium storing a computer program that implements the steps in any of the above-described method embodiments of the distributed computing method when executed by a processor.

[0084] Those skilled in the art can further realize that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be implemented by electronic hardware, computer software, or a combination of the two. To clearly illustrate the interchangeability of hardware and software, the components and steps of each example have been generally described according to their functions in the above description. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. Skilled professionals can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present invention.

[0085] The above has introduced in detail a distributed computing system, method, electronic device, storage medium, and program product provided by the present invention. Specific examples are used herein to elaborate on the principles and implementation manners of the present invention. The description of the above embodiments is only used to help understand the method and its core idea of the present invention. It should be noted that for those of ordinary skill in the art in this technical field, without departing from the principle of the present invention, several improvements and modifications can be made to the present invention, and these improvements and modifications also fall within the protection scope of the claims of the present invention.

Claims

1. A distributed computing system, characterized in that, Including: A first device and a second device with a side model deployed locally; An input device for inputting the inference task of the side model into the first device; The first device splits the inference task into a first task and a second task, inputs the first task into the side model deployed locally on the first device, and inputs the second task into the side model deployed locally on the second device. The first device and the second device execute their respective tasks in parallel, and the first task and the second task are different types of tasks.

2. The distributed computing system according to claim 1, wherein The first device includes an encoder. The encoder receives the text data of the inference task input by the input device and encodes the text data of the inference task into the first task and the second task.

3. The distributed computing system according to claim 1, wherein The first device and the second device call the side models deployed locally on them to iteratively calculate the inference task of the side model. In each iteration of calculating the inference task, the first device calls the side model deployed locally to execute the first task, and the second device calls the side model deployed locally to execute the second task.

4. The distributed computing system according to claim 1, wherein In each iteration of calculating the inference task, the first device fuses the respective task execution results of the first device and the second device and sends the fusion results to the side models deployed locally on the first device and the second device respectively.

5. The distributed computing system according to claim 1, wherein The first device and the second device are provided with a synchronization tool, and the synchronization tool is used to synchronize the task triggering and task execution between the first device and the second device.

6. The distributed computing system according to claim 1, wherein Both the first device and the second device include a plurality of processors and a memory architecture. The plurality of processors are connected to the memory architecture, and the plurality of processors share memory through the memory architecture.

7. The distributed computing system according to claim 6, wherein The memory architectures between the first device and the second device are connected, and the first device and the second device share memory through the memory architecture.

8. The distributed computing system according to claim 7, wherein Data transfer is achieved between the first device and the second device through direct memory access technology.

9. The distributed computing system according to claim 6, wherein The plurality of processors include a plurality of processors of different types.

10. The distributed computing system according to claim 1, wherein The duration for the first device to complete the first task is the first inference duration, and the duration for the second device to complete the first task is the second inference duration. One of the first device and the second device is provided with a calculation optimization module, where The calculation optimization module obtains the first inference duration of the first device and the second inference duration of the second device, calculates the duration difference between the first inference duration and the second inference duration, and adjusts the operating power consumption of at least one of the first device and the second device according to the duration difference.

11. A distributed computing method, characterized in that, The method is applied to the first device in the distributed computing system according to any one of claims 1-10. The first device and the second device call the side models deployed locally on them to iteratively calculate the inference task. The method includes: Obtaining the inference task of the side model; Splitting the inference task into the first task and the second task; Input the first task into the edge model locally deployed on the first device, and input the second task into the edge model locally deployed on the second device. The first device and the second device execute their respective tasks in parallel, and the first task and the second task are different types of tasks. During the current iteration of calculating the inference task, identify the status of the first device and the second device executing their respective tasks. If the status of the first device and the second device executing their respective tasks is the execution completion status, then initiate the next iteration of calculating the inference task.

12. A distributed computing method, characterized in that, The method is applied to the second device in the distributed computing system according to any one of claims 1-10. The first device and the second device call the edge models locally deployed on them to iteratively calculate the inference task. The method includes: Obtain the second task sent by the first device. Among them, the first device obtains the inference task of the edge model, splits the inference task into the first task and the second task, and the first device calls the edge model locally deployed on it to execute the first task. Call the edge model locally deployed on the second device to execute the second task. The first device and the second device execute their respective tasks in parallel, and the first task and the second task are different types of tasks. During the current iteration of calculating the inference task, send the status of the second device executing the task to the first device. If the status of the first device and the second device executing their respective tasks is the execution completion status, then initiate the next iteration of calculating the inference task.

13. An electronic device, characterized in that, Includes: A memory for storing computer programs; A processor for implementing the steps of the distributed computing method according to any one of claims 11 to 12 when executing the computer program.

14. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, wherein the computer program implements the steps of the distributed computing method according to any one of claims 11 to 12 when executed by a processor.

15. A computer program product, comprising a computer program, characterized in that, The computer program implements the steps of the distributed computing method according to any one of claims 11 to 12 when executed by a processor.

Images