Data processing method and device, electronic equipment and storage medium

Abstract

The application provides a data processing method and device, electronic equipment and storage medium; the method comprises the following steps: initializing a first semaphore and a second semaphore; calling a computing thread to perform computing processing on an i-th data block to obtain an i-th computing result; updating the value of the first semaphore to i; in the process of calling the computing thread to perform computing processing on the i-th data block, calling a first communication thread to load an i+1-th data block and obtain i-1-th network data; updating the second semaphore to i-1 when the first communication thread finishes obtaining the i-1-th network data; in the case that the first semaphore is greater than or equal to i and the second semaphore is greater than or equal to i-1, accumulating the i-th computing result and the i-1-th network data to obtain i-th accumulated data; and sending the i-th accumulated data to a next computing node. Through the application, the data computing performance can be improved and the communication delay can be reduced.

Description

Technical Field

[0001] This application relates to computer technology, and more particularly to a data processing method, apparatus, electronic device, and storage medium. Background Technology

[0002] In large-scale distributed parallel computing systems, tasks are typically broken down and distributed across multiple computing nodes for collaborative processing. The typical execution process of a distributed computing architecture involves alternating interactions between core arithmetic computations and data communication (such as local data loading, cross-node data transfer, and result aggregation). Current distributed computing architectures mostly employ a single control flow or coarse-grained multi-threaded scheduling, relying on system-level synchronization mechanisms such as mutexes and global barriers to maintain the temporal dependencies between computational and communication steps.

[0003] In the actual operation of the above architecture, due to the strong timing correlation between computation and communication operations, the computing unit is in an idle and blocked state while performing frequent data transfers or waiting for communication synchronization signals, resulting in low continuous utilization of the computing hardware. Furthermore, relying on coarse-grained synchronization mechanisms to handle high-frequency computation and communication coordination leads to prolonged thread blocking and overhead in the distributed parallel computing system. During the adjustment of synchronization constraints, the distributed computing architecture may experience data being read and sent over the network before computation is complete, or remote data being extracted and aggregated before it is fully written to the local buffer, impacting computational performance and increasing communication latency. Summary of the Invention

[0004] This application provides a data processing method, apparatus, electronic device, storage medium, and computer program product that can improve data computing performance and reduce communication latency.

[0005] The technical solution of this application embodiment is implemented as follows: This application provides a data processing method, the method comprising: Initialize the first semaphore and the second semaphore; The computation thread is invoked to perform computation on the i-th data block and obtain the i-th computation result. Update the value of the first semaphore to i, where i is a positive integer; During the process of calling the computing thread to perform calculation on the i-th data block, the first communication thread is called to load the (i+1)-th data block and obtain the (i-1)-th network data, wherein the (i-1)-th network data is the (i-1)-th accumulated data sent by the previous computing node; After the first communication thread has finished acquiring the (i-1)th network data, the second semaphore is updated to i-1; When the first semaphore is greater than or equal to i and the second semaphore is greater than or equal to i-1, the i-th calculation result and the i-1 network data are accumulated to obtain the i-th accumulated data; The i-th accumulated data is sent to the next computing node.

[0006] This application provides a data processing apparatus, including: The semaphore update module is used to initialize the first semaphore and the second semaphore; The calculation module is used to call the calculation thread to perform calculations on the i-th data block and obtain the i-th calculation result; The semaphore update module is further configured to update the value of the first semaphore to i, wherein i is an integer; The communication module is used to call the first communication thread to load the (i+1)th data block and obtain the (i-1)th network data during the process of calling the computing thread to perform calculation processing on the i-th data block. The (i-1)th network data is the (i-1)th accumulated data sent by the previous computing node. The semaphore update module is also used to update the second semaphore to i-1 after the first communication thread has finished acquiring the (i-1)th network data; The calculation module is used to accumulate the i-th calculation result and the (i-1)-th network data when the first semaphore is greater than or equal to i and the second semaphore is greater than or equal to i-1, to obtain the i-th accumulated data; A communication module is used to send the i-th accumulated data to the next computing node.

[0007] This application provides an electronic device, the electronic device comprising: Memory is used to store executable instructions or computer programs. The processor, when executing computer-executable instructions or computer programs stored in the memory, implements the data processing method provided in the embodiments of this application.

[0008] This application provides a computer-readable storage medium storing computer-executable instructions or computer programs, which, when executed by a processor, implement the data processing method provided in this application.

[0009] This application provides a computer program product, including a computer program or computer executable instructions. When the computer program or computer executable instructions are executed by a processor, they implement the data processing method provided in this application.

[0010] The embodiments of this application have the following beneficial effects: Through a multi-threaded asynchronous collaborative mechanism, the computation thread performs the computation for the current index i, while the first communication thread is scheduled to execute the data loading task for the subsequent index (i+1) and the network data acquisition task for the previous index (i-1) in parallel. This parallel architecture masks the memory access latency and the communication latency of cross-node network transmission with the execution time of arithmetic operations, thus significantly reducing the total processing latency of distributed computing tasks. A dual asynchronous triggering mechanism using a first and second semaphore decouples the dependency between local computation results and externally input network data. Dynamic updates of semaphore values ​​automatically trigger the accumulation operation as soon as data is ready, improving the effective utilization of hardware computing units and communication interfaces. Acquiring the next data block during computation (loading the i+1th data block) avoids instantaneous congestion caused by concentrated memory access after the computation task ends, ensuring performance stability when processing extremely large amounts of data. Attached Figure Description

[0011] Figure 1 This is a schematic diagram illustrating the application mode of the data processing method provided in the embodiments of this application; Figure 2 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application; Figure 3 This is a first flowchart illustrating the data processing method provided in an embodiment of this application; Figure 4 This is a second flowchart illustrating the data processing method provided in the embodiments of this application; Figure 5 This is a first schematic diagram of multi-threaded computing communication provided in an embodiment of this application; Figure 6 This is a second schematic diagram of multi-threaded computational communication provided in an embodiment of this application; Figure 7 This is a schematic diagram illustrating the first principle of synchronization of computing and communication provided in an embodiment of this application; Figure 8 This is a schematic diagram illustrating the second principle of synchronization of computing and communication provided in an embodiment of this application; Figure 9 This is a schematic diagram of the third principle of synchronization of computing and communication provided in the embodiments of this application.

[0012] It should be noted that the terms "first" and "second" mentioned above are only used to distinguish between different options and do not represent the degree of superiority or inferiority of the options or their priority in the implementation process. Detailed Implementation

[0013] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. The described embodiments should not be regarded as limitations on this application. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0014] In the following description, references are made to “some embodiments,” which describe a subset of all possible embodiments. However, it is understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0015] In the following description, the terms "first, second, third" are used merely to distinguish similar objects and do not represent a specific ordering of objects. It is understood that "first, second, third" may be interchanged in a specific order or sequence where permitted, so that the embodiments of this application described herein can be implemented in an order other than that illustrated or described herein.

[0016] In the embodiments of this application, the terms "module" or "unit" refer to a computer program or part of a computer program that has a predetermined function and works with other related parts to achieve a predetermined goal, and can be implemented wholly or partially using software, hardware (such as processing circuitry or memory), or a combination thereof. Similarly, a processor (or multiple processors or memory) can be used to implement one or more modules or units. Furthermore, each module or unit can be part of an overall module or unit that includes the functionality of that module or unit.

[0017] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0018] Before providing a further detailed description of the embodiments of this application, the nouns and terms involved in the embodiments of this application will be explained, and the nouns and terms involved in the embodiments of this application shall be interpreted as follows.

[0019] 1) Semaphore: A semaphore is a synchronization tool used in computer science for multi-threaded / multi-process environments. Its purpose is to ensure that critical code segments are not called concurrently, and to avoid race conditions and deadlocks. Semaphores are mainly divided into two categories: binary semaphores and counting semaphores. They manage resource access through numerical values ​​and wait queues. Typical semaphore functions include initialization (sem_init), waiting (sem_wait / P operation), releasing (sem_post / V operation), and cleanup (sem_destroy).

[0020] 2) Credit semaphore: In a distributed parallel computing system, a credit semaphore is a monotonically increasing value used to characterize the progress of task execution and the availability of buffer resources. In this embodiment, the credit semaphore is physically represented as an integer variable stored at a specific memory address or register, providing non-blocking timing constraints between different execution units (such as computing threads and communication threads) through a "producer-consumer" handshake protocol.

[0021] 3) Communication-Computing Fusion Operator: This is a mathematical tool used to integrate operational rules from different computing systems into a unified framework, often referred to as a universal computing fusion operator. The core idea of ​​the universal computing fusion operator is to achieve collaborative work between heterogeneous algorithms through specific mapping relationships, making it valuable in distributed computing, cross-platform data integration, and complex system modeling. The domain of the universal computing fusion operator includes scalars, vectors, tensors, and higher-order abstract data objects, satisfying the three fundamental properties of commutativity, associativity, and distributivity, while also supporting dynamic weight adjustment mechanisms.

[0022] 4) Ping-Pong Mechanism: This is a double-buffering technique widely used in high-speed data stream processing (such as audio, video, ADC sampling, DMA transfer, etc.). Its core advantage is avoiding waiting and enabling parallel processing. It maintains two buffers (usually called "Ping" and "Pong"). When one buffer is written to (or processed), the other buffer can be read from (or processed), and the two buffers alternately perform processing.

[0023] With the explosive growth in the scale of AI model parameters, the demand for computing power and memory bandwidth has far exceeded the carrying capacity of the related computing architecture. In this architecture, data is stored in memory, and computation is performed in the arithmetic logic unit (ALU) of the central processing unit (CPU). This architecture requires frequent data transfers between memory and the computing unit when performing computational tasks. The enormous data transfer overhead and access latency generated in this process severely limit the performance of the CPU due to memory limitations. The energy consumption for data transfer can be hundreds of times greater than that for computation itself, thus restricting the overall energy efficiency and performance of the computing architecture.

[0024] In distributed training, on the communication side, when locally computed data (such as gradients) needs to be sent to other nodes, it must be ensured that the data has already been computed; otherwise, communication errors will occur. On the computation side, when data (such as gradients) from other nodes needs to be aggregated for local updates or subsequent computations, it must be ensured that all remote data involved in the computation has been delivered completely and correctly. This state dependency between "computation completion" and "data delivery" requires an efficient and low-overhead synchronization mechanism. Coarse-grained locking or global barrier synchronization in related technologies introduces significant performance jitter and cannot meet the high-performance and low-latency goals required for integrated computing.

[0025] This application provides a data processing method, a data processing apparatus, an electronic device, a computer-readable storage medium, and a computer program product, which can improve data computing performance and reduce communication latency.

[0026] The following describes exemplary applications of the electronic devices provided in the embodiments of this application. These electronic devices can be implemented as terminal devices, such as laptops, tablets, desktop computers, set-top boxes, smart TVs, in-vehicle terminals, virtual reality (VR) devices, augmented reality (AR) devices, and other various types of terminals, or as servers. The following will describe exemplary applications of the electronic devices when implemented as terminal devices.

[0027] refer to Figure 1 , Figure 1 This is a schematic diagram illustrating the application mode of the data processing method provided in the embodiments of this application; for example, Figure 1 The system involves server 200, network 300, terminal device 400, and database 500. Terminal device 400 is connected to server 200 through network 300. Network 300 can be a wide area network, a local area network, or a combination of both.

[0028] Server 200 is the server for the artificial intelligence application. Server 200 is used to send data to terminal device 400, which can be a user's mobile phone or computer. The terminal device 400 has artificial intelligence-related applications installed on it.

[0029] Based on the data sent by the server 200, the terminal device 400 calls the data processing method provided in this application embodiment to perform data communication and calculation on the local on-chip network, obtains the final calculation result, and presents the final calculation result to the user using the terminal device 400.

[0030] In some embodiments, server 200 may be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms. Terminals and servers can be connected directly or indirectly via wired or wireless communication, which is not limited in this embodiment.

[0031] In some embodiments, the server can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. Electronic devices can be smartphones, tablets, laptops, desktop computers, smart speakers, smartwatches, etc., but are not limited to these. Terminal devices and servers can be directly or indirectly connected via wired or wireless communication, which is not limited in this embodiment.

[0032] See Figure 2 , Figure 2 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. The electronic device may be... Figure 1 400 terminal devices Figure 2 The terminal device 400 shown includes at least one processor 410, a memory 450, at least one network interface 420, and a user interface 430. The various components in the terminal device 400 are coupled together via a bus system 440. It is understood that the bus system 440 is used to implement communication between these components. In addition to a data bus, the bus system 440 also includes a power bus, a control bus, and a status signal bus. However, for clarity, ... Figure 2 The general labeled all buses as Bus System 440.

[0033] Processor 410 can be an integrated circuit chip with signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor can be a microprocessor or any conventional processor, etc.

[0034] User interface 430 includes one or more output devices 431 that enable the presentation of media content, including one or more speakers and / or one or more visual displays. User interface 430 also includes one or more input devices 432, including user interface components that facilitate user input, such as a keyboard, mouse, microphone, touch screen display, camera, other input buttons and controls.

[0035] The memory 450 may be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid-state storage, hard disk drives, optical disk drives, etc. The memory 450 may optionally include one or more storage devices physically located away from the processor 410.

[0036] The memory 450 may include volatile memory or non-volatile memory, or both. The non-volatile memory may be read-only memory (ROM), and the volatile memory may be random access memory (RAM). The memory 450 described in this application embodiment is intended to include any suitable type of memory.

[0037] In some embodiments, memory 450 is capable of storing data to support various operations, examples of which include programs, modules, and data structures or subsets or supersets thereof, as illustrated below.

[0038] Operating system 451 includes system programs for handling various basic system services and performing hardware-related tasks, such as the framework layer, core library layer, driver layer, etc., for implementing various basic business functions and handling hardware-based tasks; The network communication module 452 is used to reach other electronic devices via one or more (wired or wireless) network interfaces 420, exemplary network interfaces 420 including: Bluetooth, WiFi, and Universal Serial Bus (USB), etc. Presentation module 453 is configured to enable the presentation of information (e.g., a user interface for operating peripheral devices and displaying content and information) via one or more output devices 431 associated with user interface 430 (e.g., a display screen, a speaker, etc.). The input processing module 454 is used to detect and translate one or more user inputs or interactions from one or more input devices 432.

[0039] In some embodiments, the apparatus provided in this application can be implemented in software. Figure 2A data processing device 455 stored in memory 450 is shown. It may be software in the form of programs and plug-ins, including the following software modules: semaphore update module 4551, calculation module 4552, and communication module 4553. These modules are logical and can therefore be arbitrarily combined or further divided according to the functions they implement. The functions of each module will be described below.

[0040] In other embodiments, the data processing apparatus provided in this application can be implemented in hardware. As an example, the data processing apparatus provided in this application can be a processor in the form of a hardware decoding processor, which is programmed to execute the data processing method provided in this application. For example, the processor in the form of a hardware decoding processor can be one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs), or other electronic components.

[0041] The data processing method provided in this application will be described in conjunction with exemplary applications and implementations of the terminal devices or servers provided in the embodiments of this application.

[0042] The data processing method provided in the embodiments of this application will be described below. As mentioned above, the electronic device implementing the data processing method of the embodiments of this application can be a terminal device or a server, or a combination of both. Therefore, the executing entity of each step will not be described again below.

[0043] See Figure 3 , Figure 3 This is a flowchart illustrating the data processing method provided in the embodiments of this application, which will be combined with... Figure 3 The steps shown are explained. Figure 3 The entity responsible for executing the steps is Figure 1 Terminal device 400.

[0044] In step 301, the first semaphore and the second semaphore are initialized.

[0045] For example, semaphores are synchronization tools used in computer science for multi-threaded / multi-process environments. The purpose of semaphores is to ensure that critical code segments are not called concurrently and to avoid race conditions and deadlocks. The first semaphore is used to indicate the completion of computation for a data block, and the second semaphore is used to indicate the completion of communication (complete reception of network data).

[0046] In some embodiments, prior to step 301, the method further includes: allocating a first buffer and a second buffer in a static random access memory; for ease of understanding, the hardware and application scenarios involved in the embodiments of this application are explained and described. Reference Figure 7 , Figure 7 This is a schematic diagram illustrating the principle of synchronization between computation and communication provided in this application embodiment. Dynamic random access memory (DRAM) 701 and static random access memory (SRAM) 702 are used to perform computation and communication processes, respectively. DRAM 701 includes a buffer 7011, which is a destination buffer used to write the final calculation result. SRAM is divided into a resident storage area, a first on-chip buffer (first buffer), a second on-chip buffer (second buffer), and a communication buffer for temporarily storing network data. In this application embodiment, data is loaded and computed alternately using a ping-pong mechanism, a double-buffering mechanism to achieve read-write separation and role alternation. The resident storage area is used to permanently store data, such as the left-hand matrix needed for each calculation in matrix multiplication. The first and second buffers are used to alternately load and read data. For example, when the communication thread loads data into the first buffer, the data already loaded in the second buffer is read by the computation thread, and computation is performed based on the read data. Once the data in the second buffer has been calculated and the data in the first buffer has been loaded, the communication thread loads data into the second buffer, and the calculation thread reads data from the first buffer and performs calculations based on the read data.

[0047] In this embodiment, the resident storage area ensures extremely low latency retrieval of frequently reused data. Alternating read and write operations between the first and second buffers eliminate the need for system downtime to wait for data transfer, achieving physical overlap between off-chip loading time and on-chip computation time.

[0048] Step 301 is implemented as follows: the first semaphore is set to -1, and the second semaphore is set to -1. The first semaphore is used to indicate whether the computation is complete. When the computation of the initial data block is complete, the first semaphore is updated to 0. The second semaphore is a communication semaphore used to indicate whether the network data from the previous computing node (rank) in the communication domain has been completely received. It is represented as -1 when no network data has been received.

[0049] In some embodiments, the computation thread includes: a first computation thread; after step 301, the following processing is performed: The first communication thread is invoked to load the initial data block into the first buffer of the static random access memory, where the initial data block is labeled 0; the first calculation thread is invoked to extract the initial data block from the first buffer and perform matrix multiplication calculation based on the initial data block to obtain the 0th calculation result; during the matrix multiplication calculation based on the initial data block, the first communication thread is invoked to load the 1st data block into the second buffer of the static random access memory; the first semaphore is updated to 0, and the 0th calculation result is sent to the next calculation node.

[0050] For example, the data processing method of this application embodiment involves multiple threads. In this application embodiment, three threads are used as an example for illustration. The first communication thread loads the initial data block labeled 0 into the first buffer. The first calculation thread is used to perform matrix multiplication calculation based on the initial data block to obtain the initial 0th calculation result. The initial calculation node does not have a previous calculation node, so there is no need to accumulate data. The 0th calculation result is directly sent to the next calculation node.

[0051] Figure 5 This is a first schematic diagram of multi-threaded computational communication provided in this application embodiment. Taking three threads as an example, the horizontal direction represents the timing. Thread 0 (the first communication thread) loads the A ping1 and B ping1 data blocks. The A ping1 data block is stored in the resident storage area of ​​the static random access memory, and the B ping1 data block (data block 0) is stored in the first buffer of the static random access memory. Then, thread 1 (the first computation thread) starts to extract the B ping1 data from the first buffer and performs matrix multiplication operation 1 (GEMM) based on the A ping1 and B ping1 data blocks. At the same time, during the matrix multiplication operation 1 performed by thread 1, thread 0 loads the B pong1 data block (the first data block) into the second buffer of the static random access memory and performs communication 1.

[0052] In step 302, the calculation thread is invoked to perform calculations on the i-th data block to obtain the i-th calculation result.

[0053] For example, the computation process can be matrix multiplication, which is implemented through a matrix multiplication engine (GemmEngine). The computation thread calls the matrix multiplication computation engine to perform matrix multiplication calculations on the extracted first data block. There are multiple computation threads, which perform the calculations alternately.

[0054] In some embodiments, the computing thread includes: a first computing thread and a second computing thread; Step 302 can be implemented as follows: when i is even, the first calculation thread is called to extract the i-th data block from the first buffer of the static random access memory, and matrix multiplication is performed based on the i-th data block to obtain the i-th calculation result; when i is odd, the second calculation thread is called to extract the i-th data block from the second buffer of the static random access memory, and matrix multiplication is performed based on the i-th data block to obtain the i-th calculation result.

[0055] For example, when i is 0, the first computation thread is invoked to retrieve the 0th data block from the first buffer of the static random access memory (SRAM), and matrix multiplication is performed based on the 0th data block to obtain the 0th computation result, which is temporarily stored in the SRAM. When i is 1, the first computation thread is invoked to retrieve the 1st data block from the second buffer of the SRAM, and matrix multiplication is performed based on the 1st data block to obtain the 1st computation result, which is temporarily stored in the SRAM. For example: see [link to relevant documentation] Figure 5 When i is 0, the first computing thread (thread 1) is called to extract the Bping data block from the first buffer of the static random access memory. When i is 1, the second computing thread (thread 2) is called to extract the Bpong data block from the second buffer of the static random access memory.

[0056] In step 303, the value of the first semaphore is updated to i.

[0057] In the example, i is an integer. Semaphore updates can be implemented using a hardware synchronization register. Upon obtaining the calculation result for the i-th data block, a write instruction is generated, addressing the hardware synchronization register storing the first semaphore, and updating the latched binary value in the hardware synchronization register to i. For example, after the calculation of the 0th data block is completed, the initial value -1 is updated to 0; after the calculation of the 1st data block is completed, (first semaphore = 0) is updated to (first semaphore = 1).

[0058] In step 304, during the process of calling the calculation thread to perform calculation on the i-th data block, the first communication thread is called to load the (i+1)-th data block and obtain the (i-1)-th network data.

[0059] Here, the (i-1)th network data is the (i-1)th accumulated data sent by the previous computing node.

[0060] In some embodiments, step 304 is implemented as follows: when i+1 is even, the first communication thread is invoked to load the i+1th data block from the dynamic random access memory into the first buffer of the static random access memory; when i+1 is odd, the first communication thread is invoked to load the i+1th data block from the dynamic random access memory into the second buffer of the static random access memory.

[0061] For example, in the ping-pong buffer mechanism, computation is performed alternately, and the loading of the next data block is also performed alternately, thus achieving separation of reading and writing. The reading and writing times overlap, improving the efficiency of distributed computing.

[0062] For example, when i is 0, the first computation thread is invoked to retrieve the 0th data block from the first buffer of the static random access memory (SRAM) and perform matrix multiplication based on the 0th data block. During the computation, the first communication thread is invoked to load the 1st data block into the second buffer. The 0th computation result is sent to the next computation node. When i is 1, the first computation thread is invoked to retrieve the 1st data block from the second buffer of the SRAM and perform matrix multiplication based on the 1st data block. During the computation, the first communication thread is invoked to load the 2nd data block into the first buffer, and network data sent by the previous computation node is also received. For example: see reference [link / reference] Figure 5 After the communication thread (thread 0) finishes loading the A ping and B ping data, it calls the first computing thread (thread 1) to extract the B ping data block from the first buffer of the static random access memory and perform the calculation. During the calculation of the first computing thread, the communication thread loads the B pong data block and receives the network data sent by the previous computing node. When the first computing thread finishes the calculation, it calls the second computing thread (thread 2) to extract the B pong data block from the second buffer of the static random access memory.

[0063] In step 305, after the first communication thread has acquired the (i-1)th network data, the second semaphore is updated to i-1.

[0064] For example, the second semaphore is used to indicate that network data reception is complete. The Direct Memory Access (DMA) controller stores the data stream sent from the previous node byte by byte into the network receive buffer of static random access memory. When the DMA controller confirms that the last byte of network data (i-1) has been received, it generates a high-level signal, such as a completion interrupt or a status register setting, to indicate that the write is complete. Upon receiving the high-level signal, the hardware synchronization register storing the second semaphore is addressed, and the binary value latched inside the hardware synchronization register is updated to i-1. For example, upon receiving network data (0th byte), the initial value -1 is updated to 0; upon receiving network data (1st byte), (second semaphore = 0) is updated to (second semaphore = 1).

[0065] In step 306, if the first semaphore is greater than or equal to i and the second semaphore is greater than or equal to i-1, the i-th calculation result and the (i-1)-th network data are accumulated to obtain the i-th accumulated data.

[0066] For example, if the first semaphore is greater than or equal to i, it indicates that the calculated data is complete. If the second semaphore is greater than or equal to i-1, it indicates that the network data has been received completely. Accumulation is performed when both conditions are met.

[0067] In some embodiments, step 306 is implemented by: calling a computation thread to extract the i-th computation result and the (i-1)-th network data from the static random access memory; calling a computation thread to perform element-wise reduction addition on the i-th computation result and the (i-1)-th network data to obtain the i-th accumulated data.

[0068] For example, the result of the i-th calculation is stored in the static random access memory (SRAM) storage area for calculation results, and the (i-1)-th network data is stored in the network receive buffer of SRAM. Accumulation can be implemented using a vector engine. Reduce refers to aggregating local data distributed across different physical nodes into globally consistent data. The vector engine implements element-wise reduction addition in the following way: without changing the arrangement of the data, it calculates the values ​​at the same physical coordinates in the two data blocks independently, one-to-one. For example, the 0th byte of the local calculation result block is added only to the 0th byte of the network data block; the 1024th byte is added only to the corresponding 1024th byte.

[0069] In step 307, the i-th accumulated data is sent to the next computing node.

[0070] For example, the current compute node sends the i-th accumulated data to the static random access memory (SRAM) of the next compute node, and the next compute node stores the i-th accumulated data in the network receive buffer of the SRAM. The current compute node and the next compute node are different chips located within a single computer device. Alternatively, the current compute node and the next compute node are chips located in different computer devices.

[0071] In some embodiments, after step 307, if the first semaphore is greater than or equal to n and the second semaphore is greater than or equal to n-1, the nth calculation result and the (n-1)th network data are accumulated to obtain the nth accumulated data, where n is the upper limit of the data block label; the first communication thread is called to store the nth accumulated data in the target buffer of the dynamic random access memory.

[0072] For example, the first communication thread is scheduled to write the nth accumulated data to dynamic random access memory (DRAM). The nth accumulated data is then retrieved from static random access memory (SRAM) and stored in a target buffer. The data in the target buffer of DRAM is used as the final calculation result to generate a visualization for display to the user.

[0073] In this embodiment, a multi-threaded asynchronous collaborative mechanism is used to perform the computation processing for the current sequence number i in the computation thread, while the first communication thread is scheduled to execute the data loading task for the subsequent sequence number (i+1) and the network data acquisition task for the previous sequence number (i-1) in parallel. This parallel architecture masks the memory access latency and the communication latency of cross-node network transmission with the execution time of arithmetic operations, thereby significantly reducing the total processing latency of the distributed computing task. A dual asynchronous triggering mechanism using a first semaphore and a second semaphore decouples the dependency between the local computation result and the externally input network data. Dynamic updates of the semaphore values ​​automatically trigger the accumulation operation as soon as the data is ready, improving the effective utilization of the hardware computing unit and communication interface. Acquiring the next data block during the computation process (loading the i+1th data block) avoids the instantaneous congestion caused by concentrated memory access after the computation task ends, ensuring performance stability when processing extremely large amounts of data.

[0074] The following will describe an exemplary application of the data processing method of this application in a real-world application scenario.

[0075] With the explosive growth in the scale of AI model parameters, the demand for computing power and memory bandwidth has far exceeded the carrying capacity of the related computing architecture. In the von Neumann architecture, data is stored in memory, and computation is performed in the arithmetic logic unit (ALU) of the central processing unit (CPU), with the CPU and ALU separated by a bus. This architecture requires frequent data transfers between memory and the computation unit when performing computational tasks. The enormous data transfer overhead and access latency generated in this process constitute a bottleneck that severely limits CPU performance due to memory limitations. The energy consumption of data transfer can be hundreds of times greater than the computation itself, severely restricting the overall energy efficiency and performance of the computing architecture.

[0076] To alleviate the CPU performance limitations caused by memory performance, related technologies reduce unnecessary data transmission through converged computing, thereby achieving high energy efficiency, low latency, and high throughput computing. When converged computing technology is applied to distributed training (especially in ensemble communication scenarios), it still faces challenges: (1) Utilization of computing units. In typical converged computing designs, computing resources (such as specific processing units) are usually scarce and unique. How to efficiently and continuously utilize computing units in complex environments where computing tasks and communication tasks are intertwined or even concurrent, and avoid computing units being idle due to waiting for data or synchronization signals, is the key to improving overall performance. If the scheduling is not proper, the converged computing architecture may result in idle computing resources. (2) Fine synchronization between computing and communication. In distributed training, converged computing operations are not performed in isolation. Taking gradient synchronization as an example: On the communication side, when data (such as gradients) that has been computed locally needs to be sent to other nodes, it must be ensured that the data has been computed, otherwise communication errors will occur. On the computation side, when data (such as gradients) from other nodes needs to be aggregated for local updates or subsequent calculations, it is essential to ensure that all remote data involved in the computation has been delivered completely and correctly. This state dependency between "computation completion" and "data delivery" requires an efficient and low-overhead synchronization mechanism. Coarse-grained locking or global barrier synchronization in related technologies introduces significant performance jitter and cannot meet the high-performance and low-latency goals required for integrated computing.

[0077] Current convergence solutions for computing and general computing technologies have not yet resolved the aforementioned collaborative optimization issues. The lack of fine-grained scheduling strategies for individual computing units leads to low utilization; or, the synchronization primitives used (such as mutexes and condition variables) cannot meet the high-frequency, fine-grained synchronization requirements of distributed training, thus fundamentally limiting the potential of convergence technologies in distributed deep learning. Therefore, current technologies lack a solution that can ensure high utilization of the unique computing unit in a convergence architecture without introducing excessive overhead, while simultaneously providing a lightweight, non-blocking, and precise synchronization solution between computation and communication.

[0078] This application addresses the aforementioned problems in related technologies by providing a data processing method. By introducing a credit semaphore as a flow control and synchronization tool through a communication-computing fusion operator, a highly efficient "producer-consumer" pipeline is constructed within the communication-computing fusion operator. This processing method can overlap communication delays, transforming the originally serial computation and communication process into a highly parallel execution flow, fundamentally improving the continuous utilization of computing hardware.

[0079] See Figure 4 , Figure 4 This is a flowchart illustrating the data processing method provided in the embodiments of this application, which will be combined with... Figure 4The steps shown are explained. Figure 4 The entity responsible for executing the steps is Figure 1 Terminal device 400.

[0080] In step 401, the communication thread, the calculation thread, and the memory are configured, and the calculation semaphore and the communication semaphore are initialized in the memory space.

[0081] To facilitate understanding, the principle of multi-threaded computation and communication fusion involved in the embodiments of this application is explained. In this embodiment, data processing is performed on a data block basis. To accommodate the limited physical storage capacity of the underlying hardware's on-chip cache (SRAM), large-scale macroscopic input data (such as large-scale weight matrices or activation tensors in deep neural networks) is physically cut (Tiling / Chunking) according to preset dimensions and sizes, forming a series of sub-data sets with contiguous memory address spaces. These sub-data sets are data blocks. This embodiment uses three threads as an example: Thread 0 loads and communicates data blocks; Thread 1 performs computation based on A ping and B ping data blocks; and Thread 2 performs computation based on A ping and B pong data blocks. Therefore, when Thread 1 starts computation, it needs to ensure that Thread 0's A ping and B ping data blocks have been loaded; when Thread 2 performs computation, it needs to ensure that Thread 0's A ping and B pong data blocks have been loaded; simultaneously, it needs to ensure that communication operations with members in adjacent communication domains are completed within the time difference between data loading and computation completion.

[0082] In this embodiment, an execution unit with separate computation and communication threads is created. This functional separation is the basis for enabling parallel execution of communication and computation preparation processes. Within the computation thread, a ping-pong switching mechanism with double or multiple buffers is used for consecutive data blocks. This allows the computation unit to process the computation of the next data block immediately while waiting for one data block to be communicated, achieving continuous supply of computational tasks and maximizing resource utilization. During the time window when data has been transmitted to the computation unit but has not yet been processed by the computation thread, concurrent communication between nodes is executed, improving communication efficiency and saving communication overhead.

[0083] Figure 5This is a first schematic diagram of multi-threaded computational communication provided in an embodiment of this application. Taking three threads as an example, the horizontal direction represents the timing. Thread 0 finishes loading data blocks A ping1 and B ping1, then thread 1 begins to perform matrix multiplication operation 1 (GEMM) based on data blocks A ping1 and B ping1. Simultaneously, during the matrix multiplication operation 1 performed by thread 1, thread 0 loads data block B pong1 and performs communication 1. Subsequently, thread 0 loads data blocks A ping2 and B ping2, and synchronously, thread 2 performs matrix multiplication operation 2 based on data blocks A ping1 and B pong1. In response to thread 0 finishing loading data blocks A ping2 and B ping2, thread 1 begins to perform matrix multiplication operation 3 based on data blocks A ping2 and B ping2. During the matrix multiplication operation 3 performed by thread 1, thread 0 loads data block B pong2 and performs communication 2. When data blocks A ping2 and B ping2 are fully loaded, thread 2 performs matrix multiplication operation 4 based on data blocks A ping2 and B pong2.

[0084] Figure 6 This is a second schematic diagram of multi-threaded computational communication provided in an embodiment of this application; Figure 5 and Figure 6 This is a diagram illustrating the same thread under different data volumes. When the data volume is compared to... Figure 5 As growth occurs, communication times increase, and the communication pipeline can become full, for example... Figure 6 The circled area shows the communication pipeline 601, which is filled with cables.

[0085] In multi-threaded computational communication, this embodiment uses a credit semaphore for synchronization of computation and communication. Taking Reduce-Scatter as an example, the Reduce-Scatter operation is a many-to-many collective communication operation. Assume that computation semaphore ci1 represents the semaphore indicating computation completion, and communication semaphore ci2 represents the semaphore indicating communication completion. The initial values ​​of both computation semaphore ci1 and communication semaphore ci2 are -1. For ease of understanding, the following describes any computation node (i.e., a member of the communication domain) performing Reduce-Scatter collective communication. The computation node includes: Dynamic Random Access Memory (DRAM), which serves as both main memory and off-chip memory, used for persistent storage of raw input data; and Static Random Access Memory (SRAM), which serves as both on-chip cache and shared memory.

[0086] In this embodiment, a credit semaphore is introduced as the sole state medium for inter-thread coordination, replacing the locking mechanism of related technologies. The credit semaphore serves as a beacon for data readiness and buffer writability, precisely controlling the execution order of computation and communication in a non-blocking, event-driven manner, avoiding data races and reducing communication overhead.

[0087] In step 402, the computation semaphore is updated according to the computation processing steps, and the communication semaphore is updated according to the communication processing steps. In response to the computation semaphore and the communication semaphore meeting preset conditions, the computation processing between the locally loaded data and the data received from communication is performed.

[0088] refer to Figure 7 To the diagram, Figures 7 to 9 This is a schematic diagram illustrating the principle of synchronization of computing and communication provided in the embodiments of this application. Figures 7 to 9 This represents different stages of synchronization between computation and communication. Dynamic random access memory (DRAM) 701 and static random access memory (SRAM) 702 are used to perform computation and communication processes, respectively. DRAM 701 includes a buffer 7011, which is a destination buffer used to write the final calculation result. SRAM is divided into a resident storage area (for storing input data A ping), a first on-chip buffer (for storing input data B ping), a second on-chip buffer (for storing input data B pong), and a communication buffer for temporarily storing network data.

[0089] The hardware execution engines involved in the computation and communication process include: matrix multiplication calculation engine 703 and vector engine 704.

[0090] A Gemm engine typically refers to a hardware or software system specifically designed for efficiently performing General Matrix Multiply (GEMM). GEMM is a core operation in deep learning, scientific computing, and high-performance computing, and its performance directly impacts the efficiency of AI model training and inference. A vector engine is a computational system specifically designed for processing high-dimensional vector data. Its core function is to efficiently perform similarity search (i.e., quickly finding several vectors that are closest to a query vector in a massive dataset). It is widely used in artificial intelligence, recommender systems, image / speech recognition, natural language processing, and other scenarios.

[0091] Thread 0 is used to perform data transfer and cross-node communication between dynamic random access memory 701 and static random access memory 702; Thread 1 and Thread 2 are used to perform computational processing based on data 702 in static random access memory, such as matrix multiplication and vector accumulation calculation.

[0092] Continue to refer to Figure 7 To explain, Figure 7 The data flow between the static random access memory 702 and the dynamic random access memory 701 can be used to represent Figure 5 The data loading process of thread 0. Thread 0 reads input data A ping and the 0th block of input data B ping from dynamic random access memory, loads input data A ping into the resident storage area of ​​static random access memory, and loads the 0th block of input data B ping into the first on-chip buffer (B ping) of static random access memory.

[0093] After confirming that data A (ping) and input data B (ping) of block 0 have been loaded, thread 1 reads the data from the first on-chip buffer in the static random access memory (SRAM) and performs the calculation of the first block of data (block 0). The calculation result is temporarily stored in the SRAM. The calculated semaphore is then updated to ci1=0.

[0094] After thread 0 detects that ci1≥0 in real time, it retrieves the 0th block of computation results from static random access memory (SRAM) and sends it directly to the next member (next rank) in the communication domain via the network interface. Simultaneously, thread 0 retrieves the 1st block of input data, B pong, from dynamic random access memory (DRAM) and loads the B pong data block into the second on-chip buffer (B pong) of SRAM. This synchronous processing allows the processing time of each block to overlap, masking loading delays. At the same time, thread 0 receives the 0th block of network data from the previous member and stores it in SRAM. Subsequently, it updates the communication semaphore to ci2=0.

[0095] Threads 0 and 2 execute the processing in a loop. Thread 2 reads the x-th block of data from the second on-chip buffer (B pong) in the static random access memory and performs calculations with the resident input data. The result is stored in the static random access memory. After the calculation is completed, the computation semaphore is updated to ci1=x, where x is an integer.

[0096] The data flow between Static Random Access Memory (SRAM) and Dynamic Random Access Memory (DRAM) is local loading, which can be represented as follows: Under the scheduling of thread 0, the local data to be accumulated in step x is loaded from DRAM to SRAM. Thread 0 is also used to receive network data sent by the previous node. The network data sent by the previous computing node is the result of accumulating the local data of the previous computing node with the received network data.

[0097] The computer's low-level logic gates or synchronization primitives are continuously verified. The hardware execution engine (the matrix multiplication engine mentioned above) performs an accumulation operation on the already loaded local data and the received network data within the static random access memory (SRAM) if and only if the computation semaphore ci1 ≥ x and the communication semaphore ci2 ≥ x - 1 is satisfied. The computation semaphore ci1 ≥ x indicates that the x-th block of computation in the SRAM has been completed, and the communication semaphore ci2 ≥ x - 1 indicates that the x-th block of computation from the previous member in the SRAM has been completely received. One block of network data, and the local data loading from dynamic random access memory to static random access memory has been completed.

[0098] After the accumulation operation is completed, thread 0 sends the accumulated result from the static random access memory to the next member (next rank). After sending, the communication semaphore is updated to ci2=x, and the idle B ping buffer is used to continue loading the next block of input data from the dynamic random access memory.

[0099] In step 403, if the data block being processed is the last data block, the calculation result is stored in the target buffer of the dynamic random access memory in response to the completion of the calculation processing of the last data block.

[0100] When the processed data block is the last data block (the nth block, where n is the upper limit of the data block number), in response to the completion of the calculation of the nth block by threads 1 and 2, the calculation result is stored in the static random access memory (SRAM), and the calculation semaphore ci1 = n is updated. The data flow between SRAM and DRAM is represented as follows: thread 0 extracts the last local data block required for step n from DRAM to SRAM, and then updates the communication semaphore ci2 = n-1. When the verification conditions of calculation semaphore ci1 ≥ n and communication semaphore ci2 ≥ n-1 are met, the local data in SRAM and the network data received in step n-1 are accumulated. Calculation semaphore ci1 ≥ n indicates that the calculation of the last data block in SRAM has been completed, and communication semaphore ci2 ≥ n-1 indicates that the nth data block from the previous member has been completely received in SRAM. One block of network data, and the local data loading from dynamic random access memory to static random access memory has been completed.

[0101] After the final accumulation is completed, network communication commands are terminated. Thread 0 or the Direct Memory Access (DMA) controller extracts the final accumulated result data from the static random access memory and writes it unidirectionally to the target buffer allocated in the dynamic random access memory. The final accumulated result data is used as the final output data to be displayed to the user, completing the entire set communication task.

[0102] In some embodiments, the present application can also be applied to the following general matrix fusion operators: Gemm-Reduce-Scatter, Gemm-All Gather-Left, Gemm-All Gather-Right, and Ring-attention.

[0103] All Gather is a collective communication operation where each GPU sends its local data to all other GPUs, ultimately resulting in each GPU possessing the concatenated data from all GPUs. The left input matrix refers to the left input matrix of the general matrix multiplication operation performed by the All Gather operation (if calculated as C=A×B, then matrix A). The main purpose of Ring-attention is to extend the sequence length of the Transformer model. The core idea of ​​Ring-attention is that each GPU computes only a local attention, and the global attention is calculated using the acquired attention. Because the length of the attention matrix computed by each GPU is finite, but the overall attention matrix length is extended, and this extension increases linearly with the number of GPUs.

[0104] The embodiments of this application have the following beneficial effects: (1) Significantly improved Transactions Per Second (TPS) (a measure of system throughput) and computing resource utilization, resulting in performance improvement. By separating computing and communication functions through a multi-threaded architecture and combining it with ping-pong pipeline scheduling for different data blocks, the computing units can operate at full capacity. The above processing overlaps communication latency, freeing the computing units from idle waiting and bringing the utilization of the computing units close to the theoretical limit, thereby directly improving the system throughput.

[0105] (2) It achieves deep overlap between communication and computation in the time dimension, greatly improving the overall execution efficiency. It utilizes the time window between data transmission to the computing unit and the actual computation performed by the computing thread, and performs communication operations between different nodes concurrently within this time window. This makes communication no longer an independent stage that must wait for computation to completely stop, but rather communication and computation preparation processes occur in parallel. This deep, fine-grained time overlap reduces communication overhead, thereby significantly shortening the total completion time of a single training iteration and improving the overall efficiency of distributed training.

[0106] (3) It ensures strict data consistency and operational correctness with low overhead, thereby improving the robustness of the distributed parallel computing system. By introducing a lightweight credit semaphore mechanism, a "producer-consumer" mechanism and synchronization rules are established between the computing and communication threads. According to the credit semaphore, it is ensured that the communication sending thread will not touch the data before the computing thread completes the computation, and it is also ensured that the computing thread will not start the aggregation computation before the communication receiving thread has prepared all the data. This synchronization fundamentally avoids data disorder scenarios such as "data being transmitted before the computation is completed" and "the receiving end starting to compute before the data is fully transmitted", ensuring the correctness and reliability of the distributed training results. Moreover, the overhead of the credit semaphore mechanism is much lower than that of the lock synchronization of related technologies, ensuring high robustness under high performance.

[0107] (4) The optimization of the data processing method provided in this application is implemented at the operator and runtime levels. The ping-pong pipeline and synchronization mechanism do not depend on the layer structure or connection method of a specific neural network model and are independent of the model structure, thus having wide applicability. Whether it is a linear model or a network with complex residual connections, it can effectively improve performance and provide a general performance acceleration solution for various large-scale deep learning models.

[0108] The following description continues to illustrate the exemplary structure of the data processing apparatus 455 provided in the embodiments of this application as a software module. In some embodiments, such as Figure 2 As shown, the software modules stored in the data processing device 455 of the memory 450 may include: Semaphore update module 4551 is used to initialize a first semaphore and a second semaphore; calculation module 4552 is used to call a calculation thread to perform calculation processing on the i-th data block and obtain the i-th calculation result; semaphore update module 4551 is also used to update the value of the first semaphore to i, where i is an integer; communication module 4553 is used to call a first communication thread to load the (i+1)-th data block and obtain the (i-1)-th network data during the process of calling the calculation thread to perform calculation processing on the i-th data block, where the (i-1)-th network data is... The (i-1)th accumulated data sent by the previous computing node; the semaphore update module 4551 is further configured to update the second semaphore to i-1 after the first communication thread has completed acquiring the (i-1)th network data; the calculation module 4552 is configured to accumulate the i-th calculation result and the (i-1)th network data to obtain the i-th accumulated data when the first semaphore is greater than or equal to i and the second semaphore is greater than or equal to i-1; the communication module 4553 is configured to send the i-th accumulated data to the next computing node.

[0109] In some embodiments, the semaphore update module 4551 is configured to allocate a first buffer and a second buffer in a static random access memory before initializing the first semaphore and the second semaphore; the semaphore update module 4551 is configured to set the first semaphore to -1 and the second semaphore to -1.

[0110] In some embodiments, the computation thread includes: a first computation thread; and a communication module 4553, configured to, after initializing the first semaphore and the second semaphore, call the first communication thread to load an initial data block into a first buffer of a static random access memory, wherein the initial data block is labeled 0; call the first computation thread to extract the initial data block from the first buffer and perform matrix multiplication based on the initial data block to obtain a 0th computation result; during the process of calling the first computation thread to perform matrix multiplication based on the initial data block, call the first communication thread to load a 1st data block into a second buffer of a static random access memory; update the first semaphore to 0, and send the 0th computation result to the next computation node.

[0111] In some embodiments, the computation thread includes a first computation thread and a second computation thread; the computation module 4552 is configured to, when i is even, call the first computation thread to extract the i-th data block from the first buffer of the static random access memory, and perform matrix multiplication calculation based on the i-th data block to obtain the i-th computation result; when i is odd, call the second computation thread to extract the i-th data block from the second buffer of the static random access memory, and perform matrix multiplication calculation based on the i-th data block to obtain the i-th computation result.

[0112] In some embodiments, the communication module 4553 is configured to, when i+1 is an even number, call the first communication thread to load the i+1th data block from the dynamic random access memory to a first buffer of the static random access memory; and when i+1 is an odd number, call the first communication thread to load the i+1th data block from the dynamic random access memory to a second buffer of the static random access memory.

[0113] In some embodiments, the calculation module 4552 is used to call the calculation thread to extract the i-th calculation result and the (i-1)-th network data from the static random access memory; and to call the calculation thread to perform element-wise reduction addition on the i-th calculation result and the (i-1)-th network data to obtain the i-th accumulated data.

[0114] In some embodiments, the calculation module 4552 is configured to, when the first semaphore is greater than or equal to n and the second semaphore is greater than or equal to n-1, accumulate the nth calculation result and the (n-1)th network data to obtain the nth accumulated data, wherein n is the upper limit of the data block label; and call the first communication thread to store the nth accumulated data in the target buffer of the dynamic random access memory.

[0115] This application provides a computer program product, which includes a computer program or computer-executable instructions stored in a computer-readable storage medium. The processor of an electronic device reads the computer-executable instructions from the computer-readable storage medium and executes the computer-executable instructions, causing the electronic device to perform the data processing method described in this application.

[0116] This application provides a computer-readable storage medium storing computer-executable instructions or a computer program. When the computer-executable instructions or the computer program are executed by a processor, the processor will execute the data processing method provided in this application, for example... Figure 3 The data processing method is shown.

[0117] In some embodiments, the computer-readable storage medium may be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, flash memory, magnetic surface memory, optical disk, or CD-ROM; or it may be a variety of devices including one or any combination of the above-mentioned memories.

[0118] In some embodiments, computer-executable instructions may take the form of programs, software, software modules, scripts, or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as stand-alone programs or as modules, components, subroutines, or other units suitable for use in a computing environment.

[0119] As an example, computer-executable instructions may, but do not necessarily, correspond to files in a file system. They may be stored as part of a file that holds other programs or data, for example, in one or more scripts in a HyperText Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple co-located files (e.g., a file that stores one or more modules, subroutines, or code sections).

[0120] As an example, computer-executable instructions can be deployed to execute on a single electronic device, or on multiple electronic devices located at one location, or on multiple electronic devices distributed across multiple locations and interconnected via a communication network.

[0121] In summary, this application's embodiment utilizes a multi-threaded asynchronous collaborative mechanism. The computation thread executes the computation for the current sequence number i, while the first communication thread is scheduled to execute the data loading task for the subsequent sequence number (i+1) and the network data acquisition task for the previous sequence number (i-1) in parallel. This parallel architecture masks the memory access latency and the communication latency of cross-node network transmission with the execution time of arithmetic operations, thus significantly reducing the total processing latency of the distributed computing task. The dual asynchronous triggering mechanism using a first semaphore and a second semaphore decouples the dependency between local computation results and externally input network data. Dynamic updates to the semaphore values ​​automatically trigger the accumulation operation as soon as the data is ready, improving the effective utilization of the hardware computing unit and communication interface. Acquiring the next data block during computation (loading the i+1th data block) avoids instantaneous congestion caused by concentrated memory access after the computation task ends, ensuring performance stability when processing extremely large amounts of data.

[0122] The above description is merely an embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, and improvements made within the spirit and scope of this application are included within the scope of protection of this application.

Claims

1. A data processing method, characterized in that, Applied to a computing node, the method includes: Initialize the first semaphore and the second semaphore; The computation thread is invoked to perform computation on the i-th data block and obtain the i-th computation result. Update the value of the first semaphore to i, where i is an integer greater than or equal to 0; When i is greater than 0, during the process of calling the computing thread to perform calculation on the i-th data block, the first communication thread is called to load the (i+1)-th data block and obtain the (i-1)-th network data, wherein the (i-1)-th network data is the (i-1)-th accumulated data sent by the previous computing node; After the first communication thread has finished acquiring the (i-1)th network data, the second semaphore is updated to i-1; When the first semaphore is greater than or equal to i and the second semaphore is greater than or equal to i-1, the i-th calculation result and the i-1 network data are accumulated to obtain the i-th accumulated data; The i-th accumulated data is sent to the next computing node.

2. The method according to claim 1, characterized in that, Before initializing the first semaphore and the second semaphore, the method further includes: Allocate a first buffer and a second buffer in the static random access memory; The initialization of the first semaphore and the second semaphore includes: Set the first semaphore to -1 and the second semaphore to -1.

3. The method according to claim 2, characterized in that, The computation thread includes: a first computation thread; after initializing the first semaphore and the second semaphore, the method further includes: The first communication thread is invoked to load the initial data block into the first buffer of the static random access memory, wherein the initial data block is labeled 0; The first computing thread is invoked to extract the initial data block from the first buffer, and matrix multiplication is performed based on the initial data block to obtain the 0th calculation result; During the process of calling the first computing thread to perform matrix multiplication calculation based on the initial data block, the first communication thread is called to load the first data block into the second buffer of the static random access memory. The first semaphore is updated to 0, and the 0th calculation result is sent to the next computing node.

4. The method according to claim 2, characterized in that, The computing threads include: a first computing thread and a second computing thread; The invocation of the computation thread to perform computation on the i-th data block and obtain the i-th computation result includes: When i is an even number, the first calculation thread is invoked to extract the i-th data block from the first buffer of the static random access memory, and matrix multiplication is performed based on the i-th data block to obtain the i-th calculation result; When i is an odd number, the second calculation thread is invoked to extract the i-th data block from the second buffer of the static random access memory, and matrix multiplication is performed based on the i-th data block to obtain the i-th calculation result.

5. The method according to claim 2, characterized in that, The step of calling the first communication thread to load the (i+1)th data block includes: When i+1 is an even number, the first communication thread is invoked to load the i+1th data block from the dynamic random access memory into the first buffer of the static random access memory; When i+1 is an odd number, the first communication thread is invoked to load the i+1th data block from the dynamic random access memory into the second buffer of the static random access memory.

6. The method according to claim 1, characterized in that, The step of accumulating the i-th calculation result and the (i-1)-th network data to obtain the i-th accumulated data includes: The computation thread is invoked to retrieve the i-th computation result and the (i-1)-th network data from the static random access memory. The computation thread is invoked to perform element-wise reduction addition on the i-th computation result and the (i-1)-th network data to obtain the i-th accumulated data.

7. The method according to claim 1, characterized in that, The method further includes: When the first semaphore is greater than or equal to n and the second semaphore is greater than or equal to n-1, the nth calculation result and the (n-1)th network data are accumulated to obtain the nth accumulated data, where n is the upper limit of the data block number; The first communication thread is invoked to store the nth accumulated data into the target buffer of the dynamic random access memory.

8. A data processing apparatus, characterized in that, The device includes: The semaphore update module is used to initialize the first semaphore and the second semaphore; The calculation module is used to call the calculation thread to perform calculations on the i-th data block and obtain the i-th calculation result; The semaphore update module is further configured to update the value of the first semaphore to i, wherein i is an integer greater than or equal to 0; The communication module is used to, when i is greater than 0, during the process of calling the computing thread to perform calculation on the i-th data block, call the first communication thread to load the (i+1)-th data block and obtain the (i-1)-th network data, wherein the (i-1)-th network data is the (i-1)-th accumulated data sent by the previous computing node; The semaphore update module is further configured to update the second semaphore to i-1 after the first communication thread has finished acquiring the (i-1)th network data; The calculation module is used to accumulate the i-th calculation result and the i-1-th network data to obtain the i-th accumulated data when the first semaphore is greater than or equal to i and the second semaphore is greater than or equal to i-1. The communication module is used to send the i-th accumulated data to the next computing node.

9. An electronic device, characterized in that, The electronic device includes: Memory is used to store executable instructions or computer programs. A processor, when executing computer-executable instructions or computer programs stored in the memory, implements the data processing method according to any one of claims 1 to 7.

10. A computer-readable storage medium storing computer-executable instructions or a computer program, characterized in that, When the computer-executable instructions or computer program are executed by a processor, they implement the data processing method according to any one of claims 1 to 7.

Images