Data processing method, apparatus, system, device, medium, and program product

Abstract

The application provides a data processing method, device, system, equipment, medium and program product. The data processing method comprises: in the case of triggering an event, a first tensor processor sends first data to a second tensor processor based on a preset rule, the preset rule being used to indicate a destination address of the first data; and after the first data is sent, a value of a flag bit in the second tensor processor is set to a first value to indicate that the first data is sent. In the application, the first tensor processor and the second tensor processor directly acquire whether the other party has completed data sending or data receiving by reading the value of the flag bit, so that communication delay can be reduced, and the performance and stability of the data processing system can be improved.

Description

Technical Field

[0001] This application relates to the field of communication technology, and in particular to a data processing method, apparatus, system, device, medium, and program product. Background Technology

[0002] In a distributed data processing system, the master processor is connected to multiple tensor processing units (TPUs). The master processor controls the data communication between the multiple TPUs to execute distributed communication tasks.

[0003] Typically, the main control processor first sends data communication instructions to each TPU. These instructions include information such as the sender and receiver of the communication data, as well as the storage address and length of the communication data. Then, each TPU executes a communication task based on these instructions, for example, TPU1 sends data to TPU2. Once a TPU completes its communication task, it sends an interrupt signal to the main control processor to confirm completion, allowing the next step to proceed. However, due to the high load on the main control processor, it may control multiple TPUs to communicate simultaneously, potentially receiving interrupt signals from multiple TPUs concurrently. This simultaneous interrupt signal reception by the main control processor due to multiple TPUs can cause a delay, leading to poor performance and instability in the data processing system. Summary of the Invention

[0004] This application provides a data processing method, apparatus, system, device, medium, and program product that can reduce communication latency and improve the performance and stability of the data processing system.

[0005] In a first aspect, this application provides a data processing method executed by a first tensor processor. The method includes: sending first data to a second tensor processor based on a preset rule when a triggering event occurs, the preset rule being used to indicate the destination address of the first data; and setting the value of a flag bit in the second tensor processor to a first value after the first data is sent to indicate that the first data has been sent.

[0006] In some possible implementations, sending first data to the second tensor processor upon the occurrence of a trigger event includes: obtaining the value of a flag bit upon the occurrence of the trigger event; and sending the first data to the second tensor processor if the value of the flag bit is a second value, wherein the second value is used to indicate that the second tensor processor is capable of receiving the first data sent by the first tensor processor.

[0007] In some possible implementations, sending the first data to the second tensor processor includes: reading the first data from the first memory of the first tensor processor, the first memory storing the data to be sent by the first tensor processor; and sending the first data to the destination address of the second tensor processor.

[0008] In some possible implementations, after the first data transmission is completed, the value of the flag bit in the second tensor processor is set to a first value to indicate that the first data transmission is complete. This includes: after the first data transmission is completed, writing the first value into a first address of the first tensor processor, the first address corresponding to a second address of the flag bit memory where the flag bit is stored; and writing the first value into the second address by synchronizing the first address and the second address, so that the value of the flag bit is set to the first value.

[0009] In some possible implementations, after setting the value of the flag bit in the second tensor processor to a first value, the method further includes: obtaining the value of the flag bit; and determining that the second tensor processor has received the first data if the value of the flag bit is a second value.

[0010] In some possible implementations, when a triggering event occurs, sending first data to the second tensor processor based on preset rules includes: receiving a communication instruction sent by the master processor, the communication instruction being used to instruct the first tensor processor to perform a communication task; and sending the first data to the second tensor processor based on preset rules.

[0011] In some possible implementations, when a triggering event occurs, sending first data to the second tensor processor based on preset rules includes: receiving first data sent by a third tensor processor, where the third tensor processor is the processor of the previous node; and sending the first data to the second tensor processor based on preset rules.

[0012] Secondly, this application provides a data processing method executed by a second tensor processor. The method includes: receiving a communication instruction sent by a master processor, the communication instruction being used to instruct the second tensor processor to perform a communication task; receiving first data sent by a first tensor processor; and setting the value of a flag bit to a second value after the first data is received, to indicate that the first data reception is complete.

[0013] In some possible implementations, after the first data reception is completed, the value of the flag bit is set to a second value, including: after the first data reception is completed, the first value of the flag bit is set to a second value, where the first value is set by the first tensor processor and is used to indicate that the first data transmission is complete.

[0014] In some possible implementations, before receiving the first data sent by the first tensor processor, the method further includes: after receiving a communication instruction, setting the value of a flag bit to a second value, the second value being used to indicate that the second tensor processor is capable of receiving the first data sent by the first tensor processor.

[0015] Thirdly, this application provides a data processing system, which includes a main control processor, a first tensor processor, and a second tensor processor. The main control processor is used to send communication instructions to the first tensor processor and the second tensor processor, and the communication instructions are used to instruct the first tensor processor and the second tensor processor to perform communication tasks. The first tensor processor is used to perform the steps of the data processing method of the first aspect. The second tensor processor is used to perform the steps of the data processing method of the second aspect.

[0016] Fourthly, this application provides a data processing apparatus, comprising: a sending module, configured to send first data to a second tensor processor based on a preset rule when a triggering event occurs, the preset rule being used to indicate the destination address of the first data; and a first processing module, configured to set the value of a flag bit in the second tensor processor to a first value after the first data is sent, so as to indicate that the first data has been sent.

[0017] Fifthly, this application provides a data processing apparatus, comprising: a first receiving module for receiving a communication instruction sent by a main control processor, the communication instruction being used to instruct a second tensor processor to perform a communication task; a second receiving module for receiving first data sent by a first tensor processor; and a second processing module for setting the value of a flag bit to a second value after the first data reception is completed, so as to indicate that the first data reception is complete.

[0018] Sixthly, this application provides a computer device including a memory and a processor, wherein the memory stores a computer program executable on the processor, and the processor executes the program to implement the steps in the above-described data processing method.

[0019] In a seventh aspect, this application provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed, implements the steps in the above-described data processing method.

[0020] Eighthly, this application provides a computer program product, including a non-transitory computer-readable storage medium storing a computer program, which, when executed, implements the steps in the above-described data processing method.

[0021] The advantages of the technical solution provided in this application compared with the prior art are as follows:

[0022] In this embodiment, when a trigger event occurs, the first tensor processor first sends the first data to the second tensor processor based on the destination address of the first data indicated by a preset rule. After the first data is sent, the value of the flag bit in the second tensor processor is set to a first value, so that the second tensor processor determines that the first data transmission is complete based on the first value. Thus, when the first and second tensor processors communicate, both tensor processors only need to modify the value of the flag bit in the receiver when data transmission or reception is complete. The other tensor processor can then confirm the completion of data transmission or reception based on the modified value. Therefore, there is no need to notify the main control processor of the completion of data transmission or reception, and there is no delay caused by the main control processor's interruption of the receive task. This reduces communication latency and improves system performance and stability.

[0023] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description

[0024] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0025] Figure 1 This is a schematic diagram of the architecture of a data processing system provided in an embodiment of this application;

[0026] Figure 2 This is a schematic diagram illustrating an implementation flow of the data processing method provided in an embodiment of this application;

[0027] Figure 3 This is a schematic diagram of data transmission provided in an embodiment of this application;

[0028] Figure 4 This is a schematic diagram of another implementation flow of the data processing method provided in the embodiments of this application;

[0029] Figure 5 This is a schematic diagram of another implementation flow of the data processing method provided in the embodiments of this application;

[0030] Figure 6 This is a schematic diagram of another implementation flow of the data processing method provided in the embodiments of this application;

[0031] Figure 7 This is a schematic diagram of another implementation flow of the data processing method provided in the embodiments of this application;

[0032] Figure 8 This is a schematic diagram of the composition structure of a data processing device provided in an embodiment of this application;

[0033] Figure 9 This is a schematic diagram of another component structure of the data processing device provided in the embodiments of this application. Detailed Implementation

[0034] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples consistent with some aspects of this application as detailed in the appended claims.

[0035] Before providing a detailed explanation of the data processing methods provided in the embodiments of this application, the application scenarios of the embodiments of this application will be introduced first.

[0036] In complex and efficient distributed data processing systems, the main control processor, as the central nervous system, plays a crucial role in coordinating and monitoring data exchange between various TPUs. However, with the expansion of data processing system scale and the surge in communication demands, the challenges faced by the main control processor are becoming increasingly severe. Especially when managing real-time data communication between multiple TPUs simultaneously, the load on the main control processor increases significantly, testing not only its processing power and efficiency but also directly affecting the response speed and stability of the entire system.

[0037] Currently, while the process of the main control processor distributing data communication instructions to each TPU is highly automated and precise, the subsequent interrupt signal management mechanism has become a potential bottleneck. Each TPU, after completing its assigned communication task, immediately sends an interrupt signal to the main control processor to report its task completion status and request the next instruction. However, when multiple TPUs complete their tasks and send interrupts almost simultaneously, the main control processor's interrupt handling queue quickly fills up, leading to the so-called "interrupt storm" phenomenon. This phenomenon not only causes delays in the main control processor's processing of interrupt requests but may also trigger additional context switching overhead due to resource contention, thus affecting the timely response to subsequent interrupt requests. This delay not only slows down the overall system efficiency but may also introduce data consistency problems due to disordered communication timing, reducing the reliability and stability of the data processing system. More seriously, the accumulation of delays and errors over a long period may cause the data processing system to enter an unstable state or even experience service interruption, severely impacting businesses that rely on this data processing system for data processing and analysis.

[0038] To address the aforementioned issues, this application provides a data processing method. This method involves setting a flag bit at the receiving end. After both tensor processors complete their communication tasks, they modify the value of the flag bit to notify the other party that the communication task has been completed. This reduces reliance on the main control processor, lowers communication latency, and improves system performance and stability.

[0039] In some embodiments, the above-described data processing method can be applied to a data processing system. This data processing system includes a main control processor and multiple TPUs. In one example, Figure 1 This is a schematic diagram of the architecture of a data processing system provided in an embodiment of this application. See also... Figure 1 The data processing system includes a main control processor 100 and TPUs 200-1, 200-2, ..., 200-n, where n is an integer. The main control processor 100 sends communication commands to TPUs 200-1, 200-2, ..., 200-n, instructing each TPU to perform a communication task. TPUs 200-1, 200-2, ..., 200-n autonomously initiate communication commands to complete the transmission of communication data.

[0040] In some embodiments, when TPU200-1 and TPU200-2 are communicating, with TPU200-1 acting as the data sender and TPU200-2 as the data receiver, TPU200-1, upon receiving a communication instruction from the main control processor to execute a communication task, sends first data to TPU200-2 based on the destination address of the first data indicated by a preset rule. After the first data transmission is complete, TPU200-1 sets the value of a flag bit in TPU200-2 to a first value, so that TPU200-2 determines that TPU200-1 has completed sending the first data based on the first value. TPU200-2, upon receiving a communication instruction from the main control processor to execute a communication task, receives the first data sent by TPU200-1. After the first data reception is complete, TPU200-2 sets the value of a flag bit to a second value, so that TPU200-1 determines that TPU200-2 has received the first data based on the second value. The process by which TPU200-1 and TPU200-2 perform the corresponding operations will be described in detail later and will not be repeated here.

[0041] In some embodiments, when TPU200-1, TPU200-2, and TPU200-3 communicate data, and TPU200-1 is determined to be the data sender, TPU200-2 to be the data relay, and TPU200-3 to be the data receiver based on preset rules, TPU200-1, upon receiving a communication instruction from the main control processor to execute a communication task, sends first data to TPU200-2. After the first data transmission is completed, TPU200-1 sets the value of a flag bit in TPU200-2 to a first value, so that TPU200-2 determines that TPU200-1 has completed sending the first data based on the first value. TPU200-2 receives the first data sent by TPU200-1, and after the first data reception is completed, sets the value of a flag bit to a second value, so that TPU200-1 determines that TPU200-2 has received the first data based on the second value. Furthermore, after receiving the first data, TPU200-2 sends the first data to TPU200-3. After the first data transmission is complete, TPU200-3 sets the value of a flag bit in TPU200-3 to a first value, allowing TPU200-3 to determine that TPU200-2 has completed sending the first data. TPU200-3 receives the first data sent by TPU200-2 and, after receiving the first data, sets the value of a flag bit to a second value, allowing TPU200-2 to determine that TPU200-3 has received the first data. The processes by which TPU200-1, TPU200-2, and TPU200-3 perform their respective operations will be explained in detail later and will not be elaborated upon here.

[0042] In some embodiments, the main control processor can be a central processing unit (CPU), a neural processing unit (NPU), or other processors.

[0043] Figure 2 This is a schematic diagram illustrating one implementation flow of the data processing method provided in this application. Figure 2 This explanation uses a tensor processor, including a first tensor processor (TPU200-1) and a second tensor processor (TPU200-2), as an example. Figure 2 As shown, the data processing method includes the following steps.

[0044] Step S201: The main control processor sends communication instructions to the first tensor processor and the second tensor processor.

[0045] In some embodiments, communication instructions are used to instruct the first tensor processor and the second tensor processor to perform communication tasks.

[0046] Understandably, a data processing system includes multiple tensor processors. When executing data processing tasks, data transfer may occur between two tensor processors or between multiple tensor processors. Therefore, the first tensor processor can be either the source sender or a data relay. The second tensor processor is the data receiver.

[0047] Understandably, distributed communication between processors involves complex communication behavior achieved by combining and stacking multiple communication instructions. The main control processor sends different communication instructions to each tensor processor. These instructions can include information such as the sender and receiver of the communication data, as well as the storage address and length of the communication data. After receiving the communication instruction, each tensor processor executes the corresponding task based on it. However, this places a heavy load on the main control processor, which may lead to instability in the communication system. Therefore, in this embodiment, the main control processor sends only one identical communication instruction to each tensor processor, which instructs each tensor processor to perform a communication task.

[0048] Step S202: The first tensor processor and the second tensor processor receive communication instructions.

[0049] Understandably, each tensor processor can autonomously begin executing distributed communication tasks after receiving the data processing instruction.

[0050] In some embodiments, a data transmission rule (preset rule) can be preset. This rule may include the execution order of each tensor processor during data transmission, i.e., the data transmission address. Each tensor processor can generate a routing table based on this rule during runtime, and then transmit data based on this routing table. The routing table is an information base in the network device used to determine the data forwarding path. It may include information such as the data's destination address, next-hop address, metric value, and routing source. Thus, after receiving data, a tensor processor can forward the data to the next node according to the next-hop address in its locally generated routing table.

[0051] Step S203: Based on preset rules, the first tensor processor sends the first data to the second tensor processor.

[0052] As is understandable, a tensor processor is a processor designed for deep learning and other types of tensor computations. Tensor processors are widely used to accelerate the training and inference of machine learning models. Therefore, the first data that a tensor processor communicates with is the data required during the training and inference of the machine learning model. For example, the first data may include model parameters, gradient information, weights, activation values, control information, and other data.

[0053] In some embodiments, when the first tensor processor is the source sender of the first data, after receiving the communication instruction, the first tensor processor can determine the recipient of the first data based on the next-hop address in the local routing table generated by preset rules, and then send the first data to the second tensor processor.

[0054] In some embodiments, after receiving a communication instruction from the main processor, the first tensor processor can begin executing a communication task. The first tensor processor can prepare first data to be sent; for example, the first tensor processor can retrieve the first data from internal storage or a cache and send the first data to the second tensor processor.

[0055] In some embodiments, when the first tensor processor acts as a relay for the first data, after receiving the communication instruction, the first tensor processor can determine the recipient of the first data based on the next-hop address in the local routing table generated by the third tensor processor when it receives the first data sent by the third tensor processor, thereby sending the first data to the second tensor processor. The third tensor processor is the tensor processor of the previous node.

[0056] Understandably, when the first tensor processor acts as an intermediate tensor processor, the source of the first data should be the tensor processor located at the node above it. Therefore, in this case, after receiving the first data from the tensor processor at the node above it, the first tensor processor can send the first data to the tensor processor at the next node, i.e., the second tensor processor.

[0057] In some embodiments, each tensor processor can perform data transmission preparation operations after receiving a data processing instruction from the main processor. For example, a tensor processor acting as the data sender can prepare the data to be sent and configure the hardware unit for data transmission. A tensor processor acting as the data receiver can set a flag bit on its side, the value of which indicates the state of the tensor processor. For example, a flag bit value of 0 (second value) indicates that the tensor processor can receive data, and a flag bit value of 1 (first value) indicates that the tensor processor cannot receive data. After setting the different meanings of the flag bit values, the second tensor processor can send these meanings to the first tensor processor to determine data synchronization with the first tensor processor based on the flag bit values.

[0058] In some embodiments, after receiving a communication instruction, the second tensor processor can set the value of the flag bit to a second value. The second value indicates that the second tensor processor can receive the first data sent by the first tensor processor, corresponding to step S2021. Thus, when the first tensor processor obtains that the value of the flag bit is the second value, it can send the first data to the second tensor processor.

[0059] In some embodiments, when a triggering event occurs, the first tensor processor can also obtain the value of a flag bit; if the flag bit value is a second value, it sends first data to the second tensor processor, the second value indicating that the second tensor processor can receive the first data sent by the first tensor processor. Wherein, when the first tensor processor is the source sender of the first data, the triggering event is the event where the first tensor processor receives a communication instruction. When the first tensor processor is the relayer of the first data, the triggering event is the event where the first tensor processor receives the first data sent by the tensor processor of the previous node.

[0060] Understandably, the first tensor processor can determine whether the second tensor processor can receive the first data sent by the first tensor processor by obtaining the value of a flag bit. After determining that the second tensor processor can receive the first data sent by the first tensor processor based on the value of the flag bit, the first data to be sent can be sent to the second tensor processor across chips through the configured hardware unit.

[0061] In some embodiments, a certain address in the data sender can be pre-set to correspond to the address of the flag bit memory in the data receiver, that is, different addresses on the two tensor processors correspond to each other. For example, a first address in the first tensor processor can be pre-set to correspond to a second address of the flag bit memory in the second tensor processor. In this way, the first tensor processor can obtain the value of the flag bit by accessing its own first address. If the value of the flag bit set by the second tensor processor is the first value, it indicates that the second tensor processor cannot receive the first data sent by the first tensor processor; if the value of the flag bit is the second value, it indicates that the second tensor processor can receive the first data sent by the first tensor processor. Thus, when the value of the flag bit is obtained as the first value, the first tensor processor can determine that the second tensor processor may be busy and unable to receive new data. At this time, the first tensor processor can continue to obtain the value of the flag bit. When the value of the flag bit is obtained as the second value, it determines that the second tensor processor can now receive the first data sent by the first tensor processor, and then the first tensor processor can send the first data to the second tensor processor.

[0062] In some embodiments, the process of the first tensor processor sending the first data to the second tensor processor in step S203 can be as follows: reading the first data from the first memory of the first tensor processor, wherein the first memory stores the data to be sent by the first tensor processor; and sending the first data to the destination address of the second tensor processor.

[0063] Understandably, the first tensor processor can first retrieve the first data from the first memory storing the data to be sent, and then send the first data to the destination address of the second tensor processor. For example, the first tensor processor can send a data read request to the first memory, which carries information indicating the first data. After receiving the data read request, the first memory can return the first data to the first tensor processor. Then, the first tensor processor sends the received first data to the memory address of the second tensor processor or a specific data receiving interface through a data processing mechanism. The data processing mechanism may include mechanisms such as the system bus, peripheral component interconnect express (PCIe) bus, and network interface.

[0064] In some embodiments, after receiving the first data, the first tensor processor may perform preprocessing operations on the first data before sending it to the second tensor processor to ensure the integrity and security of the first data to be sent. For example, the first tensor processor may perform formatting or encoding operations on the first data; furthermore, it may perform compression or encryption operations on the first data.

[0065] In some embodiments, the first tensor processor can read first data from the first memory through the address access unit and send the first data to the destination address of the second tensor processor through the address forwarding unit.

[0066] In some embodiments, after receiving a data processing instruction, the first tensor processor, acting as the data sender, can configure the hardware units required for data transmission and store these hardware units in registers. Exemplarily, the first tensor processor can be configured with an address access unit and an address forwarding unit. The address access unit is used to access the first tensor processor's first memory and read first data, while the address forwarding unit is used for data forwarding. Exemplarily, the address access unit can be a chip direct memory access (CDMA) unit, and the address forwarding unit can be an address translation unit (ATU) in a PCIe bus. The ATU unit is a key component in the PCIe bus, responsible for address translation to ensure that data is correctly routed to the target device.

[0067] In one example, after receiving a data processing instruction, the master control unit in the first tensor processor can configure the CDMA unit and the ATU unit in the PCIe bus. Taking ATU configuration as an example, the master control unit can send a configuration command to the PCIe switch via the PCIe bus. This configuration command includes information such as the data source address, destination address, and address translation rules. The ATU unit of the PCIe switch updates its internal table entries according to the configuration command to support subsequent data forwarding. After configuring the CDMA unit and ATU unit, the master control unit can obtain the first data from the first memory through the CDMA unit, and then encapsulate the first data according to the PCIe protocol, including adding necessary header information, such as the data destination address, data source address, and data length. The encapsulated first data is then sent to the PCIe switch via the PCIe bus. The PCIe switch routes the first data to the destination address of the second tensor processor according to the address translation rules configured in the ATU unit. For example, the master control unit in the first tensor processor can be a scalar unit.

[0068] Figure 3 This is a schematic diagram of data transmission provided in an embodiment of this application, such as... Figure 3 As shown, after configuring the CDMA unit and ATU unit, the first tensor processor (TPU200-1) can read first data from the first memory of TPU200-1, and then send the first data to a temporary storage address (bar) on the main control processor. The ATU unit then forwards the first data to the destination address (e.g., the second memory) of the second tensor processor (TPU200-2). In some embodiments, the first data may not be sent to the bar on the main control processor; instead, it can be directly forwarded to the destination address of the second tensor processor via the ATU unit. This reduces the involvement of the main control processor and improves system stability.

[0069] Step S204: After the first data transmission is completed, the first tensor processor sets the value of the flag bit in the second tensor processor to the first value to indicate that the first data transmission is complete.

[0070] In some embodiments, step S204 can be implemented as follows: after the first data is sent, the first tensor processor writes the first value into the first address of the first tensor processor, and the first address corresponds to the second address of the flag bit memory that stores the flag bit; by synchronizing the first address and the second address, the first value is written into the second address so that the value of the flag bit is set to the first value.

[0071] Understandably, in this embodiment, a first address in the first tensor processor is pre-set to correspond to a second address in the memory storing the flag bits in the second tensor processor. This ensures that the two addresses are logically or physically related. Therefore, when the first processor writes the first value to the first address, the system can use this address correspondence to synchronize the first value to the second address. This allows the value of the flag bits in the second tensor processor to be updated to the first value, enabling the second tensor processor to determine that the first data transmission is complete based on the first value.

[0072] Step S205: The second tensor processor receives the first data.

[0073] In some embodiments, after setting the value of the flag bit to a second value, the second tensor processor can continuously acquire the value of the flag bit. When the acquired flag bit value is updated from the second value to the first value, the second tensor processor determines that the first data transmission is complete. In this way, the second tensor processor can receive the first data so that it can be used for subsequent model training and inference.

[0074] Step S206: After the first data reception is completed, the second tensor processor sets the value of the flag bit to a second value to indicate that the first data reception is complete.

[0075] Understandably, after receiving the first data, the second tensor processor can modify the value of the flag bit so that the first tensor processor can determine that the second processor has received the first data based on the modified value, thus enabling data synchronization between the two tensor processors.

[0076] In some embodiments, after the first tensor processor sets the value of the flag bit in the second tensor processor to the first value, it obtains the value of the flag bit, corresponding to step S207.

[0077] Understandably, after the first tensor processor sets the value of the flag bit in the second tensor processor to the first value, it can continuously acquire the value of the flag bit to determine whether the second tensor processor has received the first data based on the value of the flag bit.

[0078] In some embodiments, when the value of the flag bit is a second value, the first tensor processor determines that the second tensor processor has received the first data, corresponding to step S208.

[0079] Understandably, if the first tensor processor obtains that the value of the flag bit has been updated from the first value to the second value, then the first tensor processor determines that the second tensor processor has received the first data.

[0080] In this embodiment, the main control processor only needs to send communication instructions to the first and second tensor processors to instruct each tensor processor to begin executing communication tasks. The first tensor processor can send first data to the second tensor processor. After the first data is sent, the value of the flag bit in the second tensor processor is set to a first value, so that the second tensor processor determines that the first data transmission is complete based on the first value. The second tensor processor receives the first data sent by the first tensor processor, and after the first data is sent, it updates the value of the flag bit from the first value to a second value, so that the first tensor processor determines that the first data reception is complete. In this way, when the first and second tensor processors are communicating, both tensor processors only need to modify the value of the flag bit in the receiver when data transmission or reception is complete. The other tensor processor can confirm the completion of data transmission or reception based on the modified value. Thus, there is no need to notify the main control processor when data transmission or reception is complete, and there is no delay caused by the main control processor receiving interruption tasks. This can reduce communication latency and improve system performance and stability.

[0081] The above describes the data processing method provided in this application using the interaction between processors in a data operating system as an example. The following section focuses specifically on the first tensor processor as the execution entity to introduce the data processing method provided in this application.

[0082] Figure 4 This is a schematic diagram illustrating an implementation flow of the data processing method provided in an embodiment of this application. This data processing method is executed by a first tensor processor. Figure 4 As shown, the method includes steps S401 to S402.

[0083] Step S401: When a trigger event occurs, send the first data to the second tensor processor based on preset rules.

[0084] In some embodiments, a preset rule is used to indicate the destination address of the first data.

[0085] In some embodiments, when the first tensor processor is the source sender of the data, the implementation process of step S401 can be: receiving a communication instruction sent by the main control processor, the communication instruction being used to instruct the first tensor processor to perform a communication task; and sending the first data to the second tensor processor based on a preset rule.

[0086] Understandably, distributed communication between processors is a complex communication behavior achieved by combining and stacking multiple communication instructions. The main control processor sends different communication instructions to each tensor processor. These instructions can include information such as the sender and receiver of the communication data, as well as the storage address and length of the communication data. After receiving the communication instruction, each tensor processor executes the corresponding task based on the instruction. However, this places a heavy load on the main control processor, which may lead to instability in the data processing system. Therefore, in this embodiment, the main control processor sends only one identical communication instruction to each tensor processor. This instruction instructs each tensor processor to execute a communication task. After receiving this instruction, each tensor processor can autonomously begin executing the distributed communication task.

[0087] In some embodiments, after receiving a communication instruction from the main control processor, the first tensor processor determines to begin executing a communication task. The first tensor processor may prepare first data to be sent. For example, the first tensor processor may retrieve the first data from internal storage or cache, and determine the recipient of the first data based on the next-hop address in the local routing table generated by preset rules, thereby sending the first data to the second tensor processor.

[0088] In some embodiments, when a triggering event occurs, the first tensor processor can also obtain the value of a flag bit; if the value of the flag bit is a second value, it sends first data to the second tensor processor, the second value being used to indicate that the second tensor processor is capable of receiving the first data sent by the first tensor processor.

[0089] In some embodiments, each tensor processor, upon receiving a data processing instruction from the main processor, can perform data transmission preparation operations. For example, a tensor processor acting as the data sender can prepare the data to be sent and configure hardware units for data transmission. A tensor processor acting as the data receiver can set a flag bit on its side, the value of which indicates the state of the tensor processor. For instance, a flag bit value of 0 (second value) indicates that the tensor processor can receive data, and a flag bit value of 1 (first value) indicates that the tensor processor cannot receive data. After setting the different meanings of the flag bit values, the second tensor processor can send these meanings to the first tensor processor to synchronize data with it based on the flag bit values. Thus, the first processor can obtain the flag bit values ​​to determine whether the second tensor processor can receive the first data sent by the first tensor processor. After determining that the second tensor processor can receive the first data sent by the first tensor processor based on the flag bit values, the first data to be sent can be transmitted across chips to the second tensor processor through the configured hardware units.

[0090] In some embodiments, a certain address in the data sender can be pre-set to correspond to the address of the flag bit memory in the data receiver, that is, different addresses on the two tensor processors correspond to each other. For example, a first address in the first tensor processor can be pre-set to correspond to a second address of the flag bit memory in the second tensor processor. In this way, the first tensor processor can obtain the value of the flag bit by accessing its own first address. If the value of the flag bit set by the second tensor processor is the first value, it indicates that the second tensor processor cannot receive the first data sent by the first tensor processor; if the value of the flag bit is the second value, it indicates that the second tensor processor can receive the first data sent by the first tensor processor. Thus, when the value of the flag bit is obtained as the first value, the first tensor processor determines that the second tensor processor may be busy and unable to receive new data. At this time, the first tensor processor can continue to obtain the value of the flag bit. When the value of the flag bit is obtained as the second value, it determines that the second tensor processor can now receive the first data sent by the first tensor processor, and then the first tensor processor can send the first data to the second tensor processor.

[0091] In some embodiments, the process of the first tensor processor sending the first data to the second tensor processor in step S401 can be as follows: reading the first data from the first memory of the first tensor processor, the first memory storing the data to be sent by the first tensor processor; and sending the first data to the destination address of the second tensor processor. This implementation process can be found above. Figure 2 The relevant content in step S203 will not be repeated here.

[0092] The above describes the process of the first tensor processor acting as the source sender of the first data to transmit data. When the first tensor processor acts as an intermediary for the first data transmission, step S401 can be implemented as follows: receiving the first data sent by the third tensor processor, where the third tensor processor is the tensor processor of the previous node; and sending the first data to the second tensor processor based on a preset rule.

[0093] Understandably, when the first tensor processor acts as an intermediate tensor processor, the source of the first data should be the tensor processor located at the node above it. Therefore, in this case, after receiving the first data from the tensor processor at the node above it, the first tensor processor can determine the recipient of the first data based on the next-hop address in its local routing table generated by preset rules, and then send the first data to the tensor processor at the next node, i.e., the second tensor processor. The process of the first tensor processor sending the first data to the second tensor processor can be found in the section on the first tensor processor as the source sender of the first data sending to the second tensor processor, and will not be repeated here.

[0094] Step S402: After the first data transmission is completed, set the value of the flag bit in the second tensor processor to the first value to indicate that the first data transmission is complete.

[0095] In some embodiments, step S402 can be implemented as follows: after the first data is sent, the first value is written into the first address of the first tensor processor, and the first address corresponds to the second address of the flag bit memory that stores the flag bit; by synchronizing the first address and the second address, the first value is written into the second address so that the value of the flag bit is set to the first value.

[0096] Understandably, in this embodiment, a first address in the first tensor processor is pre-defined to correspond to a second address in the memory storing the flag bit in the second tensor processor. This ensures that the two addresses are logically or physically related. Therefore, when the first processor writes a first value to the first address, the system can utilize this address correspondence to synchronize the first value to the second address, thus updating the value of the flag bit in the second tensor processor to the first value. Understandably, after setting the value of the flag bit to the second value, the second tensor processor can continuously acquire the value of the flag bit. When it acquires a flag bit value updated from the second value to the first value, the second tensor processor determines that the first data transmission is complete. This enables data synchronization between the two tensor processors.

[0097] The above describes a scheme for the data sender to promptly notify the data receiver that data transmission is complete. Additionally, in this embodiment, the data receiver can also promptly notify the data sender that data reception is complete.

[0098] In some embodiments, after setting the value of the flag bit in the second tensor processor to a first value, the first tensor processor can also obtain the value of the flag bit; if the value of the flag bit is a second value, it is determined that the second tensor processor has received the first data.

[0099] Understandably, after setting the flag value in the second tensor processor to the first value, the first tensor processor can continuously acquire the flag value to determine whether the second tensor processor has received the first data. If the flag value is updated from the first value to the second value, the first tensor processor determines that the second tensor processor has received the first data. The operation of updating the flag value to the second value is performed by the second tensor processor.

[0100] In some embodiments, after setting the value of the flag bit to the second value, the second tensor processor can continuously acquire the value of the flag bit. When the value of the flag bit is acquired and updated to the first value, the second tensor processor can determine that the first data transmission is complete. In this way, the second tensor processor can receive the first data, and after the first data is received, modify the value of the flag bit to the second value to indicate that the second tensor processor has received the first data.

[0101] The above describes a method for the first tensor processor to send first data to the second tensor processor and notify the second tensor processor that the first data transmission is complete. The following section will use... Figure 5 For example, the above method will be further explained.

[0102] Figure 5 This is a schematic diagram of another implementation flow of the data processing method provided in this application embodiment. The method is executed by a first tensor processor, such as... Figure 5 As shown, the method includes steps S501 to S507.

[0103] Step S501: If the trigger event occurs, obtain the value of the flag bit;

[0104] Step S502: If the value of the flag bit is the second value, read the first data from the first memory.

[0105] Step S503: Send the first data to the destination address of the second tensor processor based on the preset rules.

[0106] Step S504: After the first data is sent, the first value is written into the first address, which corresponds to the second address of the memory storing the flag bit.

[0107] Step S505: Synchronize the first address with the second address, write the first value to the second address, so that the value of the flag bit is set to the first value.

[0108] Step S506: Obtain the value of the flag bit.

[0109] Step S507: If the value of the flag bit is the second value, determine that the second tensor processor has received the first data.

[0110] In this embodiment, when a trigger event occurs, the first tensor processor sends first data to the second tensor processor. After the first data transmission is complete, the value of a flag bit in the second tensor processor is set to a first value, so that the second tensor processor determines that the first data transmission is complete based on the first value. Thus, when the first and second tensor processors communicate, both processors only need to modify the value of the flag bit in the receiver when data transmission or reception is complete. The other tensor processor can then confirm the completion of data transmission or reception based on the modified value. Therefore, there is no need to notify the main control processor of the completion of data transmission or reception, and there is no delay caused by the main control processor's interruption of the receive task. This reduces communication latency and improves system performance and stability.

[0111] The above describes a data processing method with the data sender as the executing entity. The following describes a data processing method according to an embodiment of this application with the data receiver as the executing entity.

[0112] Figure 6 This is a schematic diagram of another implementation flow of the data processing method provided in this application embodiment, which is executed by a second tensor processor. Figure 6 As shown, the method includes steps S601 to S603.

[0113] Step S601: Receive the communication instruction sent by the main control processor. The communication instruction is used to instruct the second tensor processor to perform the communication task.

[0114] Understandably, in a data processing system, when executing a communication task, the master processor sends the same communication instruction to each tensor processor. This instruction directs each tensor processor to execute the communication task, and upon receiving it, each tensor processor can autonomously begin executing the distributed communication task. Therefore, the second tensor processor first receives the communication instruction from the master processor and then begins executing the communication task.

[0115] In some embodiments, after receiving a communication command, the second tensor processor can set a flag on its own end. The value of this flag indicates the state of the second tensor processor. After setting the meaning of different flag values, the second tensor processor can send this meaning to the first tensor processor to determine data synchronization based on the flag values.

[0116] In some embodiments, the second tensor processor can set the value of a flag bit to a second value, which indicates that the second tensor processor can receive the first data sent by the first tensor processor. Thus, when the first tensor processor receives the second value as the flag bit, it can send the first data to the second tensor processor.

[0117] Step S602: Receive the first data sent by the first tensor processor.

[0118] In some embodiments, after setting the value of the flag bit to a second value, the second tensor processor can continuously acquire the value of the flag bit. When the acquired flag bit value is updated from the second value to the first value, the second tensor processor determines that the first data transmission is complete. In this way, the second tensor processor can receive the first data so that it can be used for subsequent model training and inference.

[0119] Step S603: After the first data reception is completed, set the value of the flag bit to the second value to indicate that the first data reception is complete.

[0120] Understandably, after receiving the first data, the second tensor processor can modify the value of the flag bit so that the first tensor processor can determine that the second processor has received the first data based on the modified value, thus enabling data synchronization between the two tensor processors.

[0121] The following is based on Figure 7 For example, the above method will be further explained.

[0122] Figure 7 This is a schematic diagram of another implementation flow of the data processing method provided in this application embodiment. The method is executed by a second tensor processor, such as... Figure 7 As shown, the method includes steps S701 to S704.

[0123] Step S701: Receive communication instructions sent by the main control processor;

[0124] Step S702: Set the value of the flag bit to the second value to instruct the first tensor processor to send the first data.

[0125] Step S703: Receive the first data sent by the first tensor processor.

[0126] Step S704: After the first data reception is completed, set the first value of the flag bit to the second value so that the first tensor processor can determine that the first data reception is complete.

[0127] In this embodiment, the second tensor processor receives a communication instruction sent by the main control processor, sets the value of the flag bit to a second value to instruct the first tensor processor to send first data, and then receives the first data sent by the first tensor processor. After the first data is sent, the value of the flag bit is updated from the first value to the second value so that the first tensor processor can determine that the first data reception is complete. In this way, data synchronization can be achieved between the two tensor processors without the need for the main control processor to participate in the data synchronization operation, which can reduce communication latency and improve system performance and stability.

[0128] This application also provides a data processing apparatus. Figure 8 This is a schematic diagram of the composition structure of a data processing apparatus provided in an embodiment of this application. For example... Figure 8 As shown, the data processing device 800 includes a sending module 801 and a first processing module 802. The sending module 801 is used to send first data to a second tensor processor based on a preset rule when a trigger event occurs. The preset rule indicates the destination address of the first data. The first processing module 802 is used to set the value of a flag bit in the second tensor processor to a first value after the first data transmission is completed, to indicate that the first data transmission is complete.

[0129] In some possible implementations, the sending module 801 is further configured to: obtain the value of a flag bit when a trigger event occurs; and send first data to the second tensor processor when the value of the flag bit is a second value, wherein the second value is used to indicate that the second tensor processor is capable of receiving the first data sent by the first tensor processor.

[0130] In some possible implementations, the sending module 801 is further configured to: read first data from the first memory of the first tensor processor, the first memory storing data to be sent by the first tensor processor; and send the first data to the destination address of the second tensor processor.

[0131] In some possible implementations, the first processing module 802 is further configured to: after the first data is sent, write the first value into the first address of the first tensor processor, the first address corresponding to the second address of the flag bit memory storing the flag bit; and write the first value into the second address by synchronizing the first address and the second address, so that the value of the flag bit is set to the first value.

[0132] In some possible implementations, the apparatus further includes: an acquisition module for acquiring the value of a flag bit; and a determination module for determining that the second tensor processor has received the first data if the value of the flag bit is a second value.

[0133] In some possible implementations, the sending module 801 is further configured to: receive a communication instruction sent by the main control processor, the communication instruction being used to instruct the first tensor processor to perform a communication task; and send first data to the second tensor processor based on a preset rule.

[0134] In some possible implementations, the sending module 801 is further configured to: receive first data sent by a third tensor processor, the third tensor processor being the processor of the previous node; and send the first data to a second tensor processor based on a preset rule.

[0135] This application also provides a data processing apparatus. Figure 9 This is a schematic diagram of another structural composition of the data processing apparatus provided in the embodiments of this application. For example... Figure 9 As shown, the data processing device 900 includes: a first receiving module 901, a second receiving module 902, and a second processing module 903. The first receiving module 901 is used to receive communication instructions sent by the main control processor, which instruct the second tensor processor to perform a communication task. The second receiving module 902 is used to receive first data sent by the first tensor processor. The second processing module 903 is used to set the value of a flag bit to a second value after the first data reception is completed, to indicate that the first data reception is complete.

[0136] In some possible implementations, the second processing module 903 is further configured to: after the first data reception is completed, set the first value of the flag bit to a second value, wherein the first value is set by the first tensor processor and is used to indicate that the first data transmission is complete.

[0137] In some possible implementations, the apparatus further includes a third processing module, configured to set the value of a flag bit to a second value after receiving a communication instruction, the second value being used to indicate that the second tensor processor is capable of receiving the first data sent by the first tensor processor.

[0138] The descriptions of the communication device embodiments above are similar to those of the method embodiments above, and have similar beneficial effects. In some embodiments, the functions or modules included in the communication device provided in this disclosure can be used to perform the methods described in the method embodiments above. For technical details not disclosed in the communication device embodiments of this application, please refer to the descriptions of the method embodiments of this application for understanding.

[0139] This application provides a computer device, including a memory and a processor. The memory stores a computer program that can run on the processor, and the processor executes the program to implement the steps in the above-described data processing method.

[0140] This application provides a computer-readable storage medium storing a computer program thereon. When executed by a processor, the computer program implements some or all of the steps in the data processing method described above. The computer-readable storage medium can be transient or non-transient.

[0141] This application provides a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program. When the computer program is read and executed by a computer, it implements some or all of the steps in the data processing method described above. This computer program product can be implemented specifically through hardware, software, or a combination thereof. In some embodiments, the computer program product is specifically embodied as a computer storage medium; in other embodiments, the computer program product is specifically embodied as a software product, such as a software development kit (SDK), etc.

[0142] It should be noted that other embodiments of this application will readily conceive of by those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein. The specification and embodiments are to be considered exemplary only, and the true scope and spirit of this application are indicated by the following claims.

[0143] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.

Claims

1. A data processing method, executed by a first tensor processor, characterized in that, The method includes: When a triggering event occurs, first data is sent to the second tensor processor based on a preset rule, wherein the preset rule is used to indicate the destination address of the first data. After the first data transmission is completed, the value of the flag bit in the second tensor processor is set to the first value to indicate that the first data transmission is complete; Obtain the value of the flag bit; If the value of the flag bit is the second value, it is determined that the second tensor processor has received the first data, and the second value is set by the second tensor processor.

2. The method according to claim 1, characterized in that, The step of sending the first data to the second tensor processor upon the occurrence of a triggering event includes: When the triggering event occurs, obtain the value of the flag bit; When the value of the flag bit is the second value, the first data is sent to the second tensor processor, and the second value is used to indicate that the second tensor processor can receive the first data sent by the first tensor processor.

3. The method according to claim 1 or 2, characterized in that, Sending the first data to the second tensor processor includes: Read the first data from the first memory of the first tensor processor, wherein the first memory stores the data to be sent by the first tensor processor; The first data is sent to the destination address of the second tensor processor.

4. The method according to claim 1, characterized in that, The step of setting the value of the flag bit in the second tensor processor to a first value after the first data transmission is completed, to indicate that the first data transmission is complete, includes: After the first data is sent, the first value is written into the first address of the first tensor processor, and the first address corresponds to the second address of the flag bit memory that stores the flag bit. By synchronizing the first address and the second address, the first value is written to the second address so that the value of the flag bit is set to the first value.

5. The method according to claim 1, characterized in that, The step of sending first data to the second tensor processor based on preset rules when a triggering event occurs includes: The system receives communication instructions sent by the main control processor, the communication instructions being used to instruct the first tensor processor to perform a communication task; The first data is sent to the second tensor processor based on the preset rules.

6. The method according to claim 1, characterized in that, The step of sending first data to the second tensor processor based on preset rules when a triggering event occurs includes: Receive the first data sent by the third tensor processor, wherein the third tensor processor is the tensor processor of the node preceding the first tensor processor; The first data is sent to the second tensor processor based on the preset rules.

7. A data processing method, executed by a second tensor processor, characterized in that, The method includes: The system receives communication instructions sent by the main control processor, which are used to instruct the second tensor processor to perform a communication task. Receive the first data sent by the first tensor processor; After the first data reception is completed, the first value of the flag bit in the second tensor processor is set to the second value to indicate that the first data reception is complete. The first value is set by the first tensor processor and is used to indicate that the first data transmission is complete.

8. The method according to claim 7, characterized in that, Before receiving the first data sent by the first tensor processor, the method further includes: After receiving the communication instruction, the value of the flag bit is set to the second value, which is used to indicate that the second tensor processor can receive the first data sent by the first tensor processor.

9. A data processing system, characterized in that, The data processing system includes a main control processor, a first tensor processor, and a second tensor processor. The main control processor is used to send communication instructions to the first tensor processor and the second tensor processor, and the communication instructions are used to instruct the first tensor processor and the second tensor processor to perform communication tasks. The first tensor processor is used to perform the steps in the data processing method as described in any one of claims 1 to 6; The second tensor processor is used to perform the steps in the data processing method as described in claim 7 or 8.

10. A data processing apparatus, applied to a first tensor processor, characterized in that, The data processing device includes: The sending module is used to send first data to the second tensor processor based on a preset rule when a triggering event occurs, wherein the preset rule is used to indicate the destination address of the first data; The first processing module is configured to set the value of the flag bit in the second tensor processor to a first value after the first data transmission is completed, so as to indicate that the first data transmission is complete; The acquisition module is used to acquire the value of the flag bit; The determination module is configured to determine, when the value of the flag bit is a second value, that the second tensor processor has received the first data, wherein the second value is set by the second tensor processor.

11. A data processing apparatus, applied to a second tensor processor, characterized in that, The data processing device includes: The first receiving module is used to receive communication instructions sent by the main control processor, the communication instructions being used to instruct the second tensor processor to perform a communication task; The second receiving module is used to receive the first data sent by the first tensor processor; The second processing module is configured to set the first value of the flag bit in the second tensor processor to the second value after the first data reception is completed, so as to indicate that the first data reception is completed. The first value is set by the first tensor processor and is used to indicate that the first data transmission is completed.

12. A computer device comprising a memory and a processor, the memory storing a computer program executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the data processing method as described in any one of claims 1 to 6 or claim 7 or 8.

13. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed, it implements the steps in the data processing method as described in any one of claims 1 to 6 or claim 7 or 8.

14. A computer program product comprising a non-transitory computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed, it implements the steps in the data processing method as described in any one of claims 1 to 6 or claim 7 or 8.

Images