Data processing methods and chips
By introducing proxy messages into the server cluster, the stability problem of data transmission in low-reliability cross-server scenarios is solved, enabling rapid fault identification and handling, preventing resource waste and fault propagation, and improving the overall stability of the server cluster.
Patent Information
- Application Number
- CN202510626219.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-05-15
- Publication Date
- 2026-11-17
AI Technical Summary
In low-reliability scenarios across servers, data transmission loss and fault propagation caused by existing memory access instructions affect the stability of server clusters, especially in AI clusters where poor optical connection reliability can easily lead to overall paralysis.
By introducing acknowledgment messages during data transmission, the cause of the fault can be identified and measures can be taken, such as retransmission, discarding instructions, releasing resources, and reporting the fault, thereby enhancing the stability of the server cluster.
It improves the stability of the server cluster during data transmission, prevents the spread of faults, and enhances the resource utilization and reliability of the server cluster.
Smart Images

Figure CN123123296A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of computer technology, and more particularly to data processing methods and chips. Background Technology
[0002] A server cluster is a system composed of multiple servers. Leveraging a distributed architecture, it significantly improves overall processing power, reliability, and resource utilization, and is widely used in fields such as cloud computing, big data analytics, and high-performance computing. In this highly collaborative system, the individual server chips within the cluster need to perform a large number of data transmission operations.
[0003] Currently, the industry is attempting to improve communication efficiency by using chip memory access instructions to achieve efficient data transfer between chips within a cluster. While this method performs well in high-reliability environments within servers, in low-reliability scenarios across servers, packet loss due to abnormal network transmission paths can lead to data loss. This not only affects the stability of the server cluster but may also cause serious service loss.
[0004] For example, AI chips in an artificial intelligence (AI) cluster need to perform a large amount of data transmission. In order to improve the communication capability between AI chips in the AI cluster, the industry has gradually adopted the direct use of chip memory access instructions for data transmission between AI chips. This simple communication mechanism is more conducive to improving the effective payload of communication data. However, this memory semantic transmission method is generally used in servers with high interconnection reliability. In AI clusters, sometimes the physical medium for interconnection is optical connection, and the reliability of optical connection is relatively poor. This can lead to data transmission failure or loss, resulting in the spread of faults and causing the entire AI cluster to be paralyzed. Summary of the Invention
[0005] This application provides a data processing method and a chip to enhance the stability of server clusters during data transmission. To achieve the above objectives, this application adopts the following technical solution:
[0006] In a first aspect, embodiments of this application provide a data processing method applied to a first chip in a server cluster. The first chip is a computing chip or a switching chip. The server cluster includes multiple servers connected via a secondary switching chip. Each of the multiple servers includes multiple computing chips connected via a primary switching chip. The method includes: acquiring a data transmission instruction; sending or executing the data transmission instruction; and generating or receiving a response message in the event of a data transmission instruction failure to be sent or executed. The data transmission instruction is a read instruction or a write instruction, and the response message indicates the reason for the failure to send or execute the data transmission instruction.
[0007] The solution provided in this application, when a fault occurs during data transmission in a server cluster (such as a data transmission command failing to be sent or executed successfully), will clearly indicate the specific cause of the fault through a proxy message. This allows relevant chips (such as the command initiator chip or the switching chip) to promptly learn the cause of the fault and take corresponding measures (such as releasing occupied resources, retransmitting the command, or discarding the command), thus avoiding resource waste and fault propagation. Therefore, by introducing proxy messages, this solution enhances the stability of the server cluster during data transmission.
[0008] In one possible implementation, generating the response message includes: generating the response message based on first information, wherein the first information includes at least one of the following: port number, sender information of the data transmission instruction, address range, or message format error information.
[0009] It can be understood that the port number, the sender information of the data transmission command, the address range, and message format error information are all closely related to the cause of the fault, and therefore, a response message can be generated based on this information. Specifically:
[0010] Port numbers are used to precisely locate the specific interface or link channel through which data transmission commands are transmitted. By using port numbers, it is possible to identify faults such as a port being inactive, occupied, or incompatible with the target device, and thus generate corresponding response messages based on the port number.
[0011] The sender information of a data transmission command reveals key information such as the identity, permissions, or type of the command's origin. By analyzing the sender information, the causes of failures such as authentication failure, insufficient permissions, or type mismatch can be determined, and a response message can be generated based on this information.
[0012] The address range defines the destination address range of the data transmission command. By checking the address range, it can be determined whether the destination address is within a valid routing or access control range, and a reply message can be generated based on the address range.
[0013] Error messages in the message format reflect the compliance status of associated protocols or instruction structures. These error messages can be used to identify errors in the instruction structure or associated protocols, and to generate appropriate response messages accordingly.
[0014] In one possible implementation, the aforementioned response message includes at least one of response type, fault code, response value, response content, or device tag.
[0015] It is understandable that information such as response type, fault code, response value, response content, or device markings can reveal details related to the fault. Therefore, the response message should ideally include these details, as explained below:
[0016] The response type is used to distinguish whether the response message is a normal feedback after the successful execution of the data transmission command, or an abnormal response after a failure occurs during the execution of the command.
[0017] Fault codes can pinpoint the specific reason why a data transmission command failed.
[0018] The response value displays the execution result of the data transmission command, which can be used to determine whether the command was successfully completed.
[0019] The device marker reveals the transmission path of the acknowledgment message. Since the transmission paths of the acknowledgment message and the data transmission command are basically the same, the actual transmission path of the data transmission command can be determined from this.
[0020] In one possible implementation, the method further includes performing a first operation based on the aforementioned response message.
[0021] The first operation includes at least one of the following: retransmitting the data transmission instruction, discarding the data transmission instruction, releasing the resources related to the requested data transmission instruction, reporting a fault, adding a device tag to the response message, or sending the response message.
[0022] Understandably, performing the first operation based on the response message helps to quickly release resources, prevent the spread of faults, and resolve them, thereby further enhancing the stability of the server cluster during data transmission. Specifically:
[0023] Data retransmission commands can be divided into two types: original path retransmission and path switching retransmission. Original path retransmission commands can effectively solve data transmission failures caused by temporary network fluctuations or brief equipment malfunctions; while path switching retransmission commands provide solutions for data transmission failures caused by path failures.
[0024] Discarding data transmission commands can terminate unrecoverable invalid data transmission commands (such as continuous failures caused by unreachable target nodes, protocol incompatibility, or unauthorized access), avoiding the consumption of server cluster resources due to repeated attempts at invalid operations, thereby reducing overall performance loss.
[0025] Release the resources associated with the aforementioned data transmission command, release the resource occupation bound to the data transmission command, prevent resource leakage, and ensure that server cluster resources can be efficiently reused by other normal tasks.
[0026] When a server cluster encounters a data transmission command failure that it cannot repair on its own, reporting the failure (e.g., reporting the failure information to the software) can promptly notify maintenance personnel for handling, which can effectively improve the human-machine collaborative processing capability of the server cluster.
[0027] Sending a proxy message (such as sending a proxy message to the source node) notifies nodes with fault handling capabilities (such as the source node) of the current transmission failure status and reason, so as to handle the fault in a timely manner and improve the collaborative processing capability of the server cluster.
[0028] Adding device tags to the response message helps to identify and locate the fault path when handling faults, thereby reducing the time required for fault handling and improving fault handling efficiency.
[0029] In one possible implementation, the aforementioned server cluster is a heterogeneous system.
[0030] In one possible implementation, the aforementioned heterogeneous system is an AI cluster.
[0031] In one possible implementation, the aforementioned data transfer refers to a memory access instruction.
[0032] Understandably, artificial intelligence will be a major direction for future technological development, and AI network model training is the foundation of artificial intelligence. AI cluster training can quickly train more accurate and powerful network models. However, as models become increasingly complex, larger, and have more parameters, AI clusters with tens of thousands of processors are needed to perform AI training. Furthermore, AI chips within these clusters require significant data transmission. To improve communication between AI chips in a cluster, the industry is increasingly adopting direct memory access instructions for data transmission. This simpler communication mechanism is more conducive to increasing the effective payload of communication data. However, this memory-based transmission method is generally used in servers with high interconnect reliability. In AI clusters, the physical interconnection medium is sometimes optical, which has poor reliability. This can lead to data transmission failures or losses, causing fault propagation and potentially paralyzing the entire AI cluster. Therefore, this solution can be adopted to enhance the stability of the AI cluster during data transmission.
[0033] Secondly, embodiments of this application provide a data processing chip, which is a computing chip or a switching chip in a server cluster. The server cluster includes multiple servers, which are connected through a secondary switching chip. Each of the multiple servers includes multiple computing chips, which are connected through a primary switching chip. The data processing chip includes a transceiver module and a processing module.
[0034] The aforementioned transceiver unit is used to acquire data transmission instructions, which are either read or write instructions.
[0035] The aforementioned transceiver unit is also used to send or execute the aforementioned data transmission instructions.
[0036] The aforementioned processing unit is configured to generate or receive a substitute message in the event that the aforementioned data transmission instruction fails to be sent or executed. The substitute message is used to indicate the reason for the failure of the aforementioned data transmission instruction to be sent or executed.
[0037] In one possible implementation, the processing unit is specifically used to: generate a response message based on first information, wherein the first information includes at least one of port number, sender information of the data transmission instruction, address range, or message format error information.
[0038] In one possible implementation, the aforementioned response message includes at least one of response type, fault code, response value, response content, or device tag.
[0039] In one possible implementation, the processing unit is further configured to: perform a first operation based on the response message, the first operation including retransmitting the data transmission instruction, discarding the data transmission instruction, releasing the resources related to the request for the data transmission instruction, reporting a fault, adding a device tag to the response message, or sending at least one of the response messages.
[0040] In one possible implementation, the aforementioned server cluster is a heterogeneous system.
[0041] In one possible implementation, the aforementioned heterogeneous system is an AI cluster.
[0042] Thirdly, embodiments of this application also provide a server cluster, which includes a first chip and a second chip, wherein the first chip and the second chip are data processing chips in the second aspect or any possible implementation thereof.
[0043] The first chip is used to send the aforementioned response message to the second chip, and the response message is used to indicate the reason for the failure to send or execute the aforementioned data transmission instruction.
[0044] The second chip is configured to perform a first operation based on the response message. The first operation includes at least one of the following: retransmitting the data transmission instruction, discarding the data transmission instruction, releasing the resources related to the data transmission instruction, reporting a fault, adding a device tag to the response message, or sending the response message.
[0045] In one possible implementation, the second chip is specifically used to: execute at least one of the following when the above-mentioned reply message indicates that the link is disconnected: retransmit the above-mentioned data transmission instruction, report the fault, or add a device tag to the above-mentioned reply message.
[0046] It is understandable that a link failure may be a localized problem. By switching paths and retransmitting the data transmission commands, the fault point can be bypassed, and the data transmission commands can be successfully sent.
[0047] Reporting faults makes it easier for operations and maintenance personnel to be aware of link failures and trigger maintenance processes (such as replacing fiber optic cables), thus preventing faults from affecting the performance of the server cluster.
[0048] Adding device tags can record fault paths to facilitate subsequent diagnosis and pinpoint the specific location of link faults.
[0049] In one possible implementation, the second chip is specifically used to: execute at least one of the following when the aforementioned response message indicates that the destination chip does not respond: discard the aforementioned data transmission instruction, release the resources related to the request for the aforementioned data transmission instruction, or report a fault.
[0050] It is understandable that unresponsive data transfer commands will consume chip performance (such as buffers and processing power), and releasing resources can avoid affecting chip performance.
[0051] If the target chip fails to respond multiple times, the data transmission command is discarded to prevent further retransmission of the data transmission command from consuming server cluster resources.
[0052] Reporting faults can facilitate the rescheduling of tasks to other chips or marking the destination chip as a faulty chip.
[0053] In one possible implementation, the second chip is specifically used to: discard the data transmission instruction, release the resources related to the data transmission instruction, or report a fault when the response message indicates that the data transmission instruction format is incorrect.
[0054] Understandably, instructions with incorrect formats cannot be executed, and continuing to process them would waste resources, so they need to be discarded.
[0055] Freeing up resources ensures the chip can handle other tasks.
[0056] Formatting errors are usually software configuration errors. Reporting the fault makes it easier for maintenance personnel to fix the error.
[0057] In one possible implementation, the second chip is specifically used to: perform at least one of the following when the above-mentioned reply message indicates that the forwarding of the above-mentioned data transmission instruction has timed out: retransmit the above-mentioned data transmission instruction, report a fault, or add a device flag to the above-mentioned reply message.
[0058] It is understandable that timeouts may be due to temporary congestion, and retransmission or path switching can enable the data transmission command to be sent successfully.
[0059] By adding device tags, path information can be recorded to help analyze the causes of timeouts (such as excessive load on the switching chip).
[0060] By reporting faults, maintenance personnel can be notified to adjust the server cluster load or optimize the transmission path.
[0061] Fourthly, embodiments of this application also provide a data processing apparatus, the apparatus comprising: at least one processor, which, when the at least one processor executes program code or instructions, implements the method in the first aspect or any possible implementation thereof.
[0062] Alternatively, the data processing device may be a chip or a chip system.
[0063] Optionally, the device may further include at least one memory for storing the program code or instructions.
[0064] Fifthly, embodiments of this application also provide a chip, including: an input interface, an output interface, and at least one processor. Optionally, the chip further includes a memory. The at least one processor is used to execute code in the memory, and when the at least one processor executes the code, the chip implements the method in the first aspect or any possible implementation thereof.
[0065] Alternatively, the chip described above can also be an integrated circuit.
[0066] In a sixth aspect, embodiments of this application also provide a computer-readable storage medium for storing a computer program, the computer program including methods for implementing the first aspect or any possible implementation thereof.
[0067] In a seventh aspect, embodiments of this application also provide a computer program product containing instructions that, when run on a computer, cause the computer to implement the method described in the first aspect or any possible implementation thereof.
[0068] The data processing device, computer storage medium, computer program product, and chip provided in this embodiment are all used to execute the data processing method provided above. Therefore, the beneficial effects they can achieve can be referred to the beneficial effects in the data processing method provided above, and will not be repeated here. Attached Figure Description
[0069] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0070] Figure 1 This is a schematic diagram of a server cluster structure provided in an embodiment of this application;
[0071] Figure 2 This is a schematic diagram of another server cluster structure provided in an embodiment of this application;
[0072] Figure 3 This is a schematic diagram of another server cluster structure provided in an embodiment of this application;
[0073] Figure 4 This is a schematic diagram of another server cluster structure provided in an embodiment of this application;
[0074] Figure 5 A flowchart illustrating a data processing method provided in an embodiment of this application;
[0075] Figure 6 This is a schematic diagram illustrating an application scenario of the solution proposed in this application.
[0076] Figure 7 This is a schematic diagram illustrating another application scenario of the solution proposed in this application.
[0077] Figure 8 This is a schematic diagram illustrating another application scenario of the solution proposed in this application.
[0078] Figure 9 This is a schematic diagram illustrating another application scenario of the solution proposed in this application.
[0079] Figure 10 This is a schematic diagram illustrating another application scenario of the solution proposed in this application.
[0080] Figure 11 This is a schematic diagram illustrating another application scenario of the solution proposed in this application.
[0081] Figure 12 This is a schematic diagram illustrating another application scenario of the solution proposed in this application.
[0082] Figure 13 This is a schematic diagram illustrating another application scenario of the solution proposed in this application.
[0083] Figure 14 This is a schematic diagram illustrating another application scenario of the solution proposed in this application.
[0084] Figure 15 This is a schematic diagram illustrating another application scenario of the solution proposed in this application.
[0085] Figure 16 This is a schematic diagram illustrating another application scenario of the solution proposed in this application.
[0086] Figure 17 This is a schematic diagram of the structure of a data processing chip provided in an embodiment of this application;
[0087] Figure 18 This is a schematic diagram of the structure of a chip provided in an embodiment of this application;
[0088] Figure 19 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation
[0089] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the protection scope of the embodiments of this application.
[0090] In this article, the term "and / or" is merely a description of the relationship between related objects, indicating that there can be three relationships. For example, A and / or B can represent three situations: A exists alone, A and B exist simultaneously, and B exists alone.
[0091] The terms "first" and "second," etc., in the specification and drawings of the embodiments of this application are used to distinguish different objects or to distinguish different treatments of the same object, rather than to describe a specific order of objects.
[0092] Furthermore, the terms "comprising" and "having," and any variations thereof, used in the description of the embodiments of this application are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units is not limited to the steps or units listed, but may optionally include other steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or devices.
[0093] It should be noted that in the description of the embodiments of this application, the words "exemplarily" or "for example" are used to indicate examples, illustrations, or explanations. Any embodiment or design scheme described as "exemplarily" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design schemes. Specifically, the use of the words "exemplarily" or "for example" is intended to present the relevant concepts in a specific manner.
[0094] For ease of understanding, the terms used in the embodiments of this application are explained below for reference.
[0095] Server cluster: A system consisting of multiple servers.
[0096] Heterogeneous system: A system consisting of multiple heterogeneous servers.
[0097] An AI cluster is an AI system consisting of multiple AI servers connected via a network or bus.
[0098] Heterogeneous servers: Systems composed of different types of chips. For example, a system composed of a general-purpose central processing unit (CPU) and an AI chip.
[0099] Device node: refers to a chip (such as an AI chip) or a host in a heterogeneous server.
[0100] Switching chip (bus switching chip): It is responsible for forwarding chip bus commands between multiple chips interconnected based on the chip bus.
[0101] Proxy answer: A message that is answered on behalf of another or as a substitute for the answer.
[0102] Advanced Reduced Instruction Set Computing (RISC) Machines (ARM) are low-power, high-performance processor architectures designed based on the Reduced Instruction Set Computer (RISC) principle. Their core features include a simplified instruction set, high execution efficiency, and excellent power consumption control. The ARM architecture employs a modular design, supports both 16-bit and 32-bit instruction sets, and significantly reduces power consumption while maintaining performance by reducing instruction complexity and optimizing execution processes. This makes it particularly suitable for energy-intensive applications such as mobile devices and embedded systems.
[0103] Load / Store instructions are a core operating mechanism in RISC architecture processors, specifically responsible for data exchange between registers and memory. Load instructions load data from memory into registers (e.g., LDR R0, [R1], which reads 32 bits of data from the address pointed to by R1 into R0), while Store instructions write data from registers into memory (e.g., STRR0, [R1], which stores the data in R0 into the address pointed to by R1). These instructions employ various addressing modes (such as pre-indexed and post-indexed) to optimize memory access efficiency and strictly separate data access and computation operations to prevent processor performance degradation due to direct access to slow memory. In architectures such as ARM, Load / Store instructions support data transfer of different bit widths, including bytes, half-words, and full words, and can adapt to data type requirements through sign extension or zero extension, making them widely used in embedded systems, mobile devices, and high-performance computing scenarios.
[0104] A server cluster is a system composed of multiple servers. Its distributed architecture significantly improves overall processing power, reliability, and resource utilization, and it is widely used in cloud computing, big data analytics, and high-performance computing. In this highly collaborative architecture, a large amount of data needs to be transferred between the server chips within the cluster.
[0105] Currently, the industry is attempting to improve communication efficiency by using memory access instructions within chips to achieve efficient data transfer between chips within a cluster. This method performs well in high-reliability environments within servers; however, in low-reliability scenarios across servers, it is prone to data loss due to abnormal transmission paths, which can affect the stability of the entire server cluster and even cause it to crash.
[0106] For example, in the field of artificial intelligence, AI network model training is the foundation of AI. AI cluster training can quickly train more accurate and powerful network models. However, as models become more complex, larger, and have more parameters, AI clusters with tens of thousands of processors are needed to perform AI training. Furthermore, AI chips within these clusters need to perform a large amount of data transmission. To improve the communication capabilities between AI chips in a cluster, the industry has gradually adopted direct memory access instructions for data transmission between AI chips. This simple communication mechanism is more conducive to improving the effective payload of communication data. However, this memory semantic transmission method is generally used in servers with high interconnect reliability. In AI clusters, the physical medium for interconnection is sometimes optical connection. Optical connections have poor reliability, which can lead to data transmission failures or losses, causing fault propagation and paralyzing the entire AI cluster.
[0107] To this end, embodiments of this application provide a data processing method and a chip for enhancing the stability of server clusters (such as AI clusters) during data transmission.
[0108] The technical solutions provided in this application can be applied to server clusters.
[0109] Figure 1 A schematic diagram of a possible, non-limiting server cluster as described above is shown. Figure 1 As shown, the server cluster 10 includes multiple servers 100, which are connected through a secondary switching chip 200. Each server 100 includes multiple computing chips 300, which are connected through a primary switching chip 400.
[0110] Among them, the computing chips 300 within the same server can exchange data using data transmission commands through a primary switching chip 400.
[0111] The computing chips 300 in different servers can exchange data using data transmission commands through the primary switching chip 400 and the secondary switching chip 200.
[0112] For example, the computing chip 300 may be a central processing unit (CPU), ARM CPU, AI chip, graphics processing unit (GPU), tensor processing unit (TPU), application processor (AP), modem processor, image signal processor (ISP), video codec, digital signal processor (DSP), baseband processor, or neural processing unit (NPU).
[0113] For example, the switching chips (primary switching chip 400 and secondary switching chip 200) can be switches (SW).
[0114] A Level 1 switch chip can also be called a Level 1 (L1) switch chip or L1 SW, and a Level 2 switch chip can also be called a Level 2 (L2) switch chip or L2 SW.
[0115] The solution provided in this application can be applied to heterogeneous systems, such as an AI cluster consisting of multiple AI servers. This AI cluster provides AI accelerator computing capabilities to the outside world and can be used in different AI application scenarios, including inference and training scenarios.
[0116] It needs to be explained that, Figure 1 The server cluster shown is merely one implementation of the embodiments of this application. In actual applications, the server cluster may include more or fewer components, and the embodiments of this application are not limited here.
[0117] like Figure 2 As shown, the solution provided in this application embodiment can also be applied to AI clusters. An AI cluster includes multiple AI servers. An AI server can consist of a host and an AI chip. The host is a system based on a general-purpose CPU. The host and the AI chip are interconnected through a chip bus. One host is interconnected with multiple AI chips. Multiple AI servers are interconnected through a chip bus switch to realize communication between servers. Multiple AI servers form an AI cluster to provide AI computing capabilities.
[0118] When AI chips within an AI server or across AI servers perform aggregated communication and data transmission via a chip bus, if a bus link or device malfunctions on the current transmission path, the chip bus switching chip or AI chip will generate a response message to ensure that requests on the bus are quickly answered and that bus and accelerator resources are released rapidly.
[0119] Among them, the HOST is a system based on a general-purpose CPU, on which network models, inference or training applications run.
[0120] It needs to be explained that, Figure 2 The AI cluster shown is merely one implementation of the embodiments of this application. In actual applications, the AI cluster may include more or fewer components, and the embodiments of this application are not limited here.
[0121] like Figure 3 As shown, the AI chip in the AI cluster can include multiple accelerators, which can be interconnected via a chip bus.
[0122] It needs to be explained that, Figure 3 The AI cluster shown is merely one implementation of the embodiments of this application. In actual applications, the AI cluster may include more or fewer components, and this embodiment of the application does not impose any limitations.
[0123] In some implementations, AI chips may also include Direct Memory Access (DMA), a memory controller, and memory chips.
[0124] like Figure 4 As shown, the solution provided in this application embodiment can also be applied to heterogeneous systems. For example, a heterogeneous system can be formed by a routing engine, a response generation module and a response processing module in a bus switching chip (primary and secondary switching chips), and a response generation module and a response processing module in an AI chip. Figure 4 As shown, this heterogeneous system includes a routing engine, a response generation module, and a response processing module. These functional modules enable the reliability of the heterogeneous system. The description of each functional module is as follows:
[0125] The routing engine is responsible for data forwarding between AI chips and general-purpose CPUs, between AI chips, and between general-purpose CPUs in the bus switching chip. It forwards data based on routing rules and retransmits messages based on retransmission strategies.
[0126] The response generation module exists in both the bus switching chip and the AI chip. In the bus switching chip, it generates response messages for bus message request messages based on bus link anomalies, bus port anomalies, and abnormal messages reported by the routing engine, such as forwarding timeouts.
[0127] The response processing module exists in both the bus switching chip and the AI chip. In the bus switching chip, it is responsible for processing the response messages passing through the bus switch; in the AI chip, it processes the response messages transmitted to the accelerator on the bus.
[0128] Figure 5 This application illustrates a data processing method provided by an embodiment of the present application. This method can be executed by a first chip (computing chip or switching chip) in the aforementioned server cluster, such as... Figure 5 As shown, the method includes:
[0129] S501, Obtain data transmission command.
[0130] Among them, the above data transmission instructions are read instructions (i.e., load instructions) or write instructions.
[0131] For example, the data transfer instructions mentioned above can be ARM read instructions or ARM write instructions.
[0132] Understandably, in a server cluster, different servers are interconnected via secondary switching chips, and different chips within a single server are interconnected via a primary switching chip based on a chip bus. Throughout the entire server cluster, all chips can be uniformly addressed, and each chip can have its own independent address space. Different chips can directly access each other's address spaces using data transfer instructions (such as load / store).
[0133] Optionally, the aforementioned data transfer instructions can be memory-semantic data transfer instructions.
[0134] In one possible implementation, the aforementioned data transmission instruction can specifically be: generating a data transmission instruction.
[0135] For example, the source node of the data transmission instruction can generate the data transmission instruction.
[0136] like Figure 6 As shown, suppose computing chip 1-1 in server 1 of the server cluster needs to read data from computing chip 2-1 in server 2 of the same server cluster. Figure 6 In the scenario shown, chip 1-1 is the source node of the data transmission command, that is, the initiator of the data transmission command. Therefore, computing chip 1-1 needs to generate a read command (such as a read command for the AI chip's memory space).
[0137] In another possible implementation, the aforementioned data transmission instruction can specifically be: receiving a data transmission instruction.
[0138] For example, the intermediate node or the destination node of the data transmission instruction can receive the data transmission instruction.
[0139] exist Figure 6 In the scenario shown, the primary switching chip 1 and secondary switching chip of server 1 and the primary switching chip 2 of server 2 are intermediate nodes for data transmission instructions, that is, the transmitters of data transmission instructions. Therefore, the primary switching chip 1, secondary switching chip and primary switching chip 2 need to receive and transmit (i.e. forward) read instructions.
[0140] Specifically, such as Figure 6 As shown:
[0141] The first-level switching chip 1 can receive read commands sent by the computing chip 1-1.
[0142] The secondary switching chip can receive read commands sent by the primary chip 1.
[0143] The first-level switching chip 2 can receive read commands sent by the second-level switching chip.
[0144] exist Figure 6 In the scenario shown, chip 2-1 is the destination node of the data transmission command, that is, the executor of the data transmission command, so chip 2-1 needs to receive the read command.
[0145] Specifically, such as Figure 6 As shown: Chip 2-1 can receive read commands sent by primary switching chip 2.
[0146] S502, Send or execute data transmission instructions.
[0147] For example, the source node and intermediate node of the data transmission command can send the data transmission command.
[0148] Specifically, such as Figure 6 As shown:
[0149] The computing chip 1-1 can send the read command generated by the computing chip 1-1 to the first-level switching chip 1.
[0150] The first-level switching chip 1 can send the read command sent by the computing chip 1-1 to the second-level switching chip.
[0151] The secondary switching chip can send the read command sent by the primary switching chip 1 to the primary switching chip 2.
[0152] The primary switching chip 2 can send read commands sent by the secondary switching chip to the computing chip 2-1.
[0153] As another example, the destination node of the data transmission instruction can execute the data transmission instruction.
[0154] Specifically, such as Figure 6 As shown:
[0155] The computing chip 2-1 can execute the read command sent by the first-level switching chip 2.
[0156] S503. In the event that the data transmission command fails to be sent or executed, generate a substitute reply message or receive a substitute reply message.
[0157] The reply message is used to indicate the reason why the data transmission command failed to be sent or why it failed to be executed.
[0158] In one possible implementation, a substitute reply message can be generated if the data transmission command fails to be sent.
[0159] For example, the source node and intermediate node of the data transmission command can generate a substitute reply message if the data transmission command fails to be sent.
[0160] Specifically, such as Figure 6 As shown:
[0161] The computing chip 1-1 can send a read command to the first-level switching chip 1. If the read command fails to be sent, the computing chip 1-1 can generate a reply message indicating the reason for the failure (such as a link failure in the transmission path, a bus port failure, a module function failure, or a timeout in the request to wait for a response).
[0162] The first-level switching chip 1 can send a read command to the second-level switching chip. If the read command fails to be sent, the first-level switching chip can generate a reply message.
[0163] The secondary switching chip can send a read command to the primary switching chip 2. If the read command fails to be sent, the secondary switching chip can generate a reply message.
[0164] The primary switching chip 2 can send a read command to the computing chip 2-1. If the read command fails to be sent, the primary switching chip 2 can generate a reply message.
[0165] In one possible implementation, the failure to send a data transmission command can be determined based on a timer or exception information.
[0166] For example, a timer can be started after a data transmission command is sent. If no response message or data corresponding to the data transmission command is received before the timer expires, it can be determined that the data transmission command failed to be sent.
[0167] For example, if an error message is received after sending a data transmission command, it can be determined that the data transmission command failed to be sent.
[0168] For example, if an error message is received from an on-chip bus connection point or an off-chip bus connection point after a data transmission command is sent, it can be determined that the data transmission command failed to be sent.
[0169] In another possible implementation, a substitute message can be received if the data transmission command fails to be sent.
[0170] For example, the source node and intermediate node of the data transmission command can receive a substitute message if the data transmission command fails to be sent.
[0171] Specifically, such as Figure 6 As shown:
[0172] The computing chip 1-1 can send a read command to the primary switching chip. If the read command fails to be sent, the computing chip 1-1 can receive a reply message from other chips (such as the primary switching chip).
[0173] The first-level switching chip 1 can send a read command to the second-level switching chip. If the read command fails to be sent, the first-level switching chip 1 can receive a reply message sent by other chips (such as the second-level switching chip).
[0174] The secondary switching chip can send a read command to the primary switching chip 2. If the read command fails to be sent, the secondary switching chip can receive a reply message sent by other chips (such as the primary switching chip 2).
[0175] The primary switching chip 2 can send a read command to the computing chip 2-1. If the read command fails to be sent, the primary switching chip 2 can receive a reply message sent by other chips (such as computing chip 2-1).
[0176] In another possible implementation, a substitute reply message can be generated if the data transmission command fails to execute.
[0177] For example, the destination node of a data transmission command can generate a substitute message if the data transmission command fails to execute.
[0178] Specifically, such as Figure 6 As shown:
[0179] The computing chip 2-1 can execute read instructions. If the read instruction fails to execute, the computing chip 2-1 can generate a response message indicating the reason for the failure of the read instruction.
[0180] If the read instruction fails, computing chip 2-1 can return the data corresponding to the read instruction to computing chip 1-1. The data is forwarded through primary switching chip 2, secondary switching chip and primary switching chip 1 before returning to computing chip 1-1.
[0181] As can be seen from S501 to S503 of the above method, the solution provided in this application embodiment, when a fault occurs during data transmission in the server cluster (such as failure to send or execute a data transmission command), the system will clearly indicate the specific cause of the fault through a proxy message. This allows relevant chips (such as the command initiator chip or the switching chip) to promptly learn the cause of the fault and take corresponding measures (such as releasing occupied resources, retransmitting the command, or discarding the command), thus avoiding resource waste and fault propagation. Therefore, by introducing proxy messages, this solution enhances the stability of the server cluster during data transmission.
[0182] In one possible implementation, the aforementioned response message includes at least one of response type, fault code, response value, response content, or device tag.
[0183] It is understandable that information such as response type, fault code, response value, response content, or device markings can reveal details related to the fault. Therefore, the response message should ideally include these details, as explained below:
[0184] The response type is used to distinguish whether the response message is a normal feedback after the successful execution of the data transmission command, or an abnormal response after a failure occurs during the execution of the command.
[0185] Fault codes can pinpoint the specific reason why a data transmission command failed.
[0186] The response value displays the execution result of the data transmission command, which can be used to determine whether the command was successfully completed.
[0187] The device marker reveals the transmission path of the acknowledgment message. Since the transmission paths of the acknowledgment message and the data transmission command are basically the same, the actual transmission path of the data transmission command can be determined from this.
[0188] In one possible implementation, the method further includes performing a first operation based on the aforementioned response message.
[0189] The first operation includes at least one of the following: retransmitting the data transmission instruction, discarding the data transmission instruction, releasing the resources related to the requested data transmission instruction, reporting a fault, adding a device tag to the response message, or sending the response message.
[0190] Understandably, performing the first operation based on the response message helps to quickly release resources, prevent the spread of faults, and resolve them, thereby further enhancing the stability of the server cluster during data transmission. Specifically:
[0191] Data retransmission commands can be divided into two types: original path retransmission and path switching retransmission. Original path retransmission commands can effectively solve data transmission failures caused by temporary network fluctuations or brief equipment malfunctions; while path switching retransmission commands provide solutions for data transmission failures caused by path failures.
[0192] Discarding data transmission commands can terminate unrecoverable invalid data transmission commands (such as continuous failures caused by unreachable target nodes, protocol incompatibility, or unauthorized access), avoiding the consumption of server cluster resources due to repeated attempts at invalid operations, thereby reducing overall performance loss.
[0193] Release the resources associated with the aforementioned data transmission command, release the resource occupation bound to the data transmission command, prevent resource leakage, and ensure that server cluster resources can be efficiently reused by other normal tasks.
[0194] When a server cluster encounters a data transmission command failure that it cannot repair on its own, reporting the failure (e.g., reporting the failure information to the software) can promptly notify maintenance personnel for handling, which can effectively improve the human-machine collaborative processing capability of the server cluster.
[0195] Sending a proxy message (such as sending a proxy message to the source node) notifies nodes with fault handling capabilities (such as the source node) of the current transmission failure status and reason, so as to handle the fault in a timely manner and improve the collaborative processing capability of the server cluster.
[0196] Adding device tags to the response message helps to identify and locate the fault path when handling faults, thereby reducing the time required for fault handling and improving fault handling efficiency.
[0197] In one possible implementation, generating the proxy response message specifically includes: generating the proxy response message based on the first information.
[0198] The first information mentioned above includes at least one of the following: port number, sender information of the data transmission instruction, address range, or message format error information.
[0199] It can be understood that the port number, the sender information of the data transmission command, the address range, and message format error information are all closely related to the cause of the fault, and therefore, a response message can be generated based on this information. Specifically:
[0200] Port numbers are used to precisely locate the specific interface or link channel through which data transmission commands are transmitted. By using port numbers, it is possible to identify faults such as a port being inactive, occupied, or incompatible with the target device, and thus generate corresponding response messages based on the port number.
[0201] The sender information of a data transmission command reveals key information such as the identity, permissions, or type of the command's origin. By analyzing the sender information, the causes of failures such as authentication failure, insufficient permissions, or type mismatch can be determined, and a response message can be generated based on this information.
[0202] The address range defines the destination address range of the data transmission command. By checking the address range, it can be determined whether the destination address is within a valid routing or access control range, and a reply message can be generated based on the address range.
[0203] Error messages in the message format reflect the compliance status of associated protocols or instruction structures. These error messages can be used to identify errors in the instruction structure or associated protocols, and to generate appropriate response messages accordingly.
[0204] The following is combined Figure 7 The scenario shown illustrates the solution provided by the embodiments of this application. Figure 7 In the scenario shown, the DMA in AI chip A accesses the memory space of AI chip B on another server. This data transfer instruction (a memory semantics-based data transfer instruction) goes through the DMA, the chip's internal bus, the first-level switch chip, the second-level switch chip, the AI chip's internal bus, and the AI chip's memory controller before accessing the AI chip's memory space. In this cross-chip memory access request process based on memory semantics, reliability issues can easily occur, leading to access failures and thus affecting the reliability of the AI cluster system.
[0205] like Figure 8 As shown, proxy answer processing modules and proxy answer generation modules can be deployed on various modules of the AI chip (such as DMA, memory controller, accelerator, etc.) and the routing engine to improve the reliability of the AI cluster system.
[0206] like Figure 9 As shown, in this scenario, AI chip A's DMA needs to read data from AI chip B. AI chip A's DMA sends a data transfer command (e.g., a read command) to AI chip B. This data transfer command, sent by the DMA, reaches the on-chip bus connection point of AI chip A via the response generation module. The data transfer command is then sent to the off-chip bus connection point through the on-chip bus connection point. If the bus connecting AI chip A's off-chip bus connection point and the first-level switching chip is faulty, the data transfer command sent by the DMA will not receive a response. Therefore, exception handling is required. The exception handling process in this scenario includes:
[0207] When an external bus connection point of AI chip A detects an anomaly, it notifies the DMA response generation module. For example, if an external bus connection point of AI chip A detects an anomaly, it can use an interrupt method to notify the DMA response generation module.
[0208] The DMA's response generation module obtains abnormal information from the external bus based on the notification, generates and sends a response message.
[0209] DMA releases resources associated with data transfer instructions based on the reply message.
[0210] The DMA's response processing module reports the fault to the fault management module of AI chip A based on the response message.
[0211] like Figure 10 As shown, in this scenario, the data transmission command sent by the DMA in AI chip A first passes through the response generation module, then reaches the on-chip bus connection point of AI chip A, and then through the off-chip bus connection point, finally reaching the routing engine of the first-level switching chip. The routing engine of the first-level switching chip forwards the data transmission command to the output connection point (OUTPUT connection point) of the first-level switching chip. If an anomaly occurs in the bus between the output connection point (OUTPUT connection point) of the first-level switching chip and the input connection point of the second-level switching chip, the data transmission command forwarded by the routing engine of the first-level switching chip will not receive a response. Therefore, it is necessary to handle the anomaly. The anomaly handling process in this scenario includes:
[0212] If the output connection point of the primary switching chip detects an anomaly, it notifies the routing engine's response generation module. For example, if the output connection point of the primary switching chip detects an anomaly, it can use an interrupt method to notify the routing engine's response generation module.
[0213] The routing engine's response generation module obtains abnormal information from the off-chip bus based on notifications, generates and sends response messages.
[0214] DMA releases resources associated with data transfer instructions based on the reply message.
[0215] The DMA's response processing module reports the fault to the fault management module of AI chip A based on the response message.
[0216] like Figure 11 As shown, in this scenario, when the routing engine of the first-level switching chip receives a data transmission instruction, it forwards the instruction to the next connection point A. Connection point A then checks the bus status between itself and connection point B. If a bus anomaly is detected, the routing engine's data forwarding fails, and it then generates a proxy message and sends it to the proxy processing module.
[0217] The proxy response processing module processes proxy response messages. If the fault code in the proxy response message indicates that the fault is caused by a link abnormality, the proxy response processing module notifies the routing engine to retransmit the data transmission instruction.
[0218] The routing engine selects a retransmission strategy based on the cause of the fault, that is, it reselects a normal port to send data transmission commands.
[0219] like Figure 12 As shown, under normal circumstances, if the link message carried in the reply message indicates that the bus link has been completely disconnected, the retransmission strategy is to select another next-hop connection point for retransmission, allowing the message to be transmitted through a new transmission path. In a multi-port interconnected bus, the routing engine needs to select a normal port for data transmission based on the abnormal port status of the bus. Figure 12 In the scenario shown, two connection points are interconnected via three buses. The link on port 0 of the first-level switching chip is down due to physical contact or other reasons. When the routing engine on the first-level switching chip retransmits data transmission instructions based on the reply message, it does not select the port with the broken link for retransmission. For example, it could use the link between port 2 of the output connection point in the first-level switching chip and port 2 of the input connection point in the second-level switching chip to retransmit the data transmission instruction.
[0220] like Figure 13 As shown, in this scenario, after receiving a data transmission command, the routing engine of the secondary switching chip forwards the command to the next connection point. If the routing engine does not receive a response from the receiving end for an extended period after forwarding the data, a forwarding timeout timer will be triggered, and exception handling will be required. Exception handling in this scenario includes:
[0221] The routing engine of the secondary switching chip notifies the secondary switching chip's response processing module that the data transmission command forwarding has timed out. For example, the routing engine of the secondary switching chip can notify the secondary switching chip's response processing module that the data transmission command forwarding has timed out by sending a timeout message.
[0222] The forwarding timeout timer serves, but is not limited to, triggering message (data transmission instruction) retransmission and triggering the generation of a response message.
[0223] Upon receiving the notification, the response processing module of the secondary switching chip generates a response message and adds the device identifier of the current device to the message. It then sends this response message to the response processing module of the primary switching chip.
[0224] After receiving the reply message, the reply processing module of the first-level switching chip adds the device identifier of the current device to the reply message, and then sends the reply message to the reply processing module of AI chip A.
[0225] After receiving the proxy answering message, the proxy answering module of AI chip A reports the fault to the fault management module based on the proxy answering message.
[0226] like Figure 14 As shown, in this scenario, after receiving a data transmission command from the transmission module (e.g., DMA) of AI chip A, the routing engine of the first-level switching chip forwards the data transmission command to the output connection point of the first-level switching chip. If the routing engine of the first-level switching chip does not receive a response from the receiving end for a long time after forwarding the data, it will trigger a forwarding timeout timer and need to handle the exception. The exception handling in this scenario includes:
[0227] The routing engine of the primary switching chip notifies the response generation module of the primary switching chip that the data transmission instruction forwarding has timed out.
[0228] The first-level switching chip's response generation module generates a response message and sends it to the AI chip A's response processing module.
[0229] After receiving the proxy answering message, the proxy answering module of AI chip A reports the fault to the fault management module based on the proxy answering message.
[0230] After receiving the reply message, the transmission module of AI chip A releases the resources related to the data transmission command based on the reply message.
[0231] It can be seen that, Figure 13 and Figure 14 In the scenario shown, when a proxy response generation module generates a proxy response message, it adds the current device's tag to the message. When this proxy response message is transmitted to the initiating end, passes through the next-hop device, and is processed by the next-hop device, the proxy response processing module of the next-hop device will also add the device tag of the next-hop device.
[0232] In some scenarios, a transmission anomaly between the output connection point of the secondary switching chip and the input connection point of the primary switching chip can trigger the response generation module of the secondary switching chip's routing engine to generate a response message. The anomaly handling procedure for this scenario can be found in [reference needed]. Figure 13 The processing flow is shown below.
[0233] In some scenarios, transmission anomalies between the off-chip connection points of the primary switching chip and AI chip B can trigger the response generation module of the primary switching chip's routing engine to generate a response message. The anomaly handling process for this scenario can be found by referring to... Figure 13 The processing flow is shown below.
[0234] In one possible implementation, when a proxy response generation module generates a proxy response message, it can generate the proxy response message content based on the request initiator information (such as initiator type, initiator ID, or bus port), so that the response message has different response values (response codes) and response content. For example, if the message indicates that the data in the response is abnormal, the response content is all F or all 0.
[0235] like Figure 15 As shown, in this scenario, after receiving data transfer instructions from the AI chip's DMA and CPU, the routing engine of the primary switching chip forwards the data transfer instructions to the next connection point (e.g., the output connection point of the primary switching chip). If the routing engine of the primary switching chip does not receive a response from the receiving end for an extended period after forwarding the data, it will trigger a forwarding timeout timer and need to handle the exception. The exception handling in this scenario includes:
[0236] The routing engine of the primary switching chip notifies the response generation module of the primary switching chip that the data transmission instruction forwarding has timed out.
[0237] For the CPU, the response generation module of the primary switching chip generates and sends the response message to the CPU. The response value of this response message is the same as that of a normal message response, and the response content is all "F".
[0238] For the DMA of the AI chip, the response generation module of the first-level switching chip generates and sends the response message to the DMA of the AI chip. The response value of this response message is the response value of the abnormal message response, and the response content is all 0.
[0239] Understandably, since the CPU is on the host side, it needs to run the operating system (OS) and manage the entire AI server. Processing bus responses with error codes can cause CPU malfunctions, leading to OS crashes and ultimately rendering the entire AI server unusable. Therefore, the CPU expects all received response messages to be normal. In this case, the response generation module needs to identify that the request message originated from the CPU based on the connected port or the initiator's information (device ID), and then reply with a normal response message when a forwarding message error occurs. However, this response is not from the destination. During the data reading process, the CPU doesn't ultimately read the data. Therefore, to allow the software running on the CPU to recognize this anomaly, the response message carries all "F" characters. This allows the software on the CPU to determine if the current read operation is normal by recognizing the message content and then perform further processing.
[0240] Figure 15In the scenario shown, the AI chip is a slave device. When the AI chip's DMA and other accelerators are transmitting data, the failure of the DMA and other accelerators does not affect the stability of the entire AI chip or the AI server. Therefore, the response generation module can carry error codes and data anomaly markers in the response messages sent by the DMA and other accelerators. When the current DMA and other accelerators process such response messages, they can quickly detect the anomaly and report it to the fault management module. The fault management module then reports it to the business software, enabling the business software to quickly perform business retry or business redeployment as a recovery action.
[0241] Figure 16 In the scenario shown, the routing engine routes messages according to the access address. If the requested access address is not within the valid range of the routing policy, the routing engine will refuse to route the message and then notify the response generation module to send a response message to the initiating end. Additionally, for requests with invalid message formats, both the routing engine and the receiving accelerator will notify the response generation module to generate a response message for the requesting end. The specific exception handling process includes:
[0242] When the routing engine of a switching chip (such as a primary or secondary switching chip) receives a data transfer instruction sent by the DMA in the AI chip, it finds that the data transfer instruction does not conform to the routing rules (e.g., the destination address is empty, the destination address is unauthorized, the data transfer instruction format is incorrect, etc.). The routing engine of the switching chip will refuse to forward (route) the data transfer instruction and notify the routing engine's response generation module that the forwarding of the data transfer instruction has failed.
[0243] Upon receiving the notification, the routing engine's response generation module generates a response message and sends it to the AI chip's DMA. The response value in the response message indicates that the data transmission command forwarding failed (routing failure).
[0244] Upon receiving a data transfer command from the DMA in the AI chip, the destination device (e.g., a module within the AI chip) verifies the validity of the destination address. If the destination address is invalid, the destination device will refuse to execute the data transfer command, generate a response message, and then send the response message to the DMA of the AI chip. The response generation module of the switching chip (e.g., a primary or secondary switching chip) through which the response message passes during transmission will add a device identifier to the response message.
[0245] In summary, the solution provided in this application, in an AI cluster system based on memory semantic bus interconnection, enables the rapid completion of abnormal requests by generating a proxy message when the bus experiences link failures, the routing engine malfunctions, or the receiver malfunctions. Related modules for transmitting and processing request messages can quickly release resources, and the initiator can obtain the cause of failure and choose recovery measures to restore business operations. Furthermore, retransmitting request messages allows them to be resent to the destination, improving system reliability.
[0246] In an AI cluster system based on memory semantics and bus interconnection, when a bus or forwarding anomaly occurs on the transmission path, different response messages can be generated according to different anomaly situations. The response messages carry various anomaly information such as device information on the transmission path and bus anomaly information. Different retransmission strategies can be selected based on the response messages. The retransmission strategies can include reselecting the transmission path, reselecting the transmission port, and other retransmission strategies.
[0247] The solution provided in this application generates different types of response messages based on (port, accelerator type of the message sender, address range, message format error). The messages carry information such as response type, fault code, response value, and device markers along the path of the response, in order to meet the response requirements of the CPU, device, and different accelerators of the equipment while maintaining normal operation; and to meet the requirement of services that need to know the exact cause of transmission anomalies in order to quickly restore services.
[0248] The solution provided in this application also responds with different fault codes according to the fault type. Then, the transmission accelerator selects the transmission path and port number for data retransmission according to the different fault codes, so that a single point of failure does not affect data transmission and does not affect the stability of the entire system.
[0249] The solution provided in this application embodiment, in an AI cluster system based on memory semantic data transmission, can generate different response messages for transmission requests when an anomaly occurs in the transmission path, preventing the abnormal request on the bus from spreading to the entire system; it can also retransmit data according to the abnormal situation to prevent transmission interruption and avoid business interruption.
[0250] The solutions provided in this application relate to the reliability of data transmission in server clusters, especially AI cluster systems. They are mainly applied to chip bus protocols, but are not limited to chip bus protocols. They can also be used in other transmission protocols such as network protocols for data transmission.
[0251] The following describes the data processing chip used to perform the above data processing methods.
[0252] It is understood that, in order to achieve the above-mentioned functions, the data processing chip includes hardware and / or software modules that perform the respective functions. Based on the algorithm steps of the examples described in the embodiments disclosed herein, the embodiments of this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed by hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application in conjunction with the embodiments, but such implementation should not be considered beyond the scope of the embodiments of this application.
[0253] This application embodiment can divide the data processing chip into functional modules according to the above method example. For example, each function can be divided into its own functional modules, or two or more functions can be integrated into one processing module. The integrated modules can be implemented in hardware. It should be noted that the module division in this embodiment is illustrative and only represents one logical functional division; other division methods may be used in actual implementation.
[0254] When dividing each function into modules according to its corresponding function. Figure 17 The diagram illustrates a possible configuration of the data processing chip involved in the above embodiments. This data processing chip is applied as a first chip in a server cluster. The first chip is either a computing chip or a switching chip. The server cluster includes multiple servers, which are connected through a secondary switching chip. Each of the multiple servers includes multiple computing chips, which are connected through a primary switching chip. The device can be the first chip, a module applied to the first chip, or a logic node, logic module, or software that can implement all or part of the functions of the first chip.
[0255] like Figure 17 As shown, the data processing chip 1700 may include a transceiver module 1701 and a processing module 1702.
[0256] The aforementioned acquisition unit 1701 is used to acquire service data packets.
[0257] The aforementioned transceiver unit 1702 is used to acquire data transmission instructions, which are read instructions or write instructions.
[0258] The aforementioned transceiver unit 1702 is also used to send or execute the aforementioned data transmission instructions.
[0259] The aforementioned processing unit is configured to generate or receive a substitute message in the event that the aforementioned data transmission instruction fails to be sent or executed. The substitute message is used to indicate the reason for the failure of the aforementioned data transmission instruction to be sent or executed.
[0260] In one possible implementation, the processing unit is specifically used to: generate a response message based on first information, wherein the first information includes at least one of port number, sender information of the data transmission instruction, address range, or message format error information.
[0261] In one possible implementation, the aforementioned response message includes at least one of response type, fault code, response value, response content, or device tag.
[0262] In one possible implementation, the processing unit is further configured to: perform a first operation based on the response message, the first operation including retransmitting the data transmission instruction, discarding the data transmission instruction, releasing the resources related to the request for the data transmission instruction, reporting a fault, adding a device tag to the response message, or sending at least one of the response messages.
[0263] In one possible implementation, the aforementioned server cluster is a heterogeneous system.
[0264] In one possible implementation, the aforementioned heterogeneous system is an AI cluster.
[0265] This application embodiment also provides a server cluster, which includes a first chip and a second chip, wherein the first chip and the second chip can be used as... Figure 17 The data processing chip shown.
[0266] The first chip is used to send the aforementioned response message to the second chip, and the response message is used to indicate the reason for the failure to send or execute the aforementioned data transmission instruction.
[0267] The second chip is configured to perform a first operation based on the response message. The first operation includes at least one of the following: retransmitting the data transmission instruction, discarding the data transmission instruction, releasing the resources related to the data transmission instruction, reporting a fault, adding a device tag to the response message, or sending the response message.
[0268] In one possible implementation, the second chip is specifically used to: execute at least one of the following when the above-mentioned reply message indicates that the link is disconnected: retransmit the above-mentioned data transmission instruction, report the fault, or add a device tag to the above-mentioned reply message.
[0269] It is understandable that a link failure may be a localized problem. By switching paths and retransmitting the data transmission commands, the fault point can be bypassed, and the data transmission commands can be successfully sent.
[0270] Reporting faults makes it easier for operations and maintenance personnel to be aware of link failures and trigger maintenance processes (such as replacing fiber optic cables), thus preventing faults from affecting the performance of the server cluster.
[0271] Adding device tags can record fault paths to facilitate subsequent diagnosis and pinpoint the specific location of link faults.
[0272] In one possible implementation, the second chip is specifically used to: execute at least one of the following when the aforementioned response message indicates that the destination chip does not respond: discard the aforementioned data transmission instruction, release the resources related to the request for the aforementioned data transmission instruction, or report a fault.
[0273] It is understandable that unresponsive data transfer commands will consume chip performance (such as buffers and processing power), and releasing resources can avoid affecting chip performance.
[0274] If the target chip fails to respond multiple times, the data transmission command is discarded to prevent further retransmission of the data transmission command from consuming server cluster resources.
[0275] Reporting faults can facilitate the rescheduling of tasks to other chips or marking the destination chip as a faulty chip.
[0276] In one possible implementation, the second chip is specifically used to: discard the data transmission instruction, release the resources related to the data transmission instruction, or report a fault when the response message indicates that the data transmission instruction format is incorrect.
[0277] Understandably, instructions with incorrect formats cannot be executed, and continuing to process them would waste resources, so they need to be discarded.
[0278] Freeing up resources ensures the chip can handle other tasks.
[0279] Formatting errors are usually software configuration errors. Reporting the fault makes it easier for maintenance personnel to fix the error.
[0280] In one possible implementation, the second chip is specifically used to: perform at least one of the following when the above-mentioned reply message indicates that the forwarding of the above-mentioned data transmission instruction has timed out: retransmit the above-mentioned data transmission instruction, report a fault, or add a device flag to the above-mentioned reply message.
[0281] It is understandable that timeouts may be due to temporary congestion, and retransmission or path switching can enable the data transmission command to be sent successfully.
[0282] By adding device tags, path information can be recorded to help analyze the causes of timeouts (such as excessive load on the switching chip).
[0283] By reporting faults, maintenance personnel can be notified to adjust the server cluster load or optimize the transmission path.
[0284] This application also provides a chip, which can be the data processing chip described above. Figure 18 A schematic diagram of a chip 1800 is shown. The chip 1800 includes one or more processors 1801 and interface circuitry 1802. Optionally, the chip 1800 may also include a bus 1803.
[0285] The processor 1801 may be an integrated circuit chip with signal processing capabilities. In implementation, each step of the above data processing method can be completed through the integrated logic circuitry in the processor 1801 or through software instructions.
[0286] Optionally, the processor 1801 described above may be a general-purpose processor, a digital signal processing (DSP) processor, an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the various methods and steps disclosed in the embodiments of this application. The general-purpose processor may be a microprocessor or any conventional processor.
[0287] The interface circuit 1802 can be used to send or receive data, instructions or information. The processor 1801 can use the data, instructions or other information received by the interface circuit 1802 to process the data, instructions or other information, and can send the processed information out through the interface circuit 1802.
[0288] Optionally, the chip may also include memory, which may include read-only memory and random access memory, providing operation instructions and data to the processor. A portion of the memory may also include non-volatile random access memory (NVRAM).
[0289] Optionally, the memory stores executable software modules or data structures, and the processor can execute corresponding operations by calling the operation instructions stored in the memory (which may be stored in the operating system).
[0290] Optionally, the interface circuit 1802 can be used to output the execution results of the processor 1801.
[0291] The data processing methods provided by one or more embodiments of this application can be referred to the foregoing embodiments, and will not be repeated here.
[0292] It should be noted that the functions of the processor 1801 and the interface circuit 1802 can be implemented through hardware design, software design, or a combination of hardware and software; no restrictions are imposed here.
[0293] Figure 19 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. The electronic device can be a data processing chip or a functional module within the data processing chip. Figure 19 As shown, the electronic device 1900 includes a processor 1901, a transceiver 1902, and a communication line 1903.
[0294] The processor 1901 is used to execute any step of the data processing method provided in the embodiments of this application, and in the process of executing any step of the data processing method provided in the embodiments of this application, the transceiver 1902 and the communication line 1903 may be called to complete the corresponding operation.
[0295] Furthermore, the electronic device 1900 may also include a memory 1904. The processor 1901, the memory 1904, and the transceiver 1902 can be connected via a communication line 1903.
[0296] Wherein, processor 1901 is a processor, a general-purpose processor, a network processor (NP), a digital signal processor (DSP), a microprocessor, a microcontroller, a programmable logic device (PLD), or any combination thereof. Processor 1901 may also be other devices with processing functions, such as circuits, devices, or software modules, without limitation.
[0297] Transceiver 1902 is used to communicate with other devices or other communication networks, such as Ethernet, radio access network (RAN), wireless local area network (WLAN), etc. Transceiver 1902 can be a module, circuit, transceiver, or any device capable of enabling communication.
[0298] The transceiver 1902 is mainly used for sending and receiving commands and information, and may include a transmitter and a receiver to send and receive commands and information, respectively; operations other than sending and receiving commands and information are implemented by the processor.
[0299] Communication line 1903 is used to transmit information between the various components included in electronic device 1900.
[0300] In one design, the processor can be viewed as a logic circuit, and the transceiver as an interface circuit.
[0301] Memory 1904 is used to store instructions. These instructions can be computer programs.
[0302] The memory 1904 can be volatile memory or non-volatile memory, or both. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous DRAM (SDRAM), double data rate synchronous DRAM (DDR SDRAM), enhanced synchronous DRAM (ESDRAM), synchronous linked DRAM (SLDRAM), and direct rambus RAM (DRRAM). Memory 1904 can also be a compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed discs, laser discs, optical discs, digital universal discs, Blu-ray discs, etc.), magnetic disk storage media, or other magnetic storage devices. It should be noted that the memory in the systems and methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.
[0303] It should be noted that the memory 1904 can exist independently of the processor 1901, or it can be integrated with the processor 1901. The memory 1904 can be used to store instructions, program code, or some data, etc. The memory 1904 can be located inside or outside the electronic device 1900, without limitation. The processor 1901 is used to execute the instructions stored in the memory 1904 to implement the methods provided in the embodiments of this application.
[0304] In one example, processor 1901 may include one or more processor cores, for example Figure 19 The processor cores are 0 and 1.
[0305] As an optional implementation, the electronic device 1900 includes multiple processors, for example, besides Figure 19 In addition to processor 1901, it may also include processor 1907.
[0306] As an optional implementation, the electronic device 1900 also includes an output device 1905 and an input device 1906. For example, the input device 1906 is a device such as a keyboard, mouse, microphone, or joystick, and the output device 1905 is a device such as a display screen or speaker.
[0307] It should be noted that the electronic device 1900 can be a chip system or... Figure 19 Devices with similar structures. The chip system can be composed of chips or include chips and other discrete components. Actions, terminology, etc., involved in the various embodiments of this application can be referenced interchangeably without limitation. The message names or parameter names in the messages used for interaction between devices in the embodiments of this application are merely examples; other names can be used in specific implementations without limitation. Furthermore, Figure 19 The structural composition shown does not constitute a limitation on the electronic device 1900, except... Figure 19 In addition to the components shown, the electronic device 1900 may include more than Figure 19 This may indicate more or fewer components, or combinations of certain components, or different component arrangements.
[0308] The processor and transceiver described in the embodiments of this application can be implemented on integrated circuits (ICs), analog ICs, radio frequency integrated circuits, mixed-signal ICs, application-specific integrated circuits (ASICs), printed circuit boards (PCBs), electronic devices, etc. The processor and transceiver can also be manufactured using various IC process technologies, such as complementary metal oxide semiconductors (CMOS), n-type metal-oxide-semiconductor (NMOS), p-type metal oxide semiconductors (PMOS), bipolar junction transistors (BJTs), bipolar CMOS (BiCMOS), silicon germanium (SiGe), gallium arsenide (GaAs), etc.
[0309] This application also provides a data processing apparatus, which includes at least one processor. When the at least one processor executes program code or instructions, it implements the data processing method described above.
[0310] Optionally, the device may further include at least one memory for storing the program code or instructions.
[0311] This application also provides a computer-readable storage medium storing computer instructions. When the computer instructions are executed on a data processing device, the data processing device performs the aforementioned related method steps to implement the data processing method described above.
[0312] In one possible implementation, the computer-readable storage medium is a non-transitory computer-readable medium.
[0313] This application also provides a computer program product that, when run on a computer, causes the computer to perform the aforementioned steps to implement the data processing method described in the above embodiments.
[0314] This application also provides a data processing apparatus, which may specifically be a chip, integrated circuit, component, or module. Specifically, the apparatus may include a connected processor and a memory for storing instructions, or the apparatus may include at least one processor for fetching instructions from external memory. When the apparatus is running, the processor can execute the instructions to cause the chip to perform the data processing methods described in the above method embodiments.
[0315] It should be understood that in various embodiments of this application, the sequence number of each process does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of this application.
[0316] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the embodiments of this application.
[0317] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.
[0318] In the embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of the units described above is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.
[0319] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.
[0320] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.
[0321] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of this application, essentially, or the parts that contribute to the prior art, or parts of the technical solutions, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.
[0322] The above description is merely a specific implementation of the embodiments of this application, but the protection scope of the embodiments of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the embodiments of this application should be included within the protection scope of the embodiments of this application. Therefore, the protection scope of the embodiments of this application should be determined by the protection scope of the claims.
Claims
1. A data processing method, characterized in that, A first chip applied to a server cluster, the first chip being a computing chip or a switching chip, the server cluster comprising multiple servers connected via a secondary switching chip, each of the multiple servers comprising multiple computing chips connected via a primary switching chip, the method comprising: Obtain a data transmission instruction, wherein the data transmission instruction is a read instruction or a write instruction; Send or execute the data transmission command; In the event that the data transmission command fails to be sent or executed, a substitute reply message is generated or the substitute reply message is received. The substitute reply message is used to indicate the reason for the failure to send or execute the data transmission command.
2. The method according to claim 1, characterized in that, The generated response message includes: A response message is generated based on the first information, which includes at least one of the following: port number, sender information of the data transmission instruction, address range, or message format error information.
3. The method according to claim 1 or 2, characterized in that, The response message includes at least one of the following: response type, fault code, response value, response content, or device tag.
4. The method according to any one of claims 1 to 3, characterized in that, The method further includes: Perform a first operation based on the response message. The first operation includes at least one of the following: retransmitting the data transmission instruction, discarding the data transmission instruction, releasing the resources associated with the requested data transmission instruction, reporting a fault, adding a device tag to the response message, or sending the response message.
5. The method according to any one of claims 1 to 4, characterized in that, The server cluster is a heterogeneous system.
6. The method according to claim 5, characterized in that, The heterogeneous system is an artificial intelligence (AI) cluster.
7. A data processing chip, wherein the data processing chip is a computing chip or a switching chip in a server cluster, the server cluster including multiple servers connected through a secondary switching chip, each of the multiple servers including multiple computing chips connected through a primary switching chip, and the data processing chip including a transceiver module and a processing module: The transceiver unit is used to acquire data transmission instructions, which are either read instructions or write instructions. The transceiver unit is also used to send or execute the data transmission instructions; The processing unit is configured to generate a substitute message or receive the substitute message in the event that the data transmission command fails to be sent or executed. The substitute message is used to indicate the reason for the failure of the data transmission command to be sent or executed.
8. The data processing chip according to claim 7, characterized in that, The processing unit is specifically used for: A response message is generated based on the first information, which includes at least one of the following: port number, sender information of the data transmission instruction, address range, or message format error information.
9. The data processing chip according to claim 7 or 8, characterized in that, The response message includes at least one of the following: response type, fault code, response value, response content, or device tag.
10. The data processing chip according to any one of claims 7 to 9, characterized in that, The processing unit is also used for: Perform a first operation based on the response message. The first operation includes at least one of the following: retransmitting the data transmission instruction, discarding the data transmission instruction, releasing the resources associated with the requested data transmission instruction, reporting a fault, adding a device tag to the response message, or sending the response message.
11. The data processing chip according to any one of claims 7 to 10, characterized in that, The server cluster is a heterogeneous system.
12. The data processing chip according to claim 11, characterized in that, The heterogeneous system is an AI cluster.
13. A server cluster, characterized in that, It includes a first chip and a second chip, wherein the first chip and the second chip are data processing chips as described in any one of claims 7 to 12; The first chip is used to send the response message to the second chip, the response message being used to indicate the reason for the failure to send or execute the data transmission instruction; The second chip is configured to perform a first operation based on the proxy message, the first operation including at least one of retransmitting the data transmission instruction, discarding the data transmission instruction, releasing resources related to the requested data transmission instruction, reporting a fault, adding a device tag to the proxy message, or sending the proxy message.
14. The server cluster according to claim 13, characterized in that, The second chip is specifically used for: If the response message indicates that the link is disconnected, perform at least one of the following: retransmit the data transmission instruction, report the fault, or add a device tag to the response message.
15. The server cluster according to claim 13 or 14, characterized in that, The second chip is specifically used for: If the response message indicates that the destination chip is unresponsive, execute at least one of the following: discard the data transmission instruction, release the resources associated with the data transmission instruction, or report a fault.
16. The server cluster according to any one of claims 13 to 15, characterized in that, The second chip is specifically used for: If the response message indicates that the data transmission instruction is formatted incorrectly, perform at least one of the following actions: discard the data transmission instruction, release the resource associated with the request for the data transmission instruction, or report a fault.
17. The server cluster according to any one of claims 13 to 16, characterized in that, The second chip is specifically used for: If the response message indicates that the data transmission instruction forwarding has timed out, perform at least one of the following: retransmit the data transmission instruction, report the fault, or add a device flag to the response message.
18. A data processing apparatus, comprising at least one processor and a memory, characterized in that, The at least one processor executes a program or instructions stored in a memory to cause the data processing apparatus to implement the method of any one of claims 1 to 6.
19. A computer-readable storage medium for storing a computer program, characterized in that, When the computer program is run on a computer or processor, it causes the computer or processor to perform the method of any one of claims 1 to 6.
20. A computer program product, the computer program product comprising instructions, characterized in that, When the instructions are executed on a computer or processor, the computer or processor causes the computer or processor to perform the method of any one of claims 1 to 6.
