A quantum heuristic error correction method for large model training multi-card cluster communication

Abstract

This application relates to the field of ultra-high-speed data transmission technology and provides a quantum-heuristic error correction method for multi-GPU cluster communication in large model training. The method includes: first, encoding the original data using Hamming code (7,4) to generate a 7-bit sequence containing a parity bit; then, converting each Hamming code bit into an orthogonal code of length m to form an orthogonal code matrix; next, expanding the matrix into linearly entangled codes in column-major order for transmission to disperse burst errors; after the receiving end de-entangles and recovers the orthogonal code matrix, calculating the Hamming distance between each data packet and the basis, selecting the basis with the smallest distance for error correction, and then inversely mapping back to Hamming codes to extract the original data. Thus, through a dual encoding architecture of Hamming codes and orthogonal codes, combined with data entanglement and recombination technology, the system's fault tolerance and training stability can be improved without sacrificing communication efficiency, providing a reliable data link guarantee for distributed large model training.

Description

Technical Field

[0001] This application belongs to the field of ultra-high-speed data transmission technology, and in particular relates to a quantum heuristic error correction method for multi-card cluster communication in large model training. Background Technology

[0002] In the training of large-scale artificial intelligence models (such as GPT, BERT, Diffusion models, etc.), multiple GPUs or accelerator cards are usually required to form a distributed computing cluster, which uses a high-speed interconnect network for parameter synchronization and gradient communication. As the model size and training data volume continue to grow, the frequency of data exchange and bandwidth requirements between multiple cards increase dramatically, and the communication link becomes a key factor affecting the overall training performance and stability.

[0003] Currently, most mainstream error detection and correction mechanisms are based on CRC check, ARQ retransmission, or linear error correction coding. However, these traditional error correction mechanisms suffer from poor stability and low real-time performance in scenarios where both sudden and random errors coexist. Summary of the Invention

[0004] This application provides a quantum-inspired error correction method, device, terminal equipment, and storage medium for multi-card cluster communication in large model training. It can solve the problems of poor stability and low real-time performance of traditional error correction mechanisms in scenarios where sudden and random errors coexist.

[0005] In a first aspect, embodiments of this application provide a quantum heuristic error correction method for multi-card cluster communication during large model training, comprising: S1, acquiring bitstream data between multi-card clusters during large model training; S2, encoding the bitstream data into a Hamming code H using a standard Hamming code (7,4) structure; S3, orthogonally encoding each bit of the Hamming code H to obtain an orthogonal code C, and entangled the orthogonal code C into an entangled orthogonal code J; S4, obtaining the entangled orthogonal code J into an erroneous entangled orthogonal code J' during data transmission; S5, reversing the entanglement process according to the entanglement rules to correct errors in the data. S6. Deentangle the erroneous entangled orthogonal code J' to obtain the erroneous orthogonal code C'; S7. Calculate the Hamming distance between the erroneous orthogonal code C' and the two orthogonal code bases respectively, and combine the orthogonal code base with the smallest Hamming distance with the erroneous orthogonal code C' to obtain the error-correcting orthogonal code C1; S8. Decode the error-correcting orthogonal code C1 to obtain the error-correcting Hamming code H1; S9. Extract the error-corrected data D1 from the error-correcting Hamming code H1; S1. Use the error-corrected data D1 for error communication data recovery between multi-card clusters during large model training to ensure that each computing node can receive accurate training parameters.

[0006] In one possible implementation of the first aspect, the above S2 uses a standard Hamming code (7,4) structure to encode the bitstream data into Hamming code H, including:

[0007] Define encoding rules:

[0008] Define D = [d1, d2, d3, d4] as 4-bit bitstream data, define P = [p1, p2, p3] as parity bits, and define H = [p1, p2, d1, p3, d2, d3, d4] as Hamming code;

[0009] Formula for calculating the parity bit P:

[0010] Perform an XOR operation on each of the first 1 bits in the binary representation of all Hamming code positions H:

[0011] p1 = d1⊕d2⊕d4

[0012] Perform an XOR operation on each of the data bits whose second bit is 1 in the binary representation of the H position of the Hamming code:

[0013] p2 = d1⊕d3⊕d4

[0014] Perform an XOR operation on each of the data bits whose 3rd bit is 1 in the binary representation of the H position of the Hamming code:

[0015] p3 = d2⊕d3⊕d4.

[0016] Optionally, in another possible implementation of the first aspect, S3 above orthogonally encodes each bit of the Hamming code H to obtain an orthogonal code C, and entangles the orthogonal code C into an entangled orthogonal code J, including:

[0017] Convert each bit of the Hamming code H into an orthogonal code of length m to generate a 7m-bit orthogonal code C, where m is a positive integer;

[0018] Convert the orthogonal code C into a 7-row, m-column matrix M;

[0019] Expand matrix M column by column to generate entangled orthogonal code J.

[0020] Optionally, in another possible implementation of the first aspect, S5 above deentangles the erroneous entangled orthogonal code J' according to the reverse process of the entanglement rule to obtain the erroneous orthogonal code C', including:

[0021] Convert the erroneous entangled orthogonal code J' into a 7-row, m-column matrix M';

[0022] Expand matrix M' by rows to generate error orthogonal code C'.

[0023] Optionally, in another possible implementation of the first aspect, S6 calculates the Hamming distances between the erroneous orthogonal code C' and the two orthogonal code bases, and combines the orthogonal code base with the smallest Hamming distance and the erroneous orthogonal code C' to obtain the error-correcting orthogonal code C1, including:

[0024] Perform a bitwise XOR operation on each m-bit data packet in the erroneous orthogonal code C' with the first orthogonal code basis and the second orthogonal code basis respectively;

[0025] The two sets of result sequences obtained by the XOR operation are denoted as the first sequence and the second sequence, where each bit that is different from the basis is set to 1 after the XOR operation;

[0026] The number of 1s in the first and second sequences are counted and compared. The orthogonal code basis corresponding to the sequence with the fewest 1s is selected as the optimal error correction basis.

[0027] Replace the erroneous orthogonal code C' with the optimal error-correcting basis to generate the error-correcting orthogonal code C1.

[0028] Optionally, in another possible implementation of the first aspect, S7 decodes the error-correcting orthogonal code C1 to obtain the error-correcting Hamming code H1, including:

[0029] Convert each m-bit orthogonal code in the error-correcting orthogonal code C1 into a 1-bit Hamming code to obtain the error-correcting Hamming code H1.

[0030] Optionally, in another possible implementation of the first aspect, the above-mentioned S8 extracts the error-corrected data D1 from the error-correcting Hamming code H1, including:

[0031] Extract the 3rd, 5th, 6th, and 7th bits from the error-correcting Hamming code H1 in bit order;

[0032] The four extracted bits are combined sequentially to obtain the corrected data D1.

[0033] Secondly, embodiments of this application provide a quantum-heuristic error correction device for multi-card cluster communication during large model training, comprising: a first acquisition module for acquiring bitstream data between multi-card clusters during large model training; an encoding module for encoding the bitstream data into a Hamming code H using a standard Hamming code (7,4) structure; an entanglement module for orthogonally encoding each bit of the Hamming code H to obtain an orthogonal code C, and entangled the orthogonal code C into an entangled orthogonal code J; a second acquisition module for acquiring the entangled orthogonal code J into an erroneous entangled orthogonal code J' during data transmission; and a deentanglement module for reversing the entanglement rules. The process involves untangling the erroneous entangled orthogonal code J' to obtain the erroneous orthogonal code C'; a first acquisition module calculates the Hamming distances between the erroneous orthogonal code C' and the two orthogonal code bases, and combines the orthogonal code base with the smallest Hamming distance with the erroneous orthogonal code C' to obtain the error-correcting orthogonal code C1; a second acquisition module decodes the error-correcting orthogonal code C1 to obtain the error-correcting Hamming code H1; an extraction module extracts the error-corrected data D1 from the error-correcting Hamming code H1; and a recovery module uses the error-corrected data D1 to recover the error communication data between multi-card clusters during large model training, ensuring that each computing node can receive accurate training parameters.

[0034] Thirdly, embodiments of this application provide a terminal device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement a quantum heuristic error correction method for multi-card cluster communication for large model training as described above.

[0035] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements a quantum heuristic error correction method for multi-card cluster communication in large model training as described above.

[0036] In this technical solution, the original data is first encoded using Hamming code (7,4) to generate a 7-bit sequence including a parity bit. Then, each Hamming code bit is converted into an orthogonal code of length m, forming an orthogonal code matrix. Next, the matrix is ​​expanded into a linearly entangled code for transmission in column-major order to disperse burst errors. At the receiving end, after untangling and recovering the orthogonal code matrix, the Hamming distance between each data packet and the basis is calculated. The basis with the smallest distance is selected for error correction, and then the data is inversely mapped back to Hamming codes to extract the original data. Therefore, through a dual encoding architecture of Hamming codes and orthogonal codes, combined with data entanglement and recombination technology, the system's fault tolerance and training stability can be improved without sacrificing communication efficiency, providing a reliable data link guarantee for distributed large-scale model training. Attached Figure Description

[0037] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0038] Figure 1 This is a flowchart illustrating a quantum heuristic error correction method for multi-card cluster communication in large model training, provided in an embodiment of this application.

[0039] Figure 2 This is a schematic flowchart of a quantum heuristic error correction device for multi-card cluster communication in large model training, provided in one embodiment of this application.

[0040] Figure 3 This is a schematic diagram of the structure of the terminal device provided in the embodiments of this application. Detailed Implementation

[0041] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0042] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.

[0043] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0044] As used in this application specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if detected [the described condition or event]" may be interpreted, depending on the context, as meaning "once determined," "in response to determination," "once detected [the described condition or event]," or "in response to detection [the described condition or event]."

[0045] Furthermore, in the description of this application and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0046] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0047] The following description, with reference to the accompanying drawings, details a quantum-inspired error correction method, apparatus, terminal device, and storage medium for multi-GPU cluster communication in large model training, provided in this application.

[0048] Figure 1 The illustration shows a flowchart of a quantum heuristic error correction method for multi-GPU cluster communication in large model training, provided in an embodiment of this application.

[0049] like Figure 1 As shown, the quantum heuristic error correction method for multi-GPU cluster communication during large model training includes the following steps:

[0050] S1. Obtain bitstream data between multi-card cluster communications during large model training;

[0051] S2. The bitstream data is encoded into Hamming code H using the standard Hamming code (7,4) structure;

[0052] Furthermore, in this embodiment of the application, S2 includes:

[0053] S201 defines the encoding rules:

[0054] Define D = [d1, d2, d3, d4] as 4-bit bitstream data, define P = [p1, p2, p3] as parity bits, and define H = [p1, p2, d1, p3, d2, d3, d4] as Hamming code;

[0055] S202 Formula for Calculating Check Bit P:

[0056] Perform an XOR operation on each of the first 1 bits in the binary representation of all Hamming code positions H:

[0057] p1 = d1⊕d2⊕d4

[0058] Perform an XOR operation on each of the data bits whose second bit is 1 in the binary representation of the H position of the Hamming code:

[0059] p2 = d1⊕d3⊕d4

[0060] Perform an XOR operation on each of the data bits whose 3rd bit is 1 in the binary representation of the H position of the Hamming code:

[0061] p3 = d2⊕d3⊕d4.

[0062] S3. Orthogonally encode each bit of the Hamming code H to obtain the orthogonal code C, and entangle the orthogonal code C into the entangled orthogonal code J;

[0063] Furthermore, in this embodiment, S3 includes:

[0064] Convert each bit of the Hamming code H into an orthogonal code of length m to generate a 7m-bit orthogonal code C, where m is a positive integer;

[0065] Convert the orthogonal code C into a 7-row, m-column matrix M;

[0066] Expand matrix M column by column to generate entangled orthogonal code J.

[0067] S4. Obtain the entangled orthogonal code J' if an error occurs during data transmission;

[0068] S5. Deentangle the erroneous entangled orthogonal code J' by reversing the entanglement rule to obtain the erroneous orthogonal code C'.

[0069] Furthermore, in this embodiment of the application, the above-mentioned S5 includes:

[0070] Convert the erroneous entangled orthogonal code J' into a 7-row, m-column matrix M';

[0071] Expand matrix M' by rows to generate error orthogonal code C'.

[0072] S6. Calculate the Hamming distances between the erroneous orthogonal code C' and the two orthogonal code bases respectively. Combine the orthogonal code base with the smallest Hamming distance and the erroneous orthogonal code C' to obtain the error-correcting orthogonal code C1.

[0073] Furthermore, in this embodiment, S6 includes:

[0074] Perform a bitwise XOR operation on each m-bit data packet in the erroneous orthogonal code C' with the first orthogonal code basis and the second orthogonal code basis respectively;

[0075] The two sets of result sequences obtained by the XOR operation are denoted as the first sequence and the second sequence, where each bit that is different from the basis is set to 1 after the XOR operation;

[0076] The number of 1s in the first and second sequences are counted and compared. The orthogonal code basis corresponding to the sequence with the fewest 1s is selected as the optimal error correction basis.

[0077] Replace the erroneous orthogonal code C' with the optimal error-correcting basis to generate the error-correcting orthogonal code C1.

[0078] S7. Decode the error-correcting orthogonal code C1 to obtain the error-correcting Hamming code H1;

[0079] Furthermore, in this embodiment, S7 includes:

[0080] Convert each m-bit orthogonal code in the error-correcting orthogonal code C1 into a 1-bit Hamming code to obtain the error-correcting Hamming code H1.

[0081] S8. Extract the error-corrected data D1 from the error-correcting Hamming code H1;

[0082] Furthermore, in this embodiment, the above-mentioned S8 includes:

[0083] Extract the 3rd, 5th, 6th, and 7th bits from the error-correcting Hamming code H1 in bit order;

[0084] The four extracted bits are combined sequentially to obtain the corrected data D1.

[0085] S9. Use the corrected data D1 to recover the erroneous communication data between the multi-GPU cluster during the large model training process, ensuring that each computing node can receive accurate training parameters.

[0086] The following example illustrates the specific implementation of steps S1-S9 above. This example is exemplary and does not limit the scope of this application.

[0087] Step 101: Hamming encoding of the bitstream data between multi-card cluster communications during large model training.

[0088] Step 101.1: Definition of Encoding Rules

[0089] This method uses the standard Hamming code (7,4) structure, and the specific rules are as follows:

[0090] Define D = [d1, d2, d3, d4] as 4-bit raw input data, define P = [p1, p2, p3] as parity bits, and define the Hamming code as H = [p1, p2, d1, p3, d2, d3, d4].

[0091] Step 101.2: Formula for calculating the check bit P

[0092] p1: XOR the first bit of all Hamming code H positions in the binary representation of the data bits (positions 3, 5, 7) that are 1, i.e., formula (1): p1 = d1⊕d2⊕d4.

[0093] p2: XOR the data bits (positions 3, 6, 7) that are 1 in the binary representation of all Hamming code H positions, i.e., formula (2): p2 = d1⊕d3⊕d4.

[0094] p3: XOR the data bits (positions 5, 6, 7) that are 1 in the 3rd bit of the binary representation of all Hamming code H positions, i.e., formula (3): p3 = d2⊕d3⊕d4.

[0095] Note: ⊕ represents the XOR operation.

[0096] Step 102: Hamming code orthogonal encoding

[0097] Using the Hamming code H obtained in step 101 as input, each bit of the Hamming code H needs to be converted into an orthogonal code. The orthogonal code length m can be passed as a parameter in practical applications to ensure dynamic adjustment capability. In the practical application example, m = 8 is taken. There are two permutations of the orthogonal code, namely 01010101 and 10101010, which guarantees the maximum Hamming distance. The orthogonal code mapping rule is: 0 maps to 01010101, and 1 maps to 10101010. After orthogonally encoding each bit of the Hamming code H, every 8 bits are recorded as a data packet, generating a total of 56-bit orthogonal code sequence C.

[0098] Step 103: Data Entanglement

[0099] The orthogonal code sequence C = [c1, c2, ..., c7] obtained in step 102 is used as input, where each ci represents the i-th orthogonal code packet. The orthogonal code sequences C are arranged into a 7×8 matrix M (each row representing a data packet), i.e.:

[0100]

[0101] The entanglement operation involves expanding this matrix column-wise, writing it as an entangled orthogonal code J, as shown in the formula:

[0102] J = [c 1,1 ,c 2,1 ,…,c 7,1 ,c 1,2 ,c 2,2 ,…,c 1,8 ,…,c 7,8 ].

[0103] Step 104: Data Error

[0104] The entangled orthogonal code J obtained in step 103 is input. Due to interference such as channel noise and electromagnetic pulse (EMP), the entangled orthogonal code J is corrupted during data transmission. The corrupted entangled orthogonal code is recorded as the erroneous entangled orthogonal code J'.

[0105] Step 105: The receiving end performs data deentanglement.

[0106] Input the erroneous entangled orthogonal code J' obtained in step 104, and deentangle it according to the reverse of the entanglement rule in step 3, that is: arrange the erroneous entangled orthogonal code J' into a 7×m matrix M' in column-major order, i.e.:

[0107]

[0108] Expanding further according to row order, we obtain the error orthogonal code C', as shown in the formula:

[0109] C' = [c 1,1 ,c 1,2 ,…,c 1,m ,c 2,1 ,c 2,2 ,…,c 2,m ,…,c 7,m ]

[0110] If we still consider m bits as one data packet, then C' has 7 data packets, totaling 7 × m bits.

[0111] Step 106: Data Correction

[0112] Using the erroneous orthogonal code C' obtained in step 105 as input, the core task is to calculate the Hamming distance between each data packet of C' and the two orthogonal code bases. Each 8-bit data packet is XORed with the two orthogonal code bases 01010101 and 10101010 respectively. If any bit differs from the base, the XOR result is "1". This yields two XOR results, denoted as Sequence 1 and Sequence 2. The number of "1"s in Sequence 1 and Sequence 2 is counted. The sequence with the fewest "1"s (i.e., the smallest Hamming distance) is found. The erroneous data is most similar to the orthogonal code base corresponding to this sequence, and the data can be considered to have been caused by an error in that orthogonal code base. Finally, the original data packet is replaced with this orthogonal code base to obtain the corrected orthogonal code, denoted as C1.

[0113] Step 107: Orthogonal code decoding

[0114] Using the error-correcting orthogonal code C1 obtained in step 106 as input, decode it according to the inverse rule of the mapping rule in step 102, that is: 01010101 maps to 0; 10101010 maps to 1. The resulting 7-bit Hamming code after error correction is denoted as H1.

[0115] Step 108: Extract the correct data from the Hamming code

[0116] Using the error-correcting Hamming code H1 obtained in step 107 as input, the 3rd, 5th, 6th, and 7th bits of the error-correcting Hamming code H1 = [p1,p2,d1,p3,d2,d3,d4] are the real data, which can be extracted. Thus, the error-corrected data D1 = [d1,d2,d3,d4] is obtained.

[0117] This application provides a quantum-heuristic error correction method for multi-GPU cluster communication in large-scale model training. First, the original data is encoded using Hamming code (7,4) to generate a 7-bit sequence including a parity bit. Then, each Hamming code bit is converted into an orthogonal code of length m, forming an orthogonal code matrix. Next, the matrix is ​​expanded into linearly entangled codes in column-major order for transmission to disperse burst errors. At the receiving end, after untangling and recovering the orthogonal code matrix, the Hamming distance between each data packet and the basis is calculated. The basis with the smallest distance is selected for error correction, and then the data is inversely mapped back to Hamming codes to extract the original data. Thus, through a dual encoding architecture of Hamming codes and orthogonal codes, combined with data entanglement and recombination technology, the system's fault tolerance and training stability can be improved without sacrificing communication efficiency, providing a reliable data link guarantee for distributed large-scale model training.

[0118] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0119] Corresponding to the above embodiment, a quantum heuristic error correction method for multi-GPU cluster communication in large model training, Figure 2 This diagram illustrates a structural block diagram of a quantum heuristic error correction device for multi-card cluster communication in large model training, provided by an embodiment of this application. For ease of explanation, only the parts relevant to the embodiments of this application are shown.

[0120] Reference Figure 2 The device 2000 includes:

[0121] The first acquisition module 2001 is used to acquire bitstream data between multi-card cluster communications during the training of a large model.

[0122] Encoding module 2002 is used to encode bitstream data into Hamming code H using the standard Hamming code (7,4) structure;

[0123] Entanglement module 2003 is used to orthogonally encode each bit of Hamming code H to obtain orthogonal code C, and then entangle orthogonal code C into entangled orthogonal code J;

[0124] The second acquisition module 2004 is used to acquire the entangled orthogonal code J that has been erroneously entangled orthogonal code J' during data transmission.

[0125] The deentanglement module 2005 is used to deentangle the erroneous entangled orthogonal code J' according to the reverse process of the entanglement rule to obtain the erroneous orthogonal code C'.

[0126] The first obtaining module 2006 is used to calculate the Hamming distance between the erroneous orthogonal code C' and the two orthogonal code bases respectively, and combine the orthogonal code base with the smallest Hamming distance and the erroneous orthogonal code C' to obtain the error-correcting orthogonal code C1;

[0127] The second acquisition module 2007 is used to decode the error-correcting orthogonal code C1 to obtain the error-correcting Hamming code H1.

[0128] Extraction module 2008 is used to extract the error-corrected data D1 from the error-correcting Hamming code H1;

[0129] Recovery module 2009 is used to recover the erroneous communication data between multi-GPU clusters during the training of large models using the corrected data D1, ensuring that each computing node can receive accurate training parameters.

[0130] In practical use, the quantum heuristic error correction device for multi-card cluster communication for large model training provided in this application embodiment can be configured in any terminal device to execute the aforementioned quantum heuristic error correction method for multi-card cluster communication for large model training.

[0131] This application provides a quantum-heuristic error correction device for multi-GPU cluster communication in large-scale model training. First, the original data is encoded using Hamming code (7,4) to generate a 7-bit sequence including a parity bit. Then, each Hamming code bit is converted into an orthogonal code of length m, forming an orthogonal code matrix. Next, the matrix is ​​expanded into linearly entangled codes in column-major order for transmission to disperse burst errors. At the receiving end, after untangling and recovering the orthogonal code matrix, the Hamming distance between each data packet and the basis is calculated. The basis with the smallest distance is selected for error correction, and then the data is inversely mapped back to Hamming codes to extract the original data. Thus, through a dual encoding architecture of Hamming codes and orthogonal codes, combined with data entanglement and recombination technology, the fault tolerance and training stability of the system can be improved without sacrificing communication efficiency, providing a reliable data link guarantee for distributed large-scale model training.

[0132] It should be noted that the information interaction and execution process between the above-mentioned devices / units are based on the same concept as the method embodiments of this application. For details on their specific functions and technical effects, please refer to the method embodiments section, and they will not be repeated here.

[0133] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0134] To implement the above embodiments, this application also proposes a terminal device.

[0135] Figure 3 This is a schematic diagram of the structure of a terminal device according to an embodiment of this application.

[0136] like Figure 3 As shown, the terminal device 200 includes:

[0137] The system includes a memory 210 and at least one processor 220, and a bus 230 connecting different components (including the memory 210 and the processor 220). The memory 210 stores a computer program, and when the processor 220 executes the program, it implements a quantum heuristic error correction method for multi-card cluster communication for large model training as described in the embodiments of this application.

[0138] Bus 230 represents one or more of several bus architectures, including a memory bus or memory controller, a peripheral bus, a graphics acceleration port, a processor, or a local bus using any of the various bus architectures. Examples of these architectures include, but are not limited to, the Industry Standard Architecture (ISA) bus, the Micro Channel Architecture (MAC) bus, the Enhanced ISA bus, the Video Electronics Standards Association (VESA) local bus, and the Peripheral Component Interconnect (PCI) bus.

[0139] Terminal device 200 typically includes various electronically readable media. These media can be any available media that can be accessed by terminal device 200, including volatile and non-volatile media, removable and non-removable media.

[0140] Memory 210 may also include computer system readable media in the form of volatile memory, such as random access memory (RAM) 240 and / or cache memory 250. Terminal device 200 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, storage system 260 may be used to read and write non-removable, non-volatile magnetic media (…). Figure 3 Not shown; usually referred to as a "hard drive"). Although Figure 3 Not shown, a disk drive for reading and writing to a removable non-volatile disk (e.g., a "floppy disk") and an optical disk drive for reading and writing to a removable non-volatile optical disk (e.g., a CD-ROM, DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to bus 230 via one or more data media interfaces. Memory 210 may include at least one program product having a set (e.g., at least one) of program modules configured to perform the functions of the embodiments of this application.

[0141] A program / utility 280 having a set (at least one) of program modules 270 may be stored in, for example, memory 210. Such program modules 270 include—but are not limited to—an operating system, one or more application programs, other program modules, and program data. Each or some combination of these examples may include an implementation of a network environment. Program modules 270 typically perform the functions and / or methods described in the embodiments of this application.

[0142] Terminal device 200 can also communicate with one or more external devices 290 (e.g., keyboard, pointing device, display 291, etc.), and with one or more devices that enable a user to interact with terminal device 200, and / or with any device that enables terminal device 200 to communicate with one or more other computing devices (e.g., network card, modem, etc.). This communication can be performed via input / output (I / O) interface 292. Furthermore, terminal device 200 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 293. As shown, network adapter 293 communicates with other modules of terminal device 200 via bus 230. It should be understood that, although not shown in the figures, other hardware and / or software modules can be used in conjunction with terminal device 200, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0143] The processor 220 performs various functional applications and data processing by running programs stored in the memory 210.

[0144] It should be noted that the implementation process and technical principles of the terminal device in this embodiment are explained in the foregoing description of a quantum heuristic error correction method for multi-card cluster communication for large model training in this application embodiment, and will not be repeated here.

[0145] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps described in the various method embodiments above.

[0146] This application provides a computer program product that, when run on a terminal device, enables the terminal device to implement the steps described in the various method embodiments above.

[0147] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of this application can be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include at least: any entity or device capable of carrying computer program code to a photographing device / terminal device, a recording medium, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium. Examples include USB flash drives, portable hard drives, magnetic disks, or optical disks. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electrical carrier signals or telecommunication signals.

[0148] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0149] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0150] In the embodiments provided in this application, it should be understood that the disclosed devices / terminal equipment and methods can be implemented in other ways. For example, the device / terminal equipment embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling or direct coupling or communication connection may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0151] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0152] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A quantum-inspired error correction method for large model training multi-card cluster communication, characterized in that, include: S1. Obtain bitstream data between multi-card cluster communications during large model training; S2. The bitstream data is encoded into Hamming code H using the standard Hamming code (7,4) structure; S3. Orthogonally encode each bit of the Hamming code H to obtain the orthogonal code C, and then entangle the orthogonal code C into an entangled orthogonal code J. The obtained Hamming code H is used as input, and each bit of the Hamming code H needs to be converted into an orthogonal code. The orthogonal code length m can be passed as a parameter in practical applications. In this practical example, m=8. There are two arrangements of the orthogonal code: 01010101 and 10101010. The orthogonal code mapping rule is: 0 maps to 01010101, and 1 maps to 10101010. After orthogonally encoding each bit of the Hamming code H, every 8 bits are recorded as a data packet, generating a 56-bit orthogonal code sequence C. The obtained orthogonal code sequence C=[c1,c2,…,c7] is used as input, where each ci represents the i-th orthogonal code packet. The orthogonal code sequences C are arranged into a 7×8 matrix M, with each row representing a data packet. ； The entanglement operation involves expanding the matrix column-wise, resulting in an entangled orthogonal code J, as shown in the formula: ； S4. Obtain the entangled orthogonal code J' which is erroneous during data transmission; wherein, the obtained entangled orthogonal code J is input, and due to channel noise and electromagnetic pulse interference, the entangled orthogonal code J is erroneous during data transmission, and the entangled orthogonal code after the error is recorded as the erroneous entangled orthogonal code J'. S5. Deentangle the erroneous entangled orthogonal code J' according to the reverse process of the entanglement rule to obtain the erroneous orthogonal code C'; wherein, the obtained erroneous entangled orthogonal code J' is input, and deentangled according to the reverse of the entanglement rule in step S3, that is: the erroneous entangled orthogonal code J' is arranged into a 7×m matrix M' in column-major order, i.e.: ； Expanding in row-major order, we obtain the error-prone orthogonal code C', as shown in the formula: ； If we still consider m bits as one data packet, then C' has 7 data packets, totaling 7 × m bits; S6. Calculate the Hamming distances between the erroneous orthogonal code C' and the two orthogonal code bases respectively. Combine the orthogonal code base with the smallest Hamming distance with the erroneous orthogonal code C' to obtain the error-correcting orthogonal code C1. Using the obtained erroneous orthogonal code C' as input, calculate the Hamming distance between each data packet of C' and the two orthogonal code bases. XOR each 8-bit data packet with the two orthogonal code bases 01010101 and 10101010 respectively. If any bit is different from the base, the XOR result is "1". The two XOR results are denoted as Sequence 1 and Sequence 2. Count the number of "1"s in Sequence 1 and Sequence 2. Find the sequence with the fewest "1"s, i.e., the sequence with the smallest Hamming distance. The erroneous data is most similar to the orthogonal code base corresponding to this sequence, and the data is considered to be caused by an error in this orthogonal code base. Replace the original data packet with this orthogonal code base to obtain the error-correcting orthogonal code, denoted as C1. S7. Decode the error-correcting orthogonal code C1 to obtain the error-correcting Hamming code H1. The obtained error-correcting orthogonal code C1 is used as input and decoded according to the inverse rule of the mapping rule in step S3, that is: 01010101 is mapped to 0; 10101010 is mapped to 1, to obtain the 7-bit Hamming code after error correction, denoted as H1. S8. Extract the error-corrected data D1 from the error-correcting Hamming code H1; wherein, the obtained error-correcting Hamming code H1 is used as input, and the 3rd, 5th, 6th, and 7th bits of the error-correcting Hamming code H1 = [p1,p2,d1,p3,d2,d3,d4] are the real data, and the error-corrected data D1 = [d1,d2,d3,d4] is extracted. S9. Use the corrected data D1 to recover the erroneous communication data between the multi-GPU cluster during the large model training process, ensuring that each computing node can receive accurate training parameters.

2. The method of claim 1, wherein, The S2 uses the standard Hamming code (7,4) structure to encode the bitstream data into Hamming code H, including: Define encoding rules: Define D = [d1, d2, d3, d4] as 4-bit bitstream data, define P = [p1, p2, p3] as parity bits, and define H = [p1, p2, d1, p3, d2, d3, d4] as Hamming code; Formula for calculating the parity bit P: Perform an XOR operation on each of the first 1 bits in the binary representation of all Hamming code positions H: p1 = d1⊕d2⊕d4 Perform an XOR operation on each of the data bits whose second bit is 1 in the binary representation of the H position of the Hamming code: p2=d1⊕d3⊕d4 Perform an XOR operation on each of the data bits whose 3rd bit is 1 in the binary representation of the H position of the Hamming code: p3=d2⊕d3⊕d4.

3. A quantum-inspired error correction apparatus for large model training multi-card cluster communication, characterized in that, include: The first acquisition module is used to acquire bitstream data between multi-card cluster communications during the training of a large model. The encoding module is used to encode bitstream data into Hamming code H using the standard Hamming code (7,4) structure; The entanglement module is used to orthogonally encode each bit of the Hamming code H to obtain an orthogonal code C, and then entangle the orthogonal code C into an entangled orthogonal code J. The obtained Hamming code H is used as input, and each bit of the Hamming code H needs to be converted into an orthogonal code. The orthogonal code length m can be passed as a parameter in practical applications; in this example, m=8. There are two permutations of the orthogonal code: 01010101 and 10101010. The orthogonal code mapping rule is: 0 maps to 01010101, and 1 maps to 10101010. After orthogonally encoding each bit of the Hamming code H, every 8 bits are recorded as a data packet, generating a 56-bit orthogonal code sequence C. The resulting orthogonal code sequence C=[c1,c2,…,c7] is used as input, where each ci represents the i-th orthogonal code packet. The orthogonal code sequences C are arranged into a 7×8 matrix M, with each row representing a data packet. ； The entanglement operation involves expanding the matrix column-wise, resulting in an entangled orthogonal code J, as shown in the formula: ； The second acquisition module is used to acquire the entangled orthogonal code J that has been erroneous during data transmission as the erroneous entangled orthogonal code J'. The entangled orthogonal code J is input, and due to channel noise and electromagnetic pulse interference, the entangled orthogonal code J is erroneous during data transmission. The entangled orthogonal code after the error is recorded as the erroneous entangled orthogonal code J'. The de-entanglement module is used to de-entangle the erroneous entangled orthogonal code J' according to the reverse process of the entanglement rule, to obtain the erroneous orthogonal code C'; wherein, the obtained erroneous entangled orthogonal code J' is input, and de-entanglement is performed in reverse according to the entanglement rule of step S3, that is: the erroneous entangled orthogonal code J' is arranged into a 7×m matrix M' in column-major order, i.e.: ； Expanding in row-major order, we obtain the error-prone orthogonal code C', as shown in the formula: ； If we still consider m bits as one data packet, then C' has 7 data packets, totaling 7 × m bits; The first acquisition module is used to calculate the Hamming distances between the erroneous orthogonal code C' and the two orthogonal code bases respectively. Combining the orthogonal code base with the smallest Hamming distance and the erroneous orthogonal code C', the error-correcting orthogonal code C1 is obtained. Specifically, the obtained erroneous orthogonal code C' is used as input to calculate the Hamming distance between each data packet of C' and the two orthogonal code bases. Each 8-bit data packet is XORed with the two orthogonal code bases 01010101 and 10101010 respectively. If any bit is different from the base, the result of XOR is "1". The two results after XOR are denoted as Sequence 1 and Sequence 2. The number of "1"s in Sequence 1 and Sequence 2 is counted. The sequence with the fewest "1"s, i.e., the smallest Hamming distance, is found. The erroneous data is most similar to the orthogonal code base corresponding to this sequence. It is considered that the data is caused by the error of this orthogonal code base. The original data packet is replaced with this orthogonal code base to obtain the error-correcting orthogonal code, denoted as C1. The second obtaining module is used to decode the error-correcting orthogonal code C1 to obtain the error-correcting Hamming code H1. The obtained error-correcting orthogonal code C1 is used as input and decoded according to the inverse rule of the mapping rule in step S3, that is: 01010101 is mapped to 0; 10101010 is mapped to 1, to obtain the 7-bit Hamming code after error correction, denoted as H1. The extraction module is used to extract the error-corrected data D1 from the error-correcting Hamming code H1. The obtained error-correcting Hamming code H1 is used as input. The 3rd, 5th, 6th, and 7th bits of the error-correcting Hamming code H1 = [p1,p2,d1,p3,d2,d3,d4] are the real data, and the error-corrected data D1 = [d1,d2,d3,d4] is extracted. The recovery module is used to recover the erroneous communication data between multi-GPU clusters during the training of large models using the corrected data D1, ensuring that each computing node can receive accurate training parameters.

4. A terminal device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the method as described in any one of claims 1-2.

5. A computer-readable storage medium storing a computer program, the computer program comprising instructions that, when executed by a computer, cause the computer to perform the method of any one of claims 1 to 4. When the computer program is executed by a processor, it implements the method as described in any one of claims 1-2.

Images