Transmission system and method for intermediate result of large model

Abstract

The invention provides a transmission system and method for an intermediate result of a large model, the transmission system comprises a sending end and a receiving end, the sending end comprises an encoder, the encoder comprises a first encoding module and a second encoding module, the first encoding module is configured to receive the intermediate result and encode the intermediate result, and the second encoding module is configured to encode the intermediate result; a second encoding module configured to generate differential data, the second encoding module configured to convert the differential data into ternary values; the receiving end comprises an additive operation module, the additive operation module is configured to receive the ternary value and execute one-bit accumulation operation based on the ternary value so as to output a multiplication operation result, and the multiplication operation result is an intermediate result. According to the transmission system, all coding processes are completed at a sending end, transmission is carried out in a ternary value mode, coding and decoding at a receiving end are not needed, and the problems that transmission logic is delayed, hardware resource consumption is large, and a coder and a decoder occupy a large number of logic units are solved.

Description

Technical Field

[0001] This application relates to the field of chip technology, and in particular to a system and method for transmitting intermediate results of large models. Background Technology

[0002] Multi-chip interconnect architectures need to transmit intermediate results at high speed and perform multiplication operations between multiple chips. The intermediate results include feature maps, embedding vectors, and key-value cache data. When transmitting intermediate results through the serializer interface, there is a DC balance requirement, that is, to eliminate capacitive coupling loss caused by continuous 0 / 1 signals to ensure signal integrity. In this scenario, high-speed multiplication operations of intermediate results and weights are required, as well as a transmission encoding scheme with low latency and low hardware resource consumption.

[0003] To solve the DC balance problem, 8B / 10B encoding is used. Specifically, the 8-bit data is divided into the lower 5 bits and the higher 3 bits, which are then mapped to 10-bit codewords using 5B / 6B and 3B / 4B encoding, respectively. The encoder dynamically maintains the inconsistency state during operation. The encoder uses a lookup table to convert from 8B to 10B, and forces each sub-code block to ensure that the difference in the number of 0 / 1 values ​​does not exceed 1. When the cumulative level offset exceeds the limit, a control word is inserted to reset the balance state. The decoding process requires reverse reconstruction of the original data using a 256-item lookup table.

[0004] However, 8B / 10B encoding causes transmission logic delays. Since both encoding and decoding are based on multi-level lookup table operations, hardware resources are consumed in large quantities, and the encoder and decoder occupy a large number of logic units. Summary of the Invention

[0005] This application provides a transmission system and method for intermediate results of large models to solve the problems of transmission logic delay, high hardware resource consumption, and encoders and decoders occupying a large number of logic units.

[0006] In a first aspect, this application provides a system for transmitting intermediate results of a large model, comprising:

[0007] The transmitting end includes an encoder, which includes a first encoding module and a second encoding module. The first encoding module is configured to receive intermediate results and encode the intermediate results to generate differential data. The second encoding module is configured to convert the differential data into ternary values.

[0008] The receiving end includes an addition module, which is configured to receive the ternary value and perform a one-bit accumulation operation based on the ternary value to output a multiplication result, wherein the multiplication result is an intermediate result.

[0009] In some feasible embodiments, the first encoding module, when performing encoding on the intermediate results to generate differential data, is specifically configured as follows:

[0010] Convert the intermediate results into a sequential sequence of elements;

[0011] Subtraction is performed on the code values ​​of adjacent positions in the sequential element sequence to generate a first difference sequence; wherein the first element of the sequential element sequence is subtracted from a preset benchmark value;

[0012] Convert each element of the first difference sequence into binary data of a preset bit width;

[0013] The binary data is concatenated into differential data, which is a concatenated serial bit stream.

[0014] In some feasible embodiments, the second encoding module is specifically configured to convert the differential data into ternary values ​​as follows:

[0015] The differential data is parsed to divide it into binary data segments according to a preset bit width;

[0016] The binary data segments are output as binary sequences according to the segmentation order;

[0017] The binary sequence is converted into a ternary value.

[0018] In some feasible embodiments,

[0019] The second encoding module is configured to convert the binary sequence into ternary values ​​as follows:

[0020] Subtract the previous bit value from the current bit value to calculate the difference between adjacent bits;

[0021] Based on the difference, a ternary value sequence is output, which includes a first ternary value, a second ternary value, and a third ternary value.

[0022] Specifically, if the difference is greater than 0, the first ternary value is output; if the difference is less than 0, the second ternary value is output; if the difference is equal to 0, the third ternary value is output; and the number of the first ternary value and the second ternary value are equal.

[0023] In some feasible embodiments, the ternary value includes a negative level, a zero level, and a positive level; the multiplication result includes a first multiplication result and a second multiplication result;

[0024] The addition module receives the ternary value and performs a one-bit accumulation operation based on the ternary value to output the multiplication result. Specifically, it is configured as follows:

[0025] When the three-element value is at a negative level, a negative weight value is obtained, and a one-bit accumulation operation is performed by combining the negative weight value with the negative level to output the result of the first multiplication operation.

[0026] When the three-element value is at a positive level, a positive weight value is obtained, and a one-bit accumulation operation is performed by combining the positive weight value with the positive level to output the result of the second multiplication operation.

[0027] In some feasible embodiments, the system further includes a transmission channel, which includes an input end and an output end, the input end being connected to the second encoding module;

[0028] The transmission channel is configured as follows:

[0029] Receive the ternary value sent by the second encoding module;

[0030] The ternary values ​​are mapped to voltage level signals, and the voltage level signals are transmitted to the output terminal.

[0031] In some feasible embodiments, the receiving end further includes a signal reconstruction module, and the output end is connected to the signal reconstruction module;

[0032] The signal reconstruction module is configured as follows:

[0033] Receive the voltage level signal sent by the output terminal;

[0034] Based on the data location, the voltage level signal is reassembled to output reassembled data;

[0035] The recombined data is transmitted to the addition module, so that the addition module receives the recombined data and performs a one-bit accumulation operation based on the recombined data to output the multiplication result.

[0036] Secondly, this application provides a method for transferring intermediate results of a large model, including:

[0037] The intermediate results are obtained through the first encoding module, and the intermediate results are encoded to generate differential data;

[0038] The differential data is converted into ternary values ​​by the second encoding module;

[0039] The addition module receives the ternary value and performs a one-bit accumulation operation based on the ternary value to output the multiplication result, which is an intermediate result.

[0040] In some feasible embodiments, the step of obtaining intermediate results through the first encoding module and encoding the intermediate results to generate differential data includes:

[0041] Convert the intermediate results into a sequential sequence of elements;

[0042] Subtraction is performed on the code values ​​of adjacent positions in the sequential element sequence to generate a first difference sequence; wherein the first element of the sequential element sequence is subtracted from a preset benchmark value;

[0043] Convert each element of the first difference sequence into binary data of a preset bit width;

[0044] The binary data is concatenated into differential data, which is a concatenated serial bit stream.

[0045] In some feasible embodiments, the step of converting the differential data into ternary values ​​through the second encoding module includes:

[0046] The differential data is parsed to divide it into binary data segments according to a preset bit width;

[0047] The binary data segments are output as binary sequences according to the segmentation order;

[0048] The binary sequence is converted into a ternary value.

[0049] As can be seen from the above technical solutions, this application provides a transmission system and method for intermediate results of large models. The transmission system includes a transmitting end and a receiving end. The transmitting end includes an encoder, which includes a first encoding module and a second encoding module. The first encoding module is configured to receive the intermediate results and encode them to generate differential data. The second encoding module is configured to convert the differential data into ternary values. The receiving end includes an addition module, which is configured to receive the ternary values ​​and perform a one-bit accumulation operation based on the ternary values ​​to output a multiplication result, which is the intermediate result. The transmission system completes all encoding processes at the transmitting end and transmits the data using ternary values, eliminating the need for encoding and decoding at the receiving end. This solves the problems of transmission logic delay, high hardware resource consumption, and the large number of logic units occupied by the encoder and decoder. Attached Figure Description

[0050] To more clearly illustrate the technical solution of this application, the drawings used in the embodiments will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0051] Figure 1 A schematic diagram of 8b / 10b encoding provided for an embodiment of this application;

[0052] Figure 2A schematic diagram of the transmission system provided in an embodiment of this application;

[0053] Figure 3 This is a schematic diagram illustrating the process of converting differential data into ternary values, provided in an embodiment of this application.

[0054] Figure 4 This is a flowchart illustrating the ternary values ​​corresponding to the outputs provided in the embodiments of this application.

[0055] Figure 5 This is a schematic diagram comparing channel responses provided in an embodiment of this application;

[0056] Figure 6 This is a schematic diagram of static eye diagram comparison provided in the embodiments of this application;

[0057] Figure 7 This is a schematic diagram of dynamic eye diagram comparison provided in an embodiment of this application. Detailed Implementation

[0058] The embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described below do not represent all embodiments consistent with this application. They are merely examples of systems and methods consistent with some aspects of this application as detailed in the claims.

[0059] like Figure 1 As shown, 8b / 10b encoding is a standardized encoding circuit that maps an input 8-bit binary data unit (1 byte) to a 10-bit codeword. The encoding process follows a predefined 8b / 10b encoding table (e.g., IBM or IEEE standards). This mapping ensures, on the one hand, that the number of 0s and 1s in the transmitted code stream is roughly balanced (DC balance), and on the other hand, provides sufficient signal transition density to support clock recovery at the receiver.

[0060] The 8b / 10b encoded transmitter is the starting point for data transmission. The transmitter includes an encoder, which performs specific encoding operations on the raw intermediate data to adapt it to the requirements of the transmission channel. The transmission channel is the physical medium connecting the transmitter and receiver, carrying the 10-bit codeword stream output by the encoder, transmitted as a high-speed serial signal, such as a differential signal.

[0061] The receiver includes a decoder, a Booth encoder, and an addition module. The decoder is an 8b / 10b decoding module that receives a 10-bit codeword stream from the receiving channel and restores it to the original 8-bit binary intermediate result data through table lookup or logical decoding.

[0062] The Booth encoder receives the restored original 8-bit binary intermediate result data from the decoder. Its function is to re-encode this binary data into a ternary value representation based on the Booth algorithm, in order to prepare for subsequent multiplication and accumulation calculations, because Booth encoding can simplify multiplier design.

[0063] At the transmitting end, the intermediate results (raw binary data) are sent to the transmitting end encoder. The 8b / 10b encoding module maps each 8-bit data to a 10-bit codeword through a lookup table, and the encoded 10-bit codewords are sent to the transmission channel through the drive circuit.

[0064] At the receiving end, the first step is decoding. The received signal first enters the decoder. The 8b / 10b decoding module samples, aligns, and divides the signal into 10-bit codewords. Through lookup tables or complex decoding logic, each 10-bit codeword is restored to the original 8-bit binary intermediate result data. The restored complete original binary data block is then output.

[0065] The second step is secondary encoding. The original 8-bit binary data block output by the decoder is sent to the Booth encoder. The Booth encoder processes the input 8-bit binary data. Its core operation is to scan the data bits (or bit pairs) and generate the corresponding ternary value output according to the Booth algorithm rules.

[0066] The third step is weighted calculation. The ternary value sequence output by the Booth encoder is sent to the addition module. After the addition module traverses all positions, it outputs the final accumulated calculation result.

[0067] In other words, at the receiving end, the data must first be completely restored to its original binary form by a high-latency 8b / 10b decoder. The restored original data is then sent to a separate Booth encoder to be encoded again into ternary values ​​for use by the computing unit. This results in two encoding operations on the same data: 8b / 10b at the transmitting end and Booth at the receiving end.

[0068] Both the codec (8b / 10b) and the Booth encoder require independent hardware resources (lookup tables, logic circuits). The data stream is decoded and then re-encoded, introducing a serial delay chain that significantly slows down end-to-end speed.

[0069] To solve the above problems, such as Figure 2 As shown, some embodiments of this application provide a transmission system for intermediate results of a large model, including a sending end and a receiving end. The sending end includes an encoder for encoding the intermediate result data to be transmitted.

[0070] Intermediate results are data generated during the inference or training of large models that need to be transferred between different computing chips. Common types include feature maps, embeddings, and key-value caches (Kvcache). These data are characterized by large data volume and high transmission frequency.

[0071] The encoder includes a first encoding module and a second encoding module. The first encoding module is configured to convert the intermediate result into a sequential element sequence; perform subtraction operations on the code values ​​at adjacent positions in the sequential element sequence to generate a first difference sequence; wherein the first element of the sequential element sequence is subtracted from a preset base value; convert each element of the first difference sequence into binary data of a preset bit width; and concatenate the binary data into differential data, wherein the differential data is a concatenated serial bit stream.

[0072] The second encoding module is configured to convert the differential data into ternary values, with the differential data output from the first encoding module as input. The second encoding module converts the input differential data into a special signal representation form, namely, ternary values. A ternary value is a data representation method that includes three possible values, corresponding to negative, zero, and positive states or levels.

[0073] At the sending end, the encoder processes the raw intermediate result data of the large model, such as feature map data, embedding vectors, or key-value cache data from a certain layer of the neural network. The first encoding module inside the encoder receives this raw intermediate result data. The first encoding module performs encoding transformation on the input intermediate result data. By analyzing the difference between the current intermediate result data and a reference data (which is the intermediate result data of the previous transmission cycle or time step), a compressed differential data that characterizes the data change is generated. This differential data aims to reduce the amount of data that needs to be transmitted and take advantage of the sparsity (i.e., a large number of zero values ​​or small changes) in the intermediate result data.

[0074] After generating the differential data, the first encoding module outputs it to the second encoding module inside the encoder. Upon receiving the differential data, the second encoding module parses its structure, identifies the valid change information it contains, and maps this information into ternary values. These ternary values ​​are represented using three discrete states; for example, at the physical transmission level, they could correspond to negative voltage level, zero voltage level, and positive voltage level. After completing the conversion, the second encoding module outputs a sequence of ternary value signals representing the changes in the original intermediate results.

[0075] The ternary value signal sequence is then fed into a transmission channel. This channel can be a physical wire, printed circuit board trace, or interconnect structure between integrated chips. During transmission, the ternary value signals are modulated into corresponding voltage level signals for transmission. For example, a negative ternary value corresponds to a lower negative voltage, a zero ternary value corresponds to zero voltage or an intermediate reference voltage, and a positive ternary value corresponds to a higher positive voltage. This three-level transmission method helps maintain signal quality during high-speed signal transmission.

[0076] At the receiving end, what is received is a voltage level signal transmitted through the transmission channel. The receiving end is a system component located at the data transmission destination and is connected to the sending end through the transmission channel.

[0077] The receiving end includes an addition module, which is configured to receive the ternary value and perform a one-bit accumulation operation based on the ternary value to output a multiplication result, which is an intermediate result. The main function of the addition module is to receive the ternary value signal transmitted from the transmission channel and perform an accumulation calculation operation based on the received ternary value signal. The characteristic of this accumulation calculation operation is that each operation processes only one bit or one basic unit of data.

[0078] After performing the accumulation operation, the addition module produces a calculation result, namely the multiplication result. It is important to note that the multiplication result is not the product of two independent operands. In this embodiment, the multiplication result is intermediate result data processed by the receiving end. It represents the form recovered or reconstructed from the original intermediate result data after encoding at the sending end, transmission, decoding at the receiving end, and calculation. Its content is equivalent to or corresponds to the original intermediate result data, but it is ready for subsequent operations such as matrix multiplication.

[0079] The signal processing circuitry inside the receiver samples, identifies, and recovers the level signal, converting it back into a discrete ternary value signal sequence. The recovered ternary value signal sequence is then fed into the addition module. The addition module processes the input ternary values ​​with low latency and low hardware overhead, receiving the ternary value signal stream and performing a one-bit accumulation operation based on each received ternary value.

[0080] Specifically, when the addition module receives a non-zero ternary value (positive or negative), it retrieves the corresponding weight value from locally stored or pre-loaded data based on the sign of the ternary value. For example, it retrieves a portion of the weight matrix required for subsequent multiplication with the current intermediate result and adds this weight value to the internal state according to its sign (positive ternary value corresponds to positive weight, negative ternary value corresponds to negative weight). When a zero ternary value is received, no addition operation is performed, and the accumulated state remains unchanged. This bit-by-bit accumulation process is performed step-by-step.

[0081] After performing a round of accumulation operation on all bits of the input ternary signal sequence, the addition module finally outputs a calculation result, namely the multiplication result.

[0082] For example, the sending end needs to transmit an intermediate result vector [3,0,-2]. The first encoding module calculates the difference based on the previous vector (e.g., [1,1,-1]) to obtain the change, such as [2,-1,-1], and further processes it to generate differential data. The second encoding module converts this differential data into a ternary value sequence, such as [+1,-1,-1]. This ternary value sequence reaches the receiving end through the transmission channel (mapped to levels such as +0.5V, -0.5V, -0.5V).

[0083] The receiving end recovers the ternary value sequence [+1,-1,-1]. The addition module receives this sequence, assuming the weight vector is [W1,W2,W3]. For +1, add +W1; for the first -1, add -W2; for the second -1, add -W3. The final sum (W1-W2-W3) is output as the multiplication result. This result is numerically equivalent to the difference between the original vector [3,0,-2] multiplied by the weight vector [W1,W2,W3] (3W1+0W2-2W3) and (1W1+1W2-1W3) (i.e., the dot product of the previous vector). This process effectively transmits and recovers the information of the intermediate result [3,0,-2] and completes the intermediate step of calculating its dot product with the weights. The sum output by the addition module, combined with the previous result, can reconstruct the complete dot product of the current intermediate result and the weights.

[0084] The transmission system for intermediate results of large models provided in this embodiment converts the intermediate results into differential data and further into ternary values ​​for transmission through an encoder at the sending end. At the receiving end, an addition module is used to perform an accumulation operation on the ternary values ​​and output the multiplication result. This solves the problems of high logic latency and high hardware resource consumption caused by the use of 8B / 10B encoding scheme in the prior art.

[0085] In some embodiments, the first encoding module is Delta encoding, a data compression technique that transmits the amount of change (increment) in the data rather than the complete current value. In this embodiment, the first encoding module is configured to implement Delta encoding functionality, specifically for processing intermediate results from large models.

[0086] The first encoding module converts the intermediate result into a sequence of sequential elements; performs a subtraction operation on the code values ​​of adjacent positions in the sequence of sequential elements to generate a first difference sequence; wherein the first element of the sequence of sequential elements is subtracted from a preset base value; each element of the first difference sequence is converted into binary data of a preset bit width; the binary data is concatenated into differential data, wherein the differential data is a concatenated serial bit stream.

[0087] The intermediate results are Delta-encoded input data sources, which are data blocks such as feature maps, embedding vectors, or key-value caches generated during large model inference or training and need to be transmitted between chips.

[0088] The sequential element sequence is a preprocessed form of the intermediate input results by the first encoding module. The intermediate results have a multidimensional structure, such as an H×W×C feature map. The sequential element sequence is a one-dimensional data stream that expands and arranges these multidimensional data according to a predetermined order rule, such as row-major, column-major, or channel-major order, where the position index of each element in the sequence corresponds to its original multidimensional coordinates.

[0089] For example, the module scans line by line according to a preset scanning order, such as from left to right or from top to bottom, and reads each element value of the multidimensional intermediate result in sequence, and stores them in a one-dimensional buffer or register chain in order to form a linear sequence. The sequence length is equal to the total number of elements in the intermediate result.

[0090] When the sending end needs to transmit a new intermediate result data block, the first encoding module starts, converting it into a sequential element sequence in row-major order. It calculates the difference of the first element and subsequent adjacent differences based on a preset threshold, generating a first difference sequence. Each element of the first difference sequence is then converted into an 8-bit binary format. The conversion rule follows the signed integer two's complement representation: positive value 4 is 00000100, positive value 3 is 00000011, and all elements are uniformly 8-bit fixed-length data, padded with zeros if necessary in the higher bits. The conversion process is executed in parallel by a hardware encoder, completing all element processing in a single cycle.

[0091] Binary data is concatenated into a serial bit stream. The binary codes are concatenated in their original element order; the first element (3) has the binary code 00000011, the second element (4) has the binary code 00000100, and the output is a 32-bit continuous bit stream: 0000001100000100. This bit stream is directly connected to the physical transmission channel via a high-speed SerDes interface.

[0092] For example, inputting [1,3,5,6,9] will output [1,2,2,1,3], which is then converted to binary, containing only 0s and 1s. In large model intermediate results (such as image features), adjacent elements are strongly correlated, meaning their values ​​are close. The first difference sequence generated by differencing adjacent positions is smaller and more concentrated than the original values ​​(a large number of values ​​close to 0), allowing more elements to pass through a preset threshold (marked as 0), significantly reducing the amount of data in the non-zero difference sequence.

[0093] In some embodiments, the second encoding module is Booth-Radix2 encoding, which uses the sign bit and numerical characteristics of the multiplier to simplify the generation of the partial product. In this embodiment, the second encoding module is configured to implement Booth encoding function to convert differential data into ternary value signals suitable for transmission and calculation, and incorporate DC balance control.

[0094] like Figure 3 As shown, the second encoding module converts the differential data into ternary values, including the following steps:

[0095] S310: Parse the differential data to divide the differential data into binary data segments according to a preset bit width;

[0096] S320: Output the binary sequence of the binary data segments according to the segmentation order;

[0097] S330: Convert the binary sequence into a ternary value.

[0098] The second encoding module, in converting the difference data into ternary values, first performs a parsing operation on the difference data. The difference data refers to the discrepancies between intermediate results generated during the large model calculation; these discrepancies are stored in binary form. The parsing operation involves dividing the continuous difference data stream into independent binary data segments according to a preset bit width.

[0099] The preset bit width is a fixed length value pre-set by the system to determine the number of bits contained in each data segment. For example, when the preset bit width is set to eight bits, the differential data is divided into multiple eight-bit binary data segments. The segmentation operation is performed sequentially from the beginning of the differential data to ensure the continuity of the data segments.

[0100] After segmentation, the second encoding module outputs these binary data segments in the segmentation order, forming a binary sequence. This binary sequence is the basic data carrier for subsequent conversion into ternary values.

[0101] Specifically, the second encoding module receives the differential data generated by the large model computation. The differential data consists of floating-point gradient differences passed between model layers, stored in binary form in the system buffer. The second encoding module initiates a parsing procedure, reading the preset bit-width parameter. This preset bit-width parameter is stored in the system configuration register, typically eight bits. The differential data stream is scanned sequentially, extracting the first eight-bit data segment from the beginning of the stream, and then extracting the next adjacent eight-bit data segment. During extraction, the sequential continuity of the data segments is maintained, ensuring that their relative positions in the original data stream remain unchanged. All extracted data segments are stored in the output buffer in the extraction order, forming a complete binary sequence. This binary sequence is then passed as an intermediate conversion result to the next processing stage.

[0102] Specifically, such as Figure 4 As shown, in some embodiments, the second encoding module performs the conversion of the binary sequence into ternary values, including the following steps:

[0103] S410: Subtract the previous bit value from the current bit value to calculate the difference between adjacent bits;

[0104] S420: Output a ternary value sequence based on the difference.

[0105] The ternary value sequence includes a first ternary value, a second ternary value, and a third ternary value; wherein, if the difference is greater than 0, the first ternary value is output; if the difference is less than 0, the second ternary value is output; if the difference is equal to 0, the third ternary value is output; and the number of the first ternary value and the second ternary value are equal.

[0106] The output ternary signal is a signal stream that has been Booth-coded and balanced, including three states: -1 (negative level), 0 (zero level), and +1 (positive level). The number of negative levels is equal to the number of positive levels. Dynamic adjustment ensures that the total number of -1 and +1 in the signal stream tends to be equal, eliminating the net DC component.

[0107] After obtaining the binary sequence, the second encoding module performs an adjacent bit difference calculation. The adjacent bit difference refers to the numerical difference between the value of each bit position in the binary sequence and the value of the previous bit position. The calculation process starts from the beginning of the binary sequence and proceeds sequentially. For each bit position, the value of the current bit is subtracted from the value of the previous bit to obtain the corresponding difference result.

[0108] Based on the positive or negative characteristic of the difference result, the second encoding module outputs the corresponding ternary value: when the difference is greater than zero, it outputs the first ternary value; when the difference is less than zero, it outputs the second ternary value; and when the difference is equal to zero, it outputs the third ternary value. The ternary value sequence is composed of these output values ​​in sequence. The first ternary value represents a positive level signal, the second ternary value represents a negative level signal, and the third ternary value represents a zero level signal. Mathematical relationships ensure that the total number of first and second ternary values ​​remains equal.

[0109] After acquiring the binary sequence, the second encoding module initiates the ternary value conversion program. A virtual preamble bit with a value of zero is added at the beginning of the sequence as a reference point for difference calculation. The conversion process begins from the actual start position of the sequence, subtracting the virtual preamble bit value from the first bit value to obtain the first difference. Based on the difference result, the ternary value is output: if the difference is positive, the first ternary value is output; if negative, the second ternary value is output; and if zero, the third ternary value is output. This process continues, subtracting the current bit value from the next bit value to obtain a new difference and outputting the corresponding ternary value. This process proceeds sequentially until all bits have been processed. Due to the boundary condition processing at the beginning and end of the sequence, and the mathematical difference properties, the number of generated first and second ternary values ​​is automatically kept equal. The output ternary value sequence is stored in the transmission buffer for transmission.

[0110] For example, assume the binary sequence is six bits: 1, 0, 1, 1, 0, 0. Add a virtual leading bit 0. Subtract the leading 0 from bit 1: 1 - 0 = 1 > 0, which outputs the first ternary value. Subtract bit 1 from bit 0: 0 - 1 = -1 < 0, which outputs the second ternary value. Subtract bit 0 from bit 1: 1 - 0 = 1 > 0, which outputs the first ternary value. Subtract bit 1 from bit 1: 1 - 1 = 0, which outputs the third ternary value. Subtract bit 1 from bit 0: 0 - 1 = -1 < 0, which outputs the second ternary value. Subtract bit 0 from bit 0: 0 - 0 = 0, which outputs the third ternary value. The final ternary value sequence is: [+1, -1, +1, 0, -1, 0]. There are two first ternary values ​​and two second ternary values, the number is equal.

[0111] This embodiment eliminates the signal baseline drift problem caused by the imbalance of positive and negative pulses in traditional ternary encoding, improving the stability of data transmission. Simultaneously, because a large number of zero-level signals are generated during differential conversion, the amount of effective data to be transmitted is effectively reduced, lowering bandwidth consumption. The conversion process requires only a single data scan, eliminating the need for additional balancing correction operations, significantly improving the system's real-time processing capabilities.

[0112] After generating the ternary value signal, it is sent to the addition module at the receiving end. In some embodiments, the ternary value signal consists of a series of discrete values, each value taking one of three states: negative level, zero level, or positive level. These levels represent different voltage or current levels at the physical transmission layer, and correspond to negative ternary values, zero ternary values, or positive ternary values ​​at the logic layer, respectively. This signal carries information about the intermediate result changes encoded at the transmitting end.

[0113] A negative level is a state representing a negative ternary value in a ternary signal. In a signal sequence, whenever a negative level appears, it indicates that a significant negative change has occurred in the original data corresponding to that position. A negative level triggers the receiver to perform a negative accumulation operation.

[0114] A positive level is the state representing a positive ternary value in a ternary signal. In a signal sequence, whenever a positive level appears, it indicates that a significant positive change has occurred in the original data corresponding to that position. A positive level triggers the receiver to perform a positive accumulation operation.

[0115] The multiplication result includes a first multiplication result and a second multiplication result; when the ternary value signal is at a negative level, a negative weight value is obtained, and a one-bit accumulation operation is performed between the negative weight value and the negative level to output the first multiplication result; when the ternary value signal is at a positive level, a positive weight value is obtained, and a one-bit accumulation operation is performed between the positive weight value and the positive level to output the second multiplication result.

[0116] When the ternary signal is at a negative level, the negative weight value is obtained. The negative weight value is a set of data values ​​inside the addition module. The negative weight value is related to the positive weight value, and it is obtained by taking the negative sign of the corresponding positive weight value. For example, if the positive weight value of a certain position is W, its negative weight value is -W.

[0117] After obtaining the negative weight value, the first multiplication result is generated. The first multiplication result is the sum accumulated by performing an accumulation operation using the negative weight value when all negative levels are triggered.

[0118] When the ternary signal is at a positive level, the positive weight value is acquired. The positive weight value is a set of data values ​​stored or accessed internally by the addition module at the receiving end. The weight value is part of the coefficient matrix required for subsequent matrix multiplication or linear transformation with the intermediate result of the current processing. The positive weight value is used when a positive level is received. The weight value can be pre-loaded into the local register or cache of the addition module, or it can be acquired in real time according to the calculation requirements.

[0119] After obtaining the positive weight value, the result of the second multiplication operation is generated. The result of the second multiplication operation is the sum accumulated by performing the accumulation operation using the positive weight value when all positive levels are triggered.

[0120] Specifically, at the receiving end, the ternary signal stream after signal reconstruction and recovery is sent to the addition module. The addition module includes two independent accumulator registers, which are used to accumulate calculations related to the negative level and calculations related to the positive level, respectively. These are the negative accumulator register and the positive accumulator register, and the initial values ​​of these two registers are set to zero.

[0121] The addition module sequentially reads each level state from the input ternary signal stream. The processing is performed level by level.

[0122] When the addition module reads a negative level, it indicates a significant negative change in the original data at that position. For a negative level, the addition module retrieves the corresponding negative weight value from the locally stored weight data based on the current processing position index (or sequence order). As mentioned earlier, the negative weight value is equal to the negative of the positive weight value at that position. The addition module performs a one-bit accumulation operation, adding the retrieved negative weight value to the internal negative accumulation register.

[0123] When the addition module reads a positive level, it indicates a significant positive change in the original data at that position. For a positive level, the addition module retrieves the corresponding positive weight value from the locally stored weight data based on the current processing position index (or sequence order). The addition module then performs a one-bit accumulation operation, adding the retrieved positive weight value to the internal positive accumulation register.

[0124] When the addition module reads that the current level is zero, it indicates that the original data corresponding to that position has changed little or no. For zero level, the addition module does not perform any accumulation operation, and the values ​​of the negative accumulation register and the positive accumulation register remain unchanged, and the next input level state is processed.

[0125] The addition module continues this process until all level states in the input ternary signal stream have been processed. Each level state triggers a corresponding operation or no operation based on its type. After processing the entire ternary signal stream, the addition module outputs its accumulated result.

[0126] Traditional receivers use lookup table-based decoders to restore encoded data, such as 8B / 10B, to the original binary intermediate result, and then multiply and accumulate it with weights. This process involves many steps and a long path. This solution eliminates the independent decoding step. The addition module receives the encoded ternary value signal stream, shortening the processing path from receiving the encoded signal to outputting a valid calculation result. This reduces logic latency and simplifies hardware implementation, perfectly meeting the needs of high-speed computation of large models.

[0127] In some embodiments, the transmitting end and the receiving end are connected through a transmission channel. The transmission channel is a physical medium connecting the transmitting end and the receiving end, used to carry and transmit electrical signals. It can be a metal interconnect inside an integrated circuit, a trace on a chip packaging substrate, a transmission line on a printed circuit board, or a high-speed serial link connecting different computing units. It can provide a propagation path for the encoded signal from the transmitting end to the receiving end.

[0128] The transmission channel includes an input terminal and an output terminal. The input terminal is connected to the second encoding module and receives a ternary value signal that has been balanced and controlled from the output of the second encoding module. The input terminal includes a driving circuit to ensure that the signal can be effectively injected into the transmission channel.

[0129] The transmission channel is configured to: receive a ternary value signal sent by the second encoding module; map the ternary value signal into a voltage level signal; and transmit the voltage level signal to the output terminal.

[0130] Mapping the ternary value signal to a voltage level signal is a level conversion operation performed by the transmission channel or its interface circuit. This converts the input logic ternary value (negative, zero, positive) into an analog or digital level signal with a specific voltage amplitude that is actually transmitted on the transmission channel.

[0131] In one implementation, a negative ternary value is mapped to a lower negative voltage, such as -0.5V, a zero ternary value is mapped to a reference ground voltage or an intermediate voltage, such as 0V, and a positive ternary value is mapped to a higher positive voltage, such as +0.5V.

[0132] A voltage level signal is the physical signal form that actually propagates on the transmission channel. It is a voltage waveform that changes over time, and its instantaneous amplitude corresponds to the mapped logic ternary value state. For example, a -0.5V waveform represents a negative ternary value, a 0V waveform represents a zero ternary value, and a +0.5V waveform represents a positive ternary value. This signal carries the original data information.

[0133] The transmission channel receives a ternary value logic signal stream from the second encoding module at its input. The ternary value logic signal stream consists of a series of negative ternary values, zero ternary values, or positive ternary value states arranged in time sequence. The transmission channel maps these logic states into voltage level signals suitable for transmission over a physical medium.

[0134] The mapping operation is performed by the input interface circuit, which converts the input ternary logic state into a corresponding voltage amplitude in real time according to a pre-defined mapping rule. For example, when the input is a negative ternary value, the interface circuit generates and outputs a set negative voltage level, such as -0.5 volts. When the input is a zero ternary value, the interface circuit generates and outputs a set zero reference voltage level, which is 0 volts (ground level). When the input is a positive ternary value, the interface circuit generates and outputs a set positive voltage level, such as +0.5 volts. This three-level mapping method is a specific implementation of PAM3 modulation. The mapping process is performed in real time; each input ternary logic state is converted into a constant voltage level that lasts for a specific time (one symbol period).

[0135] After level mapping is completed, the generated voltage level signal is injected into the physical medium of the transmission channel. At this point, the signal propagates along the channel as an analog voltage wave. The physical characteristics of the transmission channel, such as the resistance of the conductor material, the capacitance between the channel and the reference plane, the inductive coupling between channels, and possible radiation losses, can affect the propagating voltage signal, potentially causing signal amplitude attenuation, slower waveform rise and fall times, or the introduction of noise and crosstalk.

[0136] The voltage level signal propagates from the input to the output along the transmission channel, and the propagation speed is determined by the dielectric constant of the channel. During propagation, the signal waveform may undergo the aforementioned changes, but the voltage level should remain identifiable relative to the three states of negative, zero, and positive time.

[0137] After propagation through the channel, the voltage level signal reaches the output of the transmission channel. The output includes a receiving circuit, such as a comparator or sampling circuit with a threshold, to detect which range (negative, zero, positive) the arriving voltage level belongs to.

[0138] To ensure the correctness of computational semantics, in some embodiments, the receiving end further includes a signal reconstruction module, and the output end is connected to the signal reconstruction module to process the received voltage level signal and convert the voltage level signal into a data form suitable for the addition operation module to perform accumulation calculation.

[0139] The signal reconstruction module is configured to: receive a voltage level signal sent by the output terminal through an input interface; reconstruct the voltage level signal based on the data position to output reconstructed data; and transmit the reconstructed data to the addition module so that the addition module receives the reconstructed data and performs a one-bit accumulation operation based on the reconstructed data to output the multiplication result.

[0140] The voltage level signals transmitted through the transmission channel are discrete sequence of level values ​​arriving in chronological order. However, these level values ​​have positional attributes in the original intermediate result data, such as corresponding to a row and column index in a feature map, or an element number in a vector. The signal reconstruction module needs to reorganize the chronologically received sequence of level values ​​into a data format that matches the original intermediate result data structure and is arranged in positional order, based on the system's predefined data structure or the positional information it carries.

[0141] The output reconstructed data is the result of signal reconstruction module processing. It is a structured data block, organized in the same way as the original intermediate result from the transmitting end, such as a two-dimensional matrix or a one-dimensional vector. Each element in the reconstructed data stores the corresponding ternary value state received and identified. This state is derived from the received voltage level. The reconstructed data restores the state information of the original intermediate result after encoding and transmission at the transmitting end.

[0142] At the receiving end, the voltage level signal is sent from the output of the transmission channel to the signal reconstruction module. The signal reconstruction module identifies and samples the input voltage level signal. For example, the signal reconstruction module includes a level detection circuit, such as a comparator array, which continuously samples the input voltage waveform.

[0143] For each sampling moment, the detection circuit compares the voltage value with a preset reference voltage threshold to determine the logic state of the voltage level at that moment. If the voltage is below the negative threshold (e.g., -0.25V), it is identified as a negative level; if the voltage is between the negative and positive thresholds (e.g., -0.25V to +0.25V), it is identified as a zero level; and if the voltage is above the positive threshold (e.g., +0.25V), it is identified as a positive level. Through this process, the continuous voltage waveform is converted back into a discrete logic state sequence, i.e., a ternary value sequence (negative, zero, positive). This sequence corresponds in time to the ternary value signal stream output by the transmitting end.

[0144] After obtaining the ternary value logic state sequence arranged in chronological order, the sequence is then restored to a structured data block arranged in the original data position order according to the sequence of arrival times. For example, if the intermediate result of the current transmission is a feature map with a width of W and a height of H, the module can set a buffer that matches the target data structure, such as an H-row, W-column register array or memory block.

[0145] The signal reconstruction module fills each logical state value in the received ternary value sequence into the corresponding element position in the buffer according to a predetermined position mapping rule. In one implementation, this is a row-major order, where the first value in the sequence corresponds to position (0, 0), the second value to position (0, 1), ..., the Wth value to position (0, W-1), the W+1th value to position (1, 0), and so on, until the entire H×W buffer is filled. The filling process is sequential. The corresponding row and column indices are calculated based on the number of currently received values, and the values ​​are written to the corresponding positions in the buffer. If the transmission protocol includes position identification information, such as the position index in the packet header, the module fills the buffer based on this information.

[0146] When the signal reconstruction module processes all the ternary value states of the current data block, the reconstruction operation is complete. At this point, the data stored in the buffer is the reconstructed data, which is a complete data structure, such as a feature map matrix, where each element stores the ternary value state (negative, zero, or positive) recovered from the voltage level signal and correctly placed.

[0147] After receiving the reconstructed data block from the signal reconstruction module, the addition module begins its accumulation calculation task. The physical voltage signal is restored to a logical ternary state sequence via a level detection circuit. Based on the predicted or protocol-carried location information, the time-arriving state sequence is reorganized and filled into the corresponding memory locations according to the original data structure, such as row-major or column-major order, generating reconstructed data. This allows the addition module to access the ternary value state of each location sequentially according to the correct spatial or logical order, and accurately obtain the corresponding weight value (positive or negative) based on the state and location index to perform the accumulation operation.

[0148] This scheme does not rely on a large lookup table for fixed mapping in traditional 8B / 10B encoding. The encoding process at the transmitting end (differential encoding and ternary value conversion) and the processing process at the receiving end (accumulation based on ternary values) can both be implemented using relatively simple combinational logic circuits, such as adders, comparators, and selectors. There is no need to build and access a huge mapping table, which significantly reduces the number of logic stages required for encoding and decoding operations.

[0149] Secondly, the receiver uses an addition module instead of a decoder. The bit-by-bit accumulation operation performed by this unit is essentially an arithmetic operation, which is suitable for implementation with efficient digital circuits, such as a carry-preserving adder tree, whose logical path is much shorter than that of multi-level lookup table lookup and decoding.

[0150] Finally, the ternary value transmission itself incorporates the encoding process, which helps maintain the DC balance of the transmission channel and avoids the additional encoding overhead and latency required for balancing in traditional schemes. Therefore, this scheme reduces the logic depth and circuit complexity of the encoding and decoding operations at the hardware implementation level, thereby reducing transmission latency and saving logic unit and chip area resources.

[0151] like Figure 5 As shown in the figure, 5a is a schematic diagram of the channel response during the 8b / 10b encoding process, and 5b is a schematic diagram of the channel response provided in this application. The signal-to-noise ratio (SNR) of 8b / 10b (32 parallel transmission channels) is 2.625, and the SNR of the transmission channel (40 parallel transmission channels) provided in this application is 4.2. These are the results of actual simulation in Cadence Virtuoso, representing a 60% improvement in SNR.

[0152] Furthermore, in 5a, the unequalized signal exhibits significant attenuation, and the signal improvement after equalization is limited. In 5b, the original impulse response remains stable at 0.6V, and the main lobe amplitude of the signal after equalization reaches 0.8V. The higher impulse response amplitude indicates that the channel attenuation of the proposed solution is lower and the signal fidelity is better.

[0153] Figure 6 In the diagram, 6a is a static eye diagram of the 8b / 10b encoding process, and 6b is a static eye diagram provided in this application. Figure 7 In the diagram, 7a is a dynamic eye diagram of the 8b / 10b encoding process, and 7b is a dynamic eye diagram provided in this application. Both are results analyzed in MATLAB, showing enhanced noise immunity and temporal stability.

[0154] Based on the aforementioned system for transmitting intermediate results of large models, some embodiments of this application provide a method for transmitting intermediate results of large models, including:

[0155] The intermediate results are obtained through the first encoding module, and the intermediate results are encoded to generate differential data;

[0156] The differential data is converted into ternary values ​​by the second encoding module;

[0157] The addition module receives the ternary value and performs a one-bit accumulation operation based on the ternary value to output the multiplication result, which is an intermediate result.

[0158] In some embodiments, obtaining intermediate results through the first encoding module and encoding the intermediate results to generate differential data includes:

[0159] Convert the intermediate results into a sequential sequence of elements;

[0160] Subtraction is performed on the code values ​​of adjacent positions in the sequential element sequence to generate a first difference sequence; wherein the first element of the sequential element sequence is subtracted from a preset benchmark value;

[0161] Convert each element of the first difference sequence into binary data of a preset bit width;

[0162] The binary data is concatenated into differential data, which is a concatenated serial bit stream.

[0163] In some embodiments, converting the differential data into ternary values ​​via the second encoding module includes:

[0164] The differential data is parsed to divide it into binary data segments according to a preset bit width;

[0165] The binary data segments are output as binary sequences according to the segmentation order;

[0166] The binary sequence is converted into a ternary value.

[0167] The embodiments of this method can be found in the embodiments of the above transmission system, and will not be repeated here.

[0168] Similar parts between the embodiments provided in this application can be referred to mutually. The specific implementation methods provided above are only a few examples under the overall concept of this application and do not constitute a limitation on the scope of protection of this application. For those skilled in the art, any other implementation methods extended from the solution of this application without creative effort shall fall within the scope of protection of this application.

Claims

1. A transmission system for intermediate results of large models, characterized in that, include: The transmitting end includes an encoder, which includes a first encoding module and a second encoding module. The first encoding module is configured to receive intermediate results and encode the intermediate results to generate differential data. The second encoding module is configured to convert the differential data into ternary values. The receiving end includes an addition module, which is configured to receive the ternary value and perform a one-bit accumulation operation based on the ternary value to output a multiplication result, wherein the multiplication result is an intermediate result.

2. The transmission system for intermediate results of large models according to claim 1, characterized in that, The first encoding module is configured to encode the intermediate results to generate differential data, specifically as follows: Convert the intermediate results into a sequential sequence of elements; Subtraction is performed on the code values ​​of adjacent positions in the sequential element sequence to generate a first difference sequence; wherein the first element of the sequential element sequence is subtracted from a preset benchmark value; Convert each element of the first difference sequence into binary data of a preset bit width; The binary data is concatenated into differential data, which is a concatenated serial bit stream.

3. The transmission system for intermediate results of large models according to claim 1, characterized in that, The second encoding module is configured to convert the differential data into ternary values ​​as follows: The differential data is parsed to divide it into binary data segments according to a preset bit width; The binary data segments are output as binary sequences according to the segmentation order; The binary sequence is converted into a ternary value.

4. The transmission system for intermediate results of large models according to claim 3, characterized in that, The second encoding module is configured to convert the binary sequence into ternary values ​​as follows: Subtract the previous bit value from the current bit value to calculate the difference between adjacent bits; Based on the difference, a ternary value sequence is output, which includes a first ternary value, a second ternary value, and a third ternary value. Specifically, if the difference is greater than 0, the first ternary value is output; if the difference is less than 0, the second ternary value is output; if the difference is equal to 0, the third ternary value is output; and the number of the first ternary value and the second ternary value are equal.

5. The transmission system for intermediate results of large models according to claim 3, characterized in that, The ternary values ​​include negative level, zero level, and positive level; the multiplication result includes the first multiplication result and the second multiplication result; The addition module receives the ternary value and performs a one-bit accumulation operation based on the ternary value to output the multiplication result. Specifically, it is configured as follows: When the three-element value is at a negative level, a negative weight value is obtained, and a one-bit accumulation operation is performed by combining the negative weight value with the negative level to output the result of the first multiplication operation. When the three-element value is at a positive level, a positive weight value is obtained, and a one-bit accumulation operation is performed by combining the positive weight value with the positive level to output the result of the second multiplication operation.

6. The transmission system for intermediate results of large models according to claim 1, characterized in that, The system also includes a transmission channel, which includes an input end and an output end, and the input end is connected to the second encoding module. The transmission channel is configured as follows: Receive the ternary value sent by the second encoding module; The ternary values ​​are mapped to voltage level signals, and the voltage level signals are transmitted to the output terminal.

7. The transmission system for intermediate results of large models according to claim 6, characterized in that, The receiving end further includes a signal reconstruction module, and the output end is connected to the signal reconstruction module; The signal reconstruction module is configured as follows: Receive the voltage level signal sent by the output terminal; Based on the data location, the voltage level signal is reassembled to output reassembled data; The recombined data is transmitted to the addition module, so that the addition module receives the recombined data and performs a one-bit accumulation operation based on the recombined data to output the multiplication result.

8. A method for transmitting intermediate results of a large model, characterized in that, include: The intermediate results are obtained through the first encoding module, and the intermediate results are encoded to generate differential data; The differential data is converted into ternary values ​​by the second encoding module; The addition module receives the ternary value and performs a one-bit accumulation operation based on the ternary value to output the multiplication result, which is an intermediate result.

9. The method for transmitting intermediate results of a large model according to claim 8, characterized in that, The step of obtaining intermediate results through the first encoding module and encoding the intermediate results to generate differential data includes: Convert the intermediate results into a sequential sequence of elements; Subtraction is performed on the code values ​​of adjacent positions in the sequential element sequence to generate a first difference sequence; wherein the first element of the sequential element sequence is subtracted from a preset benchmark value; Convert each element of the first difference sequence into binary data of a preset bit width; The binary data is concatenated into differential data, which is a concatenated serial bit stream.

10. The method for transmitting intermediate results of a large model according to claim 8, characterized in that, The step of converting the differential data into ternary values ​​through the second encoding module includes: The differential data is parsed to divide it into binary data segments according to a preset bit width; The binary data segments are output as binary sequences according to the segmentation order; The binary sequence is converted into a ternary value.

Images