Compression and transmission integrated system and method for intermediate layer result of large model
By combining differential conversion module, bit-width compression module and encoding module, the compression and transmission process of intermediate layer results of large models is optimized, solving the problem of low compression efficiency of intermediate layer results of large models, and realizing efficient data compression and resource saving.
Patent Information
- Application Number
- CN202511122419.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-08-12
- Publication Date
- 2025-11-25
AI Technical Summary
Existing intermediate layer result compression methods are mainly designed for traditional neural networks, which are difficult to adapt to the complex parameter structure of large models, resulting in low compression efficiency. Furthermore, multi-level processing methods lead to hardware resource redundancy and increased latency.
An integrated processing module consisting of a differential conversion module, a bit-width compression module, and an encoding module is adopted. Through differential calculation, bit-width compression, and zero-value run-length encoding, the compression and transmission process of the intermediate layer results of the large model is optimized, including differential conversion to differential data, splitting data rows according to the effective bit width, and using the zero-value run-length encoding method to reduce the number of 0 values.
It achieves efficient data compression, reduces the bandwidth requirements, resource consumption, and latency of large models, and improves hardware resource utilization and transmission efficiency.
Smart Images
Figure CN121012518A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of data compression technology, and in particular to an integrated system and method for compressing and transmitting intermediate layer results of large models. Background Technology
[0002] Large models generate a large number of intermediate layer results during operation. These data are not only massive in scale, but also need to be frequently transmitted between different computing nodes or devices. For example, in distributed training scenarios, each node needs to exchange intermediate layer results to collaboratively complete model updates; in edge computing and cloud collaborative inference, edge devices need to upload intermediate layer results to the cloud for further processing.
[0003] To address the challenges of compressing and transmitting intermediate layer results, quantization techniques are currently employed to reduce the data volume by decreasing the number of bits used in the intermediate layer result representation. Other methods utilize encoding algorithms for lossy or lossless data compression. For transmission, common techniques include employing efficient transmission protocols, such as optimized versions of TCP / IP, and leveraging network coding techniques to improve the reliability and efficiency of data transmission. Simultaneously, some techniques treat compression encoding and transmission encoding as two independent steps, compressing the intermediate layer results first and then performing transmission encoding on the compressed data.
[0004] However, current compression methods for intermediate layer results are mainly designed for traditional neural networks such as convolutional neural networks. Large models have deeper network layers, more complex parameter structures, and larger data scales. Compression methods for intermediate layer results of neural networks are difficult to adapt to the characteristics of intermediate layer results of large models and cannot achieve efficient compression. On the other hand, separating compression encoding and transmission encoding into multiple processing stages requires configuring corresponding hardware resources to support compression and transmission encoding operations, resulting in hardware resource redundancy. Moreover, during multi-stage processing, data needs to be converted and transferred multiple times between different modules, resulting in additional latency during compression and transmission, which seriously affects the overall operating efficiency of large models. Summary of the Invention
[0005] This application provides an integrated system and method for compressing and transmitting intermediate layer results of large models, in order to solve the technical problem that current compression methods for intermediate layer results are mainly aimed at traditional neural networks such as convolutional neural networks. Due to the more complex parameter structure of large models, compression methods for intermediate layer results of neural networks are difficult to adapt to the characteristics of intermediate layer results of large models, resulting in low compression efficiency.
[0006] The first aspect of this application provides an integrated system for compressed transmission of intermediate layer results of large models, comprising:
[0007] Differential conversion module, bit width compression module, encoding module;
[0008] The differential conversion module is configured as follows:
[0009] Obtain intermediate computational result data from the decoder; the intermediate computational result data is generated from the intermediate layer result data input to the large model; the intermediate computational result data is generated from the same neural network layer in the decoder;
[0010] The intermediate calculation results are converted into difference data using a differential calculation method.
[0011] The bit-width compression module is configured as follows:
[0012] The matrix corresponding to the difference data is divided into several data rows;
[0013] The data in the data row is retained according to the effective bit width of the data row to obtain intermediate compressed data;
[0014] The encoding module is configured as follows:
[0015] By using zero-value run-length encoding, the number of effective bits in the intermediate compressed data is compressed and the number of zero values is reduced to obtain the target compressed data.
[0016] In some embodiments, the differential conversion module is further configured to:
[0017] Obtain the first and second intermediate calculation result data from the decoder; the second intermediate calculation result data is generated from the first intermediate calculation result data.
[0018] The difference data is obtained by performing a difference calculation on the first intermediate calculation result data and the second intermediate calculation result data.
[0019] In some embodiments, the bit-width compression module is further configured to:
[0020] Determine the number of rows to split;
[0021] The matrix corresponding to the difference data is divided into several data rows according to the set number of rows.
[0022] Obtain the effective bit width of the data row;
[0023] The data in the data row within the effective bit width is retained to obtain intermediate compressed data.
[0024] In some embodiments, the bit-width compression module is further configured to:
[0025] Based on the effective bit width, the reserved data of the data row is determined; the reserved data is the data within the effective bit width from right to left in the data row;
[0026] The data in the data row, excluding the reserved data, is removed to obtain the intermediate compressed data.
[0027] In some embodiments, the encoding module is further configured to:
[0028] Determine the number of valid digits;
[0029] Obtain the data to be compressed from the intermediate compressed data that is less than or equal to the set number of valid bits;
[0030] Using zero-value run-length encoding, the number of significant bits and the number of zero values in the data to be compressed are reduced according to the equalization characteristic rule to obtain the first data to be packaged.
[0031] Based on the first data to be packaged and the second data to be packaged, the target compressed data is obtained; the second data to be packaged is the data in the intermediate compressed data that is greater than the set number of valid bits.
[0032] In some embodiments, the balancing characteristic rule includes:
[0033] The difference between the number of 0 values and the number of 1 values in the data to be compressed is kept within a set range.
[0034] In some embodiments, the equilibrium characteristic rule further includes:
[0035] The number of consecutive 0 values and the number of consecutive 1 values in the data to be compressed are kept less than or equal to a preset value.
[0036] In some embodiments, the system further includes:
[0037] A packaging module, which is communicatively connected to the encoding module, is configured to:
[0038] Determine the transmission protocol;
[0039] The target compressed data is packaged according to the rules conforming to the transmission protocol to obtain a compressed data packet.
[0040] The second aspect of this application provides an integrated method for compression and transmission of intermediate layer results of a large model, applied to the integrated system for compression and transmission of intermediate layer results of a large model as described in any one of the first aspects above, comprising:
[0041] Obtain intermediate computational result data from the decoder; the intermediate computational result data is generated from the intermediate layer result data input to the large model; the intermediate computational result data is generated from the same neural network layer in the decoder;
[0042] The intermediate calculation results are converted into difference data using a differential calculation method.
[0043] The matrix corresponding to the difference data is divided into several data rows;
[0044] The data in the data row is retained according to the effective bit width of the data row to obtain intermediate compressed data;
[0045] By using zero-value run-length encoding, the effective bits of the intermediate compressed data are compressed and the number of zero values is reduced to obtain the target compressed data.
[0046] In some embodiments, the step of retaining the data of the data row according to the effective bit width of the data row to obtain intermediate compressed data includes:
[0047] Based on the effective bit width, the reserved data of the data row is determined; the reserved data is the data within the effective bit width from right to left in the data row;
[0048] The data in the data row, excluding the reserved data, is removed to obtain the intermediate compressed data.
[0049] This application provides an integrated system and method for compressing and transmitting intermediate layer results of a large model. The system includes a differential conversion module, a bit-width compression module, and an encoding module. The differential conversion module is configured to: acquire intermediate result data from the decoder; the intermediate result data is generated from the intermediate layer result data input to the large model; the intermediate result data is generated from the same neural network layer in the decoder; and convert the intermediate result data into difference data using a differential calculation method. The bit-width compression module is configured to: divide the matrix corresponding to the difference data into several data rows; retain the data of the data rows according to their effective bit width to obtain intermediate compressed data. The encoding module is configured to: compress the effective number of bits and reduce the number of zero values in the intermediate compressed data using zero-value run-length encoding to obtain target compressed data. The integrated system and method for compressing and transmitting intermediate layer results of a large model provided in this application can reduce the bandwidth requirements, resource consumption, and latency of the large model. Attached Figure Description
[0050] To more clearly illustrate the technical solution of this application, the drawings used in the embodiments will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0051] Figure 1 This is a schematic diagram of the integrated compression and transmission system for intermediate layer results of a large model in this application.
[0052] Figure 2 This is a flowchart illustrating the runtime of the integrated compression and transmission system for intermediate layer results of a large model in this application.
[0053] Explanation of reference numerals in the attached figures:
[0054] 1-Differential conversion module; 2-Bit width compression module; 3-Encoding module; 4-Packaging module. Detailed Implementation
[0055] To enable those skilled in the art to better understand the technical solutions in this application, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of this application.
[0056] In some technologies, compression methods for intermediate layer results are mainly designed for traditional neural networks such as convolutional neural networks. Since large models have more complex parameter structures, compression methods for intermediate layer results of neural networks are difficult to adapt to the characteristics of intermediate layer results in large models, resulting in low compression efficiency. To solve this technical problem, this application provides an integrated system and method for compressing and transmitting intermediate layer results of large models. The integrated system and method for compressing and transmitting intermediate layer results of large models are described below:
[0057] like Figure 1 The diagram shown is a structural schematic of the integrated compression and transmission system for intermediate layer results of a large model in this application.
[0058] The first aspect of this application provides an integrated system for compressed transmission of intermediate layer results of large models, comprising:
[0059] The communication connection includes a differential conversion module 1, a bit-width compression module 2, and an encoding module 3.
[0060] The differential conversion module 1 is configured as follows:
[0061] Obtain intermediate computational result data from the decoder; the intermediate computational result data is generated from the intermediate layer result data input to the large model; the intermediate computational result data is generated from the same neural network layer in the decoder.
[0062] By using differential calculation, the intermediate calculation result data is converted into difference data; differential calculation is performed on the intermediate calculation result data of the same neural network layer in adjacent calculation loops of the large model, and the intermediate layer result data is converted into difference data (Delta Block) by taking advantage of the strong correlation between the intermediate results of adjacent calculation loops, which significantly reduces data redundancy.
[0063] The differential conversion module 1 is further configured as follows:
[0064] Obtain the first and second intermediate calculation result data of the decoder; the second intermediate calculation result data is generated from the first intermediate calculation result data; perform difference calculation on the first and second intermediate calculation result data to obtain the difference data.
[0065] The bit-width compression module 2 is configured as follows:
[0066] The matrix corresponding to the difference data is divided into several data rows, such as... Figure 2 As shown; retain the data of the data row according to the effective bit width of the data row to obtain intermediate compressed data; divide the matrix corresponding to the difference data after differentiation into multiple data rows. Calculate the effective bit width of the data in each row, that is, the highest effective number of bits actually occupied by the data in that row. Finally, retain the data within the effective bit width and discard the high-order redundant zero values to obtain intermediate compressed data.
[0067] The bit-width compression module 2 is further configured as follows:
[0068] Determine the number of rows to split.
[0069] Specifically, each large model has a dimension parameter (dmodel) to represent the dimension of the large model's embedding vector. For example, if the dimension parameter of the large model is 768, then each block in the intermediate layer results contains 768 numbers. The width and height of the block can be changed, but its size, i.e., the product of its width and height, equals 768, such as a block using 48 rows * 16 columns. The number of numbers per row is subject to both efficiency and performance constraints. When there are fewer numbers per row (2, 4, 8), the single-pass floating bit width compression is more effective, but it can only process a small amount of data at a time, resulting in low overall efficiency. When there are more numbers per row (32, 48, and above), more data is processed at a time, resulting in higher overall efficiency, but because more numbers may result in larger maximum values and wider bit widths (larger max_bit), the floating bit width compression effect deteriorates. After multiple comparative experiments, it was determined that 16 numbers per row resulted in the best floating bit width compression effect, thus the number of split rows was set to 768 ÷ 16 = 48 (lanes). Different large models have different dimensional parameters and data characteristics. Specific experimental comparisons are needed to determine the length and width of each sub-unit, i.e., to set the number of division rows.
[0070] The matrix corresponding to the difference data is divided into several data rows according to the set number of rows; the effective bit width of the data row is obtained; the data of the data row within the effective bit width is retained to obtain intermediate compressed data.
[0071] The bit-width compression module 2 is further configured as follows:
[0072] Based on the effective bit width, the reserved data of the data row is determined; the reserved data is the data within the effective bit width from right to left in the data row; the data in the data row other than the reserved data is removed to obtain intermediate compressed data.
[0073] Specifically, taking 8-bit quantization as an example, each row of data consists of 16 8-bit numbers. After floating bit width compression, if the maximum bit width of a data row is determined to be 6, then the highest 8-6=2 bits of each data row are discarded. A maximum bit width of 6 means that in these 16 8-bit numbers, the 7th and 8th bits (high-order bits) from right to left of each integer are 0. These are redundant and do not carry information, so they need to be discarded. The data row only needs a maximum of 6 bits to be fully represented, retaining only the 6 lower-order valid bits. In other words, the discarded high-order bits that are not within the valid bit width correspond to a 0 value and can be directly discarded.
[0074] For example, before the removal of data rows 00100001, 00111111, 00110001, and 00000001, each data row is 8 bits. However, the effective bit width of the above data rows is 6 bits. In each data row, the 7th and 8th bits from right to left are all 0, so they can be removed. The data rows after removal are 100001, 111111, 110001, and 000001, each data row is 6 bits.
[0075] The encoding module 3 is configured as follows:
[0076] Zero-value run-length encoding (0-RLE) is used to compress the effective number of bits in the intermediate compressed data and reduce the number of 0 values to obtain the target compressed data. For data rows whose bit width is significantly reduced (e.g., ≤4 bits) after compression by the compression module 2, zero-value run-length encoding (0-RLE) is applied to further compress the effective number of bits and reduce the number of 0 values in the data, matching the requirement for balanced 0 and 1 values during transmission.
[0077] The encoding module 3 is further configured as follows:
[0078] Determine the set effective number of bits; obtain the data to be compressed in the intermediate compressed data that is less than or equal to the set effective number of bits; use the zero-value run-length encoding method to compress the effective number of bits of the data to be compressed and reduce the number of 0 values according to the equalization characteristic rule to obtain the first data to be packaged.
[0079] The equilibrium characteristic rules include:
[0080] The difference between the number of 0 values and the number of 1 values in the data to be compressed is kept within a set range; and the number of consecutive 0 values and the number of consecutive 1 values in the data to be compressed are kept less than or equal to a preset value.
[0081] In this embodiment, the equalization characteristic rule is characterized as DC equalization during transmission, keeping the number of 0 and 1 values in the data to be compressed basically consistent, and the number of consecutive 0 or 1 values does not exceed a certain number of bits (preset value, for example, 8b10b encoding requires no more than 5 bits).
[0082] Based on the first data to be packaged and the second data to be packaged, the target compressed data is obtained; the second data to be packaged is the data in the intermediate compressed data that is greater than the set number of valid bits.
[0083] For example, when the number of significant bits is greater than 4, it has been verified that the corresponding data stream transmission quality is already very good. Further application of zero-value run-length encoding cannot improve the compression ratio or transmission quality. Therefore, zero-value run-length encoding is no longer applied to data rows with more than 4 significant bits. It is worth noting that, similar to the steps described above for determining the number of split rows, the significant bit count of "4" is obtained through experimental verification and can be changed accordingly for different large-scale models.
[0084] For example, the goal of compression is to approximate the binary bitstream of the intermediate layer result data (the data to be compressed) as close as possible to the minimum length and number of bits. When this minimum length (compression limit) is reached, the number of 0s and 1s in the bitstream is equal. At this point, the information carried by each bit (information entropy) is 1, and it is impossible to reduce the number of bits without information loss through any compression method. On the high-speed serial transmission (SerDes) platform, due to interference caused by electromagnetic coupling between the signal and the receiver, the transmitted binary data needs to meet the requirements of low crosstalk and DC equalization; otherwise, it will result in low signal transmission quality and bit errors. To meet the transmission requirements, the data needs to avoid carrying long 0 and 1 values and periodically repeating segments as much as possible. The number of 0 and 1 values should be as equal as possible, and the distribution of 0 and 1 values should maintain a certain degree of randomness (close to white noise). Therefore, traditional transmission schemes need to perform scrambling and DC equalization coding (such as 8b10b coding) to reduce the "regularity" of the binary data.
[0085] The ultimate goal of compression is to ensure an equal number of 0 and 1 values. The highest requirements for transmission are no repeating segments, an equal number of 0 and 1 values, and a random distribution of 0 and 1 values. Compression and transmission are compatible in their requirement for an equal number of 0 and 1 values. Therefore, when designing an integrated optimization of compression and transmission, it is necessary not only to consider the compression effect but also to make the compression method more adaptable to other transmission requirements. Thus, the chosen technical approach is to construct a compression algorithm with a higher compression ratio than existing compression methods, while also taking into account the requirements of the transmission process. The compressed bitstream should not contain long 0 and 1 values, repeating segments, and the distribution of 0 and 1 values should be more random. Since the binary data stream of the intermediate results data of the large model itself has a certain degree of randomness, and the number of 0s is far greater than the number of 1s, some existing compression algorithms directly convert long sequences of 0 values into shorter sequences of 1 values. Therefore, the algorithm we constructed mainly considers improving the compression ratio: the first step is to convert the intermediate layer results data into difference data through difference, thereby reducing the overall amount of data (the difference between two large numbers may be very small, for example: 300-298=2). The second and third steps reduce the number of 0 values, reduce long sequences of 0 values, and ensure that long sequences of 1 values do not appear in the compressed data.
[0086] In summary, the essence of this application is a compression algorithm that can better adapt to transmission requirements. Its optimization measures for transmission adaptation are mainly reflected in the selection of each step of the compression algorithm in the differential conversion module 1, bit-width compression module 2, and encoding module 3. Experimental results show that applying the integrated compression and transmission method for large model intermediate layer results provided in this application can reduce the number of encoders and decoders required by the transmission module.
[0087] The system also includes:
[0088] Packaging module 4, which is communicatively connected to encoding module 3; packaging module 4 includes units such as buffers and packers; packaging module 4 is configured as follows:
[0089] The transmission protocol is determined; the target compressed data is packaged according to the rules conforming to the transmission protocol to obtain compressed data packets. The target compressed data is encapsulated into compressed data packets conforming to the transmission protocol using units such as buffers and packers.
[0090] This application provides an integrated system for compressing and transmitting intermediate layer results of large models, which has the following advantages:
[0091] 1. Improved compression efficiency: The compression algorithm used in the system proposed in this application is optimized for the characteristics of large model data, and the compression ratio is much higher than that of current compression methods.
[0092] 2. Save hardware resources: While compressing the intermediate layer result data, the transmission characteristics of the large model are considered, and the compression and transmission are integrated into one encoding, eliminating the need for a scrambling module in the large model and reducing the consumption of hardware resources.
[0093] 3. Improved hardware friendliness: The differential conversion module 1, bit width compression module 2, and encoding module 3 are all easy to implement in hardware, and their complexity and latency are controllable.
[0094] 4. Reduced transmission latency: The number of steps for processing data in a single transmission of a large model is reduced, resulting in a decrease in overall latency while ensuring transmission quality.
[0095] The second aspect of this application provides an integrated method for compression and transmission of intermediate layer results of a large model, applied to the integrated system for compression and transmission of intermediate layer results of a large model described in any of the above embodiments, comprising the following steps:
[0096] Obtain intermediate computational result data from the decoder; the intermediate computational result data is generated from the intermediate layer result data input to the large model; the intermediate computational result data is generated from the same neural network layer in the decoder;
[0097] The intermediate calculation results are converted into difference data using a differential calculation method.
[0098] The matrix corresponding to the difference data is divided into several data rows;
[0099] The data in the data row is retained according to the effective bit width of the data row to obtain intermediate compressed data;
[0100] By using zero-value run-length encoding, the effective bits of the intermediate compressed data are compressed and the number of zero values is reduced to obtain the target compressed data.
[0101] It is worth noting that the effects of the above method embodiments can be found in the effects of the above system embodiments, and will not be repeated here.
[0102] In this embodiment, the step of retaining the data of the data row according to the effective bit width of the data row to obtain intermediate compressed data includes the following sub-steps:
[0103] Based on the effective bit width, the reserved data of the data row is determined; the reserved data is the data within the effective bit width from right to left in the data row;
[0104] The data in the data row, excluding the reserved data, is removed to obtain the intermediate compressed data.
[0105] It is worth noting that the effects of the above method embodiments can be found in the effects of the above system embodiments, and will not be repeated here.
[0106] The above detailed embodiments further illustrate the purpose, technical solution, and beneficial effects of the embodiments of this application. It should be understood that the above are merely specific embodiments of the embodiments of this application and are not intended to limit the protection scope of the embodiments of this application. Any modifications, equivalent substitutions, improvements, etc., made on the basis of the technical solutions of the embodiments of this application should be included within the protection scope of the embodiments of this application.
Claims
1. A system for compressing and transmitting intermediate results of large models, characterized in that, include: Differential conversion module (1), bit width compression module (2), encoding module (3); The differential conversion module (1) is configured as follows: Obtain intermediate computational result data from the decoder; the intermediate computational result data is generated from the intermediate layer result data input to the large model; the intermediate computational result data is generated from the same neural network layer in the decoder; The intermediate calculation results are converted into difference data using a differential calculation method. The bit-width compression module (2) is configured as follows: The matrix corresponding to the difference data is divided into several data rows; The data in the data row is retained according to the effective bit width of the data row to obtain intermediate compressed data; The encoding module (3) is configured as follows: By using zero-value run-length encoding, the number of effective bits in the intermediate compressed data is compressed and the number of zero values is reduced to obtain the target compressed data.
2. The integrated compression and transmission system for intermediate layer results of a large model according to claim 1, characterized in that, The differential conversion module (1) is further configured as follows: Obtain the first and second intermediate calculation result data from the decoder; the second intermediate calculation result data is generated from the first intermediate calculation result data. The difference data is obtained by performing a difference calculation on the first intermediate calculation result data and the second intermediate calculation result data.
3. The integrated compression and transmission system for intermediate layer results of a large model according to claim 1, characterized in that, The bit-width compression module (2) is further configured as follows: Determine the number of rows to split; The matrix corresponding to the difference data is divided into several data rows according to the set number of rows. Obtain the effective bit width of the data row; The data in the data row within the effective bit width is retained to obtain intermediate compressed data.
4. The integrated compression and transmission system for intermediate layer results of a large model according to claim 3, characterized in that, The bit-width compression module (2) is further configured as follows: Based on the effective bit width, the reserved data of the data row is determined; the reserved data is the data within the effective bit width from right to left in the data row; The data in the data row, excluding the reserved data, is removed to obtain the intermediate compressed data.
5. The integrated compression and transmission system for intermediate layer results of a large model according to claim 1, characterized in that, The encoding module (3) is further configured as follows: Determine the number of valid digits; Obtain the data to be compressed from the intermediate compressed data that is less than or equal to the set number of valid bits; Using zero-value run-length encoding, the number of significant bits and the number of zero values in the data to be compressed are reduced according to the equalization characteristic rule to obtain the first data to be packaged. Based on the first and second data to be packaged, the target compressed data is obtained; The second data to be packaged is the data in the intermediate compressed data that is greater than the set number of valid bits.
6. The integrated compression and transmission system for intermediate layer results of a large model according to claim 5, characterized in that, The equilibrium characteristic rules include: The difference between the number of 0 values and the number of 1 values in the data to be compressed is kept within a set range.
7. The integrated compression and transmission system for intermediate layer results of a large model according to claim 5, characterized in that, The equilibrium characteristic rule also includes: The number of consecutive 0 values and the number of consecutive 1 values in the data to be compressed are kept less than or equal to a preset value.
8. The integrated compression and transmission system for intermediate layer results of a large model according to claim 1, characterized in that, The system also includes: Packaging module (4), which is communicatively connected to encoding module (3); the packaging module (4) is configured to: Determine the transmission protocol; The target compressed data is packaged according to the rules conforming to the transmission protocol to obtain a compressed data packet.
9. A method for integrated compression and transmission of intermediate layer results for large models, applied to the integrated compression and transmission system for intermediate layer results of large models as described in any one of claims 1 to 8, characterized in that, include: Obtain the intermediate calculation result data of the decoder; The intermediate calculation result data is generated from the intermediate layer result data input to the large model; The intermediate calculation result data is generated by the same neural network layer in the decoder; The intermediate calculation results are converted into difference data using a differential calculation method. The matrix corresponding to the difference data is divided into several data rows; The data in the data row is retained according to the effective bit width of the data row to obtain intermediate compressed data; By using zero-value run-length encoding, the effective bits of the intermediate compressed data are compressed and the number of zero values is reduced to obtain the target compressed data.
10. The method for integrated compression and transmission of intermediate layer results of a large model according to claim 9, characterized in that, The step of retaining the data of the data row according to the effective bit width of the data row to obtain intermediate compressed data includes: Based on the effective bit width, the reserved data of the data row is determined; the reserved data is the data within the effective bit width from right to left in the data row; The data in the data row, excluding the reserved data, is removed to obtain the intermediate compressed data.