Dynamic multi-modal packing method, apparatus and chip

Abstract

This application relates to the field of artificial intelligence chip data transmission technology, and provides a dynamic multimodal packaging method, apparatus, and chip, comprising: a data sending end receiving an input data stream, analyzing the semantic features of the input data stream in multiple dimensions in real time, and dynamically deciding on a target packaging strategy and a valid data count from multiple predefined packaging modes based on the analyzed multi-dimensional feature information; packaging the valid data in the input data stream according to the target packaging strategy and the valid data count to generate packaged data blocks, and appending a packaging information header to the packaged data blocks to obtain compressed data packets; and a data receiving end receiving the compressed data packets, selecting the corresponding unpacking mode according to the target packaging strategy identifier, and restoring the packaged data blocks to the original data stream. This application can perceive the semantic features of data in real time and dynamically select the optimal packaging strategy, significantly improving the data transmission bandwidth utilization and transmission efficiency.

Description

Technical Field

[0001] This application relates to the field of chip technology, and more specifically, to a dynamic multimodal packaging method, apparatus, and chip. Background Technology

[0002] In AI chips, especially accelerators for Large Language Model (LLM) inference, on-chip or inter-chip data transfer bandwidth is one of the core bottlenecks. Currently, the industry mainly employs two types of data compression or packing schemes: one is a packer based on data bit-width compression, which "squeezes" valid data from the input data bus to the lower bits based on the position of the valid data indicated by the input strb signal. This scheme only focuses on the "positional validity" of the data on the bus, without concern for the specific content and semantics of the data; the other is an assembler based on bit-width conversion, used for data adaptation between interfaces with different bit-widths. In addition, academia has also proposed several compression technologies for AI acceleration, such as entropy-aware buffer compression and tag-based precision detection schemes.

[0003] However, existing technologies suffer from significant "semantic blind spots," failing to perceive the semantic characteristics of data. For example, when consecutive zero values ​​(sparse patterns) appear in the data stream, traditional solutions still transmit these zero values, wasting bandwidth. When the actual effective bit width of the data is much smaller than the bus bit width (such as INT4 quantized data), traditional solutions still transmit it as a full bit width, resulting in low bandwidth utilization. Furthermore, existing solutions typically employ a single, fixed compression or packing strategy, unable to handle mixed sparse and quantized data patterns simultaneously, making it difficult to adapt to dynamically changing data characteristics during LLM inference. In addition, existing receivers can only unpack data according to a preset, fixed format, unable to determine the packing strategy used by the sender, preventing the system from adaptively adjusting the transmission method based on real-time data characteristics. Summary of the Invention

[0004] The purpose of this application is to provide a dynamic multimodal packaging method, apparatus, and chip to solve the above-mentioned technical problems.

[0005] In a first aspect, this application provides a dynamic multimodal packetization method, applied to intra-chip or inter-chip data transmission processes, the method comprising: The data sending end receives the input data stream, analyzes the semantic features of the input data stream in multiple dimensions in real time, and dynamically decides the target packaging strategy and effective data count from multiple predefined packaging modes based on the multi-dimensional feature information obtained from the analysis. The data sending end packages the valid data in the input data stream according to the target packaging strategy and the valid data count to generate a packaged data block, and appends a packaging information header to the packaged data block to obtain a compressed data packet. The packaging information header includes the identifier of the target packaging strategy and the valid data count. The data receiving end receives the compressed data packet, parses the packaging information header to obtain the target packaging strategy identifier and the valid data count, selects the corresponding unpacking mode according to the target packaging strategy identifier, and restores the packaged data block to the original data stream according to the valid data count.

[0006] In an optional implementation, the semantic features of the multiple dimensions include at least sparsity features and quantization bit width features; wherein, the sparsity features are obtained by detecting the length of consecutive zero values ​​in the input data stream using a consecutive zero value counter; and the quantization bit width features are obtained by detecting the length of the high-order extension bits of the input data using a high-order sign bit detector to determine the actual effective bit width of the data.

[0007] In an optional implementation, the step of dynamically determining the target packaging strategy from multiple predefined packaging modes includes: Based on the detected sparsity features and quantization bit width features, the packaging strategy corresponding to the current data block is determined through preset decision logic or a lightweight decision tree. Among them, the various predefined packaging modes include at least: Sparse packaging mode, quantitative packaging mode, hybrid packaging mode, and pass-through packaging mode; The hybrid packaging mode is a combined packaging mode that utilizes both sparsity and quantization characteristics.

[0008] In an optional implementation, when the target packing strategy is a sparse packing mode, packing the valid data in the input data stream includes: Filter out zero-value data, capture only non-zero data, and generate the position coordinates of each non-zero data in the original data stream, packing them into tuples of {coordinates, non-zero data}.

[0009] In an optional implementation, when the target packaging strategy is a quantized packaging mode, packaging the valid data in the input data stream includes: Based on the detected effective bit width, multiple low-precision data are continuously concatenated into a high-bit-width data packet for packaging.

[0010] In an optional implementation, when the target packaging strategy is a hybrid packaging mode, packaging the valid data in the input data stream includes: First, filter out zero-value data, then concatenate and pack the captured non-zero low-precision data according to the effective bit width, and pack them in the form of a tuple of {coordinate offset, concatenated data}. In the hybrid packaging mode, the coordinate offset does not correspond to the absolute periodic position in the original data stream, but rather to the relative sequence number of the non-zero data in the original data stream. The logic for generating the coordinate offset is as follows: when non-zero data is detected, its sequence number in the non-zero data sequence is generated based on the number of non-zero data currently captured. The sequence number and the concatenated data together form a tuple, so that the receiving end can determine the precise position of each non-zero data in the original data stream based on the sequence number and the effective data count.

[0011] In an optional implementation, the step of selecting the corresponding unpacking mode according to the target packaging strategy identifier and restoring the packaged data block to the original data stream according to the valid data count includes: The packaged data block is routed to the corresponding unpacking unit according to the target packaging strategy identifier; If it is the unpacking unit corresponding to the sparse packing mode, then read the {coordinates, non-zero data} tuple according to the effective data count, write the non-zero data back to the original position according to the coordinates, and fill the missing parts with zeros. If it is an unpacking unit corresponding to the quantization packaging mode, the spliced ​​data is split into the original number of low-precision data according to the effective data count, and output in order; If it is an unpacking unit corresponding to a mixed packaging mode, then read the {coordinate offset, concatenated data} tuple according to the effective data count, split the concatenated data into low-precision data, and write it back to the original position according to the coordinate offset; If it is an unpacking unit corresponding to the pass-through packaging mode, the compressed data is expanded according to the original bit width based on the effective data count.

[0012] In an optional implementation, the method further includes: The unpacked data is reordered to restore the original data stream order.

[0013] Secondly, this application provides a dynamic multimodal packaging device for performing the method described in any of the foregoing embodiments, the device comprising: The feature-aware front end is set at the data sending end to receive the input data stream, analyze the semantic features of the input data stream in multiple dimensions in real time, and make dynamic decisions and output the target packaging strategy and effective data count based on the multi-dimensional feature information obtained from the analysis. A multimodal packaging engine, located at the data sending end and connected to the feature perception front end, is used to package the valid data in the input data stream according to the target packaging strategy and the valid data count, generate packaged data blocks, and append a packaging information header containing the target packaging strategy identifier and the valid data count to the packaged data blocks to obtain compressed data packets; A configurable unpacker is set at the data receiving end and communicates with the multimodal packing engine. It is used to receive the compressed data packet, parse the packing information header to obtain the target packing strategy identifier and the valid data count, select the corresponding unpacking mode according to the target packing strategy identifier, and restore the packed data block to the original data stream according to the valid data count.

[0014] Thirdly, this application provides a chip that includes a dynamic multimodal packaging device as described in the foregoing embodiments, through which intra-chip or inter-chip data transmission is realized.

[0015] This application provides a dynamic multimodal packetization method, apparatus, and chip. It analyzes the sparsity, quantization bit width, and other multidimensional semantic features of the data stream in real time through a feature-aware front-end, and dynamically selects the optimal packetization strategy based on the analysis results. This solves the bandwidth waste problem caused by the inability of existing technologies to perceive data semantics. Compared to traditional bit-width compression schemes, this application can simultaneously utilize sparsity and low-precision quantization characteristics, effectively improving bandwidth in typical LLM scenarios. By attaching a simplified packetization header to the data packet header, the receiving end can intelligently unpack the packets, supporting dynamic strategy switching and reducing end-to-end transmission latency. Furthermore, this application reduces invalid flips on the physical channel, lowers dynamic power consumption, reduces energy consumption per unit data transmission, and has small hardware overhead, minimal area increase, and no timing degradation, demonstrating good practical value and promising prospects for widespread application. Attached Figure Description

[0016] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments of this application will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 This is a schematic flowchart of a dynamic multimodal packaging method according to an embodiment of this application; Figure 2 This is a schematic diagram of a dynamic multimodal packaging device according to an embodiment of this application; Figure 3 This is a schematic diagram of the feature-aware front-end module structure according to an embodiment of this application; Figure 4 This is a schematic diagram of the multimodal packaging engine architecture according to an embodiment of this application; Figure 5 This is a schematic diagram of the configurable unpacker data stream structure according to an embodiment of this application; Figure 6This is a performance comparison diagram of different schemes in the LLM Key tensor transfer scenario according to the embodiments of this application; Figure 7 This is a schematic diagram of a chip structure according to an embodiment of this application. Detailed Implementation

[0018] The technical solutions in the embodiments of this application will now be described with reference to the accompanying drawings.

[0019] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.

[0020] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to be limiting of this application.

[0021] This embodiment provides a dynamic multimodal packaging method. This method is applied to data transmission processes within or between chips, such as data movement within an AI accelerator or data communication between chips. Figure 1 As shown, the method specifically includes the following steps: In step S100, the data sending end receives the input data stream, analyzes the semantic features of the input data stream in multiple dimensions in real time, and dynamically decides the target packaging strategy and effective data count from multiple predefined packaging modes based on the multi-dimensional feature information obtained from the analysis.

[0022] Specifically, this step is the "brain" decision-making stage of the entire method. The input data stream typically originates from intermediate computation results during AI inference, such as the Key tensor or Value tensor in LLM inference. The so-called "multi-dimensional semantic features" refer to the statistical or structural characteristics inherent in the data stream that can guide the compression strategy. Unlike traditional solutions that only rely on strb signals to determine the validity of data positions, this embodiment delves into the data content to extract features such as sparsity and quantization bit width. For example, the data stream may contain a large number of consecutive zero values, or the actual effective bit width of the data may be much smaller than the bus bit width. The feature-aware front-end module will perform real-time detection and fusion analysis of these features. The so-called "dynamic decision-making" means that the system does not use a fixed compression algorithm, but rather adaptively selects the optimal processing mode based on the real-time characteristics of the current data block. For example, if the current data block is detected to have extremely high sparsity, the decision is to adopt a sparse packing mode; if the data is detected to be low-precision quantized data, the decision is to adopt a quantized packing mode. The decision result includes not only a strategy identifier but also a valid data count, used to inform subsequent modules of the actual amount of valid data that needs to be transmitted in the current data block.

[0023] In step S200, the data sending end packages the valid data in the input data stream according to the target packaging strategy and the valid data count to generate a packaged data block, and appends a packaging information header before the packaged data block to obtain a compressed data packet. The packaging information header includes the identifier of the target packaging strategy and the valid data count.

[0024] Specifically, this step is the execution phase of data compression and encapsulation. The multimodal packing engine routes the input data stream to the corresponding packer for processing based on the strategy ID determined in step S100. For example, if the strategy is sparse packing, the engine filters out zero values, retaining only non-zero data and their coordinates; if the strategy is quantized packing, the engine concatenates multiple low-precision data into a single high-bit-width data. This process "slims down" the data stream, eliminating redundant information. The generated packed data block is then appended with a minimal packing header. This header information is crucial; it acts as a "manual" for the data packet, recording the strategy ID used by the sender and the count of valid data. This allows the receiver to understand the sender's processing logic by parsing the header when it receives the data packet, thus enabling blind depackaging. The final compressed data packet is then sent to the receiver via the transmission bus.

[0025] In step S300, the data receiving end receives the compressed data packet, parses the packet header to obtain the target packetization strategy identifier and the valid data count, selects the corresponding unpacking mode according to the target packetization strategy identifier, and restores the packetized data block to the original data stream according to the valid data count.

[0026] Specifically, this step is the data recovery phase. The configurable unpacker first reads the header information of the compressed data packet and extracts the policy ID. Based on this identifier, the policy scheduler inside the unpacker distributes the data stream to the corresponding unpacking unit. For example, if the policy ID indicates sparse packing, the sparse unpacking unit is called to write non-zero data back to its original position according to the coordinate information and pad with zeros in the missing places; if it is quantized packing, the quantized unpacking unit is called to split and restore the concatenated data. In this way, the receiving end can intelligently adapt to the various packing strategies of the sending end without pre-determining a fixed transmission format, realizing flexible configuration of the transmission link. Finally, the recovered data stream is sent to subsequent computing units for further processing.

[0027] Through the coordinated efforts of the three stages described above, this embodiment achieves dynamic multimodal transmission based on data semantic features. Compared to traditional single-strategy packetization schemes, this application can dynamically adjust the strategy according to the real-time characteristics of the data, maximizing the utilization of transmission bandwidth and significantly reducing the transmission overhead of invalid data.

[0028] In some embodiments, the specific implementation of the semantic feature analysis process and strategy decision-making of the feature-aware front end can be achieved through the following scheme.

[0029] The feature-aware front end receives the input data stream and analyzes its semantic features across multiple dimensions in real time. These semantic features include at least sparsity features and quantization bit width features.

[0030] Specifically, the sparsity feature is obtained by detecting the length of consecutive zero values ​​in the input data stream using a consecutive zero-value counter. This consecutive zero-value counter can be a hardware counter circuit that works by performing a sliding window detection on the input data stream. When a non-zero value is detected, the counter resets; when a zero value is detected, the counter increments. This counter can calculate the maximum or average length of consecutive zero values ​​in the current data block in real time. For example, if the input data stream is [0,0, 0, A, 0, B], the consecutive zero-value counter will record a length of 3 consecutive zero values. This feature directly reflects the sparsity of the data stream, providing a quantitative basis for whether to enable sparse packing mode. If the sparsity is too high, transmitting a large number of zero values ​​will severely waste bandwidth, thus requiring a sparse packing strategy for filtering.

[0031] The quantization bit width feature is obtained by detecting the length of the high-order extension bits of the input data using a high-order sign bit detector to determine the actual effective bit width of the data. In AI inference scenarios, low-precision calculations (such as INT8 and INT4) are very common, but fixed bit widths (such as 32-bit) are often used on the transmission bus. In this case, low-precision data is sign-extended to fill the bus width. The role of the high-order sign bit detector is to identify these redundant sign extension bits. For signed numbers, if the data is positive, all high-order bits are 0; if the data is negative, all high-order bits are 1. The detector calculates the actual effective bit width of the data by comparing whether the high-order bits are consistent. For example, if an INT8 type data is transmitted on a 32-bit bus, its high 24 bits are the sign extension bits, and the high-order sign bit detector will identify that the effective bit width is only 8 bits. This feature reveals the compression potential of data in the bit width dimension. If the effective bit width is much smaller than the bus width, multiple low-precision data can be concatenated and transmitted through quantization packing mode, thereby improving bus utilization.

[0032] As one implementation, the feature-aware front-end can also include a repeating pattern detector. The repeating pattern detector uses a simple pattern matching register to detect whether repeating data patterns, such as specific padding or mask values, exist in the input data stream. If a large amount of repeating data is detected, the system can select a specific encoding strategy for deduplication and compression. This extended design further enriches the dimensions of feature awareness, enhances the system's adaptability to complex data features, and reflects the defensive expansion approach of the technical solution.

[0033] Next, the target packaging strategy can be dynamically determined from a variety of predefined packaging modes based on the multi-dimensional feature information obtained from the analysis.

[0034] Specifically, based on the detected sparsity and quantization bit width characteristics, a packaging strategy corresponding to the current data block is determined through pre-defined decision logic or a lightweight decision tree. The decision logic can be implemented using a hardware priority comparison circuit. The system pre-defined several packaging modes, including at least: sparse packaging mode, quantization packaging mode, hybrid packaging mode, and pass-through packaging mode. The hybrid packaging mode is a combination of packaging modes that utilizes both sparsity and quantization characteristics.

[0035] The decision-making process follows preset triggering conditions. For example, when the sparsity feature indicates that the length of consecutive zero values ​​exceeds a preset threshold (e.g., 4 cycles) and the quantization bit width feature indicates that the effective bit width is normal, the decision logic prioritizes triggering the sparse packing mode to eliminate bandwidth waste caused by zero values. When the quantization bit width feature indicates that the effective bit width is less than a preset threshold (e.g., less than or equal to 8 bits) and the sparsity feature indicates that the data is not sparse, the quantization packing mode is triggered to improve transmission density using bit width compression. When both the sparsity feature and the quantization bit width feature meet the conditions (i.e., the data is both sparse and low-precision), the hybrid packing mode is triggered to achieve dual compression gain. If none of the above features are significant, the process reverts to the pass-through packing mode, i.e., using the traditional bit width compression method to ensure data transmission compatibility and stability.

[0036] Through the collaborative work of feature detection and strategy decision-making described above, this embodiment achieves accurate perception and rapid response to data semantic features. This hardware-feature-based dynamic decision-making mechanism, compared to traditional single-strategy solutions, can adaptively adjust the transmission strategy according to real-time data characteristics, ensuring maximum utilization of transmission bandwidth under various data distributions. This significantly reduces the transmission overhead of invalid data, laying an efficient foundation for the subsequent packet transmission stage.

[0037] In some embodiments, when the target packaging strategy is sparse packaging mode, the specific processing logic inside the multimodal packaging engine can be implemented through the following scheme.

[0038] When the target packaging strategy determined by the feature perception front end is sparse packaging mode, the effective data in the input data stream is packaged, including: filtering zero-value data, capturing only non-zero data, and generating the position coordinates of each non-zero data in the original data stream, and packaging them in the form of tuples of {coordinates, non-zero data}.

[0039] Specifically, sparse packers are Figure 4 A key branch of the sparse packer engine has the core task of removing invalid zero values ​​from the data stream and transmitting only meaningful data payloads. In hardware implementation, this process is typically accomplished by a zero-value detector in conjunction with a counter. When the data stream enters the sparse packer, the zero-value detector checks periodically whether the current data is zero. If zero is detected, the data is skipped without being written to the output buffer, and a counter is triggered to record the number of skipped zero values ​​or to maintain the accumulation of the position counter. If a non-zero value is detected, it is immediately captured, and the current position counter value is locked as the coordinate.

[0040] Here, "coordinates" refers to the position of the non-zero data within the original data stream. It should be understood that this position can be an absolute offset relative to the start of the data block or a relative offset relative to the previous non-zero data, depending on the trade-offs in the hardware design. In the preferred absolute offset scheme of this embodiment, the position counter continuously increments with the input of the data stream. Whenever a non-zero data is captured, the current value of the counter is the coordinate of that data. This design allows the receiving end to directly write the data back to the corresponding storage address based on the coordinates, making the recovery logic simple and intuitive.

[0041] The generated {coordinates, non-zero data} tuples are then sent to the packetized output queue. This "zero-value filtering" mechanism significantly saves bandwidth that would otherwise require transmitting a large number of zero values. For example, suppose the input data stream is [0, A, 0, B], where A and B are non-zero data. In a traditional transmission scheme, this would require transmitting four data cycles of data. However, in the sparse packetization mode of this embodiment, zero values ​​are filtered out, generating only two tuples: {1, A} and {3, B}. This indicates that A is located at position 1 in the original stream, and B is located at position 3. After parsing these two tuples, the receiving end writes A at position 1 and B at position 3, padding the remaining positions with zeros to perfectly recover the original data stream. This process not only reduces the amount of data transmitted but also reduces invalid flips on the physical channel, thereby significantly reducing dynamic power consumption.

[0042] As one implementation, the sparse packer can also combine run-length encoding to further compress and describe continuous zero-value regions, or use a more compact bitmap index to replace explicit coordinate transmission, in order to further reduce metadata overhead. These variant schemes all fall within the protection scope of the "sparse packing mode" of this application, and are designed to maximize the transmission efficiency of sparse data.

[0043] In some embodiments, when the target packaging strategy is a quantitative packaging mode, the specific processing logic inside the multimodal packaging engine can be as follows.

[0044] When the target packaging strategy determined by the feature perception front end is the quantization packaging mode, the effective data in the input data stream is packaged, including: according to the detected effective bit width, multiple low-precision data are continuously concatenated into a high-bit-width data packet for packaging.

[0045] Specifically, a quantization packer is Figure 4Another core branch of the multimodal packetizing engine was designed to address the space waste problem when transmitting low-precision quantized data on high-bit-width buses. In AI inference scenarios, models are often quantized, for example, quantizing FP32 data into INT8 or even INT4. When this low-precision data is transmitted on a standard 32-bit or 64-bit bus, without special processing, the high-bit portion is usually filled with sign extension bits or directly set to zero, resulting in a significant waste of bus bandwidth. The quantization packetizer addresses this pain point by improving transmission efficiency through a "bit-width compression" mechanism.

[0046] The process begins with the extraction of the effective bit width. The high-order sign bit detector at the feature-aware front end has already identified the actual effective bit width of the data, for example, detecting that the effective bit width of the current data block is 8 bits. Upon receiving this information, the quantizer initiates the bit width extraction logic. For signed numbers, this means removing the high-order sign extension bits and retaining only the low-order effective data body. For example, an INT8 data value of +10 is represented as "000...001010" on a 32-bit bus, where the high 24 bits are redundant sign extension bits. The quantizer removes these high 24 bits through hardware logic (such as masking or shifting operations), extracting only the core payload "001010".

[0047] Subsequently, the quantizer performs data concatenation. This process is typically implemented using a barrel shifter or data concatenation circuit. The system extracts multiple low-precision data points, arranges them consecutively according to the bus width, and concatenates them into a complete high-width data packet. Taking a 32-bit transmission bus as an example, if the detected effective width is 8 bits, the quantizer can seamlessly concatenate four INT8 data points into a single 32-bit data packet. Assuming the input data stream consists of A, B, C, and D (all of type INT8), in traditional mode, this would require four clock cycles of bus bandwidth. However, after quantization and packing, these four data points are compactly arranged in the same 32-bit data packet, requiring only one clock cycle for transmission. This mechanism directly increases bus utilization by four times.

[0048] It should be understood that the above-described concatenation of INT8 data into 32 bits is merely a preferred embodiment of this application and not a limiting provision. In practical applications, the effective bit width may vary dynamically. For example, if the feature-aware front end detects that the effective bit width of the current data block is only 4 bits (such as INT4 quantized data), the quantization packetizer can concatenate 8 INT4 data points into a 32-bit data packet, or concatenate 16 INT4 data points into a 64-bit data packet. This flexibility allows this application to adaptively process quantized data of various precisions, maximizing the utilization of bandwidth potential.

[0049] Furthermore, the quantization-packaging mode is typically suitable for scenarios with low data sparsity but high quantization. Through this "consolidation" strategy, the system significantly reduces the number of transmission cycles and bus occupancy without losing any useful information, thereby creating more data transmission opportunities for other computing units and improving the overall system throughput.

[0050] In some embodiments, when the target packaging strategy is a hybrid packaging mode, the specific processing logic inside the multimodal packaging engine can be implemented through the following scheme.

[0051] When the target packing strategy is a hybrid packing mode, the valid data in the input data stream is packed, including: first filtering zero-value data, then concatenating and packing the captured non-zero low-precision data according to the valid bit width, and packing it in the form of a tuple of {coordinate offset, concatenated data}.

[0052] Specifically, a hybrid packer is Figure 4 The multimodal packing engine targets core processing units with both sparse and low-precision feature data. Its workflow employs a "dual compression" mechanism. First, a sparse filtering operation is performed, detecting and skipping zero-value data in the input data stream, capturing only valid non-zero data. This step, similar to the sparse packer, aims to eliminate invalid zero-value payloads, addressing bandwidth waste in the sparse dimension. Next, a quantization concatenation operation is performed, compressing the bit width of the captured non-zero data. Since the feature-aware front-end has detected the actual effective bit width of the data (e.g., INT8), the hybrid packer concatenates these non-zero low-precision data according to the bus bit width. For example, if the bus is 32-bit and the non-zero data is of type INT8, the hybrid packer will concatenate four consecutively captured non-zero INT8 data points into a single 32-bit data packet. This step inherits the advantages of the quantization packer, resolving space waste in the bit width dimension.

[0053] The key innovation of this embodiment lies in the special definition of "coordinate offset". In the hybrid packing mode, the coordinate offset does not correspond to the absolute periodic position in the original data stream, but rather to the relative sequence number of the non-zero data in the original data stream. Traditional sparse packing usually records the absolute address of the data in the original stream (such as the 1st or 5th position). However, in the hybrid packing mode, the data undergoes double compression. If absolute coordinates are still recorded for each non-zero data, and each non-zero data may be concatenated together, it will lead to excessive overhead of coordinate metadata, offsetting the compression benefits. Therefore, this embodiment adopts a relative sequence number encoding strategy. The logic for generating the coordinate offset is as follows: when non-zero data is detected, its sequence number in the non-zero data sequence is generated based on the number of non-zero data currently captured. The sequence number and the concatenated data together form a tuple, so that the receiving end can determine the precise position of each non-zero data in the original data stream based on the sequence number and the effective data count.

[0054] For example, suppose the input data stream is [0, A, 0, 0, B, C] (A, B, and C are all INT8 data). The mixing packer first filters out zero values ​​at positions 0, 2, and 3, capturing three non-zero data points: A, B, and C. If absolute coordinates are used, A would be recorded as position 1, B as position 4, and C as position 5. However, under the relative sequence numbering mechanism in this embodiment, A is marked as sequence number 0, B as sequence number 1, and C as sequence number 2. If the bus width allows for concatenating two INT8 data points, the packer may generate the following tuples: the first tuple is {sequence number 0, concatenated data (A, B)}, indicating that starting from the 0th non-zero data point, two consecutive non-zero data points are concatenated together; the second tuple is {sequence number 2, data C}. When the receiving end parses the data, based on sequence number 0 and the valid data count of 2, it can know that A is the 0th non-zero data point and B is the 1st non-zero data point. Combining this with the length information of the original stream or other context, the data positions can be accurately recovered. This design cleverly utilizes the continuous nature of non-zero data, significantly reducing the bit width overhead of coordinate information and achieving deep coupling and maximizing efficiency between sparse compression and quantization compression.

[0055] It should be understood that the above-mentioned number of concatenations (such as 4 INT8s) is only an example, and the actual number of concatenations depends on the ratio of the bus width to the effective data width. The hybrid packing mode, through this dual compression and special addressing mechanism, can achieve bandwidth gains far exceeding those of single sparse or quantized tensors when processing sparse quantized tensors (such as pruned and quantized key / value caches) commonly found in LLM inference.

[0056] In some embodiments, the specific process by which the data receiving end selects the corresponding unpacking mode based on the target packing strategy identifier and restores the packed data block to the original data stream based on the valid data count can be implemented through the following scheme.

[0057] The packaged data blocks are routed to the corresponding unpacking units based on the target packaging strategy identifier.

[0058] Specifically, the configurable unpacker integrates a policy scheduler. When compressed data packets arrive at the receiving end, the header parser first extracts the policy ID. Based on this ID value, the policy scheduler dynamically routes subsequent packaged data blocks to different hardware unpacking units using a multiplexer. This design enables the receiving end to intelligently adapt to the sending end's various packing policies, achieving blind unpacking without software intervention.

[0059] Each unpacking unit performs specific unpacking operations.

[0060] For unpacking units corresponding to sparse packing patterns, the {coordinates, non-zero data} tuple is read based on the valid data count. The non-zero data is then written back to its original position according to the coordinates, with zeros padded in any gaps. Specifically, the sparse unpacker internally maintains a write address generator. For each tuple read, the unpacker extracts its coordinate information as the write address and writes the non-zero data to the corresponding position in the reordering buffer. For address ranges not covered by the tuple, the unpacker automatically fills them with zeros. For example, if the tuples {1, A} and {3, B} are received, the unpacker will write A to address 1, B to address 3, and automatically set addresses 0 and 2 to zero. This process perfectly reverses the zero-value filtering operation at the sending end, restoring the sparse structure of the data.

[0061] If the unpacking unit corresponds to the quantized packing mode, it splits the concatenated data into the original number of low-precision data segments based on the valid data count and outputs them sequentially. Specifically, the quantized unpacker performs the reverse process of data concatenation. Based on the valid data count in the header information and the pre-agreed valid bit width, it uses shift operations to break down a high-bit-width data packet into multiple low-precision data segments. For example, if a 32-bit concatenated data packet is received with an 8-bit valid bit width, the unpacker splits it into four INT8 data segments and outputs them sequentially. This process restores the precision dimension of the data, ensuring that subsequent calculation units can process it correctly.

[0062] For unpacking units corresponding to mixed packing modes, the tuple {coordinate offset, concatenated data} is read based on the valid data count. The concatenated data is then split into low-precision data and written back to its original location according to the coordinate offset. This process combines the features of sparse unpacking and quantitative unpacking. The unpacker first splits the concatenated data, restoring the non-zero low-precision data sequence. Then, based on the coordinate offset information in the tuple, these non-zero data are written back to their corresponding storage locations. Here, the coordinate offset is the relative sequence number of the non-zero data. The unpacker needs to perform simple address calculations based on the valid data count to determine the precise location of each data item in the original stream.

[0063] If the unpacking unit corresponds to the pass-through packing mode, the compressed data is expanded according to the original bit width based on the valid data count. This mode is the system's fallback strategy; the unpacker performs traditional decompression operations to ensure correct transmission even when data characteristics are not significant.

[0064] The unpacked data is reordered to restore the original data stream order.

[0065] Specifically, since sparse unpacking and hybrid unpacking perform random write-back operations based on coordinates or coordinate offsets, this may lead to a discrepancy between the order in which data is written to the reorder buffer and the physical order of the original data stream, resulting in an "out-of-order" phenomenon. For example, the unpacker might process data at coordinate 5 first, followed by data at coordinate 2. To address this issue, this embodiment incorporates a reorder buffer at the unpacker's output. This buffer is organized by address index, and the unpacking unit writes data to the corresponding position in the buffer based on the calculated address. After all data has been written, the system reads the data sequentially according to address order, thus outputting a strictly ordered original data stream. This mechanism ensures that downstream computing units (such as the computing array of an AI accelerator) receive a time-correctly ordered data stream, avoiding computational errors caused by out-of-order data.

[0066] It should be understood that the depth of the reordering buffer can be configured according to the data block size in the actual application scenario. In a preferred embodiment, the reordering buffer is implemented using dual-port RAM, supporting simultaneous read and write operations to further reduce unpacking latency and improve system throughput.

[0067] In some embodiments, such as Figure 2 As shown, this embodiment provides a dynamic multimodal packaging device for performing the methods described in the foregoing embodiments. This device can be implemented in hardware circuitry or software. It is integrated into the data transmission link and specifically includes three core modules: a feature-aware front-end 201, a multimodal packaging engine 202, and a configurable unpacker 203.

[0068] The feature-aware front-end 201 is set at the data sending end to receive the input data stream, analyze the semantic features of the input data stream in multiple dimensions in real time, and make dynamic decisions and output the target packaging strategy and effective data count based on the multi-dimensional feature information obtained from the analysis.

[0069] Specifically, the feature-sensing front-end 201 is the "control center" of the entire device, and its hardware implementation corresponds to... Figure 3The structure is shown. In terms of circuit connection, the input terminal of the feature sensing front-end 201 is coupled to the data input bus to receive the raw data stream to be transmitted in parallel or serial mode. Internally, it integrates feature detection circuits such as a continuous zero-value counter, a high-order sign bit detector, and a repeating pattern detector, as well as a policy decision-maker. The feature detection circuit performs real-time sliding window detection on the input data stream, generating sparsity feature signals and quantization bit width feature signals. The policy decision-maker receives these feature signals and, through internal comparators and state machine logic, outputs the policy ID signal corresponding to the target packaging policy and a valid data count signal. These signals serve as control signals and are transmitted to the subsequent multimodal packaging engine via the control bus. It should be understood that the feature sensing front-end is not limited to detecting sparsity and quantization bit width; it can also integrate other feature analysis units such as repeating pattern detectors according to design requirements to expand the applicable scenarios of the device.

[0070] The multimodal packaging engine 202 is set at the data sending end and connected to the feature perception front end 201. It is used to package the valid data in the input data stream according to the target packaging strategy and the valid data count, generate packaged data blocks, and attach a packaging information header containing the target packaging strategy identifier and the valid data count before the packaged data blocks to obtain compressed data packets.

[0071] Specifically, such as Figure 4 As shown, the multimodal packing engine 202 is the main execution unit for data compression processing. In terms of hardware architecture, its input terminals are coupled to the data input bus (for acquiring the raw data stream) and the control signal output terminal of the feature-aware front-end 201 (for acquiring the policy ID and valid data count). The multimodal packing engine 202 internally includes a policy distributor and multiple parallel packing units, such as a sparse packer, a quantization packer, a hybrid packer, and a pass-through packer. The policy distributor routes the input data stream to the corresponding packing unit through a multiplexer based on the policy ID signal. For example, when the policy ID is "00", the data stream is sent to the sparse packer, where the zero-value filtering logic and coordinate generation circuit work together to output a tuple of {coordinates, non-zero data}. When the policy ID is "01", the data stream is sent to the quantization packer, where the barrel shifter concatenates multiple low-precision data into high-bit-width data. After packaging, the engine's internal header generation circuit encapsulates the policy ID and valid data count into a packaging header, which is appended to the packaged data blocks to form the final compressed data packet, which is then sent to the transmission channel through the output interface. This modular hardware design enables the device to complete policy switching within one clock cycle, achieving low-latency response to dynamic data characteristics.

[0072] The configurable unpacker 203 is set at the data receiving end and communicates with the multimodal packing engine. It is used to receive compressed data packets, parse the packing information header to obtain the target packing policy identifier and the valid data count, select the corresponding unpacking mode according to the target packing policy identifier, and restore the packed data blocks to the original data stream according to the valid data count.

[0073] Specifically, such as Figure 5 As shown, the configurable unpacker 203 is located at the data receiving end, and its hardware structure corresponds to the multimodal packing engine 202 at the sending end. Its input end receives compressed data packets through the transmission channel. Internally, it integrates a header parser, a policy scheduler, and multiple unpacking units (such as sparse unpackers and quantitative unpackers). The header parser first extracts the policy ID and valid data count from the data packet header. The policy scheduler activates the corresponding unpacking unit based on the policy ID. For example, if the policy ID indicates a mixed packing mode, the mixed unpacking unit is activated, and its internal splitting and reordering logic works together to split the concatenated data and write it back to its original position based on coordinate offset. The output end of the configurable unpacker is coupled to the reordering buffer, and finally outputs the restored original data stream to the subsequent calculation unit. Through this design, the receiving end can intelligently adapt to the various packing strategies of the sending end without software intervention, realizing "plug-and-play" adaptive compression of the transmission link.

[0074] Through the collaborative work of the aforementioned feature-aware front-end 201, multimodal packing engine 202, and configurable unpacker 203, this embodiment constructs a complete hardware-based data transmission optimization device. This device integrates the dynamic decision-making logic, multimodal packing method, and intelligent unpacking mechanism from the previous embodiments into the hardware circuitry. This not only ensures the efficiency and real-time performance of the method execution but also, through standardized interface design, facilitates integration into existing AI chips or inter-chip interconnect systems, significantly improving the system's data transmission bandwidth utilization.

[0075] This embodiment provides a chip that includes a dynamic multimodal packaging device as described in the foregoing embodiments, through which intra-chip or inter-chip data transmission is realized.

[0076] Specifically, the chip can be an accelerator chip specifically designed for AI inference, or a system-on-a-chip (SoC) integrating an AI processing unit. The dynamic multimodal packaging device, a key IP core in the chip's internal data path, is integrated between the chip's data bus interface unit and computing unit. In physical implementation, the feature-aware front-end and multimodal packaging engine are typically deployed on the bus master side of the data sending end, such as at the output interface of the tensor computation core, for real-time compression of the computation results to be transmitted; while the configurable unpacker is deployed on the bus slave side of the data receiving end, such as at the control interface of the on-chip cache, for restoring the compressed data and storing it in memory. This asymmetric hardware layout design minimizes modifications to existing bus protocols and reduces integration complexity.

[0077] In on-chip data transmission applications, this device is primarily used for data flow within the chip's internal network (Network on Chip). For example, during LLM inference, the Key and Value tensors of the attention layer need to be frequently moved between different storage banks or computation units. By integrating this dynamic multimodal packing device into the NoC routing node, the chip can adaptively select sparse packing or hybrid packing modes for transmission based on the real-time sparsity and quantization characteristics of the tensor data. This not only significantly reduces the congestion probability of the NoC network but also reduces memory bandwidth pressure, allowing limited on-chip storage resources to accommodate longer context sequences, thereby improving the throughput of model inference.

[0078] In applications involving inter-chip data transfer, this device is used in die-to-die interconnect interfaces within multi-chip package architectures. With the explosive growth in model parameters, a single chip often struggles to accommodate a complete model, necessitating the connection of multiple computing chips via high-speed interconnect interfaces for collaborative operation. In this case, inter-chip interconnect bandwidth becomes a major performance bottleneck. The chip in this embodiment, by deploying this device at the die-to-die interconnect interface, can perform highly efficient dynamic compression of data before cross-chip transmission. For example, for activation value data transmitted across chips, the device can automatically identify low-precision features and trigger a quantization and packing mode, compressing data that originally required four transmission cycles into one cycle. This mechanism, without increasing the number of physical pins or power consumption, significantly increases the effective bandwidth of the inter-chip interconnect, providing an efficient communication foundation for multi-chip collaborative inference of large models.

[0079] It should be understood that the above application scenarios are merely preferred embodiments of this application and not restrictive provisions. This chip can also be applied to other scenarios requiring high-bandwidth data transmission, such as pixel data flow in image processing chips or sampling data transmission in digital signal processors. Wherever data transmission processes involve sparse or quantized characteristics, integrating the dynamic multimodal packetization device described in this application can bring significant bandwidth savings and power reduction effects.

[0080] like Figure 6 As shown, to more intuitively demonstrate the technical effect of the dynamic multimodal packaging method described in this application in a specific application scenario, this embodiment takes a typical Key tensor transmission scenario in the Large Language Model (LLM) inference process as an example for detailed explanation. In this scenario, the data transmission bus width is set to 32 bits, and the input data stream exhibits significant sparsity and low precision characteristics. Specifically, about 60% of the elements in the data are zero values, and the non-zero data are low-precision data quantized by INT8.

[0081] In this application scenario, the specific execution process of the method in this embodiment is as follows: The feature-aware front end receives the input Key tensor data stream and performs real-time feature analysis.

[0082] The continuous zero-value counter inside the feature-aware front end detects a large number of consecutive zero values ​​in the data stream, resulting in a sparsity of 60%, exceeding the preset sparsity threshold. Simultaneously, the high-order sign bit detector analyzes the non-zero data, detecting that the highest 24 bits are sign extension bits, with only the lower 8 bits being valid data, thus determining the actual effective bit width of the data to be 8 bits. Based on the above multi-dimensional feature information, the policy decision-maker determines that the current data simultaneously satisfies both sparse packing and quantized packing conditions, and then dynamically decides, through preset decision logic, that the target packing strategy is a hybrid packing mode.

[0083] The multimodal packaging engine performs dual compression processing on the input data stream based on the determined hybrid packaging mode.

[0084] The engine first initiates zero-value filtering logic, skipping the 60% of zero-value elements in the data stream and capturing only non-zero data. Then, it performs quantization and concatenation on the captured non-zero data. Since the effective bit width is 8 bits, the engine uses a barrel shifter to concatenate four consecutive captured INT8 non-zero data points into a 32-bit data packet. Simultaneously, the packetizer generates corresponding coordinate offset information, recording the relative sequence number of the non-zero data in the original stream, forming a tuple sequence of {coordinate offset, concatenated data}. Finally, the engine appends a header containing the policy ID and effective data count before the concatenated data block, generating a compressed data packet for transmission.

[0085] The receiver's configurable unpacker receives compressed data packets and parses the header information to identify the mixed packing mode.

[0086] The unpacker then activates the hybrid unpacking unit, which splits the spliced ​​data according to the coordinate offset information and writes it back to the original position, automatically filling in zeros where there are gaps, thereby completely restoring the original Key tensor data stream.

[0087] Through the above process, this embodiment achieves a significant improvement in transmission efficiency. For example... Figure 6 As shown, under the same key tensor transmission task, if a traditional bit-width compression scheme is used (corresponding to the gray solid line in the figure), a large number of invalid zero values ​​and sign extension bits still need to be transmitted due to the inability to recognize data semantics, requiring up to 100 transmission cycles. If only a single sparse packing scheme is used (corresponding to the blue solid line in the figure), although zero values ​​are filtered out, the quantization characteristics are not utilized, and the transmission cycle count is still approximately 35.9 cycles. However, by using the dynamic multimodal packing method described in this embodiment (corresponding to the red solid line in the figure), the number of transmission cycles is significantly reduced to approximately 9.0 cycles by simultaneously utilizing sparsity and quantization characteristics through a hybrid packing mode.

[0088] Data comparison shows that, compared with traditional solutions, the method in this embodiment effectively increases transmission bandwidth by approximately 10 times, far exceeding the expected effect of "2-5 times bandwidth increase". This non-linear performance leap is attributed to the accurate capture of data semantics by the feature-aware mechanism and the deep coupling of sparsity and quantization characteristics by the hybrid packing strategy. This result fully demonstrates that this application can greatly tap into transmission potential, solve bandwidth bottleneck problems, and significantly reduce end-to-end transmission latency and power consumption when processing data streams with sparse and quantized characteristics in LLM inference.

[0089] It should be understood that the above-described Key tensor transmission scenario is only a preferred example among the many application scenarios of this application. In other embodiments, the method described in this application is also applicable to scenarios such as Value tensor transmission, weight matrix transfer, or activation value flow. As long as the data stream has semantic features such as sparsity, low precision, or repetitive patterns, a significant improvement in transmission efficiency can be achieved through the dynamic multimodal packaging mechanism described in this application.

[0090] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application, such as logical adjustments to the feature detection circuit, expansion of packaging mode types, or equivalent transformations of coordinate encoding methods, should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0091] In some embodiments, an electronic device is also provided for implementing the methods described in the above embodiments. Figure 7 As shown, the chip includes a processor and a memory. The chip 700 provided in this application embodiment includes at least: a processor 701, a memory 702, and a computer program stored in the memory 702 and executable on the processor 701. When the processor 701 executes the computer program, it implements the method provided in this application embodiment.

[0092] The chip 700 provided in this application embodiment may further include a bus 703 connecting different components (including processor 701 and memory 702). The bus 703 represents one or more types of bus structures, including memory bus, peripheral bus, local area bus, etc.

[0093] Memory 702 may include a readable storage medium in the form of volatile memory, such as random access memory (RAM) 7021 and / or cache memory 7022, and may further include read-only memory (ROM) 7023. Memory 702 may also include a program tool 7025 having a set (at least one) of program modules 7024, including but not limited to an operating subsystem, one or more application programs, other program modules, and program data, each or some combination of these examples may include an implementation of a network environment.

[0094] Processor 701 can be a single processing element or a collective term for multiple processing elements. For example, processor 701 can be a central processing unit (CPU) or one or more integrated circuits configured to implement the methods provided in the embodiments of this application. Specifically, processor 701 can be a general-purpose processor, including but not limited to CPUs, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.

[0095] Chip 700 can communicate with one or more external devices 704 (e.g., keyboard, remote control, etc.), and also with one or more devices that allow a user to interact with chip 700 (e.g., mobile phone, computer, etc.), and / or with any device that enables chip 700 to communicate with one or more other chips 700 (e.g., router, modem, etc.). This communication can be performed via input / output (I / O) interface 705. Furthermore, chip 700 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 706. Figure 7 As shown, network adapter 706 communicates with other modules of chip 700 via bus 703. It should be understood that, although... Figure 7 As not shown, it can be used in conjunction with chip 700 with other hardware and / or software modules, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, Redundant Arrays of Independent Disks (RAID) subsystems, tape drives, and data backup storage subsystems.

[0096] It should be noted that, Figure 7 The chip 700 shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.

[0097] The computer-readable storage medium provided in the embodiments of this application is described below. The computer-readable storage medium provided in the embodiments of this application stores computer instructions, which, when executed by a processor, implement the methods provided in the embodiments of this application. Specifically, the computer instructions may be built into or installed in a processor, so that the processor can implement the methods provided in the embodiments of this application by executing the built-in or installed computer instructions.

[0098] Furthermore, the method provided in this application embodiment can also be implemented as a computer program product, which includes program code that implements the method provided in this application embodiment when run on a processor.

[0099] The computer program product provided in this application embodiment may employ one or more computer-readable storage media, which may be, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination thereof. Specifically, more specific examples (a non-exhaustive list) of computer-readable storage media include: electrical connections having one or more wires, portable disks, hard disks, RAM, ROM, erasable programmable read-only memory (EPROM), optical fibers, portable compact disc read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0100] The computer program product provided in this application embodiment can be a CD-ROM and include program code, and can also run on electronic devices such as computers. However, the computer program product provided in this application embodiment is not limited thereto. In this application embodiment, the computer-readable storage medium can be any tangible medium that contains or stores program code, which can be used by or in conjunction with an instruction execution system, device, or apparatus.

[0101] It should be noted that although several units or sub-units of the device have been mentioned in the detailed description above, this division is merely exemplary and not mandatory. In fact, according to embodiments of this application, the features and functions of two or more units described above can be embodied in one unit. Conversely, the features and functions of one unit described above can be further divided and embodied by multiple units.

[0102] Furthermore, although the operations of the method of this application are described in a specific order in the accompanying drawings, this does not require or imply that these operations must be performed in that specific order, or that all the operations shown must be performed to achieve the desired result. Additionally or alternatively, certain steps may be omitted, multiple steps may be combined into one step, and / or one step may be broken down into multiple steps.

[0103] Although preferred embodiments of this application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of this application.

[0104] Obviously, those skilled in the art can make various modifications and variations to the embodiments of this application without departing from the spirit and scope of the embodiments of this application. Therefore, if these modifications and variations to the embodiments of this application fall within the scope of the claims of this application and their equivalents, this application also intends to include these modifications and variations.

Claims

1. A dynamic multi-modal packing method, characterized in that, The method, applied to intra-chip or inter-chip data transmission processes, includes: The data sending end receives the input data stream, analyzes the semantic features of the input data stream in multiple dimensions in real time, and dynamically decides the target packaging strategy and effective data count from multiple predefined packaging modes based on the multi-dimensional feature information obtained from the analysis. The semantic features of the multiple dimensions include at least sparsity features and quantization bit width features; wherein the sparsity features are obtained by detecting the length of consecutive zero values ​​in the input data stream using a consecutive zero value counter; and the quantization bit width features are obtained by detecting the length of the high-order extension bits of the input data using a high-order sign bit detector to determine the actual effective bit width of the data. The step of dynamically determining the target packaging strategy from multiple predefined packaging modes includes: determining the packaging strategy corresponding to the current data block based on the detected sparsity features and quantization bit width features, through a preset decision logic or lightweight decision tree; wherein, the multiple predefined packaging modes include at least: sparse packaging mode, quantization packaging mode, hybrid packaging mode, and pass-through packaging mode; the hybrid packaging mode is a combination packaging mode that simultaneously utilizes sparsity and quantization characteristics; The decision logic follows preset triggering conditions, which include: when the sparsity feature indicates that the length of consecutive zero values ​​exceeds a preset threshold and the quantization bit width feature indicates that the effective bit width is normal, the decision adopts the sparse packing mode; when the quantization bit width feature indicates that the effective bit width is less than a preset threshold and the sparsity feature indicates that the data is not sparse, the decision adopts the quantization packing mode; when both the sparsity feature and the quantization bit width feature meet the above triggering conditions, the decision adopts the hybrid packing mode; if none of the above features are significant, the decision reverts to the pass-through packing mode. The data sending end packages the valid data in the input data stream according to the target packaging strategy and the valid data count to generate a packaged data block, and appends a packaging information header to the packaged data block to obtain a compressed data packet. The packaging information header includes the identifier of the target packaging strategy and the valid data count. The data receiving end receives the compressed data packet, parses the packaging information header to obtain the target packaging strategy identifier and the valid data count, selects the corresponding unpacking mode according to the target packaging strategy identifier, and restores the packaged data block to the original data stream according to the valid data count.

2. The method of claim 1, wherein, When the target packing strategy is sparse packing mode, the packing of valid data in the input data stream includes: Filter out zero-value data, capture only non-zero data, and generate the position coordinates of each non-zero data in the original data stream, packing them into tuples of {coordinates, non-zero data}.

3. The method of claim 1, wherein, When the target packaging strategy is quantization packaging mode, packaging the valid data in the input data stream includes: Based on the detected effective bit width, multiple low-precision data are continuously concatenated into a high-bit-width data packet for packaging.

4. The method according to claim 1, characterized in that, When the target packaging strategy is a hybrid packaging mode, packaging the valid data in the input data stream includes: First, filter out zero-value data, then concatenate and pack the captured non-zero low-precision data according to the effective bit width, and pack them in the form of a tuple of {coordinate offset, concatenated data}. In the hybrid packaging mode, the coordinate offset does not correspond to the absolute periodic position in the original data stream, but rather to the relative sequence number of the non-zero data in the original data stream. The logic for generating the coordinate offset is as follows: when non-zero data is detected, its sequence number in the non-zero data sequence is generated based on the number of non-zero data currently captured. The sequence number and the concatenated data together form a tuple, so that the receiving end can determine the precise position of each non-zero data in the original data stream based on the sequence number and the effective data count.

5. The method according to claim 1, characterized in that, The step of selecting the corresponding unpacking mode according to the target packaging strategy identifier and restoring the packaged data block to the original data stream according to the valid data count includes: The packaged data block is routed to the corresponding unpacking unit according to the target packaging strategy identifier; If it is the unpacking unit corresponding to the sparse packing mode, then read the {coordinates, non-zero data} tuple according to the effective data count, write the non-zero data back to the original position according to the coordinates, and fill the missing parts with zeros. If it is an unpacking unit corresponding to the quantization packaging mode, the spliced ​​data is split into the original number of low-precision data according to the effective data count, and output in order; If it is an unpacking unit corresponding to a mixed packaging mode, then read the {coordinate offset, concatenated data} tuple according to the effective data count, split the concatenated data into low-precision data, and write it back to the original position according to the coordinate offset; If it is an unpacking unit corresponding to the pass-through packaging mode, the compressed data is expanded according to the original bit width based on the effective data count.

6. The method according to claim 1, characterized in that, The method further includes: The unpacked data is reordered to restore the original data stream order.

7. A dynamic multimodal packaging device, characterized in that, The apparatus for performing the method as described in any one of claims 1 to 6, comprising: A feature-aware front-end, located at the data sending end, receives the input data stream, analyzes the semantic features of the input data stream in real time across multiple dimensions, and dynamically decides and outputs a target packaging strategy and an effective data count based on the analyzed multi-dimensional feature information. The multi-dimensional semantic features include at least sparsity features and quantization bit width features. The sparsity features are obtained by detecting the length of consecutive zero values ​​in the input data stream using a consecutive zero-value counter. The quantization bit width features are obtained by detecting the length of the high-order extension bits of the input data using a high-order sign bit detector to determine the actual effective bit width of the data. The dynamic decision-making of the target packaging strategy from multiple predefined packaging modes includes: based on the detected sparsity features and quantization bit width features, using a preset decision logic or a lightweight decision tree. The system determines the packaging strategy corresponding to the current data block. The predefined packaging modes include at least: sparse packaging mode, quantization packaging mode, hybrid packaging mode, and pass-through packaging mode. The hybrid packaging mode is a combination of sparsity and quantization characteristics. The decision logic follows preset triggering conditions, including: when the sparsity feature indicates that the length of consecutive zero values ​​exceeds a preset threshold and the quantization bit width feature indicates that the effective bit width is normal, the sparse packaging mode is adopted; when the quantization bit width feature indicates that the effective bit width is less than a preset threshold and the sparsity feature indicates that the data is not sparse, the quantization packaging mode is adopted; when both the sparsity feature and the quantization bit width feature meet the above triggering conditions, the hybrid packaging mode is adopted; if none of the above features are significant, the system reverts to the pass-through packaging mode. A multimodal packaging engine, located at the data sending end and connected to the feature perception front end, is used to package the valid data in the input data stream according to the target packaging strategy and the valid data count, generate packaged data blocks, and append a packaging information header containing the target packaging strategy identifier and the valid data count to the packaged data blocks to obtain compressed data packets; A configurable unpacker is set at the data receiving end and communicates with the multimodal packing engine. It is used to receive the compressed data packet, parse the packing information header to obtain the target packing strategy identifier and the valid data count, select the corresponding unpacking mode according to the target packing strategy identifier, and restore the packed data block to the original data stream according to the valid data count.

8. A chip, characterized in that, The device includes the dynamic multimodal packaging device as described in claim 7, through which intra-chip or inter-chip data transmission is achieved.

Images