Method and system for executing calculation in multi-dimensional array carrying process

Abstract

The invention provides a method and a system for executing calculation in a multi-dimensional array carrying process. The method comprises the following steps: receiving a data descriptor and a carrying calculation descriptor of a multi-dimensional array issued by a superior processing core; analyzing the data descriptor to obtain array structure information; analyzing the carrying calculation descriptor to obtain a carrying starting position, a carrying block size, a carrying mode, a target address, a calculation mode and a feedback mode; generating a reading address of the source data block in the source memory according to the array structure information, the carrying starting position and the carrying mode; reading the original data block corresponding to the size of the carrying block from the source memory according to the reading address, and performing quantization or inverse quantization calculation on the original data block in the carrying process; and writing the calculated data into the target memory according to the target address. According to the method, carrying data and calculation are fused, so that the operator implementation complexity is effectively reduced, and meanwhile, the operator performance is improved in order of magnitude.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and more specifically, to a method and system for performing calculations during the handling of multidimensional arrays. Background Technology With the widespread application of artificial intelligence, especially deep learning models, computational tasks are becoming increasingly complex, and the amount of computation and data involved is increasing dramatically. Particularly in fields such as image recognition, natural language processing, and speech recognition, the training and inference of deep neural networks place extremely high demands on hardware performance. To improve efficiency while maintaining computational accuracy, researchers are gradually moving towards hardware acceleration and operator optimization to meet the processing needs of multidimensional tensor data.

[0002] In AI chip architecture, the handling and computation of multidimensional tensors are critical operations. Multidimensional tensors are a fundamental data structure in deep learning models, widely used in operators such as convolution and matrix multiplication. In practical deployments, multidimensional tensors are frequently moved between various operators, becoming a crucial component of the computational flow. To improve computational efficiency and reduce memory bandwidth consumption, quantization techniques are typically employed to convert floating-point tensor data into a low-ratio, specific-point representation. Quantization not only reduces storage and computational complexity but also helps reduce power consumption and improve inference speed.

[0003] Existing mainstream AI chip systems primarily employ the following two methods to achieve the above operations: In terms of tensor transport, some chips have basic data transport hardware acceleration modules, which can efficiently complete data transmission from source address to destination address and support multidimensional array slice transport or format conversion.

[0004] In terms of computing, most platforms still rely on programmers to write software logic using general-purpose computing units (such as GPUs or NPUs) for operation. This approach has certain advantages in terms of flexibility, but it has a significant impact on performance.

[0005] Although existing technologies can perform multidimensional tensor manipulation and computation at the functional level, they have the following significant shortcomings: High consumption of general computing resources: The calculation process is implemented using general computing units, which requires scheduling a large number of instructions to be executed, increasing the system burden and affecting the execution efficiency of operators.

[0006] Decoupling data transfer from computation incurs additional overhead: computation requires extra data read / write operations, which not only prolongs execution time but also increases power consumption. Data is first read from source memory into the computation unit before processing. This decoupling leads to frequent switching between data transfer and computation, increasing the number of read / write operations on storage media and significantly reducing overall system performance.

[0007] High implementation complexity: Programmers need to manually manage the transfer logic and computation process, and write complex low-level code based on the specific hardware architecture. This not only increases development time but also maintenance costs. Due to the high requirements for hardware and low-level control, developers are prone to errors when developing operators, leading to extended development cycles.

[0008] The aforementioned problems are particularly pronounced in applications requiring large-scale models and high real-time performance, severely limiting the operator performance and overall inference efficiency of AI chips. Therefore, there is an urgent need for an efficient method for multidimensional data transfer and computation to reduce system complexity and improve computational performance. Summary of the Invention

[0009] This invention provides a method and system for performing calculations during the transport of multidimensional arrays, aiming to solve problems such as decoupling of transport and calculation processes, high computational resource consumption, high power consumption, and high development complexity in existing technologies. By integrating data transport and calculation processes, calculations are completed directly during data transmission, avoiding redundant data reading and writing and intermediate result storage, thereby improving overall processing efficiency and system energy efficiency.

[0010] This method receives data descriptors and transport calculation descriptors for multidimensional arrays from the upper-level processing core to obtain information such as array structure, transport start position, transport mode, transport block size, target address, and feedback method. The system calculates the read address of the data block in the source memory based on the descriptors and performs calculation operations on the original data block simultaneously. The calculation process includes determining the maximum exponent value in each data block, normalizing the floating-point exponent, and converting the normalized floating-point data into fixed-point or low-bit-width format. The processed data is then written to the specified target address.

[0011] Compared with existing technologies, the method of this invention does not require computation to be performed independently of the data transport process, thereby avoiding additional data transport and instruction scheduling, improving execution efficiency, and reducing power consumption and bus bandwidth pressure. Furthermore, by employing a descriptor-driven approach for transport and computation control, developers only need to provide standardized array information and task configurations, eliminating the need to write complex low-level control code, effectively reducing the difficulty of operator development and deployment.

[0012] Furthermore, this method supports multiple transport modes, including continuous transport, block transport, and transformation transport, adapting to tensor data of different dimensions and structures; it also supports caching of descriptor information and temporary storage of intermediate calculation results, further optimizing system operating efficiency and improving the overall performance of operator execution. It is particularly suitable for data processing scenarios requiring high throughput and low latency, such as AI applications like neural network inference.

[0013] In a first aspect, the present invention provides a method for performing calculations during the handling of multidimensional arrays, characterized in that the method includes: Receives data descriptors and transport calculation descriptors for multidimensional arrays issued by the superior processing core; Parse the data descriptor to obtain array structure information; The transport calculation descriptor is parsed to obtain the transport start position, transport block size, transport mode, target address, calculation method, and feedback method; Based on the array structure information, the transport start position, and the transport mode, generate the read address of the source data block in the source memory; The original data block corresponding to the transport block size is read from the source memory according to the read address, and quantization or dequantization calculation is performed on the original data block during the transport process according to the calculation method. The calculated data is written into the target memory according to the target address; After the data is written, the transfer and calculation completion information is sent to the upper-level processing core according to the feedback method described.

[0014] Secondly, the present invention also provides a system for performing calculations during the handling of multidimensional arrays, characterized in that the system comprises: The receiving module is used to receive the data descriptors and transport calculation descriptors of the multidimensional arrays sent by the upper-level processing core; A data parsing module is used to parse the data descriptor to obtain array structure information; The parsing and calculation module is used to parse the transport calculation descriptor to obtain the transport start position, transport block size, transport mode, target address, calculation method and feedback method; A preprocessing unit is used to generate the read address of the source data block in the source memory based on the array structure information, the transport start position, and the transport mode; The system includes a read control unit and a calculation unit. The read control unit is used to read the original data block corresponding to the transport block size from the source memory according to the read address. The calculation unit is used to perform quantization or dequantization calculation on the original data block during the transport process according to the calculation method. A write data unit is used to write the calculated data into the target memory according to the target address; The post-processing unit is used to send the transfer and calculation completion information to the upper-level processing core according to the feedback method after the data writing is completed.

[0015] The present invention provides a method and system for performing calculations during the transport of multidimensional arrays: First, the present invention integrates data transport and calculation operations by directly performing calculations during the data transport process, avoiding the repetitive data reading and writing caused by the separate processing in traditional technologies, significantly reducing memory bandwidth pressure and power consumption, and improving the overall operator execution efficiency.

[0016] Second, this invention uses a descriptor-driven approach to control the transport and calculation process. The upper-level processing core only needs to issue standardized data descriptors and transport and calculation descriptors to achieve coordinated execution of transport and calculation operations, which simplifies the software development process and reduces the complexity and error rate of operator implementation.

[0017] Third, compared with traditional methods that rely on general computing resources to complete computational operations, this invention can reduce instruction scheduling and resource contention, effectively freeing up general computing unit resources to perform other core tasks, thereby improving the overall concurrent processing capability and throughput of AI chips.

[0018] Fourth, this invention supports multiple modes such as continuous data transfer, chunked data transfer, and transformed data transfer, and is adaptable to multidimensional arrays with different structures and shapes. It has good adaptability and versatility in handling typical operations such as convolution preprocessing and im2col.

[0019] Fifth, this invention supports the handling and computation of multidimensional arrays with up to five dimensions, meeting the processing needs of complex tensor structures in current mainstream neural networks and improving the system's adaptability in high-dimensional data scenarios.

[0020] Sixth, by introducing a descriptor caching mechanism and an intermediate calculation result temporary storage mechanism, this invention further reduces repeated access to external memory, effectively reduces access latency and power consumption, and improves the continuity and overall performance of the data processing process.

[0021] In summary, this invention has significant advantages over existing technologies in terms of reducing development difficulty, reducing resource consumption, and improving computing efficiency and energy efficiency. It is especially suitable for AI chips and deep learning inference applications that have high requirements for performance, power consumption and development efficiency. Attached Figure Description

[0022] To more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0023] Figure 1 This is a flowchart of the calculation method performed during the multidimensional array transfer process provided in an embodiment of the present invention; Figure 2 This is a block diagram of a computation system performed during the handling of multidimensional arrays, provided in an embodiment of the present invention. Figure 3 This is a schematic diagram of the data reading control logic provided in an embodiment of the present invention; Figure 4 This is a schematic diagram of the quantization logic provided in an embodiment of the present invention; Figure 5 This is a flowchart of the calculation process during the handling of multidimensional arrays provided in an embodiment of the present invention. Detailed Implementation

[0024] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Invention Overview As previously stated, this invention provides a method and system for performing calculations during multidimensional array transport. By directly executing calculations during data transport, it achieves the integration of data transmission and computation, avoiding the redundant access and resource waste caused by the decoupling of transport and computation in traditional technologies. When developing high-performance operators based on AI chips using this system, users do not need to concern themselves with the underlying data transport details or write complex computation programs. They only need to provide data descriptors and transport computation descriptors in a specific format, and the chip hardware will automatically complete the transport and computation processing. This not only significantly reduces the complexity of operator implementation but also fully leverages the advantages of hardware parallel computing and efficient data pathways, resulting in an order-of-magnitude improvement in operator performance and significantly enhancing the performance and energy efficiency of AI chips in deep learning inference and other scenarios.

[0026] Exemplary methods Figure 1 This is a flowchart of a calculation method performed during the handling of a multidimensional array, provided by an embodiment of the present invention. The embodiment includes the following steps: S101: Receives the data descriptor and transport calculation descriptor of the multidimensional array issued by the upper-level processing core.

[0027] The data descriptor (data_desc) is used to characterize the data structure of the multidimensional array being processed. The compute descriptor (compute_desc) is used to define the behavior of data block transport and computation involved in the current operation.

[0028] The descriptor structure in this invention features a unified format and predefined field arrangement, facilitating direct reading and parsing via register mapping, DMA controllers, or task scheduling interfaces. The descriptor passing mechanism in this invention possesses high abstraction and standardization advantages, allowing task configuration to be completely independent of hardware implementation details, thus enhancing system flexibility and scalability.

[0029] To improve the system's portability and platform adaptability, this invention adopts a standardized and universal interface protocol to receive data descriptors and transport computation descriptors from the upper-level processing core. This interface design is applicable to mainstream processor architectures, including but not limited to CPU, GPU, GPGPU, and NPU.

[0030] To reduce the number of reads and improve descriptor processing efficiency, this invention can also cache recently used data descriptors.

[0031] A descriptor cache unit is essentially a cache unit used to store and accelerate access to descriptors. Descriptors are typically stored in memory and are used to describe metadata of data structures, tasks, or other objects. In computer systems, descriptors are commonly used to represent objects such as processes, segments, pages, and network packets. They contain pointers to data or resources, status information, control flags, etc. By caching these descriptors, the number of times they are re-read from main memory or the upper-level processing core can be reduced, significantly improving descriptor access speed and overall system performance.

[0032] The descriptor cache unit receives a data_desc request. If the requested data_desc is not in the cache, it initiates an access to external storage to read the data_desc. After the data is returned, the descriptor cache unit stores it in its local cache for future accesses to the same data_desc. This mechanism reduces the frequency of accesses to external storage, improves system efficiency, and accelerates the data access process when multiple tasks are being processed.

[0033] Specifically, when performing multiple similar data transfer and quantization tasks consecutively, if the required data descriptors are the same as or highly similar to those of the previous task, the system can directly retrieve the descriptor information from the local cache, without having to reread it from main memory or the upper-level processing core each time. This approach effectively reduces the number of bus transfers and control path latency, thereby improving descriptor resolution speed.

[0034] This caching mechanism can be implemented using structures such as circular queues, register files, or small-capacity high-speed caches, configured according to actual hardware resources. In typical application scenarios, when multiple operators need to perform batch handling and quantization operations on multidimensional arrays with the same structure, the descriptor cache in the hardware module can greatly reduce the overhead of repeated configuration, improve the efficiency of task switching, and enhance the continuity of data stream processing.

[0035] This descriptor cache unit can be integrated into the transport and quantization control module, or it can exist as an intermediate cache for communication with the task scheduling unit. When the upper-level processing core issues a descriptor for the first time, it writes it into the control register and synchronizes the cache. Before executing a new task, the system automatically compares the current task request with the most recent descriptor in the cache to determine whether it can be reused, thereby improving the intelligence and response speed of hardware execution.

[0036] In addition, this caching mechanism also supports dynamic replacement strategy management for descriptor usage frequency, such as the Least Recently Used (LRU) strategy, to ensure that the cache space always retains the control information most likely to be reused, thereby further improving the overall system performance under the condition of limited resources.

[0037] S102: Parse the data descriptor to obtain array structure information.

[0038] The array structure information includes the array dimensions, the number of elements in each dimension, the amount of data in each dimension, and the size of the data block moved each time.

[0039] The array dimension indicates whether the data being processed is 1-dimensional, 2-dimensional, 3-dimensional, 4-dimensional, or 5-dimensional. By reading the dimension field, the number of axes in space for the array is determined, which in turn determines the logic for generating the transport address and the cross-dimensional traversal strategy.

[0040] To meet the demands of high-dimensional tensor structures in modern deep learning tasks, the method described in this invention supports multidimensional arrays with an upper limit of five dimensions. This makes it widely applicable to complex data formats such as images (e.g., 4D [N,C,H,W]), video sequences (5D [N,T,C,H,W]), time-series data, or multimodal inputs. Through standardized parsing of array structure information, this invention can adapt to data of different shapes and arrangements, providing a solid foundation for subsequent generalized transfer and unified format quantization.

[0041] The number of elements in each dimension, i.e. the length of the array along each axis, such as width, height, number of channels, number of time steps, etc., is used to define the boundary conditions of each dimension.

[0042] The amount of data per dimension (stride or pitch) refers to the number of bytes of data required to advance one dimension unit, used to calculate the address offset. This parameter is particularly critical for tensors or arrays of packed structures that are not stored contiguously.

[0043] The size of the data block moved each time defines the size of the sub-block processed by each task in the moving operation. This size determines the working granularity of the quantization calculation unit, such as an 8×8 sub-matrix, a 16-channel processing unit, or a time-series data block with a fixed step size.

[0044] S103: Parse the transport calculation descriptor to obtain the transport start position, transport block size, transport mode, target address, calculation method and feedback method.

[0045] The starting position for data transfer is the starting coordinate of the data block to be processed in the source array, which can be represented in multi-dimensional coordinate form (e.g., [N,C,H,W] in a four-dimensional array). This parameter determines the precise location from which data is read from the source memory.

[0046] The transport block size defines the size of the physical data block involved in each transport operation. For example, when transporting data from a 2D image, an 8×8 image block can be transported at once; when processing multi-channel feature maps, a 16-channel data block might be transported at once; for time-series data, multiple frames within a time window are transported at once. The transport block size directly affects memory access patterns and data transfer cycles, i.e., how much data is read at a time and how this data is traversed in memory.

[0047] The transport mode indicates the structured transport method used in this transport, which controls the data reading order and layout to facilitate the specific data arrangement requirements of the quantification calculation task.

[0048] The data transfer modes include continuous data transfer, chunked data transfer, and transformed data transfer.

[0049] Contiguous Transfer is a mode suitable for situations where the memory layout of the source array is linearly continuous, commonly seen between feature map channels or in time series data. By specifying the starting address and data volume of the target storage medium, the system can directly transfer data to the target storage medium according to continuous addresses. This mode typically completes the transfer of the entire data block in a single continuous DMA transfer, minimizing overhead and maximizing efficiency, making it suitable for scenarios such as batch convolution input and fully connected layer weight loading.

[0050] Strided or block transfer is used to handle the movement of data blocks that exhibit regular slices within a multidimensional array. Examples include sliding window processing, step extraction, and region partitioning. This mode, given the starting coordinates and data volume of each dimension of the target multidimensional data, calculates jump addresses based on this information, generating multiple sets of address access sequences with step information, thereby continuously saving the data to the target storage medium. This mode is commonly found in scenarios such as ROIAlign and local attention mechanisms, and is suitable for data access patterns with regularity.

[0051] Reordering Transfer is a mode used to rearrange the order or structure of data during the transfer process to meet the specific requirements of subsequent operators on the data input structure. It is particularly important in scenarios such as matrix computation, GEMM optimization, and subgraph fusion.

[0052] The aforementioned data transfer mode involves rearranging the multidimensional array using an image-to-column algorithm before transferring the transformed data. The image-to-column algorithm is a standard transformation method commonly used in deep learning to expand local receptive fields into column vectors. Its typical applications include: expanding two-dimensional image blocks (such as the receptive region of a convolution kernel) into column vectors to convert convolution operations into matrix multiplication, adapting to efficient matrix multiplication operators in hardware; and quickly constructing intermediate data formats to improve the versatility of operators when performing complex operators such as Depthwise Convolution and Grouped Convolution.

[0053] In this invention, the image transposition and rearrangement process is completed directly by the transport logic, without the need for the participation of the upper-level processing core, and without the need to copy or expand the original array to the intermediate buffer area first, which significantly reduces memory usage and bus transmission overhead.

[0054] In practice, the system dynamically constructs a mapping relationship from the source array to the rearranged target data block based on the starting position and block size set in the transport calculation descriptor, combined with the array dimension and memory layout information defined in the data descriptor.

[0055] During the data transfer operation, the system traverses the source array from a specified starting position according to the multi-dimensional window structure defined by the size of the transfer block. The array is expanded into a columnar data layout in dimensional order and then transferred sequentially to the target address. The entire process is implemented through hardware logic pipelining, avoiding the overhead of intermediate caching and repeated reads.

[0056] This invention combines data rearrangement and data transfer, eliminating the traditional phased processing mode of "rearrange first, then transfer" or "transfer first, then rearrange". This significantly reduces the complexity of the data path, improves the utilization of transfer bandwidth and the overall system throughput performance, and is particularly suitable for high-performance chip architectures that perform batch transfer and computational preprocessing of structured data.

[0057] The target address is the address of the memory where the processed data will be written. The system can directly generate a target write request based on this address parameter without an intermediate addressing process.

[0058] The calculation method includes quantization or dequantization. By specifying the specific calculation method, the actual calculation process performed in S105 is determined.

[0059] Quantization is a precision compression technique commonly used in GPU computing. Its purpose is to convert data originally represented using high-precision floating-point numbers (such as FP32) into low-precision integer forms (such as INT8). This significantly reduces the model's memory footprint, improves computation speed, and lowers power consumption. In practical applications, quantization typically selects an appropriate scaling factor based on the distribution of a dataset to scale the original floating-point values ​​to the target integer range. Quantization is mainly applied to tensors such as weights and activation values ​​in neural networks, and its improvement in runtime efficiency is particularly significant during the model deployment phase.

[0060] Dequantization is the inverse process of quantization, used to restore integer-represented data to an approximate floating-point number. Because some precision is lost during quantization, the floating-point value obtained through dequantization is only an approximation of the original value. However, for many inference tasks, this loss of precision is acceptable. Dequantization typically occurs when further high-precision calculations are needed or when outputting results to a user-readable format. Its core function is to facilitate data transitions between calculations of different precisions, maintaining the consistency and precision control of the overall computational chain of the model.

[0061] Feedback mechanisms are used to instruct the higher-level processing core to notify the task upon completion. This facilitates task-level synchronous control and improves chip-level task scheduling efficiency.

[0062] The feedback methods include notifying the upper-level processing core via an interrupt signal (suitable for high real-time control scenarios) and updating the status register for the upper-level processing core to poll and read (suitable for low-power or batch processing models).

[0063] The feedback method can be flexibly specified in the transport calculation descriptor through configuration fields. The system can dynamically switch the feedback mechanism according to actual scheduling needs, processing load and hardware resources, thereby achieving an optimized balance between performance, power consumption and control complexity.

[0064] S104: Generate the read address of the source data block in the source memory based on the array structure information, the transport start position, and the transport mode.

[0065] The array structure information represents the layout structure of the source data block in memory. The transport start position specifies the starting address or starting data block when reading data from the source memory. For example, in image data processing, the starting position might be the top left corner of the image or a specific coordinate point. The transport mode determines how the data is read, and each mode has different reading rules. For example, continuous data transport means that data is transported sequentially, one by one.

[0066] By parsing this information, one or more read addresses are generated, pointing to the location of each data block in the source memory. These addresses will be used for subsequent data read operations, ensuring that data can be efficiently moved according to the set rules.

[0067] Based on the array structure information, the transport start position, and the transport mode, multiple consecutive read requests are generated. Each read request represents an operation to read data from the source memory.

[0068] Each request maintains contiguous address space or has a fixed step size, thereby avoiding bus idleness, fully utilizing memory bandwidth, and achieving high-throughput data access. Simultaneously, the address generation logic continuously derives all read data addresses required for the current data transfer within a single calculation cycle, ensuring seamless integration between the reading process and subsequent data transfer and calculation processes, reducing waiting time, and improving overall processing efficiency.

[0069] S105: Read the original data block corresponding to the transport block size from the source memory according to the read address, and perform quantization or dequantization calculation on the original data block during the transport process according to the calculation method.

[0070] Based on the read address generated in the preceding steps, a raw data block matching the transport block size is retrieved from the source memory in a single operation. This transport block can be a multidimensional matrix, a channel data block, or a time series segment, with its specific size determined by the transport block size.

[0071] For example, to support secure data transfer and access at the boundaries of multidimensional tensors, this invention can employ methods such as... Figure 3 This paper presents a data reading control logic with boundary detection. The reading logic is based on a multi-layer nested loop structure, which traverses each dimension of the tensor sequentially according to a set step size. Combined with out-of-bounds detection and data zero-padding mechanism, it achieves robust reading of irregular data blocks.

[0072] The read control logic traverses the four dimensions of the tensor in a four-layer nested loop manner, successively (for example, dim0 to dim3, which can correspond to dimensions such as Batch, Channel, Height, Width in actual applications). The specific implementation is as follows: (1)Multi-dimensional loop traversal structure: The system uses a four-fold nested loop to control the traversal of the four dimensions from dim3 to dim0 respectively. Each loop starts from the starting address start_dimX, increments by the step size dimX_step until the termination condition end_dimX is reached.

[0073] (2)Legal access judgment and boundary handling: Before each data access, the system performs the following judgment: If the current access position is within the legal range of each dimension (i.e., r_dim1 < DIM1, r_dim2 < DIM2, r_dim3 < DIM3), then data reading is performed; If r_dim0 + dim0_step does not exceed DIM0, indicating that this access will not exceed the tail boundary of the current block, normal reading is performed; If an out-of-bounds occurs at the tail of the current data block, reading the current index position is allowed, but the exceeded part is explicitly cleared to avoid using invalid data; If the current access position is completely out of bounds, this access is directly set to zero and the reading is skipped.

[0074] (3)Support for tiled transfer: This read control logic is particularly suitable for the scenario of tensor tiling processing. The system can set the step size according to the tile size when transferring each data block, and combine the above boundary handling logic to automatically adapt to the reading of tiles with uneven tails, thereby achieving the complete processing of tensors without introducing additional risks of out-of-bounds memory access.

[0075] In summary, the present invention can prevent the reading of out-of-bounds data, ensure system stability; be compatible with the reading of data blocks with unaligned tensor tails; be applicable to the reading process of tensors with any number of dimensions; the loops and judgment conditions of each dimension can be directly expanded into a finite state machine or nested controller to adapt to various processing units (such as DMA, AI cores).

[0076] If the calculation method is quantization, the specific quantization steps include: scanning all floating-point elements in each data block, extracting their exponent parts, and determining the maximum floating-point exponent value in each data block; performing exponent normalization on the floating-point exponent of each element in the data block to map data of different orders of magnitude to a unified exponent range, reducing precision loss; and converting the normalized floating-point data into a fixed-point format or a low bit-width format (such as INT8, INT4, etc.) to reduce the amount of data stored and transmitted, while improving the processing efficiency of the computing unit.

[0077] By performing the above quantization steps in parallel during the data transfer process, the conversion from high-precision floating-point data to low-precision fixed-point data can be completed without increasing additional processing latency. This achieves deep integration of data transmission and preprocessing, reduces storage bandwidth usage, and provides necessary exponential reference information for subsequent inverse quantization calculations.

[0078] For example, if the calculation method is quantization, to achieve unified quantization processing of the raw floating-point data, the system performs a normalization operation based on the exponent information of the elements in each data block, thereby improving the effective accuracy of the fixed-point representation. Quantization operations are processed in units of two-dimensional data blocks, the size of which is determined by the `quant_row` and `quant_col` parameters, representing the number of rows and columns of the two-dimensional submatrix involved in the current quantization operation. This range can be dynamically adjusted according to the transfer descriptor to adapt to the processing requirements of different transfer requests.

[0079] like Figure 4 As shown, in the stage of determining the maximum floating-point exponent value, all floating-point elements in the current data block are traversed, their exponent (exp) parts are extracted, and the maximum exponent value max_exp is found in the entire data block as a reference benchmark for subsequent normalization. This process can be represented as follows: max_exp = -inf for (r=0;r <quant_row;r++) { for (c=0;c <quant_col;c++) { if data[r][c].exp>max_exp { max_exp = data[r][c].exp } } } During the exponent normalization and mantissa quantization stages, after obtaining max_exp, the system performs exponent normalization on each floating-point number in the current data block, that is, subtracting max_exp from the exponent value of the element, and shifting its mantissa to the right by a preset number of bits x, in order to achieve precision compression or bit width reduction. for (r=0;r <quant_row;r++){ for (c=0;c <quant_col;c++) { data[r][c].exp -= max_exp data[r][c].mant = data[r][c].mant>>x } } The exponential normalization operation (exp -= max_exp) scales all data uniformly to a relative exponential space. Right shift of the mantissa (mant>>x) compresses the mantissa precision into a low-bit format, such as INT8 or INT4, thereby reducing the burden on subsequent transmission and storage.

[0080] The quant_row and quant_col parameters in the above logic support dynamic configuration. Different data transfer requests can adjust the range and granularity of the data to be processed based on the computing power of the target processing unit, the target format (such as the output fixed bit width), or the precision requirements of the target task, thereby improving hardware flexibility and scalability.

[0081] This invention can reduce data precision loss by normalizing based on the maximum exponent of each data block, thus avoiding the truncation of small data values ​​caused by large data values ​​during quantization; improve computational consistency by processing data blocks with the same structure according to unified logic, which facilitates parallel execution and pipeline integration; and reduce resource consumption by expressing the compressed mantissa in a smaller space in low-bandwidth transmission or on-chip storage, thereby improving system efficiency.

[0082] If the calculation method is dequantization, a dequantization operation is performed to restore the value to a floating-point approximation suitable for subsequent calculations (such as convolution, activation function processing, etc.). The dequantization operation is usually performed before the target processing module starts working, using the reference exponent max_exp recorded during the quantization process for inverse transformation.

[0083] The dequantization process is also performed on a block-by-block basis, consistent with the original quantization range (i.e., quant_row × quant_col). Its basic operations include: Iterate through each quantized data element, shift its mantissa back to the left by the original right shift bits x, and restore its exponent to its absolute value relative to max_exp.

[0084] for (r = 0; r <quant_row; r++) { for (c = 0; c<quant_col; c++) { data[r][c].mant = data[r][c].mant<<x; data[r][c].exp += max_exp; } } The mantissa left shift operation (mant<<x) restores the precision; The exponent restoration (exp += max_exp) reconstructs the approximate magnitude of the original data.

[0085] The floating-point value restored by this process is an approximate representation of the original value, and the error range is controlled by the mantissa precision, which is suitable for application scenarios with a high tolerance for precision such as subsequent neural network calculations.

[0086] If a custom floating-point format or a low-bitwidth fixed-point format is used to represent data, a floating-point reconstruction operation can also be performed after dequantization to repack exp and mant into a standard IEEE-754 floating-point number or a certain soft floating-point representation to adapt to subsequent calculation modules: data[r][c].float_value = reconstruct_float(data[r][c].mant, data[r][c].exp); This function can be a hardware built-in function or a software simulation module, and can be switched according to system resources.

[0087] The present invention can restore approximate precision, restore the floating-point form within a small error range, and provide an effective input for subsequent processing; support low-bitwidth transmission, combine with the compressed representation used after front-end quantization, and the dequantization mechanism realizes the mapping between low bandwidth and high precision; can be embedded in the calculation pipeline, and the dequantization process can be integrated as a post-processing unit into the process before the data is transported to the processing core to achieve seamless data transition; hardware-friendly implementation, the exponent addition and subtraction and mantissa shift operations can be implemented in hardware through adders and shifters, supporting high-concurrency and low-latency processing.

[0088] The present invention generates transfer requests through preprocessing, and配合 with the quantization and dequantization calculation modules embedded during the transfer process,实现了 an integrated calculation process of data transfer, compression, and restoration,显著降低了 the storage and bandwidth requirements, and ensures that the calculation precision and performance are balanced in multi-data block processing tasks.

[0089] In addition, the present invention can also temporarily store the intermediate results obtained by calculation in the data cache unit until they are written into the target memory after the calculation of the corresponding data block is completed.

[0090] During the data transfer and computation process, after performing quantization, dequantization, or related transformations on the original data blocks, a set of intermediate result data is generated. To avoid bus occupation and latency caused by frequent access to the target memory, this invention can also set up a high-speed data cache unit (such as on-chip SRAM or register file) between the computing unit and the target memory to temporarily store the intermediate results of each data block.

[0091] While a data block is not yet fully computed, intermediate results are written sequentially to the corresponding cache areas in the data cache unit according to the computation order. Once all elements of the data block have been computed, the complete result of the data block is written to the target memory all at once. This reduces the number of write operations, replacing element-by-element writing with batch writing, reducing bus access frequency, and improving data transfer efficiency; it ensures data consistency, as write-back only occurs after all elements in the data block have been computed, avoiding the presence of incomplete data in the target memory; and it improves system throughput, as the introduction of the data cache unit allows computation and storage operations to be parallelized, and the computation of subsequent data blocks can be executed simultaneously during the write-back of the previous data block, thereby improving overall pipeline efficiency.

[0092] This mechanism is particularly suitable for scenarios with high bandwidth constraints or multiple cores concurrently accessing the target memory. It can significantly reduce memory access conflicts and bandwidth bottlenecks, and improve the overall system performance and real-time response capabilities.

[0093] S106: Write the calculated data into the target memory according to the target address.

[0094] The target memory can be the system's main memory, cache, hard disk, or other storage media; its specific location is determined by the target address. The write process ensures that the calculation results are accurately saved and available for subsequent processing. The choice of the target address depends on the system architecture and requirements; it can be a predefined memory address or dynamically generated through some address calculation mechanism.

[0095] S107: After the data writing is completed, send the transfer and calculation completion information to the upper-level processing core according to the feedback method.

[0096] Once data is successfully written to the target memory, this status is fed back to the higher-level processing core so that it can be informed of the current operation's progress. This information is crucial for scheduling subsequent tasks, managing resources, and controlling the flow. Through this feedback, the higher-level processing core can make the next processing decisions, ensuring smooth and efficient system operation.

[0097] Exemplary System Accordingly, embodiments of the present invention also provide a system for performing calculations during the transport of multidimensional arrays. Figure 2This is a block diagram of a computation system performed during the handling of multidimensional arrays, as provided in an embodiment of the present invention. Figure 2 As shown, the system 100 provided in this embodiment includes: The receiving module 101 is used to receive the data descriptor and transport calculation descriptor of the multidimensional array sent by the upper-level processing core; The data parsing module 102 is used to parse the data descriptor to obtain array structure information; The parsing and calculation module 103 is used to parse the transport calculation descriptor to obtain the transport start position, transport block size, transport mode, target address, calculation method and feedback method; Preprocessing unit 104 is used to generate the read address of the source data block in the source memory based on the array structure information, the transport start position and the transport mode; The system includes a read control unit 105 and a calculation unit 106. The read control unit 105 is used to read the original data block corresponding to the transport block size from the source memory according to the read address. The calculation unit 106 is used to perform quantization or dequantization calculation on the original data block during the transport process according to the calculation method. The write data unit 107 is used to write the calculated data into the target memory according to the target address; The post-processing unit 108 is used to send the transfer and calculation completion information to the upper-level processing core in accordance with the feedback method after the data writing is completed.

[0098] The array structure information includes the array dimensions, the number of elements in each dimension, the amount of data in each dimension, and the size of the data block moved each time.

[0099] The data transfer modes include continuous data transfer, chunked data transfer, and transformed data transfer.

[0100] The computing unit 106 includes: A module used to determine the maximum floating-point exponent value in each data block; A module for performing exponent normalization on the floating-point exponent of each element in the data block; This module is used to convert normalized floating-point data into fixed-point or low-bit-width formats.

[0101] The system 100 also includes: Descriptor cache unit 109 is used to cache recently used data descriptors.

[0102] The system 100 further includes a data cache unit 110, and the calculation unit 106 is also used to temporarily store the intermediate results obtained from the calculation in the data cache unit 110. The data writing unit 107 is also used to write the data to the target memory only after the corresponding data block has been calculated.

[0103] The preprocessing unit 104 further includes: A module for generating multiple consecutive read requests based on the array structure information, the transport start position, and the transport mode.

[0104] The system 100 supports a maximum array dimension of five for multidimensional arrays.

[0105] When the data transfer mode is the transformation data transfer mode, the calculation unit 106 is also used to rearrange the multidimensional array using an image transposition algorithm before transferring it.

[0106] The feedback methods include notifying the upper-level processing core via an interrupt signal and updating the status register for the upper-level processing core to poll and read.

[0107] The receiving module 101 is also used to receive the data descriptor and transport calculation descriptor of the multidimensional array issued by the upper-level processing core using a general interface design.

[0108] For example, such as Figure 5 As shown, 001 is the decode unit, which includes a receiving module 101, a data parsing module 102, and a parsing calculation module 103. In this unit, the receiving module 101 is responsible for receiving the data transfer instructions issued by the upper-level module, and the data parsing module 102 and the parsing calculation module 103 are responsible for parsing out the mode information, quantization calculation information, and data_desc read request for this data transfer.

[0109] 003 is the descriptor cache unit 109, which receives the data_desc request from 001. If the request is not in the cache, it initiates an access to external storage, reads the data_desc, and after the data is returned, it saves it in the local cache unit for the next access to the corresponding data_desc.

[0110] 002 is the preprocessing unit 104, which calculates the initial address, step size, and destination address for each dimension of the data transfer.

[0111] 004 is for reading control unit 105; the data reading logic is described above.

[0112] 005 is the calculation unit 106. Based on the preprocessing stage information, this unit performs quantization calculation on data within a specific address and range, and temporarily stores it in the local data cache unit 110. After the data calculation is completed, the data is transmitted to the next level. The quantization calculation logic is described above.

[0113] 006 is the write data unit 107, which is responsible for generating write data requests.

[0114] 007 is the post-processing unit 108, which is responsible for writing data to complete the response after the meeting and returning the transmission completion response to the upper-level module.

[0115] It should be noted that although the operations of the computational method of the present invention in the process of handling multidimensional arrays are described in a specific order in the accompanying drawings, this does not require or imply that these operations must be performed in that specific order, or that all the operations shown must be performed to achieve the desired result. Additionally or alternatively, certain steps may be omitted, multiple steps may be combined into one step, and / or one step may be broken down into multiple steps.

[0116] Furthermore, although several devices, units, or modules of the computational system are mentioned in the detailed description above for performing the computation during the handling of multidimensional arrays, this division is merely exemplary and not mandatory. In fact, according to embodiments of the invention, the features and functions of two or more modules described above can be embodied in a single module. Conversely, the features and functions of a single module described above can be further divided and embodied by multiple modules.

[0117] While the spirit and principles of the invention have been described with reference to several specific embodiments, it should be understood that the invention is not limited to the disclosed specific embodiments, and the division of aspects does not imply that features in these aspects cannot be combined for benefit; such division is merely for ease of description. The invention is intended to cover various modifications and equivalent arrangements included within the spirit and scope of the appended claims.

[0118] This invention provides: 1. A method for performing calculations during the handling of multidimensional arrays, characterized in that the method includes: Receives data descriptors and transport calculation descriptors for multidimensional arrays issued by the superior processing core; Parse the data descriptor to obtain array structure information; The transport calculation descriptor is parsed to obtain the transport start position, transport block size, transport mode, target address, calculation method, and feedback method; Based on the array structure information, the transport start position, and the transport mode, generate the read address of the source data block in the source memory; The original data block corresponding to the transport block size is read from the source memory according to the read address, and quantization or dequantization calculation is performed on the original data block during the transport process according to the calculation method. The calculated data is written into the target memory according to the target address; After the data is written, the transfer and calculation completion information is sent to the upper-level processing core according to the feedback method described.

[0119] 2. The calculation method for performing calculations during the multidimensional array transport process as described in item 1, characterized in that the array structure information includes array dimensions, the number of elements in each dimension, the amount of data in each dimension, and the size of the data block transported each time.

[0120] 3. The calculation method performed during the multidimensional array transport process as described in item 1, characterized in that the transport mode includes continuous data transport, block data transport, and transformed data transport.

[0121] 4. The calculation method for performing calculations during multidimensional array transport according to any one of items 1-3, characterized in that the step of reading the original data block corresponding to the transport block size from the source memory according to the read address, and performing quantization or dequantization calculations on the original data block during transport according to the calculation method specifically includes: Determine the maximum floating-point exponent value in each data block; Perform exponent normalization on the floating-point exponent of each element in the data block; Convert normalized floating-point data into fixed-point format or low-bit-width format.

[0122] 5. The method for performing calculations during the transport of a multidimensional array according to any one of items 1-3, characterized in that the method further includes: caching recently used data descriptors.

[0123] 6. The method for performing calculations during multidimensional array transport according to any one of items 1-3, characterized in that, before the step of writing the calculated data into the target memory according to the target address, the method further includes: The intermediate results obtained from the calculation are temporarily stored in the data cache unit until the corresponding data block is calculated and then written to the target memory.

[0124] 7. The calculation method for performing calculations during multidimensional array transport according to any one of items 1-3, characterized in that the step of generating the read address of the source data block in the source memory based on the array structure information, the transport start position, and the transport mode specifically comprises: Based on the array structure information, the transport start position, and the transport mode, multiple consecutive read requests are generated.

[0125] 8. The calculation method for performing calculations during the transport of multidimensional arrays according to any one of items 1-3, characterized in that the upper limit of the array dimension of the multidimensional array supported by the method is five dimensions.

[0126] 9. The calculation method performed during the multidimensional array transport process as described in item 3, characterized in that the transport mode is that when the transformed data is transported, an image transposition algorithm is used to rearrange the multidimensional array before transporting it.

[0127] 10. The method for performing calculations during the transport of a multidimensional array according to any one of items 1-3, characterized in that the feedback method includes notifying the upper-level processing core through an interrupt signal and updating the status register for the upper-level processing core to poll and read.

[0128] 11. The method for performing calculations during multidimensional array transport according to any one of items 1-3, characterized in that the step of receiving the data descriptor and transport calculation descriptor of the multidimensional array issued by the superior processing core specifically comprises: It adopts a general interface design to receive data descriptors and transport calculation descriptors of multidimensional arrays issued by the upper-level processing core.

[0129] 12. A computational system for handling multidimensional arrays, characterized in that the system comprises: The receiving module is used to receive the data descriptors and transport calculation descriptors of the multidimensional arrays sent by the upper-level processing core; A data parsing module is used to parse the data descriptor to obtain array structure information; The parsing and calculation module is used to parse the transport calculation descriptor to obtain the transport start position, transport block size, transport mode, target address, calculation method and feedback method; A preprocessing unit is used to generate the read address of the source data block in the source memory based on the array structure information, the transport start position, and the transport mode; The system includes a read control unit and a calculation unit. The read control unit is used to read the original data block corresponding to the transport block size from the source memory according to the read address. The calculation unit is used to perform quantization or dequantization calculation on the original data block during the transport process according to the calculation method. A write data unit is used to write the calculated data into the target memory according to the target address; The post-processing unit is used to send the transfer and calculation completion information to the upper-level processing core according to the feedback method after the data writing is completed.

[0130] 13. The computation system for performing calculations during the multidimensional array transport process as described in item 12, characterized in that the array structure information includes array dimensions, the number of elements in each dimension, the amount of data in each dimension, and the size of the data block transported each time.

[0131] 14. The computation system for performing multidimensional array transport according to item 12, characterized in that the transport mode includes continuous data transport, block data transport, and transformed data transport.

[0132] 15. A computational system for handling multidimensional arrays according to any one of claims 12-14, characterized in that the computational unit comprises: A module used to determine the maximum floating-point exponent value in each data block; A module for performing exponent normalization on the floating-point exponent of each element in the data block; This module is used to convert normalized floating-point data into fixed-point or low-bit-width formats.

[0133] 16. The computational system for handling multidimensional arrays according to any one of claims 12-14, characterized in that the system further comprises: The descriptor cache unit is used to cache recently used data descriptors.

[0134] 17. A computation system for performing calculations during multidimensional array transport according to any one of items 12-14, characterized in that the system further includes a data cache unit, the computation unit is further configured to temporarily store intermediate results obtained from the calculation in the data cache unit, and the data writing unit is further configured to write the data to the target memory only after the corresponding data block has been calculated.

[0135] 18. A computational system for handling multidimensional arrays according to any one of claims 12-14, characterized in that the preprocessing unit further comprises: A module for generating multiple consecutive read requests based on the array structure information, the transport start position, and the transport mode.

[0136] 19. A computational system for handling multidimensional arrays according to any one of claims 12-14, characterized in that the system supports a maximum array dimension of five dimensions for multidimensional arrays.

[0137] 20. The computation system for performing calculations during the transport of a multidimensional array as described in item 14, characterized in that, when the transport mode is the transformation data transport, the computation unit is further configured to rearrange the multidimensional array using an image transposition algorithm before transporting it.

[0138] 21. A computational system for handling multidimensional arrays according to any one of claims 12-14, characterized in that the feedback method includes notifying the upper-level processing core via an interrupt signal and updating the status register for the upper-level processing core to poll and read.

[0139] 22. A computation system for performing computation during the transport of a multidimensional array according to any one of claims 12-14, characterized in that the receiving module is further configured to receive, using a general interface design, the data descriptor and transport computation descriptor of the multidimensional array issued by the upper-level processing core.

Claims

1. A method for performing calculations during the handling of multidimensional arrays, characterized in that, The method includes: Receives data descriptors and transport calculation descriptors for multidimensional arrays issued by the superior processing core; Parse the data descriptor to obtain array structure information; The transport calculation descriptor is parsed to obtain the transport start position, transport block size, transport mode, target address, calculation method, and feedback method; Based on the array structure information, the transport start position, and the transport mode, generate the read address of the source data block in the source memory; The original data block corresponding to the transport block size is read from the source memory according to the read address, and quantization or dequantization calculation is performed on the original data block during the transport process according to the calculation method. The calculated data is written into the target memory according to the target address; After the data is written, the transfer and calculation completion information is sent to the upper-level processing core according to the feedback method described.

2. The method for performing calculations during multidimensional array transport according to claim 1, characterized in that, The array structure information includes the array dimensions, the number of elements in each dimension, the amount of data in each dimension, and the size of the data block moved each time.

3. The method for performing calculations during multidimensional array transport according to claim 1, characterized in that, The data transfer modes include continuous data transfer, chunked data transfer, and transformed data transfer.

4. The method for performing calculations during multidimensional array transport according to any one of claims 1-3, characterized in that, The steps of reading the original data block corresponding to the transport block size from the source memory according to the read address, and performing quantization or dequantization calculations on the original data block during the transport process according to the calculation method, specifically include: Determine the maximum floating-point exponent value in each data block; Perform exponent normalization on the floating-point exponent of each element in the data block; Convert normalized floating-point data into fixed-point format or low-bit-width format.

5. The method for performing calculations during multidimensional array transport according to any one of claims 1-3, characterized in that, The method also includes caching recently used data descriptors.

6. The method for performing calculations during multidimensional array transport according to any one of claims 1-3, characterized in that, Before the step of writing the calculated data into the target memory according to the target address, the method further includes: The intermediate results obtained from the calculation are temporarily stored in the data cache unit until the corresponding data block is calculated and then written to the target memory.

7. The method for performing calculations during multidimensional array transport according to any one of claims 1-3, characterized in that, The specific steps for generating the read address of the source data block in the source memory based on the array structure information, the transport start position, and the transport mode are as follows: Based on the array structure information, the transport start position, and the transport mode, multiple consecutive read requests are generated.

8. The method for performing calculations during multidimensional array transport according to any one of claims 1-3, characterized in that, The method supports a maximum array dimension of five for multidimensional arrays.

9. The method for performing calculations during multidimensional array transport according to claim 3, characterized in that, The transport mode is that when transporting the transformed data, an image transposition algorithm is used to rearrange the multidimensional array before transporting it.

10. A computational system for handling multidimensional arrays, characterized in that, The system includes: The receiving module is used to receive the data descriptors and transport calculation descriptors of the multidimensional arrays sent by the upper-level processing core; A data parsing module is used to parse the data descriptor to obtain array structure information; The parsing and calculation module is used to parse the transport calculation descriptor to obtain the transport start position, transport block size, transport mode, target address, calculation method and feedback method; A preprocessing unit is used to generate the read address of the source data block in the source memory based on the array structure information, the transport start position, and the transport mode; The system includes a read control unit and a calculation unit. The read control unit is used to read the original data block corresponding to the transport block size from the source memory according to the read address. The calculation unit is used to perform quantization or dequantization calculation on the original data block during the transport process according to the calculation method. A write data unit is used to write the calculated data into the target memory according to the target address; The post-processing unit is used to send the transfer and calculation completion information to the upper-level processing core according to the feedback method after the data writing is completed.

Images