Data processing method, device, equipment, medium and program product for artificial intelligence calculation

Abstract

The present disclosure provides a data processing method, device, equipment, medium and program product for artificial intelligence calculation, relates to the technical field of artificial intelligence and data processing, and achieves the following scheme: in response to a data reading request, reading to-be-processed data; determining a to-be-transposed matrix generated by preprocessing the to-be-processed data; splitting the to-be-transposed matrix into a first sub-matrix; for each first sub-matrix, sequentially performing first to third steps: writing the first sub-matrix into a buffer memory; transposing the first sub-matrix in the buffer memory to obtain a second sub-matrix; in response to a data write-back request, based on a preset target storage strategy, writing the second sub-matrix from the buffer memory back to an external memory; integrating all the second sub-matrices written back to obtain an intermediate matrix result; and determining result data generated by post-processing the intermediate matrix result, which can significantly improve the system running performance and hardware processing efficiency in high-performance computing scenarios such as artificial intelligence calculation.

Description

Technical Field

[0001] This disclosure relates to the fields of artificial intelligence and data processing technology, particularly to matrix transposition and storage optimization, and especially to a data processing method, apparatus, computer equipment, computer-readable storage medium, and computer program product for artificial intelligence computing. Background Technology

[0002] In fields such as artificial intelligence, deep learning, neural network inference and training, big data analytics, and scientific computing, the efficiency of data processing directly determines the overall performance of the system. In modern computer architectures, data processing tasks are typically performed by a central processing unit (CPU), a graphics processing unit (GPU), or a dedicated AI accelerator. Artificial intelligence computing is characterized by large data volumes, high computational parallelism, and numerous process iterations, placing extremely high demands on the efficiency of underlying data handling and transformation. Its processing speed directly determines the overall system's throughput and latency. Summary of the Invention

[0003] This disclosure provides a data processing method, apparatus, computer device, computer-readable storage medium, and computer program product for artificial intelligence computing.

[0004] According to one aspect of this disclosure, a data processing method for artificial intelligence computing is provided, applied to a data processing system including an external memory and a buffer memory, the external memory and the buffer memory being connected via a data transmission bus. The method includes: in response to a data read request, reading data to be processed from the external memory, the data to be processed including at least one of visual data, text data, and voice data; determining a transpose matrix to be generated from the data to be processed after preprocessing; dividing the transpose matrix into at least one first submatrix, wherein all spatial dimensions of the first submatrix are identical and divisible by the spatial dimensions of the transpose matrix, and the data bit width of all first submatrixes is equal to the data bit width of the transpose matrix; and processing the data in the at least one first submatrix obtained by the division... For each first submatrix, steps one through three are executed sequentially: In the first step, the first submatrix is ​​written into a buffer memory used for buffering and transposing the matrix to be transposed, wherein the spatial dimension size and data bit width of each first submatrix are less than or equal to the corresponding spatial dimension size and data bit width supported by the buffer memory; In the second step, the first submatrix is ​​transposed in the buffer memory to obtain a second submatrix; In the third step, in response to a data write-back request, based on a preset target storage strategy, the transposed second submatrix is ​​written back from the buffer memory to external memory; All the second submatrixes written back are integrated to obtain an intermediate matrix result; The result data generated by post-processing the intermediate matrix result is determined, and the result data includes at least one of visual data, text data, and voice data.

[0005] In some embodiments, dividing the matrix to be transposed into at least one first submatrix includes: the total amount of data in each first submatrix is ​​greater than or equal to the bus bandwidth of the data transmission bus.

[0006] In some embodiments, in the first step, writing the first submatrix into a buffer memory for buffering and transposing the matrix to be transposed includes: writing the first submatrix into the buffer memory for buffering and transposing the matrix to be transposed in row-major order or column-major order.

[0007] In some embodiments, in the second step, transposing the first submatrix in the buffer memory to obtain the second submatrix includes: transposing the first submatrix in the buffer memory in the same row-major or column-major order as writing the first submatrix into the buffer memory to obtain the second submatrix.

[0008] In some embodiments, in the third step, in response to a data write-back request, the transposed second submatrix is ​​written back from the buffer memory to the external memory based on a preset target storage strategy. This includes: in response to a data write-back request, determining the physical storage address and storage format of the second submatrix to be written back in the external memory; and taking the second submatrix to be written back as a whole, transposed in row-major order or column-major order, and writing it back from the buffer memory to the external memory based on the determined physical storage address and storage format.

[0009] In some embodiments, the physical storage address of the second submatrix in the external memory is either a continuous address sequence or a non-contiguous address sequence.

[0010] According to another aspect of this disclosure, a data processing apparatus for artificial intelligence computing is provided, applied to a data processing system including an external memory and a buffer memory, the external memory and the buffer memory being connected via a data transmission bus, comprising: a reading module configured to read data to be processed from the external memory in response to a data reading request, the data to be processed including at least one of visual data, text data, and voice data; a transpose matrix determination module configured to determine a transpose matrix generated from the data to be processed through preprocessing; a segmentation module configured to segment the transpose matrix into at least one first submatrix, all first submatrixes having the same spatial dimension size that is divisible by the spatial dimension size of the transpose matrix, and all first submatrixes having a data bit width equal to the data bit width of the transpose matrix; and an execution module configured to sequentially execute a step for each of the at least one first submatrix obtained by segmentation. Steps one through three, the execution module includes: a first execution module configured to, in the first step, write a first submatrix into a buffer memory used for buffering and transposing the matrix to be transposed, wherein the spatial dimension size and data bit width of the first submatrix are all less than or equal to the corresponding spatial dimension size and data bit width supported by the buffer memory; a second execution module configured to, in the second step, transpose the first submatrix in the buffer memory to obtain a second submatrix; a third execution module configured to, in the third step, in response to a data write-back request, write the transposed second submatrix from the buffer memory back to external memory based on a preset target storage strategy; an integration module configured to integrate all the second submatrixes written back to obtain an intermediate matrix result; and a result data determination module configured to determine the result data generated by post-processing the intermediate matrix result, wherein the result data includes at least one of visual data, text data, and voice data.

[0011] According to another aspect of this disclosure, a computer device is provided, comprising: a processor; and a memory having a computer program stored thereon, wherein when executed by the processor, the computer program causes the processor to perform the methods provided above in this disclosure.

[0012] According to another aspect of this disclosure, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, causes the processor to perform the methods provided above in this disclosure.

[0013] According to another aspect of this disclosure, a computer program product is provided, including a computer program that, when executed by a processor, causes the processor to perform the methods provided above in this disclosure.

[0014] According to one or more embodiments of this disclosure, the system operating performance and hardware processing efficiency in high-performance computing scenarios such as artificial intelligence computing can be significantly improved.

[0015] These and other aspects of this disclosure will be apparent from the embodiments described below, and will be elucidated with reference to the embodiments described below. Attached Figure Description

[0016] The accompanying drawings exemplify embodiments and form part of the specification, serving together with the textual description to explain exemplary implementations of the embodiments. The illustrated embodiments are for illustrative purposes only and do not limit the scope of this disclosure. Throughout the drawings, the same reference numerals refer to similar but not necessarily identical elements.

[0017] Figure 1 This is a flowchart illustrating a data processing method for artificial intelligence computing according to an embodiment of the present disclosure.

[0018] Figure 2 This is a schematic diagram illustrating the data interaction process between external storage and buffer memory according to an embodiment of the present disclosure.

[0019] Figure 3 This is a block diagram illustrating a data processing apparatus for artificial intelligence computing according to an embodiment of the present disclosure.

[0020] Figure 4 An example computer device is shown in which any of the embodiments described herein may be implemented. Detailed Implementation

[0021] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0022] In this disclosure, unless otherwise stated, the use of terms such as "first," "second," etc., to describe various elements is not intended to limit the positional, temporal, or importance relationships of these elements; such terms are merely used to distinguish one element from another. In some examples, the first element and the second element may refer to the same instance of that element, while in other cases, based on the context, they may refer to different instances.

[0023] The terminology used in the description of the various examples described in this disclosure is for the purpose of describing particular examples only and is not intended to be limiting. Unless the context explicitly indicates otherwise, an element may be one or more unless the number of elements is specifically limited. As used herein, the term "multiple" means two or more, and the term "based on" should be interpreted as "at least partially based on". Furthermore, the terms "and / or" and "at least one of..." cover any one of the listed items and all possible combinations thereof.

[0024] In related technologies, matrix transposition is a high-frequency and critical data processing operation in fields such as artificial intelligence, big data processing, scientific computing, and deep learning, and its execution efficiency directly determines the overall system performance. Currently, large-scale matrix transposition mainly employs two implementation methods: block-based double-buffered transposition requires caching the entire matrix, resulting in huge storage overhead and difficulty in adapting to lightweight and high-density computing hardware; single-row streaming transposition processes data row by row, leading to extremely low bandwidth utilization and wasted hardware resources when the data volume and bus bandwidth are mismatched. While existing small-block transposition technologies can reduce storage footprint, their versatility and scenario adaptability are insufficient, failing to simultaneously address storage overhead, bandwidth utilization, and multi-format compatibility requirements, making it difficult to meet the efficient transposition needs of diverse high-performance computing architectures such as GPGPUs and AI accelerators. Therefore, finding a balance between reducing storage overhead, improving computational efficiency, and supporting flexible transposition in multiple formats has become a key bottleneck restricting data processing efficiency.

[0025] Therefore, embodiments of this disclosure provide a data processing method for artificial intelligence computing, which can significantly improve system performance and hardware processing efficiency in high-performance computing scenarios such as AI accelerators.

[0026] Figure 1 This is a flowchart illustrating a data processing method for artificial intelligence computing according to an embodiment of the present disclosure.

[0027] like Figure 1 As shown, the data processing method for artificial intelligence computing includes steps S101, S102, S103, S104, S105, and S106. Further, step S104 includes sub-steps S104a, S104b, and S104c.

[0028] In step S101, in response to a data read request, data to be processed is read from an external memory. The data to be processed includes at least one of visual data, text data, and voice data.

[0029] In the example, a data read request can be generated and issued by the internal hardware control module based on the current task, specifying the location and length of the data to be read to the external memory. The external memory can be an external storage unit used for large-scale data storage in the system, typically a large-capacity but relatively slow main storage space, such as a computer's DDR memory or a graphics card's video memory. The data to be processed can be raw input information, including, for example, visual data captured by a camera, voice data captured by a microphone, or text data input by the user, to meet the data processing needs of scenarios such as artificial intelligence, big data processing, scientific computing, and deep learning.

[0030] In the examples, visual data typically refers to image information composed of pixel arrays or continuous video frames, which are essentially high-dimensional matrix arrangements in computers; text data can be sequential information composed of characters, word vectors, or sentence embeddings, often used in natural language processing tasks; speech data mainly refers to sampled and digitized sound wave signals, exhibiting temporal amplitude variations. Although only these three typical data forms are listed here, the data reading and processing framework constructed in this solution is not limited to them. Those skilled in the art should understand that any other type of data that can be converted into a digital matrix form and requires efficient computation, such as time-series data acquired by sensors, analog data in scientific computing, graph structure data in graph neural networks, and point cloud data acquired by lidar, are all within the scope of protection intended to be covered by this application.

[0031] In step S102, the transpose matrix to be generated from the data to be processed is determined.

[0032] In the example, since the data to be processed is raw input information obtained by the system from external sources, its format, length, and arrangement are usually inconsistent, making it unsuitable for direct use in regularized hardware operations such as matrix transposition. Therefore, preprocessing operations can be used to organize, align, and regularize the dimensions of the messy, unstructured raw data, transforming it into a two-dimensional or multi-dimensional data structure that conforms to hardware processing specifications, ensuring orderly and efficient data reading, writing, and transmission. The matrix to be transposed can then be a data set with defined spatial dimensions formed after the aforementioned preprocessing, and can be directly used as the object of the transposition operation.

[0033] In the example, the process of determining the matrix to be transposed can be widely applied to hardware scenarios requiring high-frequency, high-volume data processing, such as AI accelerators, GPGPUs, and edge computing chips. In these scenarios, the raw data sources are diverse and the formats vary, necessitating preprocessing to form a unified matrix to support high-throughput, low-latency hardware processing flows. For instance, in image data processing scenarios, the raw pixel data stream can be preprocessed through alignment, padding, and dimension integration to generate a matrix to be transposed with fixed dimensions. Similarly, in text or feature data processing scenarios, sequential data can be normalized into matrix form so that the hardware can quickly perform the transpose operation. It is understood that all matrices referred to in this paper can be two-dimensional or multi-dimensional, and this solution does not impose any restrictions on this.

[0034] In step S103, the matrix to be transposed is divided into at least one first submatrix, wherein the spatial dimensions of all the first submatrixes are the same and can be divided by the spatial dimensions of the matrix to be transposed, and the data bit width of all the first submatrixes is equal to the data bit width of the matrix to be transposed.

[0035] In the example, the matrix to be transposed can be a data set with fixed spatial dimensions and data bit width, formed by preprocessing the data to be processed, and can be used as the object of the transpose operation. Segmentation refers to dividing the matrix to be transposed into several independent smaller matrices, i.e., the first submatrices, according to predetermined rules to reduce the hardware's processing load per operation. The spatial dimensions of the first submatrices need to be consistent and able to completely divide the corresponding dimensions of the matrix to be transposed, ensuring no redundant data or size residue after segmentation, facilitating hardware read and write operations. Simultaneously, the data bit width of the first submatrices can be consistent with that of the matrix to be transposed to avoid format conversion overhead caused by bit width changes, thereby maintaining consistency in data transmission and processing. For example, when the spatial dimension of the matrix to be transposed is 64 rows × 16 columns, and the data bit width of each data element in the matrix to be transposed is 1 byte, it can be segmented into a first submatric with a spatial dimension of 16 rows × 16 columns. At this point, the spatial dimension size (16×16) of each first submatrix is ​​divisible by the corresponding spatial dimension size of the matrix to be transposed (16 is divisible by 64, and 16 is divisible by 16), and the data bit width is consistent with that of the matrix to be transposed (both are 1 byte), which facilitates efficient reading and processing by the hardware.

[0036] In step S104, for each of the first submatrices in the at least one first submatrix obtained by the segmentation, the above substeps S104a, S104b and S104c are executed sequentially.

[0037] In sub-step S104a, the first submatrix is ​​written into a buffer memory used for buffering and transposing the matrix to be transposed. The spatial dimension size and data bit width of the first submatrix are all less than or equal to the corresponding spatial dimension size and data bit width supported by the buffer memory.

[0038] In the example, the buffer memory can be a high-speed storage unit within the system used to temporarily store data and synchronously complete the transpose operation. It can provide temporary transfer space for data and support transpose functionality, serving as a carrier that balances data buffering and format conversion. Sequential execution refers to processing each first submatrix obtained from the segmentation separately according to a preset order, i.e., completing one before processing the next, to avoid hardware resource conflicts caused by multiple matrices flooding in simultaneously, ensuring a stable and reliable transpose process. To ensure that the writing process of the first submatrix to the buffer memory is smooth and does not exceed the hardware's capacity, it can be required that the spatial dimension size and data bit width of the first submatrix do not exceed the corresponding upper limit supported by the buffer memory. This writing rule can be applied to high-performance data processing scenarios such as AI accelerators, not only reducing hardware control complexity but also improving data writing and buffering efficiency, reducing latency and resource waste, and significantly improving the processing speed of computer hardware and the overall system performance.

[0039] In substep S104b, the first submatrix is ​​transposed in the buffer memory to obtain the second submatrix.

[0040] In substep S104c, in response to the data write-back request, the transposed second submatrix is ​​written back from the buffer memory to the external memory based on the preset target storage strategy.

[0041] In the example, the first submatrix is ​​transposed in the buffer memory. For instance, this could be achieved by internally swapping the row and column positions of the first submatrix using the high-speed read / write capabilities of the buffer memory, resulting in a second submatrix with swapped dimensions. The data write-back request can be an output command generated by the hardware control unit after the transposition is complete, used to initiate the outward transmission of the result data, ensuring a closed-loop processing flow. The preset target storage strategy can be a pre-defined storage address rule, arrangement method, and storage format requirement, used to standardize the storage location and organization of the transposed data, ensuring that the output results can be directly used by subsequent modules, avoiding further format adjustments.

[0042] In step S105, all the second sub-matrices written back are integrated to obtain the intermediate matrix result.

[0043] In step S106, the result data generated by post-processing the intermediate matrix results is determined, and the result data includes at least one of visual data, text data, and voice data.

[0044] In the example, integrating all the second sub-matrices written back can be done by concatenating, rearranging, or combining multiple transposed second sub-matrices that have been processed and written back to external memory in a pre-defined order and positional relationship to restore the complete intermediate matrix result. Post-processing can then perform operations such as format restoration, dimension transformation, and data reorganization on the intermediate matrix result, transforming it from a structured matrix back into a data type directly usable in the business scenario. The resulting data can be the final data output from the post-processing, and can include at least one of visual data, text data, and speech data. Visual data can include data forms used for visual perception processing such as images, videos, feature maps, and point cloud data; text data can include data forms used for natural language processing such as sentences and character sequences; and speech data can include data forms used for voice interaction processing such as audio waveforms and speech features. The above data types are merely examples and not exhaustive; the specific types of resulting data are not limited by the list in this solution.

[0045] Therefore, the data processing method for artificial intelligence computing according to the embodiments of this disclosure generates a transpose matrix by preprocessing the data to be processed and dividing it into a first sub-matrix with consistent spatial dimensions and bit width. This allows the data structure to match the hardware processing capabilities, reducing control complexity and resource overhead. Performing write, transpose, and write-back operations on the first sub-matrix one by one fully utilizes the high-speed processing capabilities of the buffer memory, avoiding data overflow and transmission conflicts, and improving transpose execution efficiency and bus utilization. By integrating all the second sub-matrixes to obtain the intermediate matrix result, and post-processing to output multiple types of result data such as visual, text, and speech data, a complete data processing flow can be formed. The entire solution can be widely applied to high-performance computing scenarios such as AI accelerators and GPGPUs, effectively improving the continuity, standardization, and throughput of data processing, and reducing hardware latency and resource consumption. Furthermore, it effectively balances the problems of low bandwidth utilization and high storage overhead in the matrix transpose process, thereby significantly improving system performance and hardware processing efficiency in high-performance computing scenarios such as artificial intelligence computing.

[0046] Figure 2 This is a schematic diagram illustrating the data interaction process between external storage and buffer memory according to an embodiment of the present disclosure.

[0047] like Figure 2As shown, in some examples, the data to be processed can first be retrieved from external memory (e.g., DDR memory) and preprocessed to generate a transpose matrix. Then, the transpose matrix can be divided into multiple first sub-matrices with consistent spatial dimensions and data bit widths, and these first sub-matrices are sequentially written into a buffer memory. Inside the buffer memory, the first sub-matrices can be transposed to generate corresponding second sub-matrices. After transposition, the second sub-matrices can be written back to external memory for use by subsequent computational units or to save results. In this scheme, intermediate matrix results can also be obtained by integrating all the written-back second sub-matrices, and then post-processed to generate the final result data, forming a complete data flow chain adapted to efficient hardware processing.

[0048] The following describes in further detail various aspects of the data processing method for artificial intelligence computing according to embodiments of the present disclosure.

[0049] In some embodiments, dividing the matrix to be transposed into at least one first submatrix may include: the total amount of data in each first submatrix is ​​greater than or equal to the bus bandwidth of the data transmission bus.

[0050] In the example, the bus bandwidth of the data transmission bus refers to the maximum amount of data that a channel in the hardware system can transmit in a single transmission, and it is a core hardware parameter that determines data transmission efficiency. To fully utilize the transmission capacity of the data transmission bus, the total data volume of each first submatrix can be set to be no less than the bus bandwidth. This means that the amount of data transmitted each time should just fill the bus's single transmission limit, thus avoiding idle bus resources due to insufficient data. For example, when the maximum single transmission volume of the data transmission bus is 128 bytes, the total data volume of each first submatrix obtained by segmentation can be no less than 128 bytes to ensure that each transmission reaches or approaches the bus's transmission limit, avoiding resource waste.

[0051] Therefore, by dividing the matrix to be transposed into a first sub-matrix whose total data volume is adapted to the bus bandwidth, the transmission capacity of the bus can be fully utilized for each data transmission, avoiding bandwidth idleness due to insufficient data volume and significantly improving data transmission efficiency.

[0052] In some embodiments, writing the first submatrix into a buffer memory for buffering and transposing the matrix to be transposed in the first step may include writing the first submatrix into the buffer memory for buffering and transposing the matrix to be transposed in row-major order or column-major order.

[0053] In the example, the buffer memory's read / write method can adapt to different data arrangement rules. When the total amount of data in the first submatrix exceeds the single transmission limit of the data transmission bus, a single transmission may not be able to completely write the entire first submatrix. Therefore, it can be transmitted in batches according to a preset order. Row priority and column priority are two typical transmission order methods. Row priority means that data is written sequentially in a "store one row first, then the next" manner, while column priority means that data is written sequentially in a "store one column first, then the next" manner. These two orders are the most common data storage forms in hardware; they are not exhaustive but merely illustrate the design concept of this solution supporting writing in a specific order.

[0054] Therefore, by writing the first submatrix sequentially into the buffer memory while ensuring that the amount of data transmitted in each transmission fills the bus's single transmission limit as much as possible, the continuity and orderliness of data transmission can be guaranteed, the bus bandwidth can be fully utilized, resource idleness caused by insufficient data can be avoided, and the data processing efficiency of computer hardware and the overall system performance can be effectively improved.

[0055] In some embodiments, the second step of transposing the first submatrix in the buffer memory to obtain the second submatrix may include: transposing the first submatrix in the buffer memory in the same row-major or column-major order as writing the first submatrix into the buffer memory to obtain the second submatrix.

[0056] Therefore, by performing transpose in the same order as when writing, it is possible to ensure that the storage location of data in the buffer memory is consistent with the reading rules, avoiding problems such as data corruption and complex address jumps caused by mismatch in order, without the need to add additional format conversion logic, thus simplifying hardware control to the greatest extent.

[0057] In some embodiments, in the third step, in response to a data write-back request, the transposed second submatrix is ​​written back from the buffer memory to the external memory based on a preset target storage strategy. This may include: in response to a data write-back request, determining the physical storage address and storage format of the second submatrix to be written back in the external memory; and taking the second submatrix to be written back as a whole, transposed in row-major or column-major order, and writing it back from the buffer memory to the external memory based on the determined physical storage address and storage format.

[0058] In the example, the preset target storage strategy can be a pre-defined storage rule used to standardize the storage location and organization of the transposed data, ensuring that the output results can be directly used by subsequent modules. Determining the physical storage address of the second submatrix to be written back in external memory can be done based on matrix partitioning rules and concatenation logic, or on the subsequent uses and operations of the transposed matrix. The specific hardware address where the second submatrix should be stored must be calculated, ensuring that multiple transposed second submatrices can be concatenated into a complete intermediate matrix result. The storage format can be used to define the write-back method of the second submatrix when writing it back to external memory. By writing back the second submatrix as a whole, the accurate storage location, standardized format, and structural integrity of the transposed data can be guaranteed, avoiding data misalignment and transmission fragmentation.

[0059] Therefore, by determining the physical storage address and storage format, and writing back the second submatrix as a whole, the continuity and reliability of data write-back can be improved, the overhead of address calculation and format conversion can be reduced, and the data processing stability of computer hardware and the overall system performance can be further enhanced.

[0060] In some embodiments, the physical storage address of the second submatrix in the external memory is either a continuous address sequence or a non-contiguous address sequence.

[0061] In the example, a contiguous address sequence can refer to the transposed second submatrices being stored sequentially in external memory according to their address order, with their storage addresses being continuous and without gaps or jumps; a non-contiguous address sequence can refer to the transposed second submatrices being stored scattered at specified address locations, with gaps or non-contiguous arrangements between their storage addresses. Both address sequence methods can be flexibly selected according to the system's storage resource allocation status. This solution does not limit the address sequence type, aiming to improve compatibility with different storage states of external memory.

[0062] Therefore, by allowing the second submatrix to be stored in external memory using contiguous or non-contiguous address sequences, the flexibility and compatibility of data storage can be greatly improved, effectively adapting to the storage space allocation states of different hardware, enhancing the stability and robustness of data processing flows, reducing the impact of hardware storage resource limitations on data processing efficiency, and thus improving the overall operating performance and reliability of the computer system.

[0063] According to embodiments of this disclosure, a data processing apparatus for artificial intelligence computing is also provided.

[0064] Figure 3 This is a block diagram illustrating a data processing apparatus for artificial intelligence computing according to an embodiment of the present disclosure.

[0065] like Figure 3As shown, in some embodiments, the data processing device 300 for artificial intelligence calculations is applied to a data processing system including external memory and buffer memory connected via a data transmission bus. The device includes a reading module 301, a matrix to be transposed determination module 302, a segmentation module 303, an execution module 304, an integration module 305, and a result data determination module 306. Furthermore, the execution module 304 includes a first execution module 304a, a second execution module 304b, and a third execution module 304c.

[0066] The reading module 301 is configured to read data to be processed from an external memory in response to a data reading request. The data to be processed includes at least one of visual data, text data, and voice data.

[0067] The transpose matrix determination module 302 is configured to: determine the transpose matrix generated from the data to be processed after preprocessing.

[0068] The segmentation module 303 is configured to: segment the matrix to be transposed into at least one first submatrix, wherein the spatial dimensions of all first submatrixes are the same and can be divided by the spatial dimensions of the matrix to be transposed, and the data bit width of all first submatrixes is equal to the data bit width of the matrix to be transposed.

[0069] The execution module 304 is configured to execute the first step to the third step sequentially for each of the first submatrices in at least one first submatrix obtained by the splitting.

[0070] The first execution module 304a is configured to: in the first step, write the first submatrix into a buffer memory for buffering and transposing the matrix to be transposed, wherein the spatial dimension size and data bit width of the first submatrix are all less than or equal to the corresponding spatial dimension size and data bit width supported by the buffer memory.

[0071] The second execution module 304b is configured to: in the second step, transpose the first submatrix in the buffer memory to obtain the second submatrix.

[0072] The third execution module 304c is configured to: in the third step, in response to a data write-back request, write the transposed second submatrix from the buffer memory back to the external memory based on a preset target storage strategy.

[0073] The integration module 305 is configured to integrate all the second submatrices written back to obtain the intermediate matrix result.

[0074] The result data determination module 306 is configured to: determine the result data generated by post-processing the intermediate matrix results, wherein the result data includes at least one of visual data, text data, and voice data.

[0075] The operations of the aforementioned reading module 301, transpose matrix determination module 302, segmentation module 303, execution module 304, integration module 305, and result data determination module 306 can be combined. Figure 1 The operations of steps S101-S106 are the same, and the operations of the first execution module 304a, the second execution module 304b, and the third execution module 304c can be combined. Figure 1 The sub-steps S104a-S104c in step S104 are the same, so the details of each aspect will not be repeated here.

[0076] In some embodiments, the first execution module 304a further includes a writing module 304a-1, configured to write the first submatrix into a buffer memory for buffering and transposing the matrix to be transposed, in row-major order or column-major order.

[0077] In some embodiments, the second execution module 304b further includes a transpose module 304b-1, configured to transpose the first submatrix in the buffer memory to obtain the second submatrix in the same row-matrix order or column-matrix order as when the first submatrix is ​​written to the buffer memory.

[0078] In some embodiments, the third execution module 304c further includes a determination module 304c-1 and a write-back module 304c-2.

[0079] The determination module 304c-1 is configured to: in response to a data write-back request, determine the physical storage address and storage format of the second submatrix currently to be written back in external memory.

[0080] The write-back module 304c-2 is configured to take the second submatrix obtained by transposing the current data to be written back in row-major or column-major order as a whole, and write it back to the external memory from the buffer memory based on the determined physical storage address and storage format.

[0081] According to one aspect of this disclosure, a computer device is also provided, including a memory, a processor, and a computer program stored in the memory. The processor is configured to execute the computer program to implement the steps of any of the method embodiments described above.

[0082] According to one aspect of this disclosure, a non-transitory computer-readable storage medium is also provided, on which a computer program is stored, which, when executed by a processor, implements the steps of any of the method embodiments described above.

[0083] According to one aspect of this disclosure, a computer program product is also provided, which includes a computer program that, when executed by a processor, implements the steps of any of the method embodiments described above.

[0084] Figure 4 An example computer device 400 is shown in which any of the embodiments described herein may be implemented. Computer device 400 may be used to implement one or more components of the systems and methods described above. Computer device 400 may include a bus 402 or other communication mechanism for communicating information, and one or more processors 404 coupled to the bus 402 for processing information. Processor 404 may be, for example, one or more general-purpose microprocessors.

[0085] Computer device 400 may also include main memory 406, such as random access memory (RAM), cache, and / or other dynamic storage devices, coupled to bus 402, for storing information and instructions to be executed by processor 404. Main memory 406 may also be used to store temporary variables or other intermediate information during the execution of instructions to be executed by processor 404. Such instructions, when stored in a storage medium accessible to processor 404, can make computer device 400 a special-purpose machine customized to perform the operations specified in the instructions. Main memory 406 may include non-volatile media and / or volatile media. Non-volatile media may include, for example, optical discs or magnetic disks. Volatile media may include dynamic memory. Common media formats may include, for example, floppy disks, collapsible disks, hard disks, solid-state drives, magnetic tapes or any other magnetic data storage media, CD-ROMs (read-only optical disc drives), any other optical data storage media, any physical media with a perforated arrangement, RAM (random access memory), DRAM (dynamic random access memory), PROM (programmable read-only memory) and EPROM (erasable programmable read-only memory), FLASH-EPROM (fast erase programmable read-only memory), NVRAM (non-volatile random access memory), any other memory chips or tape cartridges, or network versions of the above.

[0086] Computer device 400 may implement the techniques described herein using custom hardwired logic, one or more ASICs (Application-Specific Integrated Circuits) or FPGAs (Field-Programmable Gate Arrays), firmware, and / or program logic, which, when combined with computer device 400, enable computer device 400 to become a special-purpose machine or to be programmed therein. According to one embodiment, the techniques described herein are executed by computer device 400 in response to processor 404 executing one or more sequences of one or more instructions contained in main memory 406. Such instructions may be read into main memory 406 from another storage medium, such as storage device 408. Executing the sequence of instructions contained in main memory 406 causes processor 404 to perform the processing steps described herein. For example, the processes / methods disclosed herein may be implemented by computer program instructions stored in main memory 406. When these instructions are executed by processor 404, they may perform the steps shown in the corresponding figures and as described above. In alternative embodiments, hardwired circuitry may be used in place of or in combination with software instructions.

[0087] Computer device 400 also includes a network interface 410 coupled to bus 402. Network interface 410 can provide bidirectional data communication coupled to one or more network links connected to one or more networks. As another example, network interface 410 can be a local area network (LAN) card to provide data communication connectivity with a compatible LAN (or a WAN component communicating with a WAN (wide area network)). Wireless links can also be implemented.

[0088] The performance of certain operations can be distributed across processors, not just residing within a single machine, but deployed across many machines. In some exemplary embodiments, the processor or the processor-implemented engine may reside in a single geographic location (e.g., in a home environment, office environment, or server farm). In other exemplary embodiments, the processor or the processor-implemented engine may be distributed across many geographic locations.

[0089] Each process, method, and algorithm described in the preceding sections can be embodied in a code module executed by one or more computer systems or computer processors including computer hardware, and can be fully or partially automated by them. These processes and algorithms can be implemented, in part or in whole, in a specific application circuit.

[0090] When the functions disclosed herein are implemented as software functional units and sold or used as independent products, they can be stored in a processor-executable, non-volatile, computer-readable storage medium. Specific technical solutions (all or part) disclosed herein, or aspects contributing to the prior art, can be embodied in the form of a software product. This software product can be stored in a storage medium and includes instructions to cause a computer device (which may be a personal computer, server, network device, etc.) to perform all or part of the steps of the methods described in the embodiments of this application. The storage medium may include a flash drive, a portable hard drive, ROM, RAM, a magnetic disk, an optical disk, another medium suitable for storing program code, or any combination thereof.

[0091] The embodiments disclosed herein can be implemented via a cloud platform, server, or group of servers that interact with a client. The client can be a terminal device or a client registered by a user on the platform, wherein the terminal device can be a mobile terminal, a personal computer (PC), or any device that can install platform applications.

[0092] The various features and processes described above can be used independently or combined in various ways. All possible combinations and sub-combinations are intended to fall within the scope of this disclosure. Furthermore, certain method or process blocks may be omitted in some embodiments. The methods and processes described herein are not limited to any particular order, and associated blocks or states may be executed in other suitable orders. For example, described blocks or states may be executed in a non-specifically disclosed order, or multiple blocks or states may be combined in a single block or state. Exemplary blocks or states may be executed serially, in parallel, or otherwise. Blocks or states may be added to or removed from the disclosed exemplary embodiments. The exemplary systems and components described herein may be configured differently from those described. For example, elements may be added, removed, or rearranged compared to the disclosed exemplary embodiments.

[0093] The various operations of the exemplary methods described herein can be performed at least in part by an algorithm. An algorithm may consist of program code or instructions stored in memory (such as the non-transitory computer-readable storage medium described above). Such an algorithm may include a machine learning algorithm. In some embodiments, the machine learning algorithm may not be explicitly programmed into the computer to perform the function, but may learn from training data to obtain a predictive model for performing that function.

[0094] The various operations of the exemplary methods described herein can be performed at least in part by one or more processors, which are temporarily configured (e.g., by software) or permanently configured to perform the relevant operations. Whether temporarily or permanently configured, such processors can constitute the engine of a processor implementation whose operation is to perform one or more of the operations or functions described herein.

[0095] Similarly, the methods described herein can be implemented at least partially by a processor, where a specific processor or one or more processors are examples of hardware. For example, at least some operations of the methods can be performed by one or more processors or an engine implemented by a processor. Furthermore, one or more processors can also run in a “cloud computing” environment or as “Software as a Service” (SaaS) to support the execution of the relevant operations. For example, at least some operations can be performed by a group of computers (as an example of a machine including processors), which can be accessed via a network (e.g., the Internet) and through one or more appropriate interfaces (e.g., application programming interfaces (APIs)).

[0096] The performance of certain operations can be distributed across processors, not just residing within a single machine, but deployed across many machines. In some exemplary embodiments, the processor or the processor-implemented engine may reside in a single geographic location (e.g., in a home environment, office environment, or server farm). In other exemplary embodiments, the processor or the processor-implemented engine may be distributed across many geographic locations.

[0097] In this specification, multiple instances may implement components, operations, or structures described as a single instance. Although individual operations of one or more methods are described and illustrated as independent operations, one or more individual operations may be performed concurrently, and these operations are not required to be performed in the order shown. Structures and functionalities presented as independent components in the example configuration may be implemented as combined structures or components. Similarly, structures and functionalities presented as individual components may be implemented as independent components. These and other variations, modifications, additions, and improvements are all within the scope of this document.

[0098] As used herein, “or” is inclusive rather than exclusive unless explicitly stated or indicated by context. Furthermore, “and” is both common and individual unless explicitly stated or indicated by context. Moreover, multiple instances may be provided for the resources, operations, or structures described herein as a single example. Furthermore, the boundaries between various resources, operations, engines, and data stores are somewhat arbitrary, and specific operations are illustrated within the context of a particular illustrative configuration. The allocation of other functionalities is conceivable and may fall within the scope of various embodiments of this disclosure. Generally, structures and functionalities presented as independent resources in example configurations may be implemented as combined structures or resources. Similarly, structures and functionalities presented as individual resources may be implemented as independent resources. These and other variations, modifications, additions, and improvements are all within the scope of embodiments of this disclosure. Therefore, this specification and accompanying drawings should be viewed in an illustrative rather than restrictive sense.

[0099] The terms “comprising” or “including” are used to indicate the presence of a subsequently stated feature, but do not preclude the addition of other features. Conditional language, in particular, such as “may,” “can,” or “may,” unless specifically stated or otherwise understood in the context of use, is generally intended to express that certain embodiments include certain features, elements, and / or steps, while other embodiments do not. Therefore, such conditional language generally does not imply that a feature, element, and / or step is necessary in any way for one or more embodiments, or that one or more embodiments must include logic that, with or without user input or prompting, determines whether such features, elements, and / or steps are included in any particular embodiment, or whether they are to be performed in any particular embodiment.

Claims

1. A data processing method for artificial intelligence computing, characterized in that, A method applicable to a data processing system comprising external memory and buffer memory, wherein the external memory and the buffer memory are connected via a data transmission bus, the method comprising: In response to a data read request, data to be processed is read from an external memory, the data to be processed including at least one of visual data, text data, and voice data; Determine the transpose matrix generated from the preprocessed data to be processed; The matrix to be transposed is divided into at least one first submatrix, wherein the total data volume of each first submatrix is ​​greater than or equal to the bus bandwidth of the data transmission bus, all the spatial dimensions of the first submatrix are the same and can be divided by the spatial dimensions of the matrix to be transposed, and the data bit width of all the first submatrix is ​​equal to the data bit width of the matrix to be transposed. For each of the at least one first submatrix obtained by the segmentation, the first to third steps are executed sequentially: In the first step, the first submatrix is ​​written into the buffer memory used for buffering and transposing the matrix to be transposed in row-major order or column-major order, wherein the spatial dimension size and data bit width of the first submatrix are all less than or equal to the corresponding spatial dimension size and data bit width supported by the buffer memory. In the second step, the first submatrix is ​​transposed in the buffer memory to obtain the second submatrix; In the third step, in response to the data write-back request, the transposed second sub-matrix is ​​written back from the buffer memory to the external memory based on the preset target storage strategy. Integrate all the second sub-matrices written back to obtain the intermediate matrix result; The result data generated by post-processing the intermediate matrix results is determined, and the result data includes at least one of visual data, text data, and voice data.

2. The method according to claim 1, characterized in that, In the second step, transposing the first submatrix in the buffer memory to obtain the second submatrix includes: The first submatrix is ​​transposed in the buffer memory to obtain the second submatrix, following the same row-matrix or column-matrix order as when the first submatrix was written into the buffer memory.

3. The method according to claim 2, characterized in that, In the third step, in response to a data write-back request, based on a preset target storage strategy, the transposed second sub-matrix is ​​written back from the buffer memory to the external memory, including: In response to a data write-back request, determine the physical storage address and storage format of the second submatrix currently to be written back in the external memory; The second submatrix, obtained by transposing the current data to be written back in row-major or column-major order, is treated as a whole and written back from the buffer memory to the external memory based on the determined physical storage address and storage format.

4. The method according to claim 3, characterized in that, The physical storage address of the second submatrix in the external memory is either a continuous address sequence or a non-contiguous address sequence.

5. A data processing device for artificial intelligence calculations, characterized in that, Applied to a data processing system including external memory and buffer memory, wherein the external memory and the buffer memory are connected via a data transmission bus, the device includes: The reading module is configured to read data to be processed from an external memory in response to a data reading request, the data to be processed including at least one of visual data, text data, and voice data; The transpose matrix determination module is configured to determine the transpose matrix generated from the preprocessed data to be processed; The segmentation module is configured to segment the matrix to be transposed into at least one first submatrix, wherein the total data volume of each first submatrix in the at least one first submatrix is ​​greater than or equal to the bus bandwidth of the data transmission bus, all the spatial dimensions of the first submatrix are the same and can be divided by the spatial dimensions of the matrix to be transposed, and the data bit width of all the first submatrix is ​​equal to the data bit width of the matrix to be transposed. An execution module is configured to sequentially execute steps one through three for each of the at least one first submatrix obtained from the segmentation, the execution module comprising: The first execution module is configured to, in the first step, write the first submatrix into the buffer memory used for buffering and transposing the matrix to be transposed in row-major order or column-major order, wherein the spatial dimension size and data bit width of the first submatrix are all less than or equal to the corresponding spatial dimension size and data bit width supported by the buffer memory. The second execution module is configured to, in the second step, transpose the first submatrix in the buffer memory to obtain the second submatrix. The third execution module is configured to, in the third step, respond to a data write-back request and, based on a preset target storage strategy, write the transposed second sub-matrix from the buffer memory back to the external memory. The integration module is configured to integrate all the second sub-matrices written back to obtain the intermediate matrix result; The result data determination module is configured to determine the result data generated by post-processing the intermediate matrix results, wherein the result data includes at least one of visual data, text data, and voice data.

6. A computer device, characterized in that, The computer device includes: At least one processor; A memory having a computer program stored thereon, wherein, when executed by the at least one processor, the computer program causes the at least one processor to perform the method of any one of claims 1-4.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, causes the processor to perform the method of any one of claims 1-4.

8. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, causes the processor to perform the method of any one of claims 1-4.

Images