Data processing method, processing unit, processor, system on chip, and computing device

Abstract

Provided in the embodiments of the present disclosure is a data processing method. The data processing method is applied to a processing unit of a system on a chip, wherein the processing unit comprises at least one matrix operation unit. The method comprises: receiving a service data processing request, wherein the service data processing request carries a service matrix parameter; acquiring service data to be processed, and on the basis of the service matrix parameter, determining a target data processing rule in a preset data processing rule library; splitting said service data on the basis of the target data processing rule, so as to obtain at least one piece of service sub-data to be processed, and determining a matrix operation unit group corresponding to each piece of service sub-data to be processed; and on the basis of each matrix operation unit group, processing each piece of service sub-data to be processed, so as to obtain a service processing sub-result corresponding to each piece of service sub-data to be processed, and on the basis of each service processing sub-result, determining a service processing result corresponding to the service data processing request. By means of the present method, the resource utilization rate of a processing unit of a system on a chip can be improved when matrix calculation is performed on service sub-data to be processed.

Description

Data processing method, processing unit, processor, system on chip and computing device

[0001] This disclosure claims priority to the Chinese patent application filed with the China Patent Office on March 4, 2024, with application number 202410246252.6 and application name “Data processing method, processing unit, processor, system on chip and a computing device”, the entire contents of which are incorporated by reference into this disclosure. Technical Field

[0002] The embodiments of the present disclosure relate to the field of computer technology, and in particular to a data processing method. Background Art

[0003] With the rapid development of artificial intelligence technologies such as large language models (LLM) and generative AI (Artificial Intelligence), artificial intelligence models have an increasingly high demand for computing power. In the model reasoning process, matrix multiplication (GEMM) operations play a vital role. In the current reasoning hardware for large language models, the computing dimension of the matrix multiplication accelerator cannot fully utilize the computing resources of the reasoning hardware card, resulting in a waste of computing resources for large language models, slow reasoning speed of large language models, and a decline in service indicators. Therefore, how to improve the resource utilization of reasoning hardware has become an urgent problem for technical personnel to solve. Summary of the Invention

[0004] In view of this, embodiments of the present disclosure provide a data processing method. One or more embodiments of the present disclosure simultaneously relate to a processing unit, a processor, a system on a chip, and a computing device to address technical deficiencies in the prior art.

[0005] According to a first aspect of an embodiment of the present disclosure, a data processing method is provided, which is applied to a processing unit of a system on chip, wherein the processing unit includes at least one matrix operation unit, including:

[0006] receiving a service data processing request, wherein the service data processing request carries a service matrix parameter;

[0007] Acquire the business data to be processed according to the business data processing request, and determine a target data processing rule for the business data to be processed in a preset data processing rule base according to the business matrix parameters, wherein the preset data processing rule base is determined based on a processing unit specification of the processing unit;

[0008] Splitting the to-be-processed business data according to the target data processing rule to obtain at least one to-be-processed business sub-data, and determining a matrix operation unit group corresponding to each to-be-processed business sub-data;

[0009] Based on each matrix operation unit group, each to-be-processed business sub-data is processed to obtain a business processing sub-result corresponding to each to-be-processed business sub-data, and a business processing result corresponding to the business data processing request is determined according to each business processing sub-result.

[0010] According to a second aspect of an embodiment of the present disclosure, a processing unit is provided, which includes at least one matrix operation unit, and the processing unit stores computer-executable instructions, which can implement the steps of the above-mentioned data processing method when executed by a processor.

[0011] According to the third aspect of an embodiment of the present disclosure, a processor is provided, which includes at least one processing unit, the processing unit includes at least one matrix operation unit, and the processor is used to execute computer-executable instructions, which, when executed by the processor, implement the steps of the above-mentioned data processing method.

[0012] According to a fourth aspect of an embodiment of the present disclosure, a system on chip is provided, including a processor and a memory, wherein the processor includes a single-core processor or a multi-core processor, and the processor and the memory are coupled via an on-chip network;

[0013] A memory configured to store business data to be processed;

[0014] The processor includes at least one processing unit, the processing unit includes at least one matrix operation unit, and the processor is used to execute computer-executable instructions. When the computer-executable instructions are executed by the processor, the steps of the above-mentioned data processing method are implemented.

[0015] According to a fifth aspect of an embodiment of the present disclosure, a computing device is provided, which includes one or more memories storing computer-executable instructions and at least one processor, the processor including at least one processing unit, the processing unit including at least one matrix operation unit, the processor being used to execute computer-executable instructions, which, when executed by the processor, implement the steps of the above-mentioned data processing method.

[0016] According to a sixth aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided, in which computer-executable instructions are stored. When the computer-executable instructions are executed by a processor, they are used to implement the steps of the above-mentioned data processing method.

[0017] According to a seventh aspect of an embodiment of the present disclosure, a computer program product is provided, comprising a computer program, which implements the steps of the above-mentioned data processing method when executed by a processor.

[0018] A data processing method provided by an embodiment of the present disclosure is applied to a processing unit of a system on chip, wherein the processing unit includes at least one matrix operation unit, and comprises: receiving a business data processing request, wherein the business data processing request carries a business matrix parameter; obtaining business data to be processed according to the business data processing request, and determining a target data processing rule for the business data to be processed in a preset data processing rule library according to the business matrix parameter, wherein the preset data processing rule library is determined based on the processing unit specification of the processing unit; splitting the business data to be processed according to the target data processing rule to obtain at least one business sub-data to be processed, and determining a matrix operation unit group corresponding to each business sub-data to be processed; processing each business sub-data to be processed based on each matrix operation unit group to obtain a business processing sub-result corresponding to each business sub-data to be processed, and determining a business processing result corresponding to the business data processing request according to each business processing sub-result.

[0019] Through the method provided by the embodiments of the present disclosure, the corresponding data processing rules are matched to the business data to be processed based on the business matrix parameters in the business processing request, and the business data to be processed is split according to the data processing rules. The resulting business sub-data to be processed are matched to the respective matrix operation unit groups on the processing unit, so that each matrix operation unit group can fully utilize the matrix operation unit to perform operations on the respective business sub-data to be processed, thereby improving the operational efficiency of each matrix operation unit, improving the resource utilization of the processing unit in the system-on-chip, accelerating the reasoning and calculation speed of the business model, and improving the service indicators of the large model. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] FIG1 is an architecture diagram of a system on a chip provided by one embodiment of the present disclosure;

[0021] FIG2 is a flow chart of a data processing method provided by one embodiment of the present disclosure;

[0022] FIG3 is a schematic diagram of data processing in a 2X mode according to an embodiment of the present disclosure;

[0023] FIG4 is a schematic diagram of data processing in a 4X mode according to an embodiment of the present disclosure;

[0024] FIG5 is a flowchart of a data processing method according to another embodiment of the present disclosure;

[0025] FIG6 is a schematic structural diagram of a data processing device provided by an embodiment of the present disclosure;

[0026] FIG7 is a schematic structural diagram of a system on a chip provided by an embodiment of the present disclosure;

[0027] FIG8 is a structural block diagram of a processing unit provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0028] The following description sets forth many specific details to facilitate a full understanding of the present disclosure. However, the present disclosure can be implemented in many other ways than those described herein, and those skilled in the art can make similar generalizations without violating the scope of the present disclosure. Therefore, the present disclosure is not limited to the specific implementations disclosed below.

[0029] The terms used in one or more embodiments of the present disclosure are for the purpose of describing specific embodiments only and are not intended to limit one or more embodiments of the present disclosure. The singular forms "a", "the", and "the" used in one or more embodiments of the present disclosure and the appended claims are also intended to include plural forms, unless the context clearly indicates otherwise. It should also be understood that the term "and / or" used in one or more embodiments of the present disclosure refers to and includes any or all possible combinations of one or more associated listed items.

[0030] It should be understood that although the terms first, second, etc. may be used to describe various information in one or more embodiments of the present disclosure, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from each other. For example, without departing from the scope of one or more embodiments of the present disclosure, the first may also be referred to as the second, and similarly, the second may also be referred to as the first. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0031] In addition, it should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in one or more embodiments of the present disclosure are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of the relevant regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0032] The large model in one or more embodiments of the present disclosure specifically refers to a deep learning model with large-scale model parameters, which usually contains hundreds of millions, tens of billions, or even hundreds of billions of model parameters. The large model can also be called a cornerstone model / foundation model (Foundation Model). The large model is pre-trained by large-scale unlabeled corpus to produce a pre-trained model with more than 100 million parameters. This model can adapt to a wide range of downstream tasks and has good generalization capabilities, such as large-scale language models (LLMs) and multi-modal pre-training models.

[0033] When large models are used in practice, only a small number of samples are needed to fine-tune the pre-trained model and it can be applied to different tasks. Large models can be widely used in natural language processing (NLP), computer vision and other fields. Specifically, they can be applied to computer vision tasks such as visual question answering (VQA), image caption (IC), and image generation, as well as natural language processing tasks such as text-based sentiment classification, text summary generation, and machine translation. The main application scenarios of large models include digital assistants, intelligent robots, search, online education, office software, e-commerce, and intelligent design.

[0034] First, the terms involved in one or more embodiments of the present disclosure are explained.

[0035] GEMM (General Matrix Multiplication) is a matrix multiplication operation performed in LLM reasoning tasks, i.e., (M, K) * (K, N) + (M, N) = (M, N). The dimension of M is usually small. For example, a typical combination is M = 1, N = 128, and K = 4096.

[0036] Computing device: A device with computing or processing capabilities, which can be in the form of a terminal, a server, or a cluster of servers.

[0037] System on chip (SoC) refers to a technology that integrates a complete system on a single chip and groups all or part of the necessary electronic circuits, generally including a processor or acceleration unit, memory, and peripheral circuits.

[0038] A processing unit (PE) is a processing unit designed in a computing device to improve the data processing speed in some specialized areas (for example, image processing, processing various operations of deep learning models, etc.) where traditional processing units are inefficient.

[0039] Matrix operation unit: A device located within the processing unit that has the processing power to perform matrix multiplication and other related operations.

[0040] Matrix operations: Matrix operations using General Matrix Multiplication (GEMM) are fundamental to AI applications. With the development of technologies like large models and generative AI, AI's demand for computing power is increasing. The current mainstream approach is to accelerate specific algorithms using heterogeneous GPUs (Graphics Processing Units), which can improve the computing performance of AI applications.

[0041] 3D-DRAM: 3D Dynamic Random Access Memory (3DDRAM).

[0042] With the continuous advancement of computer technology, the scale of large language models has gradually increased. Some large language models have model parameters reaching trillions. Their ability to generate language has been significantly improved, enabling them to generate coherent, natural language paragraphs with higher grammatical correctness and semantic accuracy. Their enhanced ability to understand and process language enables them to perform tasks such as text classification, named entity recognition, sentiment analysis, and semantic similarity calculation.

[0043] The batch size for large language model inference refers to the number of simultaneous inference requests. For example, a batch size of 32 means 32 requests are being processed simultaneously on the same hardware inference card (CPU (Central Processing Unit), GPU, TPU (Tensor Processing Unit), FPGA (Field-Programmable Gate Array), etc.). Increasing the batch size can improve hardware resource utilization, increase throughput, and reduce costs. However, large models have a large number of weights, which consume a large amount of graphics memory. Furthermore, each request during inference generates a large amount of cache, and hardware memory capacity is limited, making it difficult to increase the batch size. Furthermore, the quality of service metrics of large models primarily include first packet return time and token generation latency. Increasing the batch size increases these two metrics, impacting the customer experience. A smaller batch size results in highly irregular matrix calculations during inference, making it difficult to efficiently utilize hardware resources. Therefore, striking a balance between increasing the batch size and fully utilizing hardware resources becomes a challenge for technical personnel.

[0044] Current matrix multiplication accelerators have low utilization when implementing GEMM calculations. The M dimension of a matrix multiplication accelerator is fixed. For example, if the M dimension of an 8*8*4 matrix multiplication accelerator is 1 in an LLM inference task, the resource utilization of the matrix multiplication accelerator is only 1 / 8. If the M dimension of an LLM inference task is 128*128*128, the resource utilization of the matrix multiplication accelerator is only 1 / 128. The resource utilization of a matrix multiplication accelerator is the ratio of the M dimension of the inference task to the M dimension of the matrix multiplication accelerator's specifications.

[0045] In the present disclosure, a data processing method is provided. The present disclosure also relates to a processing unit, a processor, a system on a chip and a computing device, which are described in detail one by one in the following embodiments.

[0046] Referring to FIG. 1 , FIG. 1 shows an architecture diagram of a system on chip provided by an embodiment of the present disclosure. As shown in FIG. 1 , the system on chip includes a memory 102 and a processor 104 , and the memory 102 stores business data to be processed.

[0047] In practical applications, the processor 104 retrieves the business data to be processed from the memory 102. Furthermore, the business data to be processed includes first business sub-data to be processed, second business sub-data to be processed, and third business sub-data to be processed. In one or more embodiments provided herein, the business data to be processed includes matrix A, matrix B, and matrix C.

[0048] In the processor 104, the input matrices A, B, and C are reorganized to input sufficient data to each matrix operation unit group. For example, for (2.16.32)@FP16, eight different 4*16 B matrices need to be sent to eight matrix operation unit groups; while for (16.16.4)@FP16, one 4*16 B matrix needs to be sent to eight matrix operation unit groups.

[0049] In each matrix operation unit, matrix multiplication is performed, that is, the calculation of (M, K)*(K, N)+(M, N)=(M, N), where (M, K) is the A matrix, (K, N) is the B matrix, the first (M, N) is the C matrix, and the second (M, N) is the D matrix.

[0050] The results obtained from each matrix operation unit are accumulated. This means that the operation results generated by multiple matrix operation unit groups are accumulated. The accumulation method is determined by the calculation mode. In 1x / 2x mode, the eight 2x16 D matrices are accumulated element by element. In 4x mode, four of the eight 2x16 D matrices are accumulated, and the remaining four matrices are not accumulated. In 16x mode, none of the eight 2x16 D matrices are accumulated. The accumulated business processing result (i.e., the D matrix) is obtained.

[0051] Finally, the accumulated business processing result (ie, the D matrix) is written into the memory 102 using the write interface.

[0052] 2 , which shows a flow chart of a data processing method provided by one embodiment of the present disclosure. The method is applied to a processing unit of a system on chip, wherein the processing unit includes at least one matrix operation unit, and specifically includes the following steps:

[0053] Step 202: Receive a service data processing request, wherein the service data processing request carries a service matrix parameter.

[0054] The method provided by the embodiment of the present disclosure is applied to the processing unit of a certain system on chip, which can also be referred to as a processing unit within the system on chip. In practical applications, the system on chip includes a processor and a memory, the processor includes at least one processing unit, and each processing unit includes a plurality of matrix operation units. For example, for a certain processing unit, there are 1024 matrix operation units, which are divided into 8 matrix operation unit groups. Then, each matrix operation unit group includes 128 matrix operation units, which are set to the matrix operation unit group specification information of (2.16.4), which means that its matrix operation unit group specification information is M=2, N=16, K=4, that is, the calculation of (2, 4)*(4, 16)+(2, 16)=(2, 16) is performed in each of its matrix operation unit groups.

[0055] A business data processing request specifically refers to a processing request for business data of a target business received by the processing unit of the system on chip. In actual applications, the target business may be an image processing business, a text processing business, an audio processing task, a video processing task, and the like. In the method provided in the present disclosure, the specific content of the target business is not limited. When the target business is an image processing business, the business data includes image feature information; when the target business is a text processing business, the business data includes text feature information. It should be noted that the data processing method executed in the processing unit of the system on chip provided in the present disclosure may be all business data corresponding to the target business, or part of the business data corresponding to the target business. In the method provided in the present disclosure, the dimensional information of the business data is not limited.

[0056] The service matrix parameter specifically refers to the parameter information of the service matrix in the service data. Typically, the service matrix parameter information is represented by (MNK). When the service matrix parameter information is (MNK), it means that the calculation (M, K) * (K, N) + (M, N) = (M, N) needs to be performed, where (M, K) is the A matrix, (K, N) is the B matrix, the first (M, N) is the C matrix, and the second (M, N) is the D matrix.

[0057] In the data processing method provided by the embodiments of the present disclosure, a processing unit performs a GEMM calculation, i.e., a matrix-related calculation. The matrix operation performed by the processing unit is the "matrix A * matrix B + matrix C = matrix D" operation, where matrices A, B, and C are input matrices, and matrix D is the output matrix. Matrix A is a business application matrix, such as a text matrix or an image matrix; matrix B is a model parameter matrix; matrix C is a model parameter matrix; and matrix D is the output matrix.

[0058] When the received business data processing request carries a business matrix parameter of (MNK), the matrix operation unit in the processing unit performs matrix calculation, which means that the processing unit needs to complete the (M, K)*(K, N)+(M, N)=(M, N) matrix operation within a single clock cycle.

[0059] In a specific embodiment provided by the present disclosure, a business data processing request is received, and the business matrix parameter carried in the business data processing request is (2.16.32), which means that the matrix operation of (2, 32) x (32, 16) + (2, 16) = (2, 16) must be completed within a single clock cycle.

[0060] In another specific embodiment provided by the present disclosure, a business data processing request is received, and the business matrix parameter carried in the business data processing request is (4.16.16), which means that the matrix operation of (4, 16)*(16, 16)+(4, 16)=(4, 16) must be completed within a single clock cycle.

[0061] Step 204: Obtain the business data to be processed according to the business data processing request, and determine the target data processing rules of the business data to be processed in a preset data processing rule base according to the business matrix parameters, wherein the preset data processing rule base is determined based on the processing unit specifications of the processing unit.

[0062] After obtaining a business data processing request, the corresponding business data to be processed can be obtained based on the business data processing request. In the method provided in the present disclosure, the business data to be processed includes the business data corresponding to the target business, and the model parameters of the data processing model that processes the business data.

[0063] As described in the above steps, the matrix operation performed by the processing unit is the operation of matrix A * matrix B + matrix C = matrix D, where matrices A, B, and C are input matrices, and matrix D is the output matrix. The business data to be processed obtained in response to the business data processing request is matrix A, matrix B, and matrix C, where matrix A is the business data feature information corresponding to the target business, and matrix B and matrix C are model parameters of the data processing model.

[0064] The preset data processing rule base specifically refers to a plurality of preset data processing rules determined based on the processing unit specifications of the processing unit. Furthermore, the preset data processing rule base stores a plurality of rules for splitting the business data to be processed. In actual applications, the processing unit specifications of the processing unit are fixed. For example, the processing unit specifications of a certain processing unit are "8 matrix operation unit groups, and the matrix operation unit group specification information is (2.16.4)", which means that the processing unit has 8 matrix operation unit groups, and each matrix operation unit group processes the calculation of (2.4)*(4.16)+(2.16)=(2.16). The preset data processing rule base is provided with data processing rules for a variety of different business data to be processed. The purpose is to split the business data to be processed into a plurality of business sub-data to be processed, so that each matrix operation unit group can process the corresponding business sub-data to be processed, and thus each matrix operation unit group can fully utilize the computing resources.

[0065] In practical applications, when performing GEMM calculations, calculations of four matrix dimensions (2.16.32), (4.16.16), (8.16.8), and (16.16.4) can be completed within one clock cycle. For larger matrix calculation dimensions, it is necessary to split it into multiple of the above calculation dimensions and then divide it into multiple clock cycles to complete the calculation.

[0066] In a specific embodiment provided by the present disclosure, the business data processing request carries data storage location information;

[0067] Obtaining the business data to be processed according to the business data processing request includes:

[0068] The service data to be processed is obtained from the memory according to the data storage location information.

[0069] In actual applications, the service data to be processed is stored in a preset storage location on the terminal, such as a memory. The service data processing request carries data storage location information. After receiving the service data processing request, the processing unit parses the request to obtain the data storage location information. Using the data storage location information, the processing unit can locate the memory corresponding to the service data to be processed and retrieve the service data from the corresponding memory.

[0070] In a specific embodiment provided by the present disclosure, determining the target data processing rule for the to-be-processed business data in a preset data processing rule library according to the business matrix parameters includes:

[0071] Determining business data dimension information in the business parameter matrix;

[0072] The target data processing rules corresponding to the to-be-processed business data are determined in a preset data processing rule library according to the business data dimension information.

[0073] In order to make full use of the computing resources of the processing unit, when the processing unit is used to perform matrix multiplication acceleration calculations, the business data dimension information (M dimension) in the business parameter matrix is ​​used to determine the target data processing rule. For example, when the business data dimension information in the business parameter matrix is ​​(2.16.32), its M=2, and its corresponding target data processing rule is "2X mode"; for another example, when the business data dimension information in the business parameter matrix is ​​(4.16.16), its M=4, and its corresponding target data processing rule is "4X mode"; for another example, when the business data dimension information in the business parameter matrix is ​​(8.16.8), its M=8, and its corresponding target data processing rule is "8X mode"; for another example, when the business data dimension information in the business parameter matrix is ​​(16.16.4), its M=16, and its corresponding target data processing rule is "16X mode".

[0074] It should be noted that when M=1, the target data processing rule selected is also 2X mode. The difference is that the input matrices A and C and the output matrix D only have one row of valid data, and the second row of data is padded with 0s. In this case, only half of the GEMM computing power is used. When M>16, the "16X mode" data processing rule is also selected.

[0075] That is, in the embodiment provided in the present disclosure, the preset data processing rule library stores data processing rules of "2X mode", "4X mode", "8X mode" and "16X mode", and the target data processing rules are determined according to the business data dimension information (M value) in the business parameter matrix.

[0076] In another specific embodiment provided by the present disclosure, the preset data processing rule base is created by the following steps:

[0077] Obtaining the number of matrix operation units on the processing unit and determining the number of matrix operation unit groups;

[0078] Determining the number of unit group matrix operation units corresponding to each matrix operation unit group according to the number of matrix operation units and the number of matrix operation unit groups, and determining matrix operation unit group specification information according to the number of unit group matrix operation units;

[0079] Acquire at least one business data dimension information, and determine a data processing rule corresponding to each business data dimension information according to each business data dimension information and the matrix operation unit group specification information;

[0080] A preset data processing rule library is generated according to each data processing rule and the number of matrix operation unit groups.

[0081] Here, the method of creating a preset data processing rule base is explained.

[0082] First, obtain the number of matrix operation units on the processing unit. Normally, the number of matrix operation units is 2 to the power of n. The method provided in the embodiment of the present disclosure is based on the idea of ​​splitting the calculation of a matrix multiplication into multiple sub-matrix multiplications. Therefore, for any processing unit, it is necessary to first determine the number of matrix operation unit groups to be split. For example, in order to utilize the computing resources of the operation node, a GEMM is split into 8 uGEMMs, and the number of matrix operation unit groups is 8; if a GEMM is split into 16 uGEMMs, the number of matrix operation unit groups is 16.

[0083] After determining the number of matrix operation units and the number of matrix operation unit groups, the number of unit group matrix operation units in each matrix operation unit group can be further determined. For example, if the number of matrix operation units is 1024 and they are divided into 8 matrix operation unit groups, the number of unit group matrix operation units in each matrix operation unit group is 128. The matrix operation unit group specifications for each matrix operation unit group can then be further determined.

[0084] For example, when the number of matrix operation unit groups is determined to be 128, the matrix operation unit group specification information can be (2.16.4), or can be set to (2.8.8)... etc. In the method provided in the present disclosure, the matrix operation unit group specification information is not further limited, and is subject to actual application.

[0085] After determining the matrix operation unit group specification information, the specifications of each matrix operation unit group for processing business data can be determined. For example, when the matrix operation unit group specification information is (2.16.4), it is used to perform the calculation of (2, 4)*(4, 16)+(2, 16)=(2, 16).

[0086] Then, based on the business data dimension information, the corresponding data processing rules are determined. For example, if the business data dimension information M=1 or 2, it is determined to be a 2X mode, that is, the splitting rule is to split the business data to be processed into matrices that are calculated in accordance with the matrix operation unit group specification information; for another example, if the business data dimension information M=4, it is determined to be a 4X mode, and the splitting rule is to split the business data to be processed into matrices that are calculated in accordance with the matrix operation unit group specification information.

[0087] Step 206: Split the to-be-processed business data according to the target data processing rule to obtain at least one to-be-processed business sub-data, and determine a matrix operation unit group corresponding to each to-be-processed business sub-data.

[0088] After the target data processing rule is determined, the business data to be processed is split according to the target data processing rule to obtain at least one business sub-data to be processed, and a matrix operation unit group corresponding to each business sub-data to be processed is determined.

[0089] In the data processing method provided in the present disclosure, the purpose is to improve the resource utilization of the processing unit of the system on chip. Therefore, involving each matrix operation unit in the operation will be more conducive to improving the resource utilization of the processing unit. Therefore, in a specific embodiment provided in the present disclosure, the target data processing rule includes matrix operation unit group specification information;

[0090] Splitting the to-be-processed business data according to the target data processing rule to obtain at least one to-be-processed business sub-data includes:

[0091] Determining a splitting rule for the service data to be processed according to the matrix operation unit group specification information and the service matrix parameters of the service data to be processed;

[0092] The business data to be processed is split according to the splitting rule to obtain at least one business sub-data to be processed.

[0093] As described in the above steps, the matrix operation unit group specification information is related to the setting of the processing unit. In the process of splitting the business data to be processed according to the target data processing rules, it is necessary to determine the splitting rules of the business data to be processed based on the matrix operation unit group specification information of the processing unit.

[0094] For example, if the matrix operation unit group specification information is (2.16.4), it is used to perform the calculation of (2, 4) * (4, 16) + (2, 16) = (2, 16). The service matrix parameter of the service data to be processed is (2.16.32), which means that it needs to complete the matrix operation of (2, 32) * (32, 16) + (2, 16) = (2, 16) in a single clock cycle, where matrix A is (2, 32) and matrix B is (32, 16). Matrix A (2, 32) is split into 8 (2, 4) matrixes, and matrix B (32, 16) is split into 8 (4, 16) matrixes.

[0095] For another example, if the matrix operation unit group specification information is (2.16.4), it is used to perform the calculation of (2, 4) * (4, 16) + (2, 16) = (2, 16). The service matrix parameter of the service data to be processed is (4.16.16), which means that the matrix operation of (4, 16) * (16, 16) + (4, 16) = (4, 16) needs to be completed in a single clock cycle, where matrix A is (4, 16) and matrix B is (16, 16). Matrix A (4, 16) is split into 8 (2, 4) matrixes, and matrix B (16, 16) is split into 4 (4, 16) matrixes.

[0096] For another example, if the matrix operation unit group specification information is (2.16.4), it is used to perform the calculation of (2, 4) * (4, 16) + (2, 16) = (2, 16). The service matrix parameter of the service data to be processed is (8.16.8), which means that the matrix operation of (8, 8) * (8, 16) + (8, 16) = (8, 16) needs to be completed in a single clock cycle, where matrix A is (8, 8) and matrix B is (8, 16). Matrix A (8, 8) is split into 8 (2, 4) matrixes, and matrix B (8, 16) is split into 2 (4, 16) matrixes.

[0097] For another example, if the matrix operation unit group specification information is (2.16.4), it is used to perform the calculation of (2, 4) * (4, 16) + (2, 16) = (2, 16). The service matrix parameter of the service data to be processed is (16.16.4), which means that the matrix operation of (16, 4) * (4, 16) + (16, 16) = (16, 16) needs to be completed in a single clock cycle, where matrix A is (16, 4) and matrix B is (4, 16). Matrix A (16, 4) is split into 8 (2, 4) matrices, and matrix B (4, 16) is split into 1 (4, 16) matrix.

[0098] According to the above examples, in the embodiment provided by the present disclosure, the business data to be processed is split according to the splitting rules, and the business data to be processed is split into business sub-data to be processed corresponding to the matrix operation unit group specification information.

[0099] In another specific embodiment provided by the present disclosure, the target data processing rule includes the number of matrix operation unit groups;

[0100] Determine the matrix operation unit group corresponding to each business sub-data to be processed, including:

[0101] Get the number of sub-data of business sub-data to be processed;

[0102] The matrix operation unit group corresponding to each to-be-processed business sub-data is determined according to the number of the sub-data and the number of the matrix operation unit groups.

[0103] In practical applications, the target data processing rules also include the number of matrix operation unit groups, that is, the number of matrix operation unit groups performing operations on the processing unit. The matrix operation unit groups are then allocated based on the number of sub-data of the business sub-data to be processed, and the matrix operation unit group corresponding to each business sub-data to be processed is determined.

[0104] Continuing with the previous example, the service matrix parameters for the service data to be processed are (2.16.32), where matrix A is (2, 32) and matrix B is (32, 16). Matrix A (2, 32) is split into eight (2, 4) matrices, and matrix B (32, 16) is split into eight (4, 16) matrices. At this point, there are eight matrix operation unit groups, and each matrix operation unit group is assigned one (2, 4) matrix and one (4, 16) matrix.

[0105] Referring to FIG3 , FIG3 shows a schematic diagram of data processing in the 2X mode provided in this embodiment. As shown in FIG3 , matrix A (2, 32) is split into 8 (2, 4) matrices, namely A0, A1, A2, A3, A4, A5, A6, and A7; and matrix B (32, 16) is split into 8 (4, 16) matrices, namely B0, B1, B2, B3, B4, B5, B6, and B7. A0 and B0 are assigned to the first matrix operation unit group for processing, A1 and B1 are assigned to the second matrix operation unit group for processing, A2 and B2 are assigned to the third matrix operation unit group for processing, A3 and B3 are assigned to the fourth matrix operation unit group for processing, A4 and B4 are assigned to the fifth matrix operation unit group for processing, A5 and B5 are assigned to the sixth matrix operation unit group for processing, A6 and B6 are assigned to the seventh matrix operation unit group for processing, and A7 and B7 are assigned to the eighth matrix operation unit group for processing.

[0106] For another example, the service matrix parameters of the service data to be processed are (4.16.16), where matrix A is (4, 16) and matrix B is (16, 16). Matrix A (4, 16) is split into 8 (2, 4) matrices, and matrix B (16, 16) is split into 4 (4, 16) matrices. At this point, there are 8 matrix operation unit groups, so each matrix operation unit group is allocated one (2, 4) matrix, and every two matrix operation unit groups share one (4, 16) matrix. Under this calculation method, each matrix operation unit group node will calculate a (2, 16) matrix.

[0107] Referring to FIG4 , FIG4 shows a schematic diagram of data processing in the 4X mode provided in this embodiment. As shown in FIG4 , matrix A (4, 16) is split into 8 (2, 4) matrices, namely A0, A1, A2, A3, A4, A5, A6, and A7; and matrix B (16, 16) is split into 4 (4, 16) matrices, namely B0, B1, B2, and B3. A0 and B0 are assigned to the first matrix operation unit group for processing, A1 and B0 are assigned to the second matrix operation unit group for processing, A2 and B1 are assigned to the third matrix operation unit group for processing, A3 and B1 are assigned to the fourth matrix operation unit group for processing, A4 and B2 are assigned to the fifth matrix operation unit group for processing, A5 and B2 are assigned to the sixth matrix operation unit group for processing, A6 and B3 are assigned to the seventh matrix operation unit group for processing, and A7 and B3 are assigned to the eighth matrix operation unit group for processing.

[0108] For another example, suppose the service matrix parameters for the service data to be processed are (8.16.8), where matrix A is (8, 8) and matrix B is (8, 16). Matrix A (8, 8) is split into eight (2, 4) matrices, and matrix B (8, 16) is split into two (4, 16) matrices. At this point, there are eight matrix operation unit groups. Each matrix operation unit group is assigned one (2, 4) matrix, and every four matrix operation unit groups share one (4, 16) matrix.

[0109] For another example, suppose the service matrix parameters for the service data to be processed are (16.16.4), where matrix A is (16, 4) and matrix B is (4, 16). Matrix A (16, 4) is split into eight (2, 4) matrices, and matrix B (4, 16) is split into one (4, 16) matrix. In this case, there are eight matrix operation unit groups. Each matrix operation unit group is assigned one (2, 4) matrix, and all eight matrix operation unit groups share one (4, 16) matrix.

[0110] Step 208: Process each business sub-data to be processed based on each matrix operation unit group to obtain a business processing sub-result corresponding to each business sub-data to be processed, and determine a business processing result corresponding to the business data processing request according to each business processing sub-result.

[0111] After determining the matrix operation unit group corresponding to each business sub-data to be processed, each matrix operation unit group processes the corresponding business sub-data to be processed to obtain the business processing sub-result corresponding to each business sub-data to be processed, and then splices the business processing sub-results to generate the final business processing result.

[0112] In a specific embodiment provided by the present disclosure, each matrix operation unit group processes each to-be-processed business sub-data to obtain a business processing sub-result corresponding to each to-be-processed business sub-data, including:

[0113] Determining a target matrix operation unit group and target to-be-processed business sub-data corresponding to the target matrix operation unit group, wherein the target matrix operation unit group is any one of the matrix operation unit groups;

[0114] Processing the target to-be-processed business sub-data based on each matrix operation unit in the target matrix operation unit group to obtain a target business processing sub-result corresponding to the target to-be-processed business sub-data;

[0115] Accordingly, determining the business processing result corresponding to the business data processing request according to each business processing sub-result includes:

[0116] The business processing result is determined according to each business processing sub-result and the third business data to be processed.

[0117] Specifically, in practical applications, during the process of processing the business sub-data to be processed based on each matrix operation unit group, one matrix operation unit group is used as an example for explanation. Specifically, a target matrix operation unit group is determined within each matrix operation unit group, and the target matrix operation unit group processes the corresponding target business sub-data to be processed. The business processing sub-results corresponding to each matrix operation unit group are then fused with the third business sub-data to be processed to obtain a business processing result. The third business sub-data to be processed is the matrix C in the above steps.

[0118] In practical applications, the to-be-processed business sub-data includes first to-be-processed business sub-data and second to-be-processed business sub-data;

[0119] In another specific embodiment provided by the present disclosure, processing the target to-be-processed business sub-data based on each matrix operation unit in the target matrix operation unit group to obtain a target business processing sub-result corresponding to the target to-be-processed business sub-data includes:

[0120] The first to-be-processed business sub-data and the second to-be-processed business sub-data are processed based on each matrix operation unit to obtain a target business processing sub-result.

[0121] Furthermore, each matrix operation unit processes the first to-be-processed business sub-data and the second to-be-processed business sub-data to obtain a target business processing sub-result, including:

[0122] Processing the first to-be-processed business sub-data and the second to-be-processed business sub-data to obtain reference business sub-data;

[0123] Accordingly, determining the business processing result according to each business processing sub-result and the third business data to be processed includes:

[0124] A business processing result is obtained according to the business processing sub-results corresponding to each matrix operation unit group and the third business data to be processed.

[0125] After the target matrix operation unit group determines the pending business sub-data to be processed, it splits the pending business sub-data into first pending business sub-data and second pending business sub-data. The first pending business sub-data is the sub-matrix split from the above-mentioned matrix A, the second pending business sub-data is the sub-matrix split from the above-mentioned matrix B, and the third pending business sub-data is the sub-matrix split from the above-mentioned matrix C.

[0126] Referring to FIG3 , as shown in FIG3 , taking the business matrix parameters of the business data to be processed as (2.16.32) as an example, the matrix A (2, 32) is split into eight (2, 4) matrices, namely A0, A1, A2, A3, A4, A5, A6, and A7; and the matrix B (32, 16) is split into eight (4, 16) matrices, namely B0, B1, B2, B3, B4, B5, B6, and B7. Among them, A0-A7 are all first business sub-data to be processed, and B0-B7 are all second business sub-data to be processed.

[0127] Assign A0 and B0 to the first matrix operation unit group for processing. Assign A1 and B1 to the second matrix operation unit group for processing, assign A2 and B2 to the third matrix operation unit group for processing, assign A3 and B3 to the fourth matrix operation unit group for processing, assign A4 and B4 to the fifth matrix operation unit group for processing, assign A5 and B5 to the sixth matrix operation unit group for processing, assign A6 and B6 to the seventh matrix operation unit group for processing, and assign A7 and B7 to the eighth matrix operation unit group for processing.

[0128] For the first matrix operation unit group, A0 is the first business sub-data to be processed, and B0 is the second business sub-data to be processed. The two are calculated to obtain the target business processing sub-result corresponding to the first matrix operation unit group. The same processing method is used for other matrix operation unit groups and will not be repeated here.

[0129] The 8 target business processing sub-results calculated in the 8 matrix operation unit groups are fused to obtain a (2, 16) matrix, and then added to the matrix C (2, 16) to obtain a (2, 16) D matrix.

[0130] Referring to FIG4 , as shown in FIG4 , taking the matrix A(4,16) being split into 8 (2,4) matrices as an example, the matrix A(4,16) is split into 8 (2,4) matrices, namely A0, A1, A2, A3, A4, A5, A6, and A7; and the matrix B(16,16) is split into 4 (4,16) matrices, namely B0, B1, B2, and B3. A0 and B0 are assigned to the first matrix operation unit group for processing, A1 and B0 are assigned to the second matrix operation unit group for processing, A2 and B1 are assigned to the third matrix operation unit group for processing, A3 and B1 are assigned to the fourth matrix operation unit group for processing, A4 and B2 are assigned to the fifth matrix operation unit group for processing, A5 and B2 are assigned to the sixth matrix operation unit group for processing, A6 and B3 are assigned to the seventh matrix operation unit group for processing, and A7 and B3 are assigned to the eighth matrix operation unit group for processing.

[0131] Matrix multiplication calculations are performed on each matrix operation unit group to obtain the result (2, 16) corresponding to each matrix operation unit group. The 8 matrix operation unit groups are grouped, and the results of the first 4 matrix operation unit groups are accumulated to obtain the first accumulated matrix (2, 16). The results of the last 4 matrix operation unit groups are accumulated to obtain the second accumulated matrix (2, 16).

[0132] The third business data matrix to be processed, C, is (4, 16). Split the matrix C (4, 16) into two (2, 16) matrices, C0 and C1. Add the first accumulated matrix (2, 16) to C0, and add the second accumulated matrix (2, 16) to C1 to obtain the matrix D (4, 16).

[0133] It should be noted that in the method provided in the embodiments of the present disclosure, the first business sub-data to be processed includes business application data, the second business sub-data to be processed includes model parameter data, and the third business data to be processed includes model parameter data. That is, the first business sub-data to be processed is the sub-matrix decomposed from matrix A, the second business sub-data to be processed is the sub-matrix decomposed from matrix B, the third business data to be processed is matrix C, and the business processing result is matrix D.

[0134] In practical applications, the fusion of corresponding results of multiple matrix operation unit groups is determined by the calculation method. For the method of 8 matrix operation unit groups, for the 2x mode, 8 (2, 16) matrices are accumulated element by element; for the 4X mode, among the 8 (2, 16) matrices, the first 4 matrices are accumulated, and the last 4 matrices are accumulated, and the accumulated results are spliced; for the 8X mode, among the 8 (2, 16) matrices, every 2 matrices are accumulated, and the accumulated results are spliced; for the 16X mode, the 8 (2, 16) matrices are not accumulated and are directly spliced.

[0135] In another specific embodiment provided by the present disclosure, after determining the business processing result corresponding to the business data processing request according to each business processing sub-result, the method further includes:

[0136] The business processing result is saved in a memory.

[0137] After obtaining the business processing result of this time, the business processing result is saved to the corresponding memory for subsequent processing.

[0138] The method provided by the embodiments of the present disclosure proposes a method for processing pending business data when the M dimension is small in matrix multiplication calculations for LLM reasoning, resulting in GEMM being unable to efficiently utilize hardware computing power. The pending business data is split according to the number of matrix operation unit groups and the M dimension of the pending business data, and the M dimension of the pending business data when participating in the matrix multiplication calculation is flexibly adjusted, thereby improving hardware resource utilization during the matrix multiplication calculation.

[0139] Through this method, a large matrix multiplication calculation is split into multiple small matrix multiplication calculations, and algorithm mapping and data sharing in different calculation modes are performed, so that a larger matrix multiplication calculation can be split into multiple small matrix multiplication calculations, so that they correspond to the corresponding matrix operation unit group specifications, thereby improving the resource utilization of the processing unit of the on-chip system.

[0140] FIG5 shows a flowchart of a data processing method provided by another embodiment of the present disclosure, which specifically includes the following steps:

[0141] Step 502: Obtain the number of matrix operation unit groups corresponding to the processing units of the system on chip, and determine the specification information of the matrix operation unit groups.

[0142] In this embodiment, there are 1024 matrix operation units on the processing unit of the system on chip, which corresponds to 16 matrix operation unit groups. Each matrix operation unit group has 64 matrix operation units. The matrix operation unit group specification information is (2.8.4), which means that the processing unit has 16 matrix operation unit groups, and each matrix operation unit group processes the calculation of (2, 4)*(4, 8)+(2, 8)=(2, 8).

[0143] Step 504: Receive a service data processing request, which carries the service matrix parameter (2.16.32).

[0144] In this embodiment, the service parameter matrix (2.16.32) represents the matrix operation of (2, 32)*(32, 16)+(2, 16)=(2, 16) that needs to be completed in a single clock cycle.

[0145] Step 506: The business data to be processed includes the A matrix (2, 32), the B matrix (32, 16) and the C matrix (2, 16).

[0146] Step 508: According to the M dimension in the business parameter matrix = 2, the 2X mode is selected and the splitting rule corresponding to the 2X mode is determined.

[0147] Step 510: Split the A matrix (2, 32) into 8 (2, 4) matrices and split the B matrix (32, 16) into 16 (4, 8) matrices according to the splitting rule and the matrix operation unit group specification information.

[0148] Step 512: Allocate one (2, 4) matrix to every two matrix operation unit groups, allocate one (4, 8) matrix to each matrix operation unit group, perform matrix multiplication calculations in each matrix operation unit group, and obtain 16 (2, 8) matrices.

[0149] Step 514: Accumulate the first 4 groups of the 16 (2, 8) matrices to obtain the first (2, 8) matrix, accumulate the last 4 groups to obtain the second (2, 8) matrix, and concatenate the first (2, 8) matrix and the second (2, 8) matrix to obtain the reference service matrix (2, 16).

[0150] Step 516: Add the reference service matrix (2, 16) and the C matrix (2, 16) to obtain the service processing result matrix D (2, 16).

[0151] The method provided by the embodiments of the present disclosure proposes a method for processing pending business data when the M dimension is small in matrix multiplication calculations for LLM reasoning, resulting in GEMM being unable to efficiently utilize hardware computing power. The pending business data is split according to the number of matrix operation unit groups and the M dimension of the pending business data, and the M dimension of the pending business data when participating in the matrix multiplication calculation is flexibly adjusted, thereby improving hardware resource utilization during the matrix multiplication calculation.

[0152] Through this method, a large matrix multiplication calculation is split into multiple small matrix multiplication calculations, and algorithm mapping and data sharing in different calculation modes are performed, so that a larger matrix multiplication calculation can be split into multiple small matrix multiplication calculations, so that they correspond to the corresponding matrix operation unit group specifications, thereby improving the resource utilization of the processing unit of the on-chip system.

[0153] Corresponding to the above method embodiments, the present disclosure also provides a data processing device embodiment. FIG6 shows a schematic diagram of the structure of a data processing device provided by an embodiment of the present disclosure. As shown in FIG6 , the device comprises:

[0154] The receiving module 602 is configured to receive a service data processing request, wherein the service data processing request carries a service matrix parameter;

[0155] a determination module 604 configured to obtain the business data to be processed according to the business data processing request, and determine a target data processing rule for the business data to be processed in a preset data processing rule base according to the business matrix parameters, wherein the preset data processing rule base is determined based on the processing unit specifications of the processing unit;

[0156] The splitting module 606 is configured to split the to-be-processed business data according to the target data processing rule to obtain at least one to-be-processed business sub-data, and determine a matrix operation unit group corresponding to each to-be-processed business sub-data;

[0157] The processing module 608 is configured to process each business sub-data to be processed based on each matrix operation unit group, obtain a business processing sub-result corresponding to each business sub-data to be processed, and determine a business processing result corresponding to the business data processing request according to each business processing sub-result.

[0158] Optionally, the business data processing request carries data storage location information;

[0159] The determining module 604 is further configured to:

[0160] The service data to be processed is obtained from the memory according to the data storage location information.

[0161] Optionally, the determining module 604 is further configured to:

[0162] Determining business data dimension information in the business parameter matrix;

[0163] The target data processing rules corresponding to the to-be-processed business data are determined in a preset data processing rule library according to the business data dimension information.

[0164] Optionally, the data processing device further includes a creation module configured to:

[0165] Obtaining the number of matrix operation units on the processing unit and determining the number of matrix operation unit groups;

[0166] Determining the number of unit group matrix operation units corresponding to each matrix operation unit group according to the number of matrix operation units and the number of matrix operation unit groups, and determining matrix operation unit group specification information according to the number of unit group matrix operation units;

[0167] Acquire at least one business data dimension information, and determine a data processing rule corresponding to each business data dimension information according to each business data dimension information and the matrix operation unit group specification information;

[0168] A preset data processing rule library is generated according to each data processing rule and the number of matrix operation unit groups.

[0169] Optionally, the target data processing rule includes matrix operation unit group specification information;

[0170] The splitting module 606 is further configured to:

[0171] Determining a splitting rule for the service data to be processed according to the matrix operation unit group specification information and the service matrix parameters of the service data to be processed;

[0172] The business data to be processed is split according to the splitting rule to obtain at least one business sub-data to be processed.

[0173] Optionally, the target data processing rule includes the number of matrix operation unit groups;

[0174] The splitting module 606 is further configured to:

[0175] Get the number of sub-data of business sub-data to be processed;

[0176] The matrix operation unit group corresponding to each to-be-processed business sub-data is determined according to the number of the sub-data and the number of the matrix operation unit groups.

[0177] Optionally, the processing module 608 is further configured to:

[0178] Determining a target matrix operation unit group and target to-be-processed business sub-data corresponding to the target matrix operation unit group, wherein the target matrix operation unit group is any one of the matrix operation unit groups;

[0179] Processing the target to-be-processed business sub-data based on each matrix operation unit in the target matrix operation unit group to obtain a target business processing sub-result corresponding to the target to-be-processed business sub-data;

[0180] Accordingly, the processing module 608 is further configured to:

[0181] The business processing result is determined according to each business processing sub-result and the third business data to be processed.

[0182] Optionally, the to-be-processed business sub-data includes first to-be-processed business sub-data and second to-be-processed business sub-data;

[0183] The processing module 608 is further configured to:

[0184] The first to-be-processed business sub-data and the second to-be-processed business sub-data are processed based on each matrix operation unit to obtain a target business processing sub-result.

[0185] Optionally, the processing module 608 is further configured to:

[0186] Processing the first to-be-processed business sub-data and the second to-be-processed business sub-data to obtain reference business sub-data;

[0187] Accordingly, the processing module 608 is further configured to:

[0188] A business processing result is obtained according to the business processing sub-results corresponding to each matrix operation unit group and the third business data to be processed.

[0189] Optionally, the first to-be-processed business sub-data includes business application data, the second to-be-processed business sub-data includes model parameter data, and the third to-be-processed business sub-data includes model parameter data.

[0190] Optionally, the data processing device further includes a storage module configured to:

[0191] The business processing result is saved in a memory.

[0192] The data processing device provided by the disclosed embodiments proposes a method for processing pending business data when the M dimension is small in matrix multiplication calculations for LLM reasoning, resulting in GEMM being unable to efficiently utilize hardware computing power. The pending business data is split according to the number of matrix operation unit groups and the M dimension of the pending business data, and the M dimension of the pending business data when participating in the matrix multiplication calculation is flexibly adjusted, thereby improving hardware resource utilization during the matrix multiplication calculation.

[0193] Through this data processing device, a large matrix multiplication calculation is split into multiple small matrix multiplication calculations, and algorithm mapping and data sharing in different calculation modes are performed, so that a larger matrix multiplication calculation can be split into multiple small matrix multiplication calculations, so that they correspond to the corresponding matrix operation unit group specifications, thereby improving the resource utilization of the processing unit of the on-chip system.

[0194] The above is a schematic diagram of a data processing device according to this embodiment. It should be noted that the technical solution of the data processing device and the technical solution of the above-mentioned data processing method are based on the same concept. For details not described in detail in the technical solution of the data processing device, please refer to the description of the technical solution of the above-mentioned data processing method.

[0195] FIG7 shows a schematic diagram of the structure of a system on chip provided by an embodiment of the present disclosure. An embodiment of the present disclosure further provides a system on chip, including a memory 702 and a processor 704. The processor includes a single-core processor or a multi-core processor. The processor and the memory are coupled via an on-chip network.

[0196] Memory 702, configured to store business data to be processed;

[0197] The processor 704 includes at least one processing unit 7042, which includes at least one matrix operation unit. The processor is used to execute computer-executable instructions, which implement the steps of the above-mentioned data processing method when executed by the processor.

[0198] In another specific embodiment provided by the present disclosure, the processor of the system on chip is a multi-core processor composed of a plurality of processing unit arrays.

[0199] 8 shows a block diagram of a processing unit 800 according to an embodiment of the present disclosure. Components of the processing unit 800 include, but are not limited to, an on-chip network control unit 810, a control scheduling unit 820, a matrix operation unit 830, a vector operation unit 840, a data processing unit 850, and an internal storage unit 860.

[0200] In the processing unit, in order to meet the demand for data stream transmission, the processing unit 800 communicates with the router and other processing units through the on-chip network control unit 810 (NoU, Network Overlay Unit) module.

[0201] The control scheduling unit 820 performs traditional processing in the processing unit. In addition, it also assumes the scheduling function of the acceleration unit, and allocates tasks that the acceleration unit needs to undertake, such as matrix operation tasks, to the processing unit.

[0202] The matrix operation unit 830 stores computer executable instructions, which implement the steps of the above-mentioned data processing method when executed by the processor.

[0203] The vector operation unit 840 stores computer executable instructions for executing related tasks that the unit needs to undertake, such as vector operation tasks.

[0204] The data processing unit 850 is provided with a direct memory access (DMA) device to implement data transmission between memories.

[0205] The processing unit further includes an internal storage unit 860 for storing instructions and data for corresponding calculations and control tasks performed by the processing unit. The internal storage unit may be a static random access memory (SRAM) for storing frequently read and written data.

[0206] The processing unit 800 is coupled to an external storage unit. The external storage unit can be implemented as, for example, a dynamic random access memory (DRAM), a synchronous dynamic random access memory (SDRAM), or a 3D-DRAM dynamic random access memory. The processing unit accesses the corresponding 3D-DRAM memory controller through an on-chip bus, and then accesses the 3D-DRAM storage unit corresponding to it in the three-dimensional direction.

[0207] The above is a schematic scheme of a processing unit of this embodiment. It should be noted that the technical scheme of the processing unit and the technical scheme of the above-mentioned data processing method belong to the same concept. For details not described in detail in the technical scheme of the processing unit, please refer to the description of the technical scheme of the above-mentioned data processing method.

[0208] An embodiment of the present disclosure further provides a processor, which includes at least one processing unit, and the processing unit includes at least one matrix operation unit. The processor is used to execute computer-executable instructions, and when the computer-executable instructions are executed by the processor, the steps of the above-mentioned data processing method are implemented.

[0209] The above is a schematic solution of a processor of this embodiment. It should be noted that the technical solution of the processor and the technical solution of the above-mentioned data processing method belong to the same concept. For details not described in detail in the technical solution of the processor, please refer to the description of the technical solution of the above-mentioned data processing method.

[0210] An embodiment of the present disclosure also provides a computing device, which includes one or more memories storing computer-executable instructions and at least one processor, wherein the processor includes at least one processing unit, and the processing unit includes at least one matrix operation unit. The processor is used to execute computer-executable instructions, and when the computer-executable instructions are executed by the processor, the steps of the above-mentioned data processing method are implemented.

[0211] An embodiment of the present disclosure further provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are executed by a processor, they are used to implement the steps of the above-mentioned data processing method.

[0212] An embodiment of the present disclosure further provides a computer program product, comprising a computer program, which implements the steps of the above-mentioned data processing method when executed by a processor.

[0213] The foregoing description describes specific embodiments of the present disclosure. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims can be performed in an order different from that described in the embodiments and still achieve the desired results. Furthermore, the processes depicted in the accompanying drawings do not necessarily require the specific order shown or the sequential order to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0214] The computer-executable instructions include computer program code, which may be in source code form, object code form, executable file, or some intermediate form. The computer-readable medium may include any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a mobile hard drive, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunications signal, and a software distribution medium.

[0215] It should be noted that for the aforementioned method embodiments, for the sake of simplicity of description, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the embodiments of the present disclosure are not limited by the order of the actions described, because according to the embodiments of the present disclosure, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the embodiments of the present disclosure.

[0216] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0217] The preferred embodiments of the present disclosure disclosed above are only used to help illustrate the present disclosure. The optional embodiments do not describe all details in detail, nor do they limit the invention to only the specific embodiments described. Obviously, many modifications and variations can be made based on the content of the embodiments of the present disclosure. The present disclosure selects and specifically describes these embodiments in order to better explain the principles and practical applications of the embodiments of the present disclosure, so that those skilled in the art can better understand and utilize the present disclosure. The present disclosure is limited only by the claims and their full scope and equivalents.

Claims

1. A data processing method, applied to a processing unit of a system on chip, wherein the processing unit includes at least one matrix operation unit, comprising: receiving a service data processing request, wherein the service data processing request carries a service matrix parameter; Acquire the business data to be processed according to the business data processing request, and determine a target data processing rule for the business data to be processed in a preset data processing rule base according to the business matrix parameters, wherein the preset data processing rule base is determined based on a processing unit specification of the processing unit; Splitting the to-be-processed business data according to the target data processing rule to obtain at least one to-be-processed business sub-data, and determining a matrix operation unit group corresponding to each to-be-processed business sub-data; Based on each matrix operation unit group, each to-be-processed business sub-data is processed to obtain a business processing sub-result corresponding to each to-be-processed business sub-data, and a business processing result corresponding to the business data processing request is determined according to each business processing sub-result.

2. The method according to claim 1, wherein the business data processing request carries data storage location information; Obtaining the business data to be processed according to the business data processing request includes: The service data to be processed is obtained from the memory according to the data storage location information.

3. The method according to claim 1 or 2, wherein determining the target data processing rule for the to-be-processed business data in a preset data processing rule library according to the business matrix parameters comprises: Determine business data dimension information in the business parameter matrix; The target data processing rules corresponding to the to-be-processed business data are determined in a preset data processing rule library according to the business data dimension information.

4. The method according to any one of claims 1 to 3, wherein the preset data processing rule base is created by the following steps: Obtaining the number of matrix operation units on the processing unit and determining the number of matrix operation unit groups; Determining the number of unit group matrix operation units corresponding to each matrix operation unit group according to the number of matrix operation units and the number of matrix operation unit groups, and determining matrix operation unit group specification information according to the number of unit group matrix operation units; Acquire at least one business data dimension information, and determine a data processing rule corresponding to each business data dimension information according to each business data dimension information and the matrix operation unit group specification information; A preset data processing rule library is generated according to each data processing rule and the number of matrix operation unit groups.

5. The method according to any one of claims 1 to 4, wherein the target data processing rule includes matrix operation unit group specification information; Splitting the to-be-processed business data according to the target data processing rule to obtain at least one to-be-processed business sub-data includes: Determining a splitting rule for the service data to be processed according to the matrix operation unit group specification information and the service matrix parameters of the service data to be processed; The business data to be processed is split according to the splitting rule to obtain at least one business sub-data to be processed.

6. The method according to claim 5, wherein the target data processing rule includes the number of matrix operation unit groups; Determine the matrix operation unit group corresponding to each business sub-data to be processed, including: Get the number of sub-data of business sub-data to be processed; The matrix operation unit group corresponding to each to-be-processed business sub-data is determined according to the number of the sub-data and the number of the matrix operation unit groups.

7. The method according to any one of claims 1 to 6, wherein each matrix operation unit group processes each to-be-processed business sub-data to obtain a business processing sub-result corresponding to each to-be-processed business sub-data, comprising: Determining a target matrix operation unit group and target to-be-processed business sub-data corresponding to the target matrix operation unit group, wherein the target matrix operation unit group is any one of the matrix operation unit groups; Processing the target to-be-processed business sub-data based on each matrix operation unit in the target matrix operation unit group to obtain a target business processing sub-result corresponding to the target to-be-processed business sub-data; Accordingly, determining the business processing result corresponding to the business data processing request according to each business processing sub-result includes: The business processing result is determined according to each business processing sub-result and the third business data to be processed.

8. The method according to claim 7, wherein the to-be-processed business sub-data comprises first to-be-processed business sub-data and second to-be-processed business sub-data; Processing the target to-be-processed business sub-data based on each matrix operation unit in the target matrix operation unit group to obtain a target business processing sub-result corresponding to the target to-be-processed business sub-data includes: The first to-be-processed business sub-data and the second to-be-processed business sub-data are processed based on each matrix operation unit to obtain a target business processing sub-result.

9. The method according to claim 8, wherein each matrix operation unit processes the first to-be-processed business sub-data and the second to-be-processed business sub-data to obtain a target business processing sub-result, comprising: Processing the first to-be-processed business sub-data and the second to-be-processed business sub-data to obtain reference business sub-data; Accordingly, determining the business processing result according to each business processing sub-result and the third business data to be processed includes: A business processing result is obtained according to the business processing sub-results corresponding to each matrix operation unit group and the third business data to be processed.

10. The method according to claim 8 or 9, wherein the first to-be-processed business sub-data comprises business application data, the second to-be-processed business sub-data comprises model parameter data, and the third to-be-processed business sub-data comprises model parameter data.

11. The method according to any one of claims 1 to 10, further comprising, after determining the business processing result corresponding to the business data processing request according to each business processing sub-result: The business processing result is saved in a memory.

12. A processing unit comprising at least one matrix operation unit, wherein the processing unit stores computer executable instructions, and when the computer executable instructions are executed by a processor, the steps of the method according to any one of claims 1 to 11 are implemented.

13. A processor, comprising at least one processing unit, each processing unit comprising at least one matrix operation unit, the processor being configured to execute computer-executable instructions, wherein the computer-executable instructions, when executed by the processor, implement the steps of the method according to any one of claims 1 to 11.

14. A system on chip, comprising a processor and a memory, wherein the processor comprises a single-core processor or a multi-core processor, and the processor and the memory are coupled via an on-chip network; A memory configured to store business data to be processed; A processor comprising at least one processing unit, wherein the processing unit comprises at least one matrix operation unit, and the processor is configured to execute computer-executable instructions, wherein the computer-executable instructions, when executed by the processor, implement the steps of the method according to any one of claims 1 to 11.

15. A computing device comprising one or more memories storing computer-executable instructions and at least one processor, wherein the processor comprises at least one processing unit, and the processing unit comprises at least one matrix operation unit, and the processor is configured to execute computer-executable instructions, wherein the computer-executable instructions, when executed by the processor, implement the steps of the method described in any one of claims 1 to 11.

16. A computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions, wherein the computer-executable instructions are used to implement the steps of the method according to any one of claims 1 to 11 when executed by a processor.

17. A computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements the steps of the method according to any one of claims 1 to 11.