Data processing method and apparatus, device, chip, and storage medium

Abstract

A data processing method and apparatus, a device, a chip, and a storage medium, relating to the technical field of computers. The method comprises: determining a first matrix and a second matrix in a memory; when the total data amount of the first matrix and the second matrix is larger than the buffer capacity of a buffer, determining a plurality of segmentation strategies, wherein a segmentation strategy comprises a segmentation manner and a loading sequence; simulating the plurality of segmentation strategies to obtain a loading duration of each segmentation strategy; determining a target segmentation strategy from the plurality of segmentation strategies, wherein the loading duration of the target segmentation strategy meets a performance condition; and segmenting at least one of the first matrix or the second matrix on the basis of the target segmentation strategy, loading the segmented matrix to the buffer, and carrying out an operation on the matrix loaded into the buffer.

Description

Data processing method, device, equipment, chip and storage medium

[0001] This application claims priority to the Chinese patent application filed on August 28, 2023, with application number 202311089778.X and invention name “Data processing method, device, equipment, chip and storage medium”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The embodiments of the present application relate to the field of computer technology, and in particular to a data processing method, apparatus, device, chip, and storage medium. Background Art

[0003] AI (Artificial Intelligence) chips are used to perform computational tasks in the field of artificial intelligence, such as the matrix multiplication required in neural networks. To increase the speed of accessing the matrices, memory stores the two matrices that require matrix multiplication. The matrices are then transferred from memory to a buffer, where they are then read by a matrix operation engine and operated on.

[0004] Summary of the Invention

[0005] The embodiments of the present application provide a data processing method, apparatus, device, chip and storage medium, and the technical solution is as follows.

[0006] In one aspect, a data processing method is provided, which is executed by a computer device, and the method comprises:

[0007] Determining a first matrix and a second matrix in a memory, where the first matrix and the second matrix are two matrices to be subjected to matrix operations in a neural network;

[0008] When the total amount of data of the first matrix and the second matrix exceeds the cache capacity of the cache, determining multiple splitting strategies, the splitting strategies including a splitting method and a loading order, the splitting method being a method of splitting at least one of the first matrix or the second matrix, and the loading order being an order of loading the split matrices from the memory into the cache;

[0009] Simulating the multiple segmentation strategies to obtain the loading time of each segmentation strategy;

[0010] Determining a target switching strategy among the multiple segmentation strategies, wherein the loading duration of the target segmentation strategy satisfies a performance condition;

[0011] At least one of the first matrix or the second matrix is ​​segmented according to the target segmentation strategy, the segmented matrix is ​​loaded into the buffer, and operations are performed on the matrix loaded into the buffer.

[0012] In another aspect, a data processing device is provided, comprising:

[0013] A matrix determination module, configured to determine a first matrix and a second matrix in a memory, where the first matrix and the second matrix are two matrices to be subjected to matrix operations in a neural network;

[0014] a strategy determination module, configured to determine, when the total amount of data of the first matrix and the second matrix exceeds the cache capacity of the cache, a plurality of splitting strategies, the splitting strategies including a splitting method and a loading order, the splitting method being a method of splitting at least one of the first matrix or the second matrix, and the loading order being an order in which the split matrices are loaded from the memory into the cache;

[0015] A simulation module, configured to simulate the plurality of segmentation strategies to obtain a loading time for each segmentation strategy;

[0016] a strategy selection module, configured to determine a target switching strategy from among the plurality of segmentation strategies, wherein the loading time of the target segmentation strategy satisfies a performance condition;

[0017] The matrix processing module is used to split at least one of the first matrix or the second matrix according to the target segmentation strategy, load the segmented matrix into the buffer, and perform operations on the matrix loaded into the buffer.

[0018] Optionally, the policy determination module is configured to:

[0019] Acquire multiple segmentation templates, wherein the segmentation templates indicate segmentation targets and segmentation directions, and the segmentation target refers to a matrix to be segmented;

[0020] For any segmentation template, based on the segmentation template, the data volume of the first matrix, and the data volume of the second matrix, a segmentation strategy corresponding to the segmentation template is searched. The segmentation strategy corresponding to the segmentation template refers to a segmentation strategy that conforms to the segmentation template and satisfies a segmentation condition. The segmentation condition refers to that after segmentation according to the segmentation strategy, the total amount of matrix data that needs to be loaded into the cache for calculation each time is not greater than the cache capacity.

[0021] Optionally, the policy determination module is configured to:

[0022] Finding a number of splits that meets the splitting condition based on a splitting target indicated by the splitting template, a splitting direction indicated by the splitting template, a data volume of the first matrix, and a data volume of the second matrix, wherein the splitting method includes the splitting target, the splitting direction, and the number of splits;

[0023] Based on the segmentation target and segmentation direction indicated by the segmentation template, the operation order of the matrix after segmentation is determined, and based on the operation order, the loading order is determined.

[0024] Optionally, the policy determination module is configured to:

[0025] Determining, based on the segmentation target, the segmentation direction, the amount of data in the first matrix, the amount of data in the second matrix, and at least one candidate segmentation number, an evaluation result for each candidate segmentation number, the evaluation result indicating whether the candidate segmentation number satisfies the segmentation condition, the candidate segmentation number being a positive integer greater than 1;

[0026] Based on the evaluation result of at least one candidate number of cuts, a number of cuts that meets the cut condition is determined from the at least one candidate number of cuts.

[0027] Optionally, the policy determination module is configured to:

[0028] Determining an evaluation result of the i-th candidate number of splits based on the split target, the split direction, the data volume of the first matrix, the data volume of the second matrix, and the i-th candidate number of splits;

[0029] Based on the segmentation target, the segmentation direction, the data volume of the first matrix, the data volume of the second matrix and the i+1th candidate segmentation number, determine the evaluation result of the i+1th candidate segmentation number until the stop search condition is met; wherein the i+1th candidate segmentation number is equal to i+1, and i is a positive integer greater than 1.

[0030] Optionally, the stop search condition includes at least one of the following:

[0031] The evaluation result of the current candidate number of splits indicates that the split condition is satisfied;

[0032] After segmentation according to the current candidate segmentation number, the data amount of the obtained minimum matrix is ​​less than the minimum operation amount of the matrix operation engine, and the matrix operation engine is used to operate on the matrix.

[0033] Optionally, the multiple segmentation templates include at least one target segmentation template and at least one non-target segmentation template, the segmentation target indicated by the target segmentation template is one of any two matrices, and the segmentation target indicated by the non-target segmentation template is each of any two matrices;

[0034] The strategy determination module is used to:

[0035] Find the segmentation strategy corresponding to each target segmentation template in turn;

[0036] When at least one segmentation strategy corresponding to the target segmentation template is found, the search stops;

[0037] When no segmentation strategy corresponding to any target segmentation template is found, the segmentation strategy corresponding to each non-target segmentation template is searched in turn.

[0038] Optionally, the plurality of segmentation templates include at least one of a first segmentation template, a second segmentation template, a third segmentation template, a fourth segmentation template, a fifth segmentation template, or a sixth segmentation template;

[0039] The first segmentation template indicates that the right matrix of any two matrices is segmented in the width direction;

[0040] The second segmentation template indicates that the left matrix of any two matrices is segmented in the height direction;

[0041] The third segmentation template indicates that the left matrix of any two matrices is segmented in the height direction and the right matrix is ​​segmented in the width direction;

[0042] The fourth segmentation template indicates that the left matrix of any two matrices is segmented in the height direction and the right matrix is ​​segmented in the height direction and the width direction;

[0043] The fifth segmentation template indicates that the left matrix of any two matrices is segmented in the height direction and the width direction and the right matrix is ​​segmented in the width direction;

[0044] The sixth segmentation template indicates that the left matrix of any two matrices is segmented in the height direction and the width direction, and the right matrix is ​​segmented in the height direction and the width direction.

[0045] Optionally, the simulation module is used to:

[0046] Determining the number of loads and the amount of data loaded each time based on the segmentation strategy, the amount of data in the first matrix, and the amount of data in the second matrix;

[0047] The loading duration of the splitting strategy is determined based on the number of load times, the amount of data loaded each time, and the read and write bandwidth of the memory.

[0048] Optionally, the matrix processing module is used to:

[0049] For the matrix currently required to be loaded, the matrix is ​​divided into a first block matrix and a second block matrix;

[0050] The first block matrix is ​​loaded into the buffer, and the first block matrix in the buffer is operated. During the operation of the first block matrix, the second block matrix is ​​loaded into the buffer in parallel.

[0051] Optionally, the segmentation strategy further includes a cache address of the matrix after segmentation, and the matrix processing module is configured to:

[0052] For a matrix currently required to be loaded, determining a cache address of the matrix;

[0053] The matrix is ​​loaded into the cache space indicated by the cache address in the cache.

[0054] Optionally, the matrix processing module is used to:

[0055] Calling a matrix operation engine to operate the matrix in the buffer to obtain an operation result;

[0056] The operation result is stored in the memory.

[0057] In another aspect, a chip is provided, the chip being communicatively connected to a memory and including a buffer; the chip being configured to perform the following operations:

[0058] Determining a first matrix and a second matrix in the memory, where the first matrix and the second matrix are two matrices to be subjected to matrix operations in a neural network;

[0059] determining, when the total amount of data of the first matrix and the second matrix exceeds the cache capacity of the cache, a plurality of splitting strategies, the splitting strategies including a splitting method and a loading order, the splitting method referring to a method of splitting at least one of the first matrix or the second matrix, and the loading order referring to an order in which the split matrices are loaded from the memory into the cache;

[0060] Simulating the multiple segmentation strategies to obtain the loading time of each segmentation strategy;

[0061] Determining a target switching strategy among the multiple segmentation strategies, wherein the loading duration of the target segmentation strategy satisfies a performance condition;

[0062] At least one of the first matrix or the second matrix is ​​segmented according to the target segmentation strategy, the segmented matrix is ​​loaded into the buffer, and operations are performed on the matrix loaded into the buffer.

[0063] On the other hand, a computer device is provided, comprising a processor and a memory, wherein the memory stores at least one computer program, and the at least one computer program is loaded and executed by the processor to implement the operations performed by the data processing method described in the above aspects.

[0064] On the other hand, a computer-readable storage medium is provided, in which at least one computer program is stored. The at least one computer program is loaded and executed by a processor to implement the operations performed by the data processing method described in the above aspects.

[0065] On the other hand, a computer program product is provided, comprising a computer program, wherein the computer program is loaded and executed by a processor to implement the operations performed by the data processing method as described in the above aspects.

[0066] In the solution provided by the embodiment of the present application, the first matrix and the second matrix to be calculated are stored in a memory. In order to facilitate the subsequent reading of the first matrix and the second matrix during the calculation, the first matrix and the second matrix need to be loaded from the memory into a buffer. If the total amount of data of the first matrix and the second matrix exceeds the cache capacity of the buffer, the first matrix and the second matrix cannot be directly loaded into the buffer at the same time for calculation, and the matrix needs to be split and loaded in batches. Therefore, a splitting strategy for the first matrix and the second matrix is ​​determined. The splitting strategy is used to indicate how to split the first matrix and the second matrix, and the order in which each matrix obtained after the split is loaded into the buffer. After obtaining multiple splitting strategies, each splitting strategy is simulated to obtain the loading time of each splitting strategy, so as to find the target splitting strategy whose loading time meets the performance conditions among the multiple splitting strategies, so as to facilitate the screening of the splitting strategy with a shorter loading time. Therefore, the splitting is performed according to the target splitting strategy, and then the split matrices are loaded into the cache in sequence for calculation. On the one hand, it solves the problem that the total amount of data of the first matrix and the second matrix is ​​too large to be loaded into the cache at one time. On the other hand, it can find the splitting strategy with a shorter loading time among the multiple splitting strategies, thereby reducing the time spent on loading multiple matrices, thereby improving the overall efficiency of data processing. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] FIG1 is a system architecture diagram of a data processing method provided in an embodiment of the present application;

[0068] FIG2 is a flow chart of a data processing method provided in an embodiment of the present application;

[0069] FIG3 is a flow chart of another data processing method provided in an embodiment of the present application;

[0070] FIG4 is a schematic diagram of a cutting template provided in an embodiment of the present application;

[0071] FIG5 is a flow chart of another data processing method provided in an embodiment of the present application;

[0072] FIG6 is a system architecture diagram of another data processing method provided in an embodiment of the present application;

[0073] FIG7 is a flowchart of another data processing method provided in an embodiment of the present application;

[0074] FIG8 is a schematic structural diagram of a data processing device provided in an embodiment of the present application;

[0075] FIG9 is a schematic structural diagram of a terminal provided in an embodiment of the present application;

[0076] FIG10 is a schematic diagram of the structure of a server provided in an embodiment of the present application. DETAILED DESCRIPTION

[0077] In order to make the objectives, technical solutions and advantages of the embodiments of the present application clearer, the implementation methods of the present application will be further described in detail below with reference to the accompanying drawings.

[0078] It is understood that the terms "first," "second," and the like used herein may be used to describe various concepts, but unless otherwise specified, these concepts are not limited by these terms. These terms are used solely to distinguish one concept from another. For example, a first matrix may be referred to as a second matrix, and similarly, a second matrix may be referred to as a first matrix without departing from the scope of this application.

[0079] Here, "at least one" refers to one or more than one. For example, the at least one matrix can be one matrix, two matrices, three matrices, or any other integer greater than or equal to one. "Multiple" refers to two or more than two. For example, the multiple matrices can be two matrices, three matrices, or any other integer greater than or equal to two. "Each" refers to each of the at least one. For example, "each matrix" refers to each matrix in the multiple matrices. If the multiple matrices are three matrices, "each matrix" refers to each matrix in the three matrices.

[0080] It should be noted that the information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data used for analysis, stored data, displayed data, etc.) and signals (including but not limited to signals transmitted between user terminals and other devices, etc.) involved in this application are all fully authorized by users or relevant parties, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards of relevant countries and regions.

[0081] In matrix operations, the matrix to be operated on is typically stored in memory. To increase the speed of matrix access, the matrix must first be moved from memory to a buffer. The matrix operation engine then reads the matrix from the buffer and performs the operation on the read matrix. However, since the buffer's cache capacity is typically small, if the matrix data size exceeds the buffer's cache capacity, the matrix to be operated on cannot be moved to the buffer, resulting in a limited data processing process.

[0082] Based on this, an embodiment of the present application proposes a data processing method, which divides the matrix and moves the matrix into batches to a buffer for calculation. The method provided by the embodiment of the present application is described in detail below.

[0083] The data processing method provided in the embodiment of the present application can be used in a computer device. Optionally, the computer device is a server or a terminal. Optionally, the server is an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms. Optionally, the terminal is a smart phone, tablet computer, laptop computer, desktop computer, smart speaker, smart watch, smart terminal, etc., but is not limited to this.

[0084] Figure 1 is a system architecture diagram of a data processing method provided in an embodiment of the present application. The execution subject of the embodiment of the present application is a computer device, such as a server or a terminal. As shown in Figure 1, the computer device includes a chip 101 and a memory 102, and the chip includes a cache 1011.

[0085] In an embodiment of the present application, it can be applied to a scenario where matrices in a neural network are operated, such as matrix multiplication operations. A first matrix and a second matrix are stored in the memory 102. The first matrix and the second matrix are two matrices to be subjected to matrix operations in the neural network. For example, the first matrix is ​​the input matrix of a layer in the neural network, and the second matrix is ​​the weight matrix included in the layer. The weight matrix is ​​used to perform convolution on the input matrix, and convolution is a special matrix multiplication operation. The chip 101 is used to determine the target segmentation strategy of the first matrix and the second matrix when the total amount of data of the first matrix and the second matrix is ​​greater than the cache capacity of the buffer 1011, and to segment at least one of the first matrix or the second matrix according to the target segmentation strategy, and then load the segmented matrix from the memory 102 into the buffer 1011. Optionally, as shown in FIG1 , the chip 101 also includes a matrix operation engine 1012, which is used to perform operations on the matrix loaded into the buffer 1011.

[0086] FIG2 is a flow chart of a data processing method provided in an embodiment of the present application. The embodiment of the present application is executed by a computer device. Referring to FIG2 , the method includes the following steps.

[0087] 201. A computer device determines a first matrix and a second matrix in a memory, where the first matrix and the second matrix are two matrices to be subjected to matrix operations in a neural network.

[0088] A computer device includes a memory storing a first matrix and a second matrix. The first matrix and the second matrix are two matrices to be subjected to a matrix operation in a neural network, such as a matrix multiplication operation. The matrix multiplication operation includes a multiplication operation of two matrices or a convolution operation of two matrices.

[0089] In one possible implementation, the first matrix is ​​the input matrix of any layer in the neural network, where the input matrix is ​​the matrix to be convolved, and the second matrix is ​​the weight matrix of the layer in the neural network, where the weight matrix is ​​the weight matrix of the convolution kernel and is used to convolve the input matrix of the layer. For example, the layer refers to any convolutional layer in the neural network.

[0090] Optionally, the first matrix is ​​an input matrix of the first layer in the neural network.

[0091] Optionally, the first matrix is ​​the input matrix of the nth layer in the neural network, n is a positive integer greater than 1, and the first matrix is ​​also the output matrix of the n-1th layer in the neural network.

[0092] The memory may be any type of memory, for example, a DDR (Double Data Rate) memory, etc., which is not limited in the embodiment of the present application.

[0093] 202. When the total amount of data in the first matrix and the second matrix is ​​greater than the cache capacity of the cache, the computer device determines multiple splitting strategies, where the splitting strategies include a splitting method and a loading order. The splitting method refers to a method of splitting at least one of the first matrix or the second matrix, and the loading order refers to an order in which the split matrices are loaded from the memory to the cache.

[0094] The computer device also includes a buffer, which is a buffer on the processor or chip of the computer device. The capacity of the buffer is smaller than the capacity of the memory, but the read and write speed of the buffer is faster than the read and write speed of the memory. In order to improve the read and write speed during the matrix operation, it is necessary to load the matrix in the memory into the buffer, that is, to move the matrix in the memory into the buffer, and then read the matrix in the buffer by the processor or chip to perform the operation. However, since the cache capacity of the buffer is small, there is a situation where the amount of data of the matrix to be loaded is greater than the cache capacity of the buffer. Therefore, in the embodiment of the present application, after determining the first matrix and the second matrix to be operated, the computer device determines the total amount of data of the first matrix and the second matrix, and judges whether the total amount of data of the first matrix and the second matrix is ​​greater than the cache capacity of the buffer.

[0095] If the total amount of data in the first matrix and the second matrix exceeds the cache capacity of the cache, the cache cannot accommodate both the first matrix and the second matrix simultaneously. This means that both the first matrix and the second matrix cannot be directly transferred to the cache for computation at once. Therefore, at least one of the first matrix or the second matrix must be segmented and loaded into the cache in batches for computation, ensuring that the total amount of data in each batch of matrices required for computation does not exceed the cache capacity of the cache. Therefore, in this case, the computer device determines multiple segmentation strategies, each of which is different and includes a segmentation method and a loading order. The segmentation method indicates how to segment the first matrix and the second matrix, and the loading order indicates the order in which the segmented matrices are loaded into the cache.

[0096] The "sliced ​​matrix" refers to the matrix remaining after slicing at least one of the first matrix or the second matrix. The "sliced ​​matrix" includes the submatrices obtained by slicing the first matrix or the second matrix, or the "sliced ​​matrix" also includes the unsliced ​​first matrix or the second matrix. For example, when only the first matrix is ​​sliced, the "sliced ​​matrix" includes the second matrix and multiple submatrices obtained by slicing the first matrix. When only the second matrix is ​​sliced, the "sliced ​​matrix" includes the first matrix and multiple submatrices obtained by slicing the second matrix. When both the first matrix and the second matrix are sliced, the "sliced ​​matrix" includes multiple submatrices obtained by slicing the first matrix and multiple submatrices obtained by slicing the second matrix.

[0097] It should be noted that the embodiments of the present application are described using an example in which the total amount of data in the first matrix and the second matrix is ​​greater than the cache capacity of the cache. In another embodiment, if the total amount of data in the first matrix and the second matrix is ​​not greater than the cache capacity of the cache, there is no need to split the first matrix and the second matrix. The first matrix and the second matrix can be directly loaded into the cache at the same time, and then the first matrix and the second matrix can be directly read from the cache and operated on.

[0098] 203. The computer device simulates multiple segmentation strategies to obtain the loading time of each segmentation strategy.

[0099] Since at least one of the first matrix or the second matrix is ​​split, the first matrix and the second matrix cannot be loaded into the cache all at once. The split matrices need to be loaded into the cache in batches. Therefore, splitting the matrix will incur additional loading time.

[0100] Different sharding strategies involve different sharding methods and loading sequences. Different sharding methods require different load times and different amounts of data per load. Consequently, different sharding methods require different loading times, meaning each sharding strategy has its own loading time. The loading time of a sharding strategy reflects its performance: longer loading times indicate lower processing efficiency, while shorter loading times indicate higher processing efficiency.

[0101] Therefore, in order to select a segmentation strategy with better processing performance from multiple segmentation strategies, the computer device simulates the multiple segmentation strategies respectively to obtain the loading time of each segmentation strategy. The loading time of the segmentation strategy is the total time required to load the segmented matrix into the cache in batches when the segmentation strategy is used for processing.

[0102] Here, simulating the partitioning strategy refers to simulating the process of partitioning at least one of the first matrix or the second matrix according to the partitioning strategy and then loading the partitioned matrices into the buffer in batches, thereby obtaining the loading duration of the entire process. It should be noted that the loading duration does not include the computation duration of performing operations on the matrices loaded into the buffer.

[0103] 204. The computer device determines a target switching strategy from a plurality of segmentation strategies, and the loading time of the target segmentation strategy satisfies the performance condition.

[0104] Since the loading time of a segmentation strategy can reflect the performance of the segmentation strategy, the computer device determines a target segmentation strategy whose loading time meets the performance conditions among multiple segmentation strategies, thereby screening out a target segmentation strategy with relatively better performance.

[0105] Optionally, the loading time satisfying the performance condition means that the loading time is less than a preset time.

[0106] Optionally, the loading duration meeting the performance condition means that the loading duration is the minimum value among the loading durations of multiple segmentation strategies.

[0107] 205. The computer device divides at least one of the first matrix or the second matrix according to the target division strategy, loads the divided matrix into a buffer, and performs operations on the matrix loaded into the buffer.

[0108] After determining the target segmentation strategy, the computer device segments at least one of the first matrix or the second matrix according to the segmentation method indicated by the target segmentation strategy to obtain a segmented matrix, and then loads the segmented matrix into the cache in batches according to the loading order indicated by the target segmentation strategy. Each time the loading is completed, the current matrix in the cache is operated.

[0109] Among them, the computer device loads the split matrix into the cache in accordance with the loading order. After completing the operation on the matrix in the cache, the computer device loads the next split matrix into the cache, and the next matrix will replace the matrix in the cache that has completed the operation. Then the operation is performed on the next matrix in the cache, and so on, until the operation of all the split matrices is completed, thereby realizing that the split matrices are loaded into the cache in batches for operation, and the total amount of data of the matrices required for operation in each batch is not greater than the cache capacity of the cache, so as to ensure that the matrices required for operation can be loaded into the cache.

[0110] In the method provided by the embodiment of the present application, the first matrix and the second matrix to be calculated are stored in a memory. In order to facilitate the subsequent reading of the first matrix and the second matrix during the calculation, the first matrix and the second matrix need to be loaded from the memory into a buffer. If the total amount of data of the first matrix and the second matrix exceeds the cache capacity of the buffer, the first matrix and the second matrix cannot be directly loaded into the buffer at the same time for calculation, and it is necessary to split the matrix and load them in batches. Therefore, a splitting strategy for the first matrix and the second matrix is ​​determined. The splitting strategy is used to indicate how to split the first matrix and the second matrix, and the order in which each matrix obtained after the splitting is loaded into the buffer. After obtaining multiple splitting strategies, each splitting strategy is simulated to obtain the loading time of each splitting strategy, so as to find the target splitting strategy whose loading time meets the performance conditions among the multiple splitting strategies, so as to facilitate the screening of the splitting strategy with a shorter loading time. Therefore, the splitting is performed according to the target splitting strategy, and then the split matrices are loaded into the cache in sequence for calculation. On the one hand, it solves the problem that the total amount of data of the first matrix and the second matrix is ​​too large to be loaded into the cache at one time. On the other hand, it can find the splitting strategy with a shorter loading time among the multiple splitting strategies, thereby reducing the time spent on loading multiple matrices, thereby improving the overall efficiency of data processing.

[0111] The embodiment shown in FIG2 above is a brief description of the data processing method provided in the embodiment of the present application. The detailed process of the data processing method can be seen in the embodiment shown in FIG3 below. FIG3 is a flowchart of another data processing method provided in the embodiment of the present application. The embodiment of the present application is executed by a computer device. Referring to FIG3, the method includes the following steps.

[0112] 301. A computer device determines a first matrix and a second matrix in a memory, where the first matrix and the second matrix are two matrices to be subjected to a matrix multiplication operation in a neural network.

[0113] The step 301 is similar to the above step 201 and will not be described again here.

[0114] 302. When the total amount of data in the first matrix and the second matrix exceeds the buffer capacity of the buffer, the computer device obtains multiple segmentation templates, where the segmentation templates indicate segmentation targets and segmentation directions, and the segmentation target refers to the matrix to be segmented.

[0115] If the total amount of data in the first matrix and the second matrix is ​​greater than the cache capacity of the cache, at least one of the first matrix or the second matrix needs to be segmented, and the computer device obtains multiple segmentation templates, which are different, and each segmentation template includes a segmentation target and a segmentation direction.

[0116] Among them, the split target refers to the matrix that needs to be split between any two matrices. For example, in a matrix multiplication operation, the two matrices to be operated are divided into a left matrix and a right matrix. Among them, the left matrix refers to the matrix on the left side in the matrix multiplication operation, and the right matrix refers to the matrix on the right side in the matrix multiplication operation. When two matrices are multiplied, the operation rule is to multiply the rows of the left matrix with the columns of the right matrix, and add the results. Then, optionally, if the split target is the left matrix, the left matrix needs to be split. Alternatively, if the split target is the right matrix, the right matrix needs to be split. Alternatively, if the split target is the left matrix and the right matrix, both the left matrix and the right matrix need to be split.

[0117] The "slicing direction" refers to the direction in which the matrix is ​​sliced, and the matrix includes a height direction and a width direction. Optionally, if the slicing direction is the height direction, the matrix needs to be sliced ​​horizontally, and the height of the matrix after slicing will be changed. Optionally, if the slicing direction is the width direction, the matrix needs to be sliced ​​vertically, and the width of the matrix after slicing will be changed. Optionally, if the slicing direction includes the height direction and the width direction, the matrix needs to be sliced ​​in a combination of the horizontal and vertical directions, and the height and width of the matrix after slicing will be changed.

[0118] In one possible implementation, the multiple segmentation templates include at least one of a first segmentation template, a second segmentation template, a third segmentation template, a fourth segmentation template, a fifth segmentation template, or a sixth segmentation template. The first segmentation template, the second segmentation template, the third segmentation template, the fourth segmentation template, the fifth segmentation template, and the sixth segmentation template are described in detail as follows.

[0119] (1) The first splitting template indicates that the right matrix of any two matrices is split in the width direction. The splitting target of the first splitting template is the right matrix, and the splitting direction is the width direction. As shown in (1) in Figure 4, the left matrix 401 does not need to be split, and the right matrix 402 needs to be split in the width direction, that is, vertical splitting.

[0120] (2) The second segmentation template indicates that the left matrix of any two matrices is segmented in the height direction. The segmentation target of the second segmentation template is the left matrix, and the segmentation direction is the height direction. As shown in (2) in Figure 4, the right matrix 402 does not need to be segmented, and the left matrix 401 needs to be segmented in the height direction, that is, horizontal segmentation.

[0121] (3) The third segmentation template indicates that the left matrix of any two matrices is segmented in the height direction and the right matrix is ​​segmented in the width direction. The segmentation targets of the third segmentation template are the left matrix and the right matrix. The segmentation direction of the left matrix is ​​the height direction, and the segmentation target of the right matrix is ​​the width direction. As shown in (3) in Figure 4, the left matrix 401 needs to be segmented in the height direction, that is, horizontal segmentation, and the right matrix 402 needs to be segmented in the width direction, that is, vertical segmentation.

[0122] (4) The fourth segmentation template indicates that the left matrix of any two matrices is segmented in the height direction and the right matrix is ​​segmented in the height and width directions. The segmentation targets of the fourth segmentation template are the left matrix and the right matrix, the segmentation direction of the left matrix is ​​the height direction, and the segmentation targets of the right matrix are the height and width directions. As shown in (4) in Figure 4, the left matrix 401 needs to be segmented in the height direction, that is, horizontal segmentation, and the right matrix 402 needs to be segmented in the height and width directions, that is, a combination of horizontal and vertical segmentation.

[0123] (5) The fifth segmentation template indicates that the left matrix of any two matrices is segmented in the height direction and the width direction, and the right matrix is ​​segmented in the width direction. The segmentation targets of the fifth segmentation template are the left matrix and the right matrix. The segmentation directions of the left matrix are the height direction and the width direction, and the segmentation target of the right matrix is ​​the width direction. As shown in (5) in Figure 4, the left matrix 401 needs to be segmented in the height direction and the width direction, that is, it is segmented in a combination of horizontal and vertical directions, and the right matrix 402 needs to be segmented in the width direction, that is, it is segmented in the vertical direction.

[0124] (6) The sixth segmentation template indicates that the left matrix of any two matrices is segmented in the height direction and the width direction, and the right matrix is ​​segmented in the height direction and the width direction. The segmentation targets of the sixth segmentation template are the left matrix and the right matrix, the segmentation directions of the left matrix are the height direction and the width direction, and the segmentation targets of the right matrix are the height direction and the width direction. As shown in (6) in Figure 4, the left matrix 401 needs to be segmented in the height direction and the width direction, that is, the horizontal and vertical directions are combined, and the right matrix 402 needs to be segmented in the height direction and the width direction, that is, the horizontal and vertical directions are combined.

[0125] It should be noted that the above six segmentation templates are only simple examples provided by the embodiments of the present application. In addition to these six segmentation templates, other types of segmentation templates may also be included, for example, a segmentation template indicating that the right matrix of any two matrices is segmented in the height direction, a segmentation template indicating that the left matrix of any two matrices is segmented in the width direction, a segmentation template indicating that the left matrix of any two matrices is segmented in the width direction and the right matrix is ​​segmented in the height direction, a segmentation template indicating that the left matrix of any two matrices is segmented in the width direction and the right matrix is ​​segmented in the height and width directions, a segmentation template indicating that the left matrix of any two matrices is segmented in the height and width directions and the right matrix is ​​segmented in the height direction, etc. The embodiments of the present application do not limit the segmentation templates.

[0126] 303. For any segmentation template, the computer device searches for a segmentation strategy corresponding to the segmentation template based on the segmentation template, the data volume of the first matrix, and the data volume of the second matrix. The segmentation strategy corresponding to the segmentation template refers to a segmentation strategy that conforms to the segmentation template and satisfies the segmentation condition. The segmentation condition refers to the total amount of matrix data that needs to be loaded into the cache for calculation each time after segmentation according to the segmentation strategy is not greater than the cache capacity.

[0127] The computer device searches for a slicing strategy corresponding to any slicing template among the plurality of slicing templates. The slicing strategy includes a slicing method and a loading order. The slicing method refers to a method for slicing at least one of the first matrix or the second matrix, and includes a slicing target and a slicing direction. The loading order refers to the order in which the sliced ​​matrices are loaded from the memory into the cache.

[0128] The segmentation strategy being consistent with the segmentation template means that the segmentation target and segmentation direction indicated by the segmentation strategy are consistent with the segmentation target and segmentation direction indicated by the segmentation template. For example, if the segmentation template indicates that the right matrix is ​​to be segmented in the width direction, then the segmentation strategy that is consistent with the segmentation template is used to indicate that the right matrix of the first matrix and the second matrix is ​​to be segmented into multiple parts in the width direction.

[0129] Among them, the splitting strategy satisfies the splitting condition, which means that after splitting according to the splitting strategy, multiple matrices are obtained after splitting, and the total amount of data of the matrices required for calculation in each batch of the multiple matrices is not greater than the cache capacity. In other words, the total amount of data of the matrices that need to be loaded into the cache for calculation each time is not greater than the cache capacity, so as to ensure that the matrices required for calculation in each batch can be successfully loaded into the cache.

[0130] In an embodiment of the present application, by providing multiple segmentation templates in advance, the process of determining the segmentation strategy can be simplified, which is conducive to quickly finding a segmentation strategy that meets the segmentation conditions in accordance with the segmentation template, and reduces the time for repeatedly adjusting the segmentation target and segmentation direction during the calculation process, thereby reducing the need for complex calculations and judgments, and is conducive to improving overall computing efficiency and decision-making efficiency.

[0131] In addition, multiple segmentation templates provide different segmentation targets and segmentation directions for selection. Different segmentation templates can adapt to different cache capacities and matrix structures, so that the segmentation strategy can be customized according to specific matrix characteristics and computing requirements, thereby improving the flexibility of data processing.

[0132] In a possible implementation, the computer device searches for a segmentation strategy corresponding to the segmentation template based on the segmentation template, the data volume of the first matrix, and the data volume of the second matrix, including the following steps 3031-3032.

[0133] 3031. The computer device searches for the number of splits that meet the splitting conditions based on the splitting target indicated by the splitting template, the splitting direction indicated by the splitting template, the data volume of the first matrix, and the data volume of the second matrix. The splitting method includes the splitting target, the splitting direction, and the number of splits.

[0134] Based on the splitting target, it is possible to determine which matrix in the first matrix and the second matrix to split. Based on the splitting direction, it is possible to determine in which direction the splitting target is split. Based on the number of splits, it is possible to determine how many parts the splitting target is split into in the splitting direction. Then, based on the splitting target, the splitting direction, the data volume of the first matrix and the data volume of the second matrix, and the number of splits, it is possible to determine the data volume of each matrix after the splitting. Furthermore, it is determined whether the total amount of matrix data required for operation in each batch of the split matrix exceeds the cache capacity. If it does not exceed the cache capacity, then the number of splits meets the splitting condition. If it exceeds the cache capacity, then the number of splits does not meet the splitting condition and the number of splits needs to be re-determined.

[0135] Optionally, the computer device determines an evaluation result of at least one candidate number of cuts based on the cut target, the cut direction, the data amount of the first matrix, the data amount of the second matrix and at least one candidate number of cuts, and determines a number of cuts that meets the cut conditions from the at least one candidate number of cuts based on the evaluation result of the at least one candidate number of cuts.

[0136] The evaluation result indicates whether the candidate number of splits meets the splitting conditions, and the candidate number of splits is a positive integer greater than 1.

[0137] For each candidate number of splits, the computer device determines an evaluation result for the candidate number of splits based on the splitting target, the splitting direction, the amount of data in the first matrix, the amount of data in the second matrix, and the candidate number of splits, thereby obtaining an evaluation result for each candidate number of splits. Because the evaluation result indicates whether the candidate number of splits meets the splitting conditions, based on the evaluation results of each candidate number of splits, a number of splits that meets the splitting conditions can be screened from at least one candidate number of splits.

[0138] In the embodiment of the present application, by evaluating whether multiple candidate split numbers meet the splitting conditions, the most appropriate split number can be flexibly selected for different matrix sizes and data volumes, so that the splitting strategy can meet the splitting conditions, thereby enhancing the adaptability of the algorithm, being able to better handle various complex computing scenarios, and improving computing speed and efficiency.

[0139] Optionally, the computer device determines an evaluation result of at least one candidate number of splits based on the splitting target, the splitting direction, the data volume of the first matrix, the data volume of the second matrix, and at least one candidate number of splits, including: determining an evaluation result of the i-th candidate number of splits based on the splitting target, the splitting direction, the data volume of the first matrix, the data volume of the second matrix, and the i-th candidate number of splits; and determining an evaluation result of the i+1-th candidate number of splits based on the splitting target, the splitting direction, the data volume of the first matrix, the data volume of the second matrix, and the i+1-th candidate number of splits, until a stop search condition is met. The i+1-th candidate number of splits is equal to i+1, where i is a positive integer greater than 1.

[0140] That is, the evaluation results of different candidate split shares are determined in order from small to large, until the stop search condition is met, and then the split share that meets the split condition is determined among the at least one candidate split share based on the evaluation result of the determined at least one candidate split share.

[0141] For example, starting from the number of candidate split shares equal to 2, determine whether the split condition is met when the number of candidate split shares equals 2, then determine whether the split condition is met when the number of candidate split shares equals 3, and then determine whether the split condition is met when the number of candidate split shares equals 4, and so on, until the stop search condition is met, then the candidate split share number that meets the split condition is determined as the final split share number found.

[0142] For example, the first matrix is ​​the left matrix, the second matrix is ​​the right matrix, and the splitting template indicates that the right matrix is ​​split in the width direction. Taking the evaluation result when the candidate split number is equal to 2 as an example, the computer device determines the data amount of submatrix A and the data amount of submatrix B obtained after splitting the second matrix in the width direction into 2 parts. Since it is necessary to perform operations on the first matrix and submatrix A and submatrix B respectively, it is necessary to load the first matrix and submatrix A into the buffer for operation, and then load the first matrix and submatrix B into the buffer for operation. If the total amount of data of the first matrix and submatrix A is not greater than the cache capacity of the buffer, and the total amount of data of the first matrix and submatrix B is not greater than the cache capacity of the buffer, then the evaluation result indicates that the splitting condition is met when the candidate split number is equal to 2. If the total amount of data of the first matrix and submatrix A is greater than the cache capacity of the buffer, or the total amount of data of the first matrix and submatrix B is greater than the cache capacity of the buffer, then the evaluation result indicates that the splitting condition is not met when the candidate split number is equal to 2. Optionally, when the second matrix is ​​evenly divided, the amount of data in submatrix A is equal to the amount of data in submatrix B.

[0143] Optionally, the stop search condition includes at least one of the following.

[0144] (1) The evaluation result of the current candidate number of splits indicates that the splitting conditions are met.

[0145] For example, if the evaluation result when the number of candidate splits is equal to 2 indicates that the splitting condition is not met, then the evaluation result when the number of candidate splits is equal to 3 is determined. If the evaluation result when the number of candidate splits is equal to 3 indicates that the splitting condition is met, there is no need to determine the evaluation result when the number of candidate splits is greater than 3.

[0146] (2) After the matrix is ​​divided according to the current candidate number of divisions, the amount of data of the minimum matrix obtained is less than the minimum amount of operation of the matrix operation engine, and the matrix operation engine is used to operate on the matrix.

[0147] The minimum operation amount of the matrix operation engine refers to the minimum amount of data processed by the matrix operation engine during one operation. For example, if the amount is less than the minimum operation amount, the utilization rate of the matrix operation engine will be too low, so the minimum operation amount of the matrix operation engine is set. In the embodiment of the present application, when the matrix is ​​divided evenly, the larger the number of divisions, the smaller the data amount of the minimum matrix after division. If the data amount of the minimum matrix obtained after division according to a certain candidate number of divisions is less than the minimum operation amount of the matrix operation engine, then the data amount of the minimum matrix obtained after division according to a number greater than the candidate number of divisions must also be less than the minimum operation amount of the matrix operation engine. For example, when the number of candidate splits is less than or equal to 5, the evaluation results all indicate that the splitting conditions are not met, and when the number of candidate splits is equal to 5, the data volume of the minimum matrix after splitting is less than the minimum computational volume of the matrix operation engine. In this case, there is no need to determine the evaluation results when the number of candidate splits is greater than 5. Since the number of splits that meets the switching conditions is not found, it can be determined that the splitting strategy corresponding to the splitting template is not found, that is, the splitting template cannot be applied to split the first matrix and the second matrix.

[0148] In the embodiment of the present application, traversing in order of the number of slices from small to large is beneficial to giving priority to a smaller number of slices. A smaller number of slices means that the matrix obtained by slicing is larger, so the number of loading times can be reduced, thereby reducing the overhead of data loading and improving processing efficiency.

[0149] Moreover, if a smaller number of splits can meet the conditions, there is no need to continue traversing a larger number of splits. Once a number of splits that meet the split conditions is found, the traversal can be stopped immediately, which simplifies the implementation logic, can determine the splitting strategy more quickly, avoids unnecessary calculations, saves computing time and resources, and thus improves processing speed.

[0150] 3032. The computer device determines the operation order of the matrix after segmentation based on the segmentation target and segmentation direction indicated by the segmentation template, and determines the loading order based on the operation order.

[0151] Matrix operations essentially involve multiplying and adding elements within a matrix. Therefore, a matrix operation order must be determined. This order refers to the order in which operations are performed on the matrix elements. The computer first determines the order of operations for the matrix after partitioning based on the partitioning target and partitioning direction. Once this order is determined, the order in which the matrix is ​​loaded after partitioning is determined. This loading order aligns with the order of operations.

[0152] Optionally, as shown in (1) in FIG4 , when the segmentation template is the first segmentation template, the right matrix 402 is traversed in the width direction, that is, starting from the first column element in the right matrix 402 in columns, each column element is operated with the left matrix 401 respectively until the operation of the last column element is completed. Then the loading order is: a sub-matrix of the left matrix 401 and the right matrix 402 is loaded into the cache for operation. After the operation is completed, the next sub-matrix of the right matrix 402 is loaded into the cache to replace the previous sub-matrix, and the operation is performed, and so on, until the operation of each sub-matrix in the right matrix 402 is completed, and each sub-matrix of the left matrix 401 and the right matrix 402 only needs to be loaded once.

[0153] Optionally, as shown in (2) in FIG4 , when the segmentation template is the second segmentation template, the left matrix 401 is traversed in the height direction, that is, the left matrix 401 is arranged in rows, starting from the first row of elements, and each row of elements is operated with the right matrix 402 respectively until the operation of the last row of elements is completed. Then the loading order is: the right matrix 402 and a submatrix of the left matrix 401 are loaded into the cache for operation. After the operation is completed, the next submatrix of the left matrix 401 is loaded into the cache to replace the previous submatrix, and the operation is performed, and so on, until the operation of each submatrix in the left matrix 401 is completed, and each submatrix of the right matrix 402 and the left matrix 401 only needs to be loaded once.

[0154] Alternatively, as shown in (3) in FIG4 , when the segmentation template is the third segmentation template, the left matrix 401 can be traversed in the height direction. Then, each submatrix of the left matrix 401 only needs to be loaded once, and each submatrix of the right matrix 402 needs to be loaded repeatedly multiple times. Then the loading order is: a submatrix of the left matrix 401 and a submatrix of the right matrix 402 are loaded into the buffer for operation. After the operation is completed, the next submatrix of the right matrix 402 is loaded into the buffer to replace the previous submatrix of the right matrix 402, and the operation is performed, and so on, until the operation of each submatrix in the right matrix 402 is completed. Then, the next submatrix of the left matrix 401 is loaded into the buffer to replace the previous submatrix of the left matrix 401, and then each submatrix in the right matrix 402 is loaded in sequence in the same manner as the above process, until each submatrix of the left matrix 401 and each submatrix of the right matrix 402 are operated. Alternatively, the right matrix 402 can be traversed in the width direction, in which case each submatrix of the left matrix 401 needs to be loaded repeatedly multiple times, while each submatrix of the right matrix 402 only needs to be loaded once. The loading order is then as follows: a submatrix of the left matrix 401 and a submatrix of the right matrix 402 are loaded into the buffer for operation. After the operation is completed, the next submatrix of the left matrix 401 is loaded into the buffer to replace the previous submatrix of the left matrix 401, and the operation is performed, and so on, until the operation of each submatrix in the left matrix 401 is completed. Then, the next submatrix of the right matrix 402 is loaded into the buffer to replace the previous submatrix of the right matrix 402, and then each submatrix in the left matrix 401 is loaded in sequence in a manner similar to the above process, until each submatrix of the left matrix 401 and each submatrix of the right matrix 402 have been operated.

[0155] Optionally, as shown in (4) in Figure 4, when the segmentation template is the fourth segmentation template, the left matrix 401 can be traversed in the height direction, and each submatrix of the left matrix 401 only needs to be loaded once, and each submatrix of the right matrix 402 needs to be loaded repeatedly multiple times.

[0156] Optionally, as shown in (5) in Figure 4, when the segmentation template is the fifth segmentation template, the right matrix 402 can be traversed in the width direction, and each submatrix of the right matrix 402 only needs to be loaded once, and each submatrix of the left matrix 401 needs to be loaded repeatedly multiple times.

[0157] Optionally, as shown in (6) in FIG4 , when the segmentation template is the sixth segmentation template, each submatrix of the left matrix 401 needs to be loaded repeatedly multiple times, and each submatrix of the right matrix 402 also needs to be loaded repeatedly multiple times.

[0158] In an embodiment of the present application, when determining the segmentation strategy based on the segmentation template, since the segmentation template has already specified the segmentation target and segmentation direction, it is only necessary to determine the appropriate number of segments. Furthermore, when the segmentation target, segmentation direction and number of segments are known, the appropriate loading order can be determined according to the matrix structure. This simplifies the process of determining the segmentation strategy, facilitates the rapid determination of a segmentation strategy that meets the segmentation conditions, and improves the efficiency of specifying the segmentation strategy.

[0159] In one possible implementation, the multiple segmentation templates include at least one target segmentation template and at least one non-target segmentation template. The target segmentation template indicates a segmentation target of one of any two matrices, and the non-target segmentation template indicates a segmentation target of each of any two matrices. For any segmentation template, the computer device determines a segmentation strategy corresponding to the segmentation template based on the segmentation template, the amount of data in the first matrix, and the amount of data in the second matrix, including: sequentially searching for a segmentation strategy corresponding to each target segmentation template; stopping the search if a segmentation strategy corresponding to at least one target segmentation template is found; and sequentially searching for a segmentation strategy corresponding to each non-target segmentation template if a segmentation strategy corresponding to any target segmentation template is not found.

[0160] For example, as shown in Figure 4, the first and second segmentation templates are target segmentation templates, and the third to sixth segmentation templates are non-target segmentation templates. If a segmentation strategy corresponding to at least one of the first or second segmentation templates has been found, there is no need to search for segmentation strategies corresponding to the third to sixth segmentation templates. If no segmentation strategies corresponding to the first and second segmentation templates have been found, it is necessary to search for segmentation strategies corresponding to the third to sixth segmentation templates.

[0161] In an embodiment of the present application, considering that the segmentation complexity of the target segmentation template is lower than that of the non-target segmentation template, the segmentation strategy corresponding to the target segmentation template is first searched. If a segmentation strategy that satisfies the target segmentation template is found, the segmentation strategy corresponding to the non-segmentation template is no longer searched, thereby saving processing resources.

[0162] In a possible implementation, the partitioning strategy further includes a cache address of each matrix after partitioning, where the cache address is used to indicate a cache space of the matrix after partitioning in a buffer.

[0163] In an embodiment of the present application, based on the found segmentation strategy, the segmentation templates used by the first matrix and the second matrix, the loading order of the segmented matrices, the data volume of the segmented matrices, and the cache address of the segmented matrices can be determined.

[0164] 304. The computer device simulates multiple segmentation strategies to obtain the loading time of each segmentation strategy.

[0165] After determining multiple splitting strategies, the computer device simulates the multiple splitting strategies separately to select a splitting strategy with better processing performance from the multiple splitting strategies, and obtains the loading time of each splitting strategy. The loading time of the splitting strategy is the total time required to load the split matrix into the cache in batches when the splitting strategy is used for processing. The loading time of the splitting strategy can reflect the processing performance of the splitting strategy.

[0166] In one possible implementation, the computer device determines the number of loads and the amount of data loaded each time based on the splitting strategy, the data volume of the first matrix, and the data volume of the second matrix, and determines the loading duration of the splitting strategy based on the number of loads, the data volume of each load, and the read and write bandwidth of the memory.

[0167] The read / write bandwidth of a memory refers to the amount of data that can be read or written per unit time. The larger the read / write bandwidth of the memory, the faster the loading speed. The smaller the read / write bandwidth of the memory, the slower the loading speed.

[0168] Optionally, the computer device uses the following formula to determine the loading time of the segmentation strategy.

[0169] Among them, T represents the loading time, f z Indicates the amount of data in the submatrix obtained after the left matrix is ​​split, f t Indicates the number of times the submatrix obtained after the left matrix is ​​split is loaded, f r Indicates the read bandwidth of the left matrix. z Indicates the amount of data in the output matrix, o t Indicates the number of times the output matrix is ​​moved to the memory, o r represents the write bandwidth to the output matrix. z Indicates the amount of data in the submatrix obtained after the right matrix is ​​split, w t Indicates the number of times the submatrix is ​​loaded after the right matrix is ​​split, w r Indicates the read bandwidth of the right matrix.

[0170] Optionally, the computer device includes an emulator, which is used to simulate the splitting strategy. The computer device inputs the splitting strategy, the data volume of the first matrix, the data volume of the second matrix, and the read and write bandwidth of the memory into the emulator. The emulator determines the number of loads and the amount of data loaded each time based on the splitting strategy, the data volume of the first matrix, and the data volume of the second matrix. Based on the number of loads, the amount of data loaded each time, and the read and write bandwidth of the memory, the emulator determines the loading time of the splitting strategy, and the emulator outputs the loading time of the splitting strategy.

[0171] 305. The computer device determines a target segmentation strategy from among multiple segmentation strategies, and the loading time of the target segmentation strategy meets the performance condition.

[0172] Since the loading time of a segmentation strategy can reflect the performance of the segmentation strategy, the computer device determines a target segmentation strategy whose loading time meets the performance conditions among multiple segmentation strategies, thereby screening out a target segmentation strategy with relatively better performance.

[0173] Optionally, the loading time meeting the performance condition means that the loading time is less than a preset time. Optionally, the loading time meeting the performance condition means that the loading time is the minimum value among the loading times of multiple segmentation strategies.

[0174] 306. The computer device divides at least one of the first matrix or the second matrix according to the target division strategy, loads the divided matrix into a buffer, and performs operations on the matrix loaded into the buffer.

[0175] After determining the target segmentation strategy, the computer device segments at least one of the first matrix or the second matrix according to the segmentation method indicated by the target segmentation strategy to obtain a segmented matrix, and then loads the segmented matrix into the cache in batches according to the loading order indicated by the target segmentation strategy. Each time a load is completed, an operation is performed on the current matrix in the cache. After the operation on the current matrix in the cache is completed, the next load is performed, and so on, until the loading and operation of all segmented matrices are completed.

[0176] For example, the first matrix is ​​the left matrix of a matrix multiplication operation, and the second matrix is ​​the right matrix of a matrix multiplication operation. If the target segmentation strategy indicates that the right matrix is ​​to be segmented into three parts in the width direction, the computer device will evenly segment the first matrix into three parts in the width direction according to the target segmentation strategy, obtaining submatrix A, submatrix B, and submatrix C. The computer device will first load the first matrix and submatrix A into the cache according to the loading order indicated by the target segmentation strategy, perform operations on the first matrix and submatrix A in the cache, and obtain operation result 1. Then, the computer device will load submatrix B into the cache to replace submatrix A, perform operations on the first matrix and submatrix B in the cache, and obtain operation result 2. Then, the computer device will load submatrix C into the cache to replace submatrix B, perform operations on the first matrix and submatrix C in the cache, and obtain operation result 3. Operation result 1, operation result 2, and operation result 3 constitute the operation result of the matrix multiplication operation of the first matrix and the second matrix. That is, by dividing the matrix, the operation process of the first matrix and the second matrix is ​​divided into three batches, and the matrices required for operation in each batch are loaded into the buffer in turn.

[0177] In one possible implementation, the computer device divides the matrix currently required to be loaded into a first block matrix and a second block matrix, loads the first block matrix into a buffer, performs operations on the first block matrix in the buffer, and loads the second block matrix into the buffer in parallel during the operation on the first block matrix.

[0178] For example, the first matrix is ​​the left matrix of the matrix multiplication operation, and the second matrix is ​​the right matrix of the matrix multiplication operation. The target segmentation strategy indicates that the right matrix is ​​divided into three parts in the width direction. Then, the computer device divides the first matrix into three parts evenly in the width direction according to the target segmentation strategy, and obtains submatrix A, submatrix B, and submatrix C. Taking the operation process of the first matrix and submatrix A as an example, the computer device divides submatrix A into submatrix A1 and submatrix A2, loads the first matrix and submatrix A1 into the cache, and in the process of operating the first matrix and submatrix A1 in the cache, loads submatrix B1 into the cache in parallel. When the operation of the first matrix and submatrix A1 is completed, the loading of submatrix B1 is also completed.

[0179] In an embodiment of the present application, the matrix to be loaded is divided into two parts. The first part is loaded first, and the second part is loaded while the first part is being calculated. Thus, the loading process and the calculation process are executed in parallel, which is conducive to improving processing efficiency.

[0180] In one possible implementation, the splitting strategy also includes a cache address of the split matrix, and the computer device loads the split matrix into the cache, including: for the matrix currently required to be loaded, determining the cache address of the matrix, and loading the matrix into the cache space indicated by the cache address in the cache.

[0181] Optionally, each time two split matrices need to be loaded into a cache, the cache address includes a first cache address and a second cache address, the first cache address indicates the cache space of one of the matrices, the second cache address indicates the cache space of the other matrix, and the cache space indicated by the first cache address and the cache space indicated by the second cache address are continuous cache spaces.

[0182] In the embodiment of the present application, the cache address in the splitting strategy indicates which cache space in the cache the split matrix is ​​cached into. According to the cache address, the split matrix is ​​loaded into the respective cache spaces, ensuring the orderly progress of the caching process and helping to reduce the error rate.

[0183] Furthermore, the segmented matrix is ​​loaded into a continuous cache space, which facilitates the reading and writing operations of the matrix and helps to improve the efficiency of reading and writing the matrix.

[0184] In one possible implementation, a computer device performs operations on a matrix loaded into a buffer, including: calling a matrix operation engine, performing operations on the matrix in the buffer, obtaining operation results, and storing the operation results in a memory. The matrix operation engine is configured to perform matrix operations.

[0185] Computer devices run a matrix operation engine, which is a system or tool for efficiently performing matrix operations. The matrix operation engine improves the speed and efficiency of matrix operations through hardware acceleration, optimized software libraries, and parallel computing technology.

[0186] In the embodiment of the present application, a matrix operation engine is used to operate on the matrix in the buffer. Since the matrix operation engine can use parallel computing technology to operate, it is beneficial to improve the efficiency and performance of matrix operations.

[0187] FIG5 is a flow chart of another data processing method provided in an embodiment of the present application. As shown in FIG5 , a first matrix and a second matrix are stored in a memory. After at least one item in the first matrix or the second matrix is ​​segmented using a target segmentation strategy, the segmented matrix is ​​loaded into a buffer. A matrix operation engine reads the matrix from the buffer, performs an operation on the read matrix, obtains an operation result, and then writes the operation result to the memory. Optionally, the first matrix, the second matrix, and the operation result are respectively stored in different storage spaces in the memory.

[0188] In the method provided by the embodiment of the present application, the first matrix and the second matrix to be calculated are stored in a memory. In order to facilitate the subsequent reading of the first matrix and the second matrix during the calculation, the first matrix and the second matrix need to be loaded from the memory into a buffer. If the total amount of data of the first matrix and the second matrix exceeds the cache capacity of the buffer, the first matrix and the second matrix cannot be directly loaded into the buffer at the same time for calculation, and it is necessary to split the matrix and load them in batches. Therefore, a splitting strategy for the first matrix and the second matrix is ​​determined. The splitting strategy is used to indicate how to split the first matrix and the second matrix, and the order in which each matrix obtained after the splitting is loaded into the buffer. After obtaining multiple splitting strategies, each splitting strategy is simulated to obtain the loading time of each splitting strategy, so as to find the target splitting strategy whose loading time meets the performance conditions among the multiple splitting strategies, so as to facilitate the screening of the splitting strategy with a shorter loading time. Therefore, the splitting is performed according to the target splitting strategy, and then the split matrices are loaded into the cache in sequence for calculation. On the one hand, it solves the problem that the total amount of data of the first matrix and the second matrix is ​​too large to be loaded into the cache at one time. On the other hand, it can find the splitting strategy with a shorter loading time among the multiple splitting strategies, thereby reducing the time spent on loading multiple matrices, thereby improving the overall efficiency of data processing.

[0189] The embodiment of the present application provides a search scheme for a splitting strategy for AI operators of matrix operations such as matrix convolution and matrix multiplication. It can search for splitting strategies with relatively short loading times, which is conducive to improving processing performance and processing efficiency. Figure 6 is a system architecture diagram of another data processing method provided by an embodiment of the present application. As shown in Figure 6, the system architecture consists of three parts: a parsing layer, an optimization layer, and a simulation layer. Among them, the parsing layer is used to obtain a neural network and perform network parsing on the neural network. The optimization layer is used to fuse and eliminate operators in the neural network obtained after network parsing, and allocate storage addresses for the input matrices and output matrices of each layer in the neural network, which indicate the storage space in the memory. The splitting strategy for any two matrices generated in the neural network that need to perform matrix operations is then determined. The simulation layer is used to simulate the splitting strategy through a first simulator or a second simulator to obtain the loading time of each splitting strategy. The first simulator and the second simulator have different accuracy. The simulation layer can also deploy the target splitting strategy obtained by the simulator screening in the chip, and the subsequent chip can split and load the two matrices to be performed on the matrix operation generated in the neural network according to the target splitting strategy.

[0190] Figure 7 is a flowchart of another data processing method provided by an embodiment of the present application. As shown in Figure 7, the method includes the following steps: Step 1, network analysis, that is, analyzing the neural network at the analysis layer; Step 2, operator fusion and elimination, that is, performing operator fusion and operator elimination on the analyzed neural network at the optimization layer; Step 3, memory allocation, that is, allocating storage addresses for the input matrix and output matrix of each layer in the neural network at the optimization layer; Step 4, splitting strategy, that is, determining the splitting strategy of any two matrices generated in the neural network that need to perform matrix multiplication operations in the optimization layer, and the splitting strategy includes a splitting method and a loading order; Step 5, simulator, that is, simulating multiple determined splitting strategies through the simulator at the simulation layer to obtain the loading time of each splitting strategy.

[0191] FIG8 is a schematic diagram of the structure of a data processing device provided in an embodiment of the present application. Referring to FIG8 , the device includes:

[0192] A matrix determination module 801 is configured to determine a first matrix and a second matrix in a memory, where the first matrix and the second matrix are two matrices to be subjected to matrix operations in a neural network;

[0193] a strategy determination module 802 configured to determine, when the total amount of data of the first matrix and the second matrix exceeds the cache capacity of the cache, a plurality of splitting strategies, wherein the splitting strategies include a splitting method and a loading order, wherein the splitting method refers to a method for splitting at least one of the first matrix or the second matrix, and the loading order refers to an order in which the split matrices are loaded from the memory into the cache;

[0194] The simulation module 803 is used to simulate multiple segmentation strategies to obtain the loading time of each segmentation strategy;

[0195] A strategy selection module 804 is configured to determine a target switching strategy from among multiple segmentation strategies, wherein the loading time of the target segmentation strategy satisfies the performance condition;

[0196] The matrix processing module 805 is configured to segment at least one of the first matrix or the second matrix according to a target segmentation strategy, load the segmented matrix into a buffer, and perform operations on the matrix loaded into the buffer.

[0197] In the data processing device provided by the embodiment of the present application, the first matrix and the second matrix to be calculated are stored in a memory. In order to facilitate the subsequent reading of the first matrix and the second matrix during the calculation, the first matrix and the second matrix need to be loaded from the memory into a buffer. If the total amount of data of the first matrix and the second matrix exceeds the cache capacity of the buffer, the first matrix and the second matrix cannot be directly loaded into the buffer at the same time for calculation, and it is necessary to split the matrix and load them in batches. Therefore, a splitting strategy for the first matrix and the second matrix is ​​determined, which is used to indicate how to split the first matrix and the second matrix, and the order in which each matrix obtained after the split is loaded into the buffer. After obtaining multiple splitting strategies, each splitting strategy is simulated to obtain the loading time of each splitting strategy, so as to find the target splitting strategy whose loading time meets the performance conditions among the multiple splitting strategies, so as to facilitate the screening of the splitting strategy with a shorter loading time. Therefore, the splitting is performed according to the target splitting strategy, and then the split matrices are loaded into the cache in sequence for calculation. On the one hand, it solves the problem that the total amount of data of the first matrix and the second matrix is ​​too large to be loaded into the cache at one time. On the other hand, it can find the splitting strategy with a shorter loading time among the multiple splitting strategies, thereby reducing the time spent on loading multiple matrices, thereby improving the overall efficiency of data processing.

[0198] Optionally, the policy determination module 802 is configured to:

[0199] Obtain multiple segmentation templates, where the segmentation templates indicate segmentation targets and segmentation directions, and the segmentation target refers to the matrix to be segmented;

[0200] For any segmentation template, based on the segmentation template, the data volume of the first matrix and the data volume of the second matrix, the segmentation strategy corresponding to the segmentation template is found. The segmentation strategy corresponding to the segmentation template refers to a segmentation strategy that conforms to the segmentation template and meets the segmentation conditions. The segmentation condition means that after segmentation according to the segmentation strategy, the total amount of matrix data that needs to be loaded into the cache for calculation each time is not greater than the cache capacity.

[0201] Optionally, the policy determination module 802 is configured to:

[0202] Based on the segmentation target indicated by the segmentation template, the segmentation direction indicated by the segmentation template, the data volume of the first matrix and the data volume of the second matrix, searching for the number of segments that meet the segmentation conditions, where the segmentation method includes the segmentation target, the segmentation direction and the number of segments;

[0203] Based on the segmentation target and segmentation direction indicated by the segmentation template, the operation order of the segmented matrix is ​​determined, and based on the operation order, the loading order is determined.

[0204] Optionally, the policy determination module 802 is configured to:

[0205] Determining an evaluation result of at least one candidate number of splits based on a splitting target, a splitting direction, a data amount of the first matrix, a data amount of the second matrix, and at least one candidate number of splits, wherein the evaluation result indicates whether the candidate number of splits satisfies a splitting condition, and the candidate number of splits is a positive integer greater than 1;

[0206] Based on the evaluation result of at least one candidate number of cuts, a number of cuts that meets the cut condition is determined from the at least one candidate number of cuts.

[0207] Optionally, the policy determination module 802 is configured to:

[0208] Determine an evaluation result of the i-th candidate number of splits based on the splitting target, the splitting direction, the data amount of the first matrix, the data amount of the second matrix, and the i-th candidate number of splits;

[0209] Based on the segmentation target, segmentation direction, the data volume of the first matrix, the data volume of the second matrix and the i+1th candidate segmentation number, the evaluation result of the i+1th candidate segmentation number is determined until the stop search condition is met; wherein the i+1th candidate segmentation number is equal to i+1, and i is a positive integer greater than 1.

[0210] Optionally, the stop search condition includes at least one of the following:

[0211] The evaluation result of the current candidate number of splits indicates that the splitting condition is met;

[0212] After segmentation according to the current candidate segmentation number, the data amount of the obtained minimum matrix is ​​less than the minimum operation amount of the matrix operation engine, and the matrix operation engine is used to operate on the matrix.

[0213] Optionally, the multiple segmentation templates include at least one target segmentation template and at least one non-target segmentation template, the segmentation target indicated by the target segmentation template is one of any two matrices, and the segmentation target indicated by the non-target segmentation template is each of any two matrices;

[0214] The policy determination module 802 is configured to:

[0215] Find the segmentation strategy corresponding to each target segmentation template in turn;

[0216] When at least one segmentation strategy corresponding to the target segmentation template is found, the search stops;

[0217] When no segmentation strategy corresponding to any target segmentation template is found, the segmentation strategy corresponding to each non-target segmentation template is searched in turn.

[0218] Optionally, the plurality of segmentation templates include at least one of a first segmentation template, a second segmentation template, a third segmentation template, a fourth segmentation template, a fifth segmentation template, or a sixth segmentation template;

[0219] The first segmentation template indicates that the right matrix of any two matrices is segmented in the width direction;

[0220] The second segmentation template indicates that the left matrix of any two matrices is segmented in the height direction;

[0221] The third segmentation template indicates that the left matrix of any two matrices is segmented in the height direction and the right matrix is ​​segmented in the width direction;

[0222] The fourth segmentation template indicates that the left matrix of any two matrices is segmented in the height direction and the right matrix is ​​segmented in the height direction and the width direction;

[0223] The fifth segmentation template indicates that the left matrix of any two matrices is segmented in the height direction and the width direction and the right matrix is ​​segmented in the width direction;

[0224] The sixth segmentation template indicates that the left matrix of any two matrices is segmented in the height direction and the width direction, and the right matrix is ​​segmented in the height direction and the width direction.

[0225] Optionally, the simulation module 803 is configured to:

[0226] Determine the number of loads and the amount of data loaded each time based on the segmentation strategy, the amount of data in the first matrix and the amount of data in the second matrix;

[0227] Determine the loading duration of the split strategy based on the number of loads, the amount of data loaded each time, and the read and write bandwidth of the memory.

[0228] Optionally, the matrix processing module 805 is configured to:

[0229] For the matrix currently required to be loaded, the matrix is ​​divided into a first block matrix and a second block matrix;

[0230] The first block matrix is ​​loaded into a buffer, and the first block matrix in the buffer is operated. During the operation of the first block matrix, the second block matrix is ​​loaded into the buffer in parallel.

[0231] Optionally, the segmentation strategy further includes a cache address of the segmented matrix. The matrix processing module 805 is configured to:

[0232] For the matrix currently required to be loaded, determine the cache address of the matrix;

[0233] The matrix is ​​loaded into the cache space indicated by the cache address in the cache.

[0234] Optionally, the matrix processing module 805 is configured to:

[0235] Call the matrix operation engine to operate the matrix in the buffer and obtain the operation result;

[0236] Store the operation results in memory.

[0237] It should be noted that the data processing device provided in the above embodiment is merely illustrated by the division of the above functional modules. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the computer device can be divided into different functional modules to complete all or part of the functions described above. In addition, the data processing device provided in the above embodiment and the data processing method embodiment are based on the same concept. The specific implementation process is detailed in the method embodiment and will not be repeated here.

[0238] The present application also provides a chip, which is communicatively connected to a memory and includes a buffer; the chip is configured to perform the following operations:

[0239] Determining a first matrix and a second matrix in a memory, where the first matrix and the second matrix are two matrices to be subjected to matrix operations in a neural network;

[0240] When the total amount of data of the first matrix and the second matrix exceeds the cache capacity of the cache, determining multiple splitting strategies, the splitting strategies including a splitting method and a loading order, the splitting method referring to a method of splitting at least one of the first matrix or the second matrix, and the loading order referring to an order in which the split matrices are loaded from the memory into the cache;

[0241] Simulate multiple sharding strategies to obtain the loading time of each sharding strategy;

[0242] Determine the target switching strategy among multiple segmentation strategies, and ensure that the loading time of the target segmentation strategy meets the performance requirements;

[0243] At least one of the first matrix or the second matrix is ​​segmented according to a target segmentation strategy, the segmented matrix is ​​loaded into a buffer, and operations are performed on the matrix loaded into the buffer.

[0244] In one possible implementation, the chip is configured in a computer device, and the chip is also used to implement the operations performed in the data processing method of the above embodiment.

[0245] Optionally, the chip is an AI chip, etc. In the scenario of application to AI chips, the embodiment of the present application can evaluate the processing performance of the entire AI chip by determining the loading time of each slicing strategy, and can provide guidance for the deployment and development of each operator in the AI ​​chip, thereby providing useful guidance for the hardware architecture design and optimization of the AI ​​chip.

[0246] An embodiment of the present application also provides a computer device, which includes a processor and a memory, wherein the memory stores at least one computer program, and the at least one computer program is loaded and executed by the processor to implement the operations performed in the data processing method of the above embodiment.

[0247] Optionally, the computer device is provided as a terminal. FIG9 shows a schematic diagram of the structure of a terminal 900 provided in an exemplary embodiment of the present application. The terminal 900 includes: a processor 901 and a memory 902.

[0248] The processor 901 may include one or more processing cores, such as a 4-core processor, an 8-core processor, and the like. The processor 901 may be implemented in at least one hardware form of a DSP (Digital Signal Processing), an FPGA (Field Programmable Gate Array), or a PLA (Programmable Logic Array). The processor 901 may also include a main processor and a coprocessor. The main processor is a processor for processing data in an awake state, also known as a CPU (Central Processing Unit); the coprocessor is a low-power processor for processing data in a standby state. In some embodiments, the processor 901 may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 901 may also include an AI (Artificial Intelligence) chip, which is used to process computing operations related to machine learning.

[0249] The memory 902 may include one or more computer-readable storage media, which may be non-transitory. The memory 902 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices and flash memory storage devices. In some embodiments, the non-transitory computer-readable storage medium in the memory 902 is used to store at least one computer program, which is used to be used by the processor 901 to implement the data processing method provided in the method embodiment of the present application.

[0250] In some embodiments, terminal 900 may optionally include a peripheral device interface 903 and at least one peripheral device. Processor 901, memory 902, and peripheral device interface 903 may be connected via a bus or signal lines. Each peripheral device may be connected to peripheral device interface 903 via a bus, signal lines, or circuit boards. Optionally, the peripheral device includes at least one of a radio frequency circuit 904, a display screen 905, a camera assembly 906, an audio circuit 907, and a power supply 908.

[0251] The peripheral device interface 903 can be used to connect at least one I / O (Input / Output)-related peripheral device to the processor 901 and the memory 902. In some embodiments, the processor 901, the memory 902, and the peripheral device interface 903 are integrated on the same chip or circuit board; in some other embodiments, any one or two of the processor 901, the memory 902, and the peripheral device interface 903 can be implemented on separate chips or circuit boards, which is not limited in this embodiment.

[0252] The RF circuit 904 is used to receive and transmit RF (Radio Frequency) signals, also known as electromagnetic signals. The RF circuit 904 communicates with communication networks and other communication devices via electromagnetic signals. The RF circuit 904 converts electrical signals into electromagnetic signals for transmission, or converts received electromagnetic signals into electrical signals. Optionally, the RF circuit 904 includes: an antenna system, an RF transceiver, one or more amplifiers, a tuner, an oscillator, a digital signal processor, a codec chipset, a user identity module card, and the like. The RF circuit 904 can communicate with other devices via at least one wireless communication protocol. Such wireless communication protocols include, but are not limited to, metropolitan area networks, various generations of mobile communication networks (2G, 3G, 4G, and 5G), wireless local area networks, and / or WiFi (Wireless Fidelity) networks. In some embodiments, the RF circuit 904 may also include circuits related to NFC (Near Field Communication), which is not limited in this application.

[0253] The display screen 905 is used to display a user interface (UI). This UI may include graphics, text, icons, videos, or any combination thereof. When the display screen 905 is a touch screen, it is also capable of collecting touch signals on or above the surface of the display screen 905. These touch signals can be input as control signals to the processor 901 for processing. In this case, the display screen 905 can also be used to provide virtual buttons and / or a virtual keyboard, also known as soft buttons and / or a soft keyboard. In some embodiments, there can be a single display screen 905, located on the front panel of the terminal 900. In other embodiments, there can be at least two display screens 905, located on different surfaces of the terminal 900 or in a foldable design. In other embodiments, the display screen 905 can be a flexible display screen, located on a curved or foldable surface of the terminal 900. Furthermore, the display screen 905 can be configured as a non-rectangular irregular shape, i.e., a special-shaped screen. The display screen 905 can be made of materials such as LCD (Liquid Crystal Display) and OLED (Organic Light-Emitting Diode).

[0254] The camera assembly 906 is used to capture images or videos. Optionally, the camera assembly 906 includes a front camera and a rear camera. The front camera is arranged on the front panel of the terminal 900, and the rear camera is arranged on the back of the terminal 900. In some embodiments, there are at least two rear cameras, which are any one of a main camera, a depth of field camera, a wide-angle camera, and a telephoto camera, so as to realize the fusion of the main camera and the depth of field camera to realize the background blur function, the fusion of the main camera and the wide-angle camera to realize panoramic shooting and VR (Virtual Reality) shooting function or other fusion shooting functions. In some embodiments, the camera assembly 906 may also include a flash. The flash can be a monochrome temperature flash or a dual-color temperature flash. The dual-color temperature flash refers to a combination of a warm light flash and a cold light flash, which can be used for light compensation under different color temperatures.

[0255] The audio circuit 907 may include a microphone and a speaker. The microphone is used to collect sound waves from the user and the environment, and convert the sound waves into electrical signals to be input into the processor 901 for processing, or input into the radio frequency circuit 904 to achieve voice communication. For the purpose of stereo sound collection or noise reduction, there may be multiple microphones, each located in different parts of the terminal 900. The microphone may also be an array microphone or an omnidirectional collection microphone. The speaker is used to convert electrical signals from the processor 901 or the radio frequency circuit 904 into sound waves. The speaker may be a traditional thin film speaker or a piezoelectric ceramic speaker. When the speaker is a piezoelectric ceramic speaker, it can not only convert electrical signals into sound waves audible to humans, but also convert electrical signals into sound waves inaudible to humans for purposes such as ranging. In some embodiments, the audio circuit 907 may also include a headphone jack.

[0256] Power supply 908 is used to power various components in terminal 900. Power supply 908 can be AC ​​power, DC power, a disposable battery, or a rechargeable battery. When power supply 908 includes a rechargeable battery, the rechargeable battery can support wired charging or wireless charging. The rechargeable battery can also be used to support fast charging technology.

[0257] Those skilled in the art will appreciate that the structure shown in FIG9 does not limit the terminal 900 and may include more or fewer components than shown, or combine certain components, or adopt a different component arrangement.

[0258] Optionally, the computer device is provided as a server. Figure 10 is a structural diagram of a server provided in an embodiment of the present application. The server 1000 may have relatively large differences due to different configurations or performances, and may include one or more processors (Central Processing Units, CPU) 1001 and one or more memories 1002, wherein the memory 1002 stores at least one computer program, and the at least one computer program is loaded and executed by the processor 1001 to implement the methods provided in the above-mentioned various method embodiments. Of course, the server may also have components such as a wired or wireless network interface, a keyboard, and an input and output interface for input and output. The server may also include other components for implementing device functions, which will not be described in detail here.

[0259] An embodiment of the present application further provides a computer-readable storage medium, in which at least one computer program is stored. The at least one computer program is loaded and executed by a processor to implement the operations performed by the data processing method of the above embodiment.

[0260] An embodiment of the present application further provides a computer program product, including a computer program, which is loaded and executed by a processor to implement the operations performed by the data processing method of the above embodiment.

[0261] Those skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware, or by a program to instruct the relevant hardware, and the program may be stored in a computer-readable storage medium, which may be a read-only memory, a disk, or an optical disk, etc.

[0262] The above description is merely an optional embodiment of the embodiments of the present application and is not intended to limit the embodiments of the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the embodiments of the present application shall be included in the scope of protection of the present application.

Claims

1. A data processing method, executed by a computer device, comprising: Determine a first matrix and a second matrix in a memory, wherein the first matrix and the second matrix are two matrices to be subjected to matrix operations in a neural network; In a case where the total amount of data of the first matrix and the second matrix is ​​greater than the cache capacity of the cache, a plurality of segmentation strategies are determined, wherein the segmentation strategies include a segmentation method and a loading order, wherein the segmentation method refers to a method of segmenting at least one of the first matrix or the second matrix, and the loading order refers to an order of loading the segmented matrices from the memory to the cache; Simulating the multiple segmentation strategies to obtain the loading time of each segmentation strategy; Determining a target switching strategy among the multiple segmentation strategies, wherein the loading duration of the target segmentation strategy satisfies the performance condition; At least one of the first matrix or the second matrix is ​​segmented according to the target segmentation strategy, the segmented matrix is ​​loaded into the buffer, and operations are performed on the matrix loaded into the buffer.

2. The method according to claim 1, wherein: In the case where the total amount of data in the first matrix and the second matrix is ​​greater than the cache capacity of the cache, determining multiple segmentation strategies includes: Acquire multiple segmentation templates, wherein the segmentation templates indicate segmentation targets and segmentation directions, and the segmentation target refers to a matrix to be segmented; For any segmentation template, based on the segmentation template, the data volume of the first matrix and the data volume of the second matrix, the segmentation strategy corresponding to the segmentation template is found. The segmentation strategy corresponding to the segmentation template refers to a segmentation strategy that conforms to the segmentation template and satisfies the segmentation condition. The segmentation condition means that after segmentation according to the segmentation strategy, the total amount of matrix data that needs to be loaded into the cache for calculation each time is not greater than the cache capacity.

3. The method according to claim 1 or 2, wherein: The searching for a segmentation strategy corresponding to the segmentation template based on the segmentation template, the data amount of the first matrix, and the data amount of the second matrix includes: Based on the segmentation target indicated by the segmentation template, the segmentation direction indicated by the segmentation template, the data volume of the first matrix and the data volume of the second matrix, searching for the number of segmentation portions that meets the segmentation condition, wherein the segmentation method includes the segmentation target, the segmentation direction and the number of segmentation portions; Based on the segmentation target and segmentation direction indicated by the segmentation template, the operation order of the segmented matrix is ​​determined, and based on the operation order, the loading order is determined.

4. The method according to any one of claims 1 to 3, wherein: The searching for the number of splits that meet the splitting condition based on the splitting target indicated by the splitting template, the splitting direction indicated by the splitting template, the data volume of the first matrix, and the data volume of the second matrix includes: Based on the segmentation target, the segmentation direction, the data amount of the first matrix, the data amount of the second matrix, and at least one candidate segmentation number, respectively determine an evaluation result of the at least one candidate segmentation number, the evaluation result indicating whether the candidate segmentation number satisfies the segmentation condition, and the candidate segmentation number is a positive integer greater than 1; Based on the evaluation result of the at least one candidate number of cuts, a number of cuts that meets the cut condition is determined from the at least one candidate number of cuts.

5. The method according to any one of claims 1 to 4, wherein: The step of determining an evaluation result of at least one candidate number of cuts based on the cut target, the cut direction, the data amount of the first matrix, the data amount of the second matrix, and at least one candidate number of cuts comprises: Determine an evaluation result of the i-th candidate number of cuts based on the cut target, the cut direction, the data amount of the first matrix, the data amount of the second matrix, and the i-th candidate number of cuts; Based on the segmentation target, the segmentation direction, the data volume of the first matrix, the data volume of the second matrix and the i+1th candidate segmentation number, determine the evaluation result of the i+1th candidate segmentation number until the stop search condition is met; wherein the i+1th candidate segmentation number is equal to i+1, and i is a positive integer greater than 1.

6. The method according to any one of claims 1 to 5, wherein: The stop search condition includes at least one of the following: The evaluation result of the current candidate number of splits indicates that the split condition is satisfied; After segmentation according to the current candidate segmentation number, the data amount of the obtained minimum matrix is ​​less than the minimum operation amount of the matrix operation engine, and the matrix operation engine is used to operate on the matrix.

7. The method according to any one of claims 1 to 6, wherein: The multiple segmentation templates include at least one target segmentation template and at least one non-target segmentation template, the segmentation target indicated by the target segmentation template is one of any two matrices, and the segmentation target indicated by the non-target segmentation template is each of any two matrices; For any segmentation template, searching for a segmentation strategy corresponding to the segmentation template based on the segmentation template, the data volume of the first matrix, and the data volume of the second matrix includes: Find the segmentation strategy corresponding to each target segmentation template in turn; When at least one segmentation strategy corresponding to the target segmentation template is found, the search is stopped; When no segmentation strategy corresponding to any target segmentation template is found, the segmentation strategy corresponding to each non-target segmentation template is searched in turn.

8. The method according to any one of claims 1 to 7, wherein: The plurality of segmentation templates include at least one of a first segmentation template, a second segmentation template, a third segmentation template, a fourth segmentation template, a fifth segmentation template, or a sixth segmentation template; The first segmentation template indicates that the right matrix of any two matrices is segmented in the width direction; The second segmentation template indicates that the left matrix of any two matrices is segmented in the height direction; The third segmentation template indicates that the left matrix of any two matrices is segmented in the height direction and the right matrix is ​​segmented in the width direction; The fourth segmentation template indicates that the left matrix of any two matrices is segmented in the height direction and the right matrix is ​​segmented in the height direction and the width direction; The fifth segmentation template indicates that the left matrix of any two matrices is segmented in the height direction and the width direction and the right matrix is ​​segmented in the width direction; The sixth segmentation template indicates that the left matrix of any two matrices is segmented in the height direction and the width direction and the right matrix is ​​segmented in the height direction and the width direction.

9. The method according to any one of claims 1 to 8, wherein: The simulating the multiple segmentation strategies to obtain the loading time of each segmentation strategy includes: Determine the number of loads and the amount of data loaded each time based on the segmentation strategy, the amount of data in the first matrix, and the amount of data in the second matrix; The loading duration of the segmentation strategy is determined based on the number of loads, the amount of data loaded each time, and the read and write bandwidth of the memory.

10. The method according to any one of claims 1 to 9, wherein: The step of loading the segmented matrix into the buffer and performing operations on the matrix loaded into the buffer includes: For the matrix currently required to be loaded, the matrix is ​​divided into a first block matrix and a second block matrix; The first block matrix is ​​loaded into the buffer, and the first block matrix in the buffer is operated. During the operation of the first block matrix, the second block matrix is ​​loaded into the buffer in parallel.

11. The method according to any one of claims 1 to 10, wherein: The segmentation strategy also includes a cache address of the segmented matrix, and the step of loading the segmented matrix into the cache includes: For a matrix currently required to be loaded, determining a cache address of the matrix; The matrix is ​​loaded into the cache space indicated by the cache address in the cache.

12. The method according to any one of claims 1 to 11, wherein: The operating of the matrix loaded into the buffer includes: Calling a matrix operation engine to operate the matrix in the buffer to obtain an operation result; The operation result is stored in the memory.

13. A data processing device, comprising: A matrix determination module, used to determine a first matrix and a second matrix in a memory, wherein the first matrix and the second matrix are two matrices to be subjected to matrix operations in a neural network; a strategy determination module, configured to determine a plurality of segmentation strategies when the total amount of data of the first matrix and the second matrix is ​​greater than the cache capacity of the cache, wherein the segmentation strategies include a segmentation method and a loading order, wherein the segmentation method refers to a method of segmenting at least one of the first matrix or the second matrix, and the loading order refers to an order of loading the segmented matrices from the memory to the cache; A simulation module, used to simulate the multiple segmentation strategies to obtain the loading time of each segmentation strategy; A strategy selection module, configured to determine a target switching strategy from among the plurality of segmentation strategies, wherein the loading duration of the target segmentation strategy satisfies a performance condition; The matrix processing module is used to segment at least one of the first matrix or the second matrix according to the target segmentation strategy, load the segmented matrix into the buffer, and perform operations on the matrix loaded into the buffer.

14. A chip, the chip being communicatively connected to a memory, the chip comprising a buffer; the chip being used to perform the following operations: Determining a first matrix and a second matrix in the memory, wherein the first matrix and the second matrix are two matrices to be subjected to matrix operations in a neural network; In a case where the total amount of data of the first matrix and the second matrix is ​​greater than the cache capacity of the cache, a plurality of segmentation strategies are determined, wherein the segmentation strategies include a segmentation method and a loading order, wherein the segmentation method refers to a method of segmenting at least one of the first matrix or the second matrix, and the loading order refers to an order of loading the segmented matrices from the memory to the cache; Simulating the multiple segmentation strategies to obtain the loading time of each segmentation strategy; Determining a target switching strategy among the multiple segmentation strategies, wherein the loading duration of the target segmentation strategy satisfies the performance condition; At least one of the first matrix or the second matrix is ​​segmented according to the target segmentation strategy, the segmented matrix is ​​loaded into the buffer, and operations are performed on the matrix loaded into the buffer.

15. A computer device, comprising a processor and a memory, wherein the memory stores at least one computer program, and the at least one computer program is loaded and executed by the processor to implement the operations performed by the data processing method according to any one of claims 1 to 12.

16. A computer-readable storage medium, wherein at least one computer program is stored in the computer-readable storage medium, and the at least one computer program is loaded and executed by a processor to implement the operations performed by the data processing method according to any one of claims 1 to 12.

17. A computer program product, comprising a computer program, wherein the computer program is loaded and executed by a processor to implement the operations performed by the data processing method according to any one of claims 1 to 12.

Images