Data processing method, device, electronic device and computer-readable storage medium

Abstract

A data processing method, a data processing device, an electronic device, and a computer-readable storage medium. The data processing method includes: determining the data granularity based on the total capacity of multiple available shared caches; based on the data granularity, dividing each dimension of the calculation data into multiple parts, so as to divide the first-level tasks corresponding to the calculation data into multiple groups of second-level tasks, each group of second-level tasks includes multiple second-level tasks, and the data volume of the calculation results of each second-level task corresponds to the total capacity of multiple shared caches; for each group of second-level tasks, using a loop code, the calculation results of multiple second-level tasks in each group are cyclically stored in multiple shared caches, and the data of the multiple shared caches are cyclically output to the data consumer operator. This method can simplify the data sharing process between producer operators and consumer operators.

Description

Technical Field

[0001] Embodiments of the present disclosure relate to a data processing method, apparatus, electronic device, and computer-readable storage medium. Background Art

[0002] In the field of efficient computing for neural network inference or training, when multiple dynamic shape operators are integrated into a computational task, resource synchronization issues arise between producer and consumer operators. The buffers within neural network accelerators are often limited in number or size, and these resources need to be reused between producers and consumers. If producer and consumer operators execute concurrently or simultaneously, it is necessary to ensure that there are no resource conflicts between them. Consumers can only begin to consume data after the producer has produced it. Only after the consumer has consumed the data can the producer continue to store new data in the shared cache. Summary of the Invention

[0003] At least one embodiment of the present disclosure provides a data processing method, comprising: determining data granularity based on the total capacity of multiple available shared caches; based on the data granularity, dividing each dimension of the calculation data into multiple parts, so as to divide the first-level tasks corresponding to the calculation data into multiple groups of second-level tasks, wherein each group of the second-level tasks includes multiple second-level tasks, and the data volume of the calculation results of each second-level task corresponds to the total capacity of the multiple shared caches; for each group of the second-level tasks, using a loop code, the calculation results of the multiple second-level tasks in each group are cyclically stored in the multiple shared caches, and the data of the multiple shared caches are cyclically output to the data consumer operator.

[0004] For example, in the data processing method provided by at least one example of the above-mentioned embodiment of the present disclosure, each group of second-level tasks includes a first task, a second task, and at least one intermediate task located between the first task and the second task; the loop code includes a first code segment, a second code segment, and an intermediate code segment. For each group of second-level tasks, the loop code is used to cyclically store the calculation results of the multiple second-level tasks in each group into the multiple shared caches, and the data of the multiple shared caches is cyclically output to the data consumer operator, including: using the first code segment to cyclically store the calculation result of the first task into the multiple shared caches, and cyclically output the calculation result of the first task from the data of the multiple shared caches to the data consumer operator; for each intermediate task in the at least one intermediate task, using the intermediate code segment to cyclically store the calculation result of the intermediate task into the multiple shared caches, and cyclically output the calculation result of the intermediate task from the data of the multiple shared caches to the data consumer operator; using the second code segment to cyclically store the calculation result of the second task into the multiple shared caches, and cyclically output the calculation result of the second task from the data of the multiple shared caches to the data consumer operator.

[0005] For example, in the data processing method provided by at least one example of the above-mentioned embodiments of the present disclosure, each second-level task includes multiple third-level tasks, and the data volume of the calculation results of each third-level task corresponds to the capacity of one of the shared caches; the loop code includes code corresponding to a data producer operator and code corresponding to a data consumer operator, wherein the data producer operator is used to execute the multiple groups of second-level tasks to obtain the calculation results of the multiple second-level tasks, and the data consumer operator is used to perform the next step of calculation based on the calculation results of the second-level tasks; the code corresponding to the data producer operator is used to store the calculation results of the multiple third-level tasks of each second-level task in the multiple shared caches in sequence, and send a storage notification to the data consumer operator after each time the calculation result of a third-level task is stored in a shared cache; the code corresponding to the data consumer operator is used to retrieve the calculation results of the multiple third-level tasks of each second-level task from the multiple shared caches in sequence, and send a retrieval notification to the data producer operator after each time the calculation result of a third-level task is retrieved from a shared cache.

[0006] For example, in the data processing method provided by at least one example of the above-mentioned embodiments of the present disclosure, the first code segment includes: storing the calculation results of multiple third-level tasks included in the first task in the multiple shared caches in sequence, and after storing the calculation result of a third-level task in a shared cache each time, sending a storage notification corresponding to the stored shared cache to the data consumer operator; after receiving the storage notification each time, retrieving data from the shared cache corresponding to the storage notification, and sending a retrieval notification corresponding to the shared cache to the data producer operator, so as to output all the calculation results of the multiple third-level tasks of the first task to the data consumer operator.

[0007] For example, in the data processing method provided by at least one example of the above-mentioned embodiments of the present disclosure, the intermediate code segment includes: after each receipt of a retrieval notification, storing the calculation result of a third-level task of the intermediate task in the shared cache corresponding to the retrieval notification, and sending a storage notification corresponding to the shared cache to the data consumer operator to store all the calculation results of multiple third-level tasks of the intermediate task in the multiple shared caches; after each receipt of a storage notification, retrieving data from the shared cache corresponding to the storage notification, and sending a retrieval notification corresponding to the shared cache to the data producer operator to output all the calculation results of multiple third-level tasks of the intermediate task to the data consumer operator.

[0008] For example, in the data processing method provided by at least one example of the above-mentioned embodiments of the present disclosure, the second code segment includes: after each receipt of a retrieval notification, storing the calculation result of a third-level task of the second task in the shared cache corresponding to the retrieval notification, and sending a storage notification corresponding to the shared cache to the data consumer operator to store all the calculation results of multiple third-level tasks of the second task in the multiple shared caches; after each receipt of a storage notification, retrieving data from the shared cache corresponding to the storage notification to output all the calculation results of multiple third-level tasks of the second task to the data consumer operator.

[0009] For example, in the data processing method provided by at least one example of the above embodiments of the present disclosure, the method further includes: determining the required number of synchronization resource locks based on the number of the multiple shared caches and the synchronization granularity.

[0010] For example, in the data processing method provided in at least one example of the above-mentioned embodiments of the present disclosure, the data granularity includes a first granularity, a second granularity, and a third granularity; determining the data granularity based on the total capacity of the multiple available shared caches includes: determining the data granularity based on the number of the multiple shared caches and the capacity of each of the shared caches.

[0011] For example, in the data processing method provided in at least one example of the above-mentioned embodiments of the present disclosure, the data granularity is determined based on the number of the multiple shared caches and the capacity of each of the shared caches, including: determining a first size and a second size based on the capacity of each shared cache; decomposing the number of the multiple shared caches into a first numerical value and a second numerical value; taking the product of the first numerical value and the first size as the first granularity, and taking the product of the second numerical value and the second size as the second granularity.

[0012] For example, in the data processing method provided by at least one example of the above-mentioned embodiments of the present disclosure, the calculation data includes a first matrix and a second matrix, and the first-level task includes multiplying the first matrix by the second matrix; wherein, based on the number of the multiple shared caches and the capacity of each of the shared caches, the data granularity is determined, and further includes: determining the third granularity based on the first granularity, the second granularity, the capacity of the multiple first caches for storing the first matrix, and the capacity of the multiple second caches for storing the second matrix. wherein, based on the first granularity, the second granularity, the capacity of the multiple first caches for storing the first matrix, and the capacity of the multiple second caches for storing the second matrix, determining the third granularity includes: performing an operation on the capacity of the multiple first caches and the first granularity to obtain a third value; performing an operation on the capacity of the multiple second caches and the second granularity to obtain a fourth value; and taking the smaller of the third value and the fourth value as the third granularity.

[0013] For example, in the data processing method provided by at least one example of the above-mentioned embodiment of the present disclosure, the calculation data includes a first matrix and a second matrix, and the first-level task includes the multiplication of the first matrix and the second matrix. Based on the data granularity, each dimension of the calculation data is divided into multiple parts to divide the first-level task corresponding to the calculation data into multiple groups of second-level tasks, including: dividing the first matrix into multiple segments based on the first granularity on the first dimension, and dividing the first matrix into multiple segments based on the third granularity on the third dimension, so that the first matrix is ​​divided into multiple first sub-matrices; dividing the second matrix into multiple segments based on the third granularity on the third dimension, and dividing the second matrix into three segments based on the second granularity on the second dimension, so that the second matrix is ​​divided into multiple second sub-matrices, wherein the first segment and the last segment of the three segments on the second dimension correspond to the second granularity; wherein any task of multiplying the first sub-matrix with three corresponding second sub-matrices can be used as a group of second-level tasks, wherein the three corresponding second sub-matrices correspond to the three segments on the second dimension respectively.

[0014] At least one embodiment of the present disclosure provides a data processing device, including a determination unit, a splitting unit and an execution unit, wherein the determination unit is configured to determine the data granularity based on the total capacity of multiple available shared caches; the splitting unit is configured to split each dimension of the calculation data into multiple parts based on the data granularity, so as to divide the first-level task corresponding to the calculation data into multiple groups of second-level tasks, wherein each group of the second-level tasks includes multiple second-level tasks, and the data volume of the calculation result of each second-level task corresponds to the total capacity of the multiple shared caches; the execution unit is configured to, for each group of the second-level tasks, use a loop code to cyclically store the calculation results of the multiple second-level tasks in each group into the multiple shared caches, and cyclically output the data of the multiple shared caches to the data consumer operator.

[0015] At least one embodiment of the present disclosure provides an electronic device, comprising a processor; a memory storing one or more computer program modules; wherein the one or more computer program modules are configured to be executed by the processor to implement the data processing method provided by any embodiment of the present disclosure.

[0016] At least one embodiment of the present disclosure provides a computer-readable storage medium storing non-transitory computer-readable instructions. When the non-transitory computer-readable instructions are executed by a computer, the data processing method provided by any embodiment of the present disclosure can be implemented. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the drawings of the embodiments will be briefly introduced below. Obviously, the drawings in the following description only relate to some embodiments of the present disclosure, rather than limiting the present disclosure.

[0018] Figure 1 A flowchart of a data processing method provided by at least one embodiment of the present disclosure is shown;

[0019] Figure 2 A schematic diagram illustrating first-level tasks and second-level tasks provided by at least one embodiment of the present disclosure is shown;

[0020] Figure 3 A schematic diagram illustrating multiplication of two underlying data blocks provided by at least one embodiment of the present disclosure is shown;

[0021] Figure 4 A schematic diagram illustrating multiplication of two inner loop data blocks provided by at least one embodiment of the present disclosure is shown;

[0022] Figure 5 A schematic diagram illustrating multiplication of a first matrix and a second matrix provided by at least one embodiment of the present disclosure is shown;

[0023] Figure 6 A schematic diagram illustrating data sharing provided by at least one embodiment of the present disclosure is shown;

[0024] Figure 7 A schematic diagram of matrix segmentation provided by at least one embodiment of the present disclosure is shown;

[0025] Figure 8 、 9A 9C is a schematic diagram showing grouping of the total outer loop tasks according to at least one embodiment of the present disclosure;

[0026] Figure 10 A schematic diagram of a loop code provided by at least one embodiment of the present disclosure is shown;

[0027] Figure 11 A flowchart of a code generation method provided by at least one embodiment of the present disclosure is shown;

[0028] Figure 12 A schematic block diagram showing a data processing device provided by at least one embodiment of the present disclosure

[0029] Figure 13 A schematic block diagram of an electronic device provided by at least one embodiment of the present disclosure is shown;

[0030] Figure 14 A schematic block diagram showing another electronic device provided by at least one embodiment of the present disclosure; and

[0031] Figure 15 A schematic diagram of a computer-readable storage medium provided by at least one embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0032] To make the purpose, technical solutions, and advantages of the embodiments of the present disclosure more clear, the technical solutions of the embodiments of the present disclosure will be clearly and completely described below in conjunction with the accompanying drawings of the embodiments of the present disclosure. Obviously, the described embodiments are part of the embodiments of the present disclosure, not all of the embodiments. Based on the described embodiments of the present disclosure, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present disclosure.

[0033] Unless otherwise defined, the technical or scientific terms used in this disclosure should have the usual meanings understood by people with ordinary skills in the field to which this disclosure belongs. The words "first", "second" and similar words used in this disclosure do not indicate any order, quantity or importance, but are only used to distinguish different components. Similarly, words such as "one", "an" or "the" do not indicate a quantity limitation, but rather indicate the existence of at least one. Words such as "include" or "comprise" mean that the elements or objects appearing before the word include the elements or objects listed after the word and their equivalents, without excluding other elements or objects. Words such as "connect" or "connected" are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the object being described changes, the relative positional relationship may also change accordingly.

[0034] The synchronization of data and resources also requires the use of certain synchronization operation resources. The use of these synchronization resources is also somewhat complex in the operator fusion process of dynamic shape classes.

[0035] When merging producer and consumer operators, a conditional statement is added to determine whether the currently used resources require synchronization. If so, the producer waits for the required resources, typically shared buffers and registers, to be released by the consumer. After the consumer has finished using the resources, all resources are released. After the consumer has finished using the data, a conditional statement is added to determine whether the relevant resources need to be released. When conditional statements are used, the compiler generates the relevant hardware branch statements after compilation. These branch statements typically execute slowly, resulting in slower performance than code without branch statements.

[0036] Therefore, how to simplify data transmission / sharing between producer operators and consumer operators is a technical problem that needs to be solved.

[0037] At least one embodiment of the present disclosure provides a data processing method, a data processing device, an electronic device, and a computer-readable storage medium. The data processing method includes: determining the data granularity based on the total capacity of multiple available shared caches; based on the data granularity, dividing each dimension of the calculation data into multiple parts, so as to divide the first-level task corresponding to the calculation data into multiple groups of second-level tasks, wherein each group of the second-level tasks includes multiple second-level tasks, and the data volume of the calculation results of each second-level task corresponds to the total capacity of the multiple shared caches; for each group of the second-level tasks, using a loop code, the calculation results of the multiple second-level tasks in each group are cyclically stored in the multiple shared caches, and the data of the multiple shared caches are cyclically output to the data consumer operator.

[0038] This data processing method divides the first-level tasks corresponding to the calculation data into multiple groups of second-level tasks. The data volume of the calculation results of each second-level task corresponds to the total capacity of the multiple available shared caches. The shared cache is reused each time a second-level task is executed, and the multiple second-level tasks in each group use the shared cache multiple times in a loop. In the fusion code of the data producer operator and the data consumer operator, the above-mentioned loop code can be reused, and the storage and retrieval process can be executed in a loop using the loop code, which simplifies the data transmission / sharing between the producer operator and the consumer operator. In addition, for the first second-level task of each group, the data producer operator can directly store data in multiple shared caches without waiting for notification that the shared cache is released. For the last second-level task of each group, the data consumer operator does not need to feedback the retrieval notification to the data producer operator. In this way, the data sharing process between the data producer operator and the data consumer operator is further simplified, and efficiency is improved.

[0039] Figure 1 A flowchart of a data processing method provided by at least one embodiment of the present disclosure is shown.

[0040] like Figure 1 As shown, the data processing method may include steps S110 to S130.

[0041] Step S110: Determine data granularity based on the total capacity of multiple available shared caches.

[0042] Step S120: Based on the data granularity, each dimension of the calculation data is divided into multiple parts, so as to divide the first-level tasks corresponding to the calculation data into multiple groups of second-level tasks, wherein each group of second-level tasks includes multiple second-level tasks, and the data volume of the calculation results of each second-level task corresponds to the total capacity of multiple shared caches.

[0043] Step S130: For each group of second-level tasks, use loop code to loop-store the calculation results of multiple second-level tasks in each group into multiple shared caches, and loop-output the data of the multiple shared caches to the data consumer operator.

[0044] For example, the available multiple shared caches refer to all shared caches used for sharing data between the data producer operator and the data consumer operator to be fused. The total capacity of the multiple shared caches can be determined according to the number of the multiple shared caches and the capacity of each shared cache.

[0045] For example, the data producer operator produces data, and the data consumer operator uses the data produced by the data producer operator to perform operations. For example, the calculation data may include a first matrix A (M×K) and a second matrix B (K×N). The data producer operator can perform the multiplication of the first matrix A and the second matrix B to obtain a third matrix C (M×N). The task of multiplying the first matrix A and the second matrix B can be used as the first-level task. The data producer operator can share the generated matrix C with the data consumer operator, and the data consumer operator can use the matrix C to perform the next operation (such as addition operation).

[0046] For example, a first-level task can be decomposed into multiple second-level tasks, and the calculation results of each second-level task can be synchronized / shared with data consumer operators through the multiple shared caches.

[0047] Figure 2 A schematic diagram of first-level tasks and second-level tasks provided by at least one embodiment of the present disclosure is shown.

[0048] like Figure 2 As shown, the first-level task corresponding to the computation data is, for example, the overall outer loop task, and the second-level task is, for example, the inner loop task. For example, if the size of the first matrix A is 1536×4096 and the size of the second matrix B is 4096×768, then the first-level task (overall outer loop task) is the multiplication of the first matrix A (1536×4096) and the second matrix B (4096×768). Each second-level task (inner loop task) is, for example, the multiplication of data block a (256×512) and data block b (512×128) to obtain data block c (256×128). The computation result of each second-level task is data block c. The data size of each second-level task's computation result corresponds to the total capacity of the multiple shared caches. It can be understood that the data size of each data block c can be equal to or less than the total capacity of the multiple shared caches. Each data block c can occupy multiple shared caches once, that is, each time a data block c is calculated, the multiple shared caches can be used cyclically. Furthermore, the multiple shared caches can be used cyclically to realize data sharing between the data producer operator and the data consumer operator.

[0049] For example, during the execution of each second-level task, each second-level task can be further divided into multiple (e.g., eight) third-level tasks (bottom-level loop tasks). Each third-level task, for example, multiplies a bottom-level data block of 64×512 by a bottom-level data block of 512×64 to obtain a data block of size 64×64. The calculation result of each third-level task can occupy a shared cache.

[0050] For example, if the size of the first matrix is ​​M×K and the size of the second matrix is ​​K×N, then the dimensions of the calculated data include the first dimension (M dimension), the second dimension (N dimension), and the third dimension (K dimension).

[0051] For example, data granularity includes first granularity, second granularity and third granularity. The first granularity is the granularity of the first dimension (M-dimensional granularity), the second granularity is the granularity of the second dimension (N-dimensional granularity), and the third granularity is the granularity of the third dimension (K-dimensional granularity).

[0052] For example, the data granularity can be the granularity of the calculation data of the second-level task. For example, the second-level task is to multiply data block a (256×512) and data block b (512×128) to obtain data block c (256×128). The size of data block a in the M dimension is 256, the size of data block b in the N dimension is 128, and the size of data block a and data block b in the K dimension is 512. Then the first granularity can be 256, the second granularity can be 128, and the third granularity can be 512.

[0053] For example, in addition to using the multiple shared caches mentioned above, the data producer operator also needs to use other caches to load computational data. For example, the computational data includes a first matrix A (M×K) and a second matrix B (K×N). Before multiplying the first matrix A and the second matrix B, the data producer operator needs to load the first matrix A and the second matrix B into multiple first caches and multiple second caches, respectively. Since first-level tasks can be divided into multiple second-level tasks, and each second-level task can be further divided into multiple third-level tasks, when loading the first matrix A and the second matrix B, a portion of the matrix data can also be loaded each time, enabling the circular use of multiple first caches and multiple second caches.

[0054] For example, the first granularity, the second granularity, and the third granularity may be determined based on the capacity of the multiple shared caches and the capacity of the caches used to store computing data (eg, the first cache and the second cache).

[0055] For example, in some embodiments, step S110 may include determining a data granularity based on the number of the multiple shared caches and the capacity of each shared cache, for example, determining at least a first granularity and a second granularity. For example, each data block c may occupy the multiple shared caches described above, and the size of the data block c may be determined based on the number and capacity of the multiple shared caches, i.e., determining a first granularity (M-dimensional granularity) and a second granularity (N-dimensional granularity).

[0056] For example, determining the data granularity based on the number of multiple shared caches and the capacity of each shared cache may include: determining a first size and a second size based on the capacity of each shared cache; decomposing the number of multiple shared caches into a first numerical value and a second numerical value; taking the product of the first numerical value and the first size as the first granularity, and taking the product of the second numerical value and the second size as the second granularity.

[0057] For example, the capacity of each shared cache is 8K. Assuming that each matrix element occupies 2 bytes, each shared cache can accommodate 4096 elements. Therefore, each shared cache can accommodate a data block with a size of, for example, 64×64. Then the first size can be 64, and the second size can be 64. Then, the first granularity and the second granularity can be further determined based on the first size, the second size and the number of shared caches.

[0058] Figure 3 A schematic diagram of multiplying two bottom-level data blocks (or third-level data blocks) provided by at least one embodiment of the present disclosure is shown.

[0059] like Figure 3 As shown, the first bottom-level data block 301 and the second bottom-level data block 302 are multiplied to obtain bottom-level result data 303. The size of the first bottom-level data block 301 is, for example, m=64 and k=32, and the size of the second bottom-level data block 302 is, for example, k=32 and n=64. The first bottom-level data block 301 and the second bottom-level data block 302 are multiplied to obtain the bottom-level calculation result 303. The size of the bottom-level calculation result 303 is m=64 and n=64. One bottom-level calculation result 303 occupies one shared cache, that is, each shared cache can store one data with a size of m=64 and n=64. For example, if the number of available shared caches is 8, then the total storage capacity of the 8 shared caches is 8 (64×64), the number 8 can be decomposed into 1 8 or 2 4, if decomposed into 2 4, multiply one of m and n by 2 and the other by 4. If it is decomposed into 2 4, multiply one of m and n by 1 and the other by 8. Then the first and second granularities in the data granularity can be: 2 m, 4 n; or 4 m, 2 n; or 1 m, 8 n; or 8 m, 1 n. Take the more balanced value 4 m, 2 For example, the first granularity is 4 m=256, the second granularity is 2 n = 128. In the following embodiments, an example is given in which the number of available shared caches is 8, but the present disclosure is not limited thereto, and the number of available shared caches may be determined according to actual needs.

[0060] Figure 4 A schematic diagram of multiplying two inner loop data blocks (second-level data blocks) provided by at least one embodiment of the present disclosure is shown.

[0061] like Figure 4 As shown, the size of the inner loop data block 401 in the first dimension is the first granularity, which is, for example, 4 m. The size of the inner loop data block 402 in the second dimension is the second granularity, which is, for example, 2 The process of multiplying the inner loop data block 401 and the inner loop data block 402 can be divided into 8 groups: Figure 3 The multiplication of the two underlying data blocks shown, therefore, the amount of data obtained by multiplying the inner loop data block 401 and the inner loop data block 402 corresponds to the total capacity of 8 shared caches, and the calculation result of the second-level task occupies 8 shared caches.

[0062] For example, the third granularity does not affect the amount of data in the calculation results, so the number and capacity of the required shared cache are not affected by the third granularity. The third granularity can be set according to demand or determined based on a predetermined method. The following embodiment describes a method for determining the third granularity.

[0063] For example, the third granularity is determined based on the first granularity, the second granularity, the capacity of multiple first caches for storing the first matrix, and the capacity of multiple second caches for storing the second matrix. In a specific embodiment, the capacity of the multiple first caches and the first granularity can be calculated to obtain a third value; the capacity of the multiple second caches and the second granularity can be calculated to obtain a fourth value; and the smaller of the third value and the fourth value is used as the third granularity.

[0064] For example, in the process of loading the first matrix and the second matrix by the data producer operator, multiple first cache cycles can be used to load the first matrix, and multiple second cache cycles can be used to load the second matrix. The total capacity of the multiple first caches is Sa, and Sa is 256K, for example. If the first granularity is 256 and each element occupies 2 bytes, then the maximum third granularity can be ka=(256 1024) / (256 2) = 512. The total capacity of the multiple second caches is Sb, Sb is, for example, 256K. If the second granularity is 128 and each element occupies 2 bytes, then the maximum third granularity can be kb = (256 1024) / (128 2) = 1024. The smaller of ka and kb, 512, is taken as the third granularity of the inner loop.

[0065] For example, after obtaining a first granularity (such as 256), a second granularity (such as 128), and a third granularity (such as 512), the calculated data can be segmented in three dimensions based on the data granularity.

[0066] Figure 5 A schematic diagram of multiplying a first matrix and a second matrix provided by at least one embodiment of the present disclosure is shown.

[0067] like Figure 5 As shown, the size of the first matrix 501 is M×K, and the size of the second matrix 502 is K×N, where M is, for example, 1536, K is, for example, 4096, and N is, for example, 768. The first-level task is to multiply the first matrix 501 by the second matrix 502. The first matrix 501 can be divided into multiple sub-matrices, and the second matrix 502 can also be divided into multiple sub-matrices. Then, the multiplication task of the first matrix 501 and the second matrix 502 can be divided into the multiplication task of multiple groups of sub-matrices.

[0068] Figure 6 A schematic diagram of data sharing provided by at least one embodiment of the present disclosure is shown.

[0069] like Figure 6 As shown, when the shared cache is idle, the data producer operator deposits data into the shared cache. Each time the data producer operator produces data, it notifies the data consumer operator that the data is ready. The data consumer operator waits for the data to be in the shared cache before retrieving it and notifying the data producer operator that a shared cache has become idle. The first time a data producer operator deposits data into the shared cache, it does not need to wait for the shared cache to become idle, as the shared cache is initially set to idle. The last time a data consumer operator consumes data from the shared cache, it does not need to notify the data producer operator.

[0070] For example, when there are 8 available shared caches, for each second-level task, the 8 third-level tasks of the second-level task can be executed in sequence, and the 8 calculation results can be stored in the 8 shared caches in sequence.

[0071] For example, the first-level tasks are divided into multiple groups of second-level tasks, each group of second-level tasks includes multiple second-level tasks, and the multiple second-level tasks in each group can reuse multiple shared caches. In the case where each group includes multiple second-level tasks, for the first second-level task in each group, data can be directly stored in multiple available shared caches without waiting for notification that the shared cache is released (retrieval notification); for the last second-level task in each group, after the data consumer operator retrieves the calculation result of the last second-level task from the multiple available shared caches, it is not necessary to feedback the retrieval notification to the data producer operator. For multiple groups of second-level tasks, loop code can be reused, and the loop code is used to cyclically store the calculation results of multiple second-level tasks in each group into the multiple shared caches, and cyclically output the data of the multiple shared caches to the data consumer operator.

[0072] For example, a loop code can be run once for each group of second-level tasks. Each time the loop code is run, the computation results of multiple second-level tasks in each group are stored in multiple shared caches in a loop, and the data shared by the multiple caches is output to the data consumer operator in a loop. This can be understood as follows: for any two adjacent second-level tasks in each group, as the computation result of the previous second-level task is retrieved from the multiple shared caches by the data consumer operator, the multiple shared caches are cleared in sequence. During this clearing process, the computation result of the next second-level task can be stored in the cleared shared cache in sequence. The process of retrieving the computation result of the previous second-level task and storing the computation result of the next second-level task can overlap in time. This cycle repeats until the computation results of the multiple second-level tasks in the current group are shared, and then the sharing process for the next group of second-level tasks begins. During this loop, each shared cache can quickly store the next data after being cleared, thereby improving the sharing rate. Since the data volume of the calculation result of each second-level task corresponds to the total capacity of multiple shared caches, each second-level task can reuse multiple shared caches once, and each group can reuse multiple shared caches multiple times.

[0073] According to the data processing method of the embodiment of the present disclosure, the first-level tasks corresponding to the calculation data are divided into multiple groups of second-level tasks, the data volume of the calculation results of each second-level task corresponds to the total capacity of the available multiple shared caches, and the shared cache is reused each time a second-level task is executed, and the multiple second-level tasks in each group use the shared cache multiple times in a loop. In the fusion code of the data producer operator and the data consumer operator, the above-mentioned loop code can be reused, and the storage and retrieval process can be executed in a loop using the loop code, which simplifies the data transmission / sharing between the producer operator and the consumer operator. In addition, for the first second-level task of each group, the data producer operator can directly store data in multiple available shared caches without waiting for notification that the shared cache is released / cleared. For the last second-level task of each group, the data consumer operator does not need to feedback the retrieval notification to the data producer operator. In this way, the data sharing process between the data producer operator and the data consumer operator is further simplified, and efficiency is improved.

[0074] For example, the computational data includes a first matrix and a second matrix, and the first-level task includes multiplying the first matrix by the second matrix. Step S120 may include: dividing the first matrix into multiple segments based on a first granularity in a first dimension (e.g., M dimension), and dividing the first matrix into multiple segments based on a third granularity in a third dimension (e.g., K dimension), so that the first matrix is ​​divided into multiple first sub-matrices; dividing the second matrix into multiple segments based on the third granularity in the third dimension (K dimension), and dividing the second matrix into three segments based on the second granularity in a second dimension (e.g., N dimension), so that the second matrix is ​​divided into multiple second sub-matrices, wherein the first and last segments of the three segments in the second dimension correspond to the second granularity; wherein the task of multiplying any first sub-matrix by three corresponding second sub-matrices can be regarded as a group of second-level tasks, wherein the three corresponding second sub-matrices correspond to the three segments in the second dimension, respectively.

[0075] Figure 7 A schematic diagram of matrix segmentation provided by at least one embodiment of the present disclosure is shown.

[0076] like Figure 7As shown, the first matrix 701 is divided into three segments in the M dimension: 0-256, 256-1280, and 1280-1536. The first matrix is ​​divided into three segments in the K dimension: 0-512, 512-3584, and 3584-4096. The first matrix is ​​divided into nine first sub-matrices. The second matrix 702 is divided into three segments in the K dimension: 0-512, 512-3584, and 3584-4096. The second matrix is ​​divided into three segments in the N dimension: 0-128, 128-640, and 640-768. The second matrix is ​​divided into nine second sub-matrices. The first and last segments of the three segments in the M dimension correspond to the first granularity of 256, for example. The first and last segments of the three segments in the N dimension correspond to the second granularity of 128, for example. The first and last segments of the three segments in the K dimension correspond to the third granularity of 512, for example.

[0077] For example, when multiplying a first matrix and a second matrix, each first submatrix can be multiplied with three second submatrices. For example, if the first submatrix a1 has M=0-256 and K=0-512, the first submatrix a1 corresponds to three segments of the second matrix with K=512: b1, b2, and b3. For another example, if the first submatrix a2 has M=0-256 and K=512-3584, the first submatrix a2 corresponds to three segments of the second matrix with K=512-3584: b4, b5, and b6. The task of multiplying a first submatrix, obtained by combining any segment in the M dimension with any segment in the K dimension, with the corresponding three second submatrices in the second matrix can be considered a group of second-level tasks, with each group of second-level tasks corresponding to three segments in the N dimension. For example, the multiplication of the first submatrix a2 with the corresponding three second submatrices b1, b2, and b3 can be considered a group of second-level tasks.

[0078] Figure 8 、 9A 9C shows a schematic diagram of grouping the total outer loop tasks provided by at least one embodiment of the present disclosure.

[0079] like Figure 8 、 9A As shown in Figure 9C, for each value range in the M dimension, the K dimension can have three possible ranges: 0–512, 512–3584, and 3584–4096. Furthermore, for each value range in the K dimension, the N dimension can have three possible ranges: 0–128, 128–640, and 640–768. When the values ​​of M and K are fixed (for example, M = 0–256 and K = 0–512), the three possible values ​​of N correspond to a set of second-level tasks. Figure 9AThe figure shows the case where M=0~256 and K=0~512. In this case, there are three segments: N=0~128, N=128~640, and N=640~768, which form three small outer loop tasks respectively. For the first outer loop task, M=0~256, K=0~512, and N=0~128, it can be used as an inner loop task (second-level task); for the second outer loop task, M=0~256, K=0~512, and N=128~640 can be divided into four inner loop tasks; for the third outer loop task, M=0~256, K=0~512, and N=640~768 can be used as an inner loop task. In this way, the six inner loop tasks can be used as a group. Figure 9B The case where M=0~256 and K=512~3584 is shown. Figure 9C The case where M=0~256 and K=3584~4096 is shown, and Figure 9A Similarly, Figure 9C The case shown may also include a set of 6 inner loop tasks. Figure 9B In , since K = 512 ~ 3584, it can be divided into more memory cycle tasks.

[0080] For example, the computational data can be divided into multiple segments along the first dimension, multiple segments along the third dimension, and three segments along the second dimension. The first and last segments along the second dimension have the second granularity. This allows the first and last tasks in each group of second-level tasks to run the first and second code segments, respectively. The first code segment does not need to determine whether each shared cache is idle, and the second code segment does not need to send a retrieval notification back to the data producer operator. This can further simplify the data sharing process between the data producer operator and the data consumer operator, improving efficiency. Furthermore, multiple intermediate tasks in each group can reuse intermediate code segments, further simplifying the code of the fusion operator.

[0081] For example, in other embodiments, the number of segments in any dimension may be greater than 3. For example, the M dimension may be segmented according to a first granularity, with each segment having the first granularity, and the K dimension may be segmented according to a third granularity, with each segment having the third granularity.

[0082] For example, it also includes: determining the number of synchronization resource locks required based on the number of multiple shared caches and the synchronization granularity. The synchronization granularity is the amount of data synchronized / shared each time by the data producer operator and the data consumer operator. For example, the synchronization granularity can be the same as the capacity of each shared cache. In this way, each time the data producer operator stores data in a shared cache, it notifies the data consumer operator to extract the data. After the data consumer operator extracts the data from the shared cache, it notifies the data producer operator that the data has been extracted, thereby completing one data synchronization / sharing. For example, in other embodiments, the synchronization granularity can also be greater than the capacity of the shared cache, for example, it can be a multiple of the shared cache capacity. The synchronization resource lock is used to achieve data synchronization between the data producer operator and the data consumer operator. The synchronization resource lock can be, for example, a barrier object. The synchronization resource lock can be inserted in the running code of the data producer operator and the data consumer operator as a synchronization node, waiting for the data producer end and the data consumer end to reach synchronization at this node before executing subsequent operations. The number of synchronization resource locks is related to the number of shared caches and the synchronization granularity. For example, if the synchronization granularity is the same as the capacity of the shared cache, two synchronization resource locks are required for each data synchronization. Therefore, the number of synchronization resource locks required can be twice the number of multiple shared caches.

[0083] For example, each group of second-level tasks includes a first task, a second task, and at least one intermediate task located between the first task and the second task. The loop code includes a first code segment, a second code segment, and an intermediate code segment. Step S130 may include: using the first code segment to cyclically store the calculation result of the first task in multiple shared caches, and cyclically outputting the calculation result of the first task from the data of the multiple shared caches to the data consumer operator; for each intermediate task in at least one intermediate task, using the intermediate code to cyclically store the calculation result of the intermediate task in multiple shared caches, and cyclically outputting the calculation result of the intermediate task from the data of the multiple shared caches to the data consumer operator; using the second code segment to cyclically store the calculation result of the second task in multiple shared caches, and cyclically outputting the calculation result of the second task from the data of the multiple shared caches to the data consumer operator.

[0084] For example, each second-level task includes multiple third-level tasks, and the amount of data for each third-level task corresponds to the capacity of a shared cache. The loop code includes code corresponding to a data producer operator and code corresponding to a data consumer operator, wherein the data producer operator is used to execute multiple groups of second-level tasks to obtain calculation results of multiple second-level tasks, and the data consumer operator is used to perform the next step of calculation based on the calculation results; the code corresponding to the data producer operator is used to store the calculation results of multiple third-level tasks of each second-level task in multiple shared caches in sequence, and send a storage notification to the data consumer operator after each time the calculation result of a third-level task is stored in a shared cache; the code corresponding to the data consumer operator is used to retrieve the calculation results of multiple third-level tasks of each second-level task from multiple shared caches in sequence, and send a retrieval notification to the data producer operator after each time the calculation result of a third-level task is retrieved from a shared cache.

[0085] For example, the first code segment includes: storing the calculation results of multiple third-level tasks included in the first task in multiple shared caches in sequence, and sending a storage notification corresponding to the stored shared cache to the data consumer operator after storing the calculation result of a third-level task in a shared cache each time; after receiving a storage notification each time, retrieving data from the shared cache corresponding to the storage notification, and sending a retrieval notification corresponding to the shared cache to the data producer operator, so as to output all the calculation results of multiple third-level tasks of the first task to the data consumer operator.

[0086] For example, the intermediate code segment includes: after each retrieval notification is received, the calculation result of a third-level task of the intermediate task is stored in the shared cache corresponding to the retrieval notification, and a storage notification corresponding to the shared cache is sent to the data consumer operator to store all the calculation results of multiple third-level tasks of the intermediate task in multiple shared caches; after each storage notification is received, data is retrieved from the shared cache corresponding to the storage notification, and a retrieval notification corresponding to the shared cache is sent to the data producer operator to output all the calculation results of multiple third-level tasks of the intermediate task to the data consumer operator.

[0087] For example, the second section of code includes: after each retrieval notification is received, the calculation result of a third-level task of the second task is stored in the shared cache corresponding to the retrieval notification, and a storage notification corresponding to the shared cache is sent to the data consumer operator to store all the calculation results of multiple third-level tasks of the second task in multiple shared caches; after each storage notification is received, data is retrieved from the shared cache corresponding to the storage notification to output all the calculation results of multiple third-level tasks of the second task to the data consumer operator.

[0088] Figure 10 A schematic diagram of a loop code provided by at least one embodiment of the present disclosure is shown.

[0089] like Figure 10 As shown, taking the number of available shared caches as 3 as an example, the available shared caches are buf0~2. In the process of data producer operators and data consumer operators sharing data, messages need to be passed. When the number of available shared caches is 3, the number of required synchronization resource locks is 6. Each time all multiple shared caches are used, the data producer operator needs to pass 3 storage notifications, and the data consumer operator needs to pass 3 retrieval notifications. The synchronization signals used are, for example, sync0~5. Producer represents the data producer operator, and Consumer represents the data consumer operator. The code segment 1001 corresponding to the Producer end and the code segment 1002 corresponding to the Consumer end can be combined as the first code segment in the loop code. The code segment 1003 corresponding to the Producer end and the code segment 1004 corresponding to the Consumer end can be combined as the intermediate code segment in the loop code. The code segment 1005 corresponding to the Producer end and the code segment 1006 corresponding to the Consumer end can be combined as the second code segment in the loop code. Figure 9A Take the group of second-level tasks shown as an example. As mentioned above, the group of second-level tasks includes 6 second-level tasks. The first second-level task (N=0~128) is used as a first task, the second to fifth second-level tasks (N=128~256, 256~384, 384~512, 512~640) are each used as an intermediate task, and the last second-level task (N=640~768) is used as a second task. The first code segment is used to share the calculation results of the first task between the Producer and the Consumer, and the intermediate code segment is used to share the calculation results of each intermediate task. Figure 10 Only one intermediate code segment is shown in FIG. 1 , but the intermediate code segment can be executed multiple times continuously as needed, and the second code segment is used to share the calculation result of the second task.

[0090] For example, in Figure 10In the example, the first task includes three third-level tasks, and the first code segment is executed once for the first task. In the first code segment, since the shared caches buf0~2 are all idle, the Producer does not need to wait for the removal notification sent by the Consumer. The Producer can directly store the calculation result of the first third-level task in the first task in buf0 and use sync0 to notify the Consumer; the Producer stores the calculation result of the second third-level task in the first task in buf1 and uses sync1 to notify the Consumer; the Producer stores the calculation result of the third third-level task in the first task in buf2 and uses Notify sync2 to notify the Consumer. After the Consumer waits for sync0, it takes out the calculation result of the first third-level task from buf0 and uses sync3 to notify the Producer; after the Consumer waits for sync1, it takes out the calculation result of the second third-level task from buf1 and uses sync4 to notify the Producer; after the Consumer waits for sync2, it takes out the calculation result of the third third-level task from buf2 and uses sync5 to notify the Producer.

[0091] For example, in Figure 10 In the example, each intermediate task includes three third-level tasks, and the intermediate code segment is executed once for each intermediate task. In the intermediate code segment, the Producer waits until sync3, stores the calculation result of the first third-level task of the intermediate task in buf0, and notifies the Consumer using sync0. The Producer waits until sync4, stores the calculation result of the second third-level task in the intermediate task in buf1, and notifies the Consumer using sync1. The Producer waits until sync5, stores the calculation result of the third third-level task in the intermediate task in buf2, and notifies the Consumer using Notify sync2. The Consumer waits until sync0, takes out the calculation result of the first third-level task from buf0, and notifies the Producer using sync3. The Consumer waits until sync1, takes out the calculation result of the second third-level task from buf1, and notifies the Producer using sync4. The Consumer waits until sync2, takes out the calculation result of the third third-level task from buf2, and notifies the Producer using sync5.

[0092] For example, in Figure 10In the example, the second task includes three third-level tasks, and the second code segment is executed once for the second task. In the second code segment, the Producer waits until sync3, stores the calculation result of the first third-level task of the second task in buf0, and notifies the Consumer using sync0. The Producer waits until sync4, stores the calculation result of the second third-level task in the second task in buf1, and notifies the Consumer using sync1. The Producer waits until sync5, stores the calculation result of the third third-level task in the second task in buf2, and notifies the Consumer using Notify sync2. The Consumer waits until sync0, takes out the calculation result of the first third-level task from buf0, and does not need to notify the Producer again. The Consumer waits until sync1, takes out the calculation result of the second third-level task from buf1, and does not need to notify the Producer again. The Consumer waits until sync2, takes out the calculation result of the third third-level task from buf2, and does not need to notify the Producer again.

[0093] For example, the data processing method of the embodiment of the present disclosure can be applied to calculation data of various shapes.

[0094] At least one embodiment of the present disclosure provides a code generation method. Figure 11 A flowchart of a code generation method provided by at least one embodiment of the present disclosure is shown.

[0095] like Figure 11 As shown, the code generation method may include steps S1110 to S1140.

[0096] Step S1110: Determine data granularity based on the total capacity of multiple available shared caches.

[0097] Step S1120: Based on the data granularity, each dimension of the calculation data is divided into multiple parts (such as three parts) to divide the first-level tasks corresponding to the calculation data into multiple groups of second-level tasks, wherein each group of second-level tasks includes multiple second-level tasks, and the data volume of the calculation results of each second-level task corresponds to the total capacity of multiple shared caches.

[0098] Step S1130: Determine a loop code for each group of second-level tasks, wherein the loop code is used to cyclically store the calculation results of multiple second-level tasks in each group into multiple shared caches, and cyclically output the data of the multiple shared caches to the data consumer operator.

[0099] Step S1140: Generate a fusion code based on the loop code.

[0100] For example, for steps S1110 to S1130, please refer to the above-mentioned relevant embodiments. For step S1140, the fusion code can be the fusion code of the data producer operator and the data consumer operator, and the fusion code includes the code for data sharing between the data producer operator and the data consumer operator. In the data sharing code portion, since the total computing task (first-level task) is divided into multiple groups of second-level tasks, the loop code can be reused in the branch code of each group of second-level tasks to implement each group of second-level tasks using the loop code, and then the code of other parts is combined to form the fusion code of the data producer operator and the data consumer operator.

[0101] For example, by using the three-step approach to simplify the data and resources shared between producers and consumers, as well as the allocation and release of synchronized resources, operators with producer-consumer relationships can be automatically generated into fused code. This simplifies the data sharing relationship between producers and consumers and reduces the use of conditional branch statements.

[0102] Figure 12 A schematic block diagram of a data processing device 1200 provided by at least one embodiment of the present disclosure is shown.

[0103] For example, Figure 12 As shown, the data processing device 1200 includes a determination unit 1210, a segmentation unit 1220 and an execution unit 1230. These components are interconnected via a bus system and / or other forms of connection mechanisms (not shown). For example, these modules can be implemented by hardware (e.g., circuit) modules, software modules, or any combination of the two. The following embodiments are the same and will not be described in detail. For example, these units can be implemented by a central processing unit (CPU), a graphics processing unit (GPU), a tensor processing unit (TPU), a field programmable gate array (FPGA), or other forms of processing units with data processing capabilities and / or instruction execution capabilities, as well as corresponding computer instructions. It should be noted that Figure 12 The components and structure of the data processing device 1200 shown are merely exemplary and non-limiting. The data processing device 1200 may also have other components and structures as needed.

[0104] The determining unit 1210 is configured to determine the data granularity based on the total capacity of the available multiple shared caches. The determining unit 1210 may, for example, execute Figure 1 Step S110 described.

[0105] The splitting unit 1220 is configured to split each dimension of the computational data into multiple parts based on the data granularity, so as to divide the first-level task corresponding to the computational data into multiple groups of second-level tasks, wherein each group of the second-level tasks includes multiple second-level tasks, and the amount of data of the computational result of each second-level task corresponds to the total capacity of the multiple shared caches. The splitting unit 1220 may, for example, execute Figure 1 Step S120 described.

[0106] The execution unit 1230 is configured to, for each group of the second-level tasks, use a loop code to loop the calculation results of the multiple second-level tasks in each group to the multiple shared caches, and loop the data of the multiple shared caches to the data consumer operator. The execution unit 1230 can, for example, execute Figure 1 Step S130 described.

[0107] For example, the determining unit 1210, the segmenting unit 1220, and the executing unit 1230 may be hardware, software, firmware, or any feasible combination thereof. For example, the determining unit 1210, the segmenting unit 1220, and the executing unit 1230 may be dedicated or general-purpose circuits, chips, or devices, or may be a combination of a processor and memory. The embodiments of the present disclosure do not limit the specific implementation of each of the above-mentioned units.

[0108] For example, the determination unit 1210, the segmentation unit 1220, and the execution unit 1230 may include codes and programs stored in a memory; the processor may execute the codes and programs to implement some or all of the functions of the image determination unit 1210, the segmentation unit 1220, and the execution unit 1230 as described above. For example, the determination unit 1210, the segmentation unit 1220, and the execution unit 1230 may be dedicated hardware devices used to implement some or all of the functions of the determination unit 1210, the segmentation unit 1220, and the execution unit 1230 as described above. For example, the determination unit 1210, the segmentation unit 1220, and the execution unit 1230 may be a circuit board or a combination of multiple circuit boards used to implement the functions as described above. In an embodiment of the present disclosure, the circuit board or the combination of multiple circuit boards may include: (1) one or more processors; (2) one or more non-temporary memories connected to the processors; and (3) firmware stored in the memory that is executable by the processors.

[0109] It should be noted that in the embodiment of the present disclosure, the various units of the data processing device 1200 correspond to the various steps of the aforementioned data processing method. For the specific functions of the data processing device 1200, please refer to the relevant description of the data processing method, which will not be repeated here. Figure 12The components and structure of the data processing device 1200 shown are merely exemplary and non-restrictive. The data processing device 1200 may include other components and structures as needed. The data processing device 1200 may include more or fewer circuits or units, and the connections between the various circuits or units are not limited and can be determined based on actual needs. The specific configuration of each circuit or unit is not limited and may be composed of analog devices, digital chips, or other suitable configurations based on circuit principles.

[0110] At least one embodiment of the present disclosure provides a code generation device, including a first determination unit, a splitting unit, a second determination unit and a fusion unit. The first determination unit is configured to determine the data granularity based on the total capacity of multiple available shared caches; the splitting unit is configured to split each dimension of the calculation data into multiple parts based on the data granularity, so as to divide the first-level task corresponding to the calculation data into multiple groups of second-level tasks, wherein each group of the second-level tasks includes multiple second-level tasks, and the data volume of the calculation results of each second-level task corresponds to the total capacity of the multiple shared caches; the second determination unit is configured to determine the loop code for each group of the second-level tasks, wherein the loop code is used to cyclically store the calculation results of the multiple second-level tasks in each group into the multiple shared caches, and cyclically output the data of the multiple shared caches to the data consumer operator; the fusion unit is configured to generate a fusion code based on the loop code.

[0111] In the embodiment of the present disclosure, each unit of the code generation device corresponds to each step of the aforementioned code generation method. For the specific functions of the code generation device, reference may be made to the relevant description of the code generation method, which will not be repeated here.

[0112] At least one embodiment of the present disclosure further provides an electronic device including a processor and a memory, wherein the memory stores one or more computer program modules configured to be executed by the processor to implement the above-mentioned data processing method.

[0113] Figure 13 This is a schematic block diagram of an electronic device provided in some embodiments of the present disclosure. Figure 13As shown, the electronic device 1300 includes a processor 1310 and a memory 1320. The memory 1320 stores non-transitory computer-readable instructions (e.g., one or more computer program modules). The processor 1310 is configured to execute the non-transitory computer-readable instructions. When the non-transitory computer-readable instructions are executed by the processor 1310, one or more steps of the data processing method described above are performed. The memory 1320 and the processor 1310 may be interconnected via a bus system and / or other form of connection mechanism (not shown). For the specific implementation of each step of the data processing method and related explanations, please refer to the above-mentioned embodiment of the data processing method, and repeated parts will not be repeated here.

[0114] It should be noted that Figure 13 The components of the electronic device 1300 shown are merely exemplary and non-limiting. The electronic device 1300 may also have other components according to actual application requirements.

[0115] For example, the processor 1310 and the memory 1320 may communicate with each other directly or indirectly.

[0116] For example, the processor 1310 and the memory 1320 may communicate via a network. The network may include a wireless network, a wired network, and / or any combination of a wireless network and a wired network. The processor 1310 and the memory 1320 may also communicate with each other via a system bus, which is not limited in this disclosure.

[0117] For example, the processor 1310 and the memory 1320 may be provided on a server side (or a cloud side).

[0118] For example, the processor 1310 can control other components in the electronic device 1300 to perform desired functions. For example, the processor 1310 can be a central processing unit (CPU), a graphics processing unit (GPU), or other processing unit with data processing capabilities and / or program execution capabilities. For example, the central processing unit (CPU) can be an X86 or ARM architecture. The processor 1310 can be a general-purpose processor or a dedicated processor and can control other components in the electronic device 1300 to perform desired functions.

[0119] For example, memory 1320 may include any combination of one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache memory. Non-volatile memory may include, for example, read-only memory (ROM), a hard disk, an erasable programmable read-only memory (EPROM), a portable compact disk read-only memory (CD-ROM), a USB memory, a flash memory, and the like. One or more computer program modules may be stored on the computer-readable storage medium, and the processor 1310 may execute one or more computer program modules to implement various functions of the electronic device 1300. The computer-readable storage medium may also store various applications and data, as well as data used and / or generated by the applications.

[0120] It should be noted that, in the embodiment of the present disclosure, the specific functions and technical effects of the electronic device 1300 can be referred to the above description of the data processing method, which will not be repeated here.

[0121] Figure 14 This is a schematic block diagram of another electronic device provided in some embodiments of the present disclosure. The electronic device 1400 is suitable for implementing the data processing method provided in the embodiments of the present disclosure. The electronic device 1400 may be a terminal device, etc. It should be noted that, Figure 14 The electronic device 1400 shown is merely an example and does not limit the functions and scope of use of the embodiments of the present disclosure.

[0122] like Figure 14 As shown, electronic device 1400 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 1410, which can perform various appropriate actions and processes according to programs stored in a read-only memory (ROM) 1420 or programs loaded from a storage device 1480 into a random access memory (RAM) 1430. Various programs and data required for the operation of electronic device 1400 are also stored in RAM 1430. Processing device 1410, ROM 1420, and RAM 1430 are connected to each other via a bus 1440. An input / output (I / O) interface 1450 is also connected to bus 1440.

[0123] Typically, the following devices may be connected to the I / O interface 1450: an input device 1460 including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 1470 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 1480 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1490. The communication device 1490 may allow the electronic device 1400 to communicate with other electronic devices wirelessly or by wire to exchange data. Although Figure 14 The electronic device 1400 is shown with various devices, but it should be understood that it is not required to implement or possess all of the devices shown, and the electronic device 1400 may alternatively implement or possess more or fewer devices.

[0124] For example, according to an embodiment of the present disclosure, the above-mentioned data processing method can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program includes a program code for executing the above-mentioned data processing method. In such an embodiment, the computer program can be downloaded and installed from the network through the communication device 1490, or installed from the storage device 1480, or installed from the ROM 1420. When the computer program is executed by the processing device 1410, the functions defined in the data processing method provided in the embodiment of the present disclosure can be implemented.

[0125] At least one embodiment of the present disclosure further provides a computer-readable storage medium storing non-transitory computer-readable instructions. When the non-transitory computer-readable instructions are executed by a computer, the above-mentioned data processing method can be implemented.

[0126] Figure 15 A schematic diagram of a storage medium provided in some embodiments of the present disclosure. Figure 15 As shown, the storage medium 1500 stores non-transitory computer-readable instructions 1510. For example, when the non-transitory computer-readable instructions 1510 are executed by a computer, one or more steps in the data processing method described above are performed.

[0127] For example, the storage medium 1500 can be applied to the electronic device 1300. Figure 13 The memory 1320 in the electronic device 1300 is shown. For example, the description of the storage medium 1500 can be referred to Figure 13 The corresponding description of the memory 1320 in the electronic device 1300 is not repeated here.

[0128] The above description is merely a preferred embodiment of the present disclosure and an illustration of the technical principles employed. Those skilled in the art should understand that the scope of the present disclosure is not limited to technical solutions formed by specific combinations of the aforementioned technical features. It also encompasses other technical solutions formed by any combination of the aforementioned technical features or their equivalents, without departing from the scope of the above disclosure. For example, a technical solution formed by replacing the aforementioned features with (but not limited to) technical features with similar functions disclosed in this disclosure.

[0129] In addition, although each operation is described in a specific order, this should not be understood as requiring these operations to be performed in the specific order shown or in a sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Similarly, although some specific implementation details have been included in the above discussion, these should not be interpreted as limiting the scope of the present disclosure. Some features described in the context of a separate embodiment can also be implemented in a single embodiment in combination. On the contrary, the various features described in the context of a single embodiment can also be implemented in multiple embodiments individually or in any suitable sub-combination mode.

[0130] Although the subject matter has been described in language specific to structural features and / or methodological logical acts, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are merely example forms of implementing the claims.

[0131] Regarding this disclosure, the following points need to be explained:

[0132] (1) The drawings of the embodiments of the present disclosure only relate to the structures involved in the embodiments of the present disclosure. Other structures may refer to conventional designs.

[0133] (2) In the absence of conflict, the embodiments of the present disclosure and the features therein may be combined with each other to form new embodiments.

[0134] The above description is only a specific embodiment of the present disclosure, but the protection scope of the present disclosure is not limited thereto. The protection scope of the present disclosure shall be based on the protection scope of the claims.

Claims

1. A data processing method, comprising: Determine the data granularity based on the total capacity of multiple shared caches available; Based on the data granularity, each dimension of the computation data is divided into a plurality of parts, so as to divide the first-level tasks corresponding to the computation data into a plurality of groups of second-level tasks, wherein each group of the second-level tasks includes a plurality of second-level tasks, and the data volume of the computation result of each second-level task corresponds to the total capacity of the plurality of shared caches; For each group of the second-level tasks, using a loop code, cyclically storing the calculation results of the plurality of second-level tasks in each group into the plurality of shared caches, and cyclically outputting the data in the plurality of shared caches to the data consumer operator; Among them, the data processing method also includes: determining the required number of synchronization resource locks based on the number and synchronization granularity of the multiple shared caches, and the loop code includes using the said number of synchronization resource locks to realize data synchronization between the data producer operator and the data consumer operator.

2. The data processing method according to claim 1, wherein: Each group of the second-level tasks includes a first task, a second task, and at least one intermediate task between the first task and the second task; The loop code includes a first code segment, a second code segment and an intermediate code segment, For each group of the second-level tasks, using loop code, cyclically storing the calculation results of the multiple second-level tasks in each group into the multiple shared caches, and cyclically outputting the data of the multiple shared caches to the data consumer operator, including: Using the first code segment, cyclically storing the calculation result of the first task to the multiple shared caches, and cyclically outputting the calculation result of the first task from the data of the multiple shared caches to the data consumer operator; For each of the at least one intermediate task, using the intermediate code segment, cyclically storing the calculation result of the intermediate task to the multiple shared caches, and cyclically outputting the calculation result of the intermediate task from the data of the multiple shared caches to the data consumer operator; The second code segment is used to circularly store the calculation result of the second task in the multiple shared caches, and the calculation result of the second task is circularly output from the data of the multiple shared caches to the data consumer operator.

3. The data processing method according to claim 2, wherein: Each of the second-level tasks includes a plurality of third-level tasks, and the data volume of the calculation results of each third-level task corresponds to the capacity of one of the shared caches; The loop code includes code corresponding to a data producer operator and code corresponding to a data consumer operator, wherein the data producer operator is used to execute the multiple groups of second-level tasks to obtain calculation results of the multiple second-level tasks, and the data consumer operator is used to perform the next step of calculation based on the calculation results of the second-level tasks; The code corresponding to the data producer operator is used to store the calculation results of multiple third-level tasks of each second-level task in the multiple shared caches in sequence, and send a storage notification to the data consumer operator after storing the calculation result of a third-level task in a shared cache each time; The code corresponding to the data consumer operator is used to retrieve the calculation results of multiple third-level tasks of each second-level task from the multiple shared caches in sequence, and send a retrieval notification to the data producer operator after each time a calculation result of a third-level task is retrieved from a shared cache.

4. The data processing method according to claim 2 or 3, wherein: The first code segment includes: storing the computation results of the plurality of third-level tasks included in the first task in the plurality of shared caches in sequence, and sending a storage notification corresponding to the stored shared cache to the data consumer operator after storing the computation result of one of the third-level tasks in the shared cache each time; Each time a storage notification is received, data is retrieved from the shared cache corresponding to the storage notification, and a retrieval notification corresponding to the shared cache is sent to the data producer operator to output all the calculation results of multiple third-level tasks of the first task to the data consumer operator.

5. The data processing method according to claim 2 or 3, wherein: The intermediate code segment includes: After receiving a pull notification each time, the operator stores the computation result of a third-level task of the intermediate task in the shared cache corresponding to the pull notification, and sends a storage notification corresponding to the shared cache to the data consumer operator to store all the computation results of multiple third-level tasks of the intermediate task in the multiple shared caches; Each time a storage notification is received, data is retrieved from the shared cache corresponding to the storage notification, and a retrieval notification corresponding to the shared cache is sent to the data producer operator to output all the calculation results of multiple third-level tasks of the intermediate task to the data consumer operator.

6. The data processing method according to claim 2 or 3, wherein: The second code segment includes: After receiving a pull notification each time, the operator stores a calculation result of a third-level task of the second task in the shared cache corresponding to the pull notification, and sends a storage notification corresponding to the shared cache to the data consumer operator to store all calculation results of multiple third-level tasks of the second task in the multiple shared caches; After receiving the storage notification each time, data is retrieved from the shared cache corresponding to the storage notification, so as to output all calculation results of multiple third-level tasks of the second task to the data consumer operator.

7. The data processing method according to claim 1 or 2, wherein: The data granularity includes a first granularity, a second granularity and a third granularity; Determine the data granularity based on the total capacity of multiple shared caches available, including: The data granularity is determined based on the number of the multiple shared caches and the capacity of each of the shared caches.

8. The data processing method according to claim 7, wherein: Determining the data granularity based on the number of the multiple shared caches and the capacity of each of the shared caches includes: Determining a first size and a second size based on a capacity of each shared cache; Decomposing the number of the plurality of shared caches into a first value and a second value; The product of the first value and the first size is used as the first particle size, and the product of the second value and the second size is used as the second particle size.

9. The data processing method according to claim 8, wherein: The calculation data includes a first matrix and a second matrix, and the first-level task includes multiplying the first matrix by the second matrix; The step of determining the data granularity based on the number of the multiple shared caches and the capacity of each shared cache further includes: determining the third granularity based on the first granularity, the second granularity, capacities of a plurality of first buffers for storing the first matrix, and capacities of a plurality of second buffers for storing the second matrix; The determining of the third granularity based on the first granularity, the second granularity, the capacity of a plurality of first caches for storing the first matrix, and the capacity of a plurality of second caches for storing the second matrix includes: performing a calculation on the capacities of the plurality of first caches and the first granularity to obtain a third value; performing calculation on the capacities of the plurality of second caches and the second granularity to obtain a fourth value; The smaller of the third value and the fourth value is used as the third particle size.

10. The data processing method according to claim 7, wherein: The calculation data includes a first matrix and a second matrix, and the first-level task includes multiplying the first matrix by the second matrix; Based on the data granularity, each dimension of the computational data is divided into multiple parts, so as to divide the first-level tasks corresponding to the computational data into multiple groups of second-level tasks, including: Divide the first matrix into multiple segments based on the first granularity in a first dimension, and divide the first matrix into multiple segments based on the third granularity in a third dimension, so that the first matrix is ​​divided into multiple first sub-matrices; Dividing the second matrix into multiple segments based on the third granularity in the third dimension, and dividing the second matrix into three segments based on the second granularity in the second dimension, so that the second matrix is ​​divided into multiple second sub-matrices, wherein the first segment and the last segment of the three segments in the second dimension both correspond to the second granularity; The task of multiplying any first sub-matrix by three corresponding second sub-matrices can be used as a group of second-level tasks, wherein the three corresponding second sub-matrices respectively correspond to three segments on the second dimension.

11. A code generation method comprising: Determine the data granularity based on the total capacity of multiple shared caches available; Based on the data granularity, each dimension of the computation data is divided into a plurality of parts, so as to divide the first-level tasks corresponding to the computation data into a plurality of groups of second-level tasks, wherein each group of the second-level tasks includes a plurality of second-level tasks, and the data volume of the computation result of each second-level task corresponds to the total capacity of the plurality of shared caches; Determining a loop code for each group of the second-level tasks, wherein the loop code is used to cyclically store computation results of the plurality of second-level tasks in each group into the plurality of shared caches, and cyclically output data from the plurality of shared caches to a data consumer operator; generating a fusion code based on the loop code; Among them, the code generation method also includes: determining the required number of synchronization resource locks based on the number of the multiple shared caches and the synchronization granularity, and the loop code includes using the said number of synchronization resource locks to realize data synchronization between the data producer operator and the data consumer operator.

12. A data processing device comprising: a determination unit configured to determine a data granularity based on a total capacity of the available multiple shared caches; a splitting unit configured to split each dimension of the computation data into a plurality of parts based on the data granularity, so as to divide the first-level task corresponding to the computation data into a plurality of groups of second-level tasks, wherein each group of the second-level tasks includes a plurality of second-level tasks, and a data volume of a computation result of each second-level task corresponds to a total capacity of the plurality of shared caches; an execution unit configured to, for each group of the second-level tasks, use a loop code to cyclically store the calculation results of the plurality of second-level tasks in each group into the plurality of shared caches, and cyclically output the data in the plurality of shared caches to a data consumer operator; The determination unit is further configured to determine the number of required synchronization resource locks based on the number of the multiple shared caches and the synchronization granularity, and the loop code includes using the number of synchronization resource locks to achieve data synchronization between the data producer operator and the data consumer operator.

13. An electronic device comprising: processor; a memory storing one or more computer program modules; The one or more computer program modules are configured to be executed by the processor to implement the method according to any one of claims 1 to 11.

14. A computer-readable storage medium storing non-transitory computer-readable instructions, wherein when the non-transitory computer-readable instructions are executed by a computer, the method according to any one of claims 1 to 11 can be implemented.

Images