Code adjustment method, data processing method, device, system, product and medium

Abstract

The specification provides a code adjustment method, a data processing method, equipment, a system, a product and a medium. The method comprises: obtaining implementation code for implementing a plurality of reduction operations; determining a calculation expression of the reduction operation based on the implementation code; converting the calculation expression of the previous reduction operation into a previous recursive expression for gradually calculating a local reduction result of the current step from the first data in the data set of the reduction operation; converting the calculation expression of the next reduction operation into a next recursive expression for gradually calculating the reduction result of the current step by relying on the local reduction result of the current step of the previous recursive expression from the first data in the data set of the next reduction operation; and adjusting the implementation code into target code for implementing: synchronously reading data in the data set of the previous reduction operation and data in the data set of the next reduction operation, and gradually calculating the reduction result by using the previous recursive expression and the next recursive expression on the read data.

Description

Technical Field

[0001] This manual relates to the field of code adjustment technology, and in particular to code adjustment methods, data processing methods, equipment, systems, products and media. Background Technology

[0002] Currently, many fields involve computations of continuously nested reduction operations. For example, mainstream deep learning models contain a large number of continuously nested reduction operations. In a reduction operation, a binary operator satisfying the associative law can be used for N input data. Produces one result. These operations include finding the minimum value, finding the maximum value, summing, sum of squares, logical AND / OR, vector dot product, etc.

[0003] Traditional implementations of nested reduction operations require independent computation for each operation. This necessitates loading the data needed for each operation, and subsequent operations must wait for the result of the previous one, resulting in low computational efficiency. Therefore, adjusting the implementation of nested reduction operations to reduce the overhead of loading data or to enable faster computation to improve efficiency is a pressing technical challenge. Summary of the Invention

[0004] To overcome the problems existing in the related technologies, this manual provides code adjustment methods, data processing methods, equipment, systems, products and media.

[0005] According to a first aspect of the embodiments of this specification, a code adjustment method is provided, the method comprising:

[0006] Obtain implementation code for implementing multiple reduction operations; wherein, the multiple reduction operations include: a preceding reduction operation and a following reduction operation that are nested consecutively, wherein the preceding reduction operation is used to traverse the data in the dataset corresponding to the reduction operation and calculate the global reduction result on the traversed data, and the following reduction operation is used to traverse the data in the dataset corresponding to the reduction operation and calculate the reduction result based on the global reduction result and the traversed data;

[0007] The computational expression for the reduction operation is determined based on the implementation code.

[0008] The computational expression of the reduction operation is transformed to obtain a recursive expression; wherein, the computational expression of the previous reduction operation is transformed into: the previous recursive expression that calculates the local reduction result of the current step step by step starting from the first data in the dataset of the reduction operation; the computational expression of the next reduction operation is transformed into: the next recursive expression that calculates the reduction result of the current step step by step, depending on the local reduction result of the previous recursive expression in the current step, starting from the first data in the dataset of the reduction operation;

[0009] Based on the recursive expression obtained from the transformation, the implementation code is adjusted into target code; wherein, the target code is used to implement: synchronously reading the data in the dataset of the previous reduction operation and the data in the dataset of the subsequent reduction operation, and gradually using the previous recursive expression and the subsequent recursive expression to calculate the reduction result on the read data.

[0010] According to a second aspect of the embodiments of this specification, a data processing method based on reduction operations is provided, the method comprising:

[0011] Load the dataset to be computed;

[0012] The dataset is read, and the target code obtained by the method described in the first aspect is executed to obtain the calculation result; wherein, the dataset includes: the dataset corresponding to the reduction operation in the target code.

[0013] According to a third aspect of the embodiments of this specification, a data processing system is provided, including a code adjustment device for performing the method embodiments of the first aspect and a data processing device for performing the method embodiments of the first aspect.

[0014] According to a fourth aspect of the embodiments of this specification, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the method embodiments described in the first or second aspect above.

[0015] According to a fifth aspect of the embodiments of this specification, a computer-readable storage medium is provided, on which a computer program is stored, which, when executed by a processor, implements the steps of the method embodiments described in the first or second aspect above.

[0016] According to a sixth aspect of the embodiments of this specification, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps of the method embodiments described in the first or second aspect above.

[0017] The technical solutions provided in the embodiments of this specification may include the following beneficial effects:

[0018] In this embodiment of the specification, for the implementation code used to implement multiple reduction operations, this embodiment can obtain the calculation expression of the reduction operation. For the calculation expression of the previous reduction operation, it is converted into a previous recursive expression that calculates the local reduction result of the current step step by step from the first data in the dataset of the reduction operation. Therefore, the recursive expression obtained by the global reduction transformation of the reduction operation can make the intermediate results of each step available in real time. Thus, the calculation expression of the next reduction operation is converted into: calculating the current step by step from the first data in the dataset of the reduction operation, depending on the local reduction result of the previous recursive expression in the current step. The subsequent recursive expression of the reduction result of the previous step allows the subsequent reduction operation to directly depend on the local reduction result of the current step, rather than the final global reduction result. Therefore, the converted target code can synchronously read data from the datasets of the previous and subsequent reduction operations, and progressively use the previous and subsequent recursive expressions to calculate the reduction result from the read data. Thus, this embodiment can eliminate the dependency on the global reduction result, and the subsequent reduction operation can directly use the current result of the previous reduction operation in each loop step, thereby achieving single-pass computation for multiple reduction operations. Compared to two-pass computations, single-pass computation reduces two data loading operations to one. The original two-pass computations required sequential execution due to dependencies, while single-pass computation can be executed in parallel, resulting in higher computational efficiency.

[0019] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this specification. Attached Figure Description

[0020] Figure 1 This is a flowchart illustrating a code adjustment method according to an exemplary embodiment of this specification.

[0021] Figure 2 This is a flowchart illustrating a data processing method based on a specification operation according to an exemplary embodiment of this specification.

[0022] Figure 3 This specification is a hardware structure diagram of a computer device containing a code adjustment device / protocol-based data processing device according to an exemplary embodiment.

[0023] Figure 4 This is a block diagram illustrating a code adjustment device according to an exemplary embodiment of this specification.

[0024] Figure 5 This is a block diagram illustrating a protocol-based data processing apparatus according to an exemplary embodiment. Detailed Implementation

[0025] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this specification. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this specification as detailed in the appended claims.

[0026] The terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to be limiting of this specification. The singular forms “a,” “the,” and “the” as used in this specification and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any and all possible combinations of one or more of the associated listed items.

[0027] It should be understood that although the terms first, second, third, etc., may be used in this specification to describe various information, this information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, without departing from the scope of this specification, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to determination."

[0028] The user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this manual are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation portals are provided for users to choose to authorize or refuse.

[0029] In recent years, deep learning models have become a core technology driving the development of various applications in artificial intelligence research. These models process complex data through multi-layered neural networks, demonstrating outstanding performance. However, as the scale of these models continues to expand, the inference process faces significant computational challenges. To address this, operator fusion techniques have emerged. This method combines multiple operators into a single computational operation to reduce memory access and improve computational efficiency, thereby accelerating the inference process of deep learning models. Algorithm fusion in the field of deep learning models typically involves operator fusion of operations such as convolution, matrix multiplication, and activation functions.

[0030] Research has revealed that some deep learning models contain a large number of consecutively nested reduction operations. Reduction operations are a type of parallel algorithm that, for N input data, uses a single associative binary operator. Produce one result. Such operations include, but are not limited to: finding the minimum value, finding the maximum value, summing, sum of squares, logical AND / OR, vector dot product, etc., but this embodiment does not limit these operations.

[0031] In traditional implementations, the computation of consecutively nested reduction operations requires independent computation for each operation. Specifically, the data to be computed for the first reduction operation is loaded from memory into the internal cache of the computing unit (e.g., the processor), and then the first reduction operation is executed. The result of the first reduction operation is written back to memory. The data to be computed for the second reduction operation and its result are then loaded into the internal cache of the computing unit (e.g., the processor), and the second reduction operation is executed. This process is repeated until all reduction operations are completed. It can be seen that traditional implementations require two data loading operations. Furthermore, after the data to be computed for the first reduction operation is loaded and computation begins, the computing unit must wait for the data to be loaded for the second reduction operation before starting computation. The two reduction operations must be executed serially and cannot be processed in parallel, resulting in poor computational efficiency.

[0032] Based on this, the embodiments of this specification utilize the concept of operator fusion technology to integrate multiple reduction operations into a single traversal calculation, thereby reducing data loading overhead and improving computational efficiency when performing multiple reduction operations. The embodiments of this specification will now be described in detail.

[0033] like Figure 1 As shown, Figure 1 This is a flowchart illustrating a code adjustment method according to an exemplary embodiment, comprising the following steps:

[0034] In step 102, the implementation code for implementing multiple reduction operations is obtained.

[0035] The multiple reduction operations include: a preceding reduction operation and a following reduction operation that are nested consecutively. The preceding reduction operation is used to traverse the data in the dataset corresponding to the reduction operation and calculate the global reduction result on the traversed data. The following reduction operation is used to traverse the data in the dataset corresponding to the reduction operation and calculate the reduction result based on the global reduction result and the traversed data.

[0036] In step 104, the computational expression for the reduction operation is determined based on the implementation code.

[0037] In step 106, the computational expression of the reduction operation is transformed to obtain the recursive expression.

[0038] The calculation expression of the previous reduction operation is converted into: the previous recursive expression for calculating the local reduction result of the current step by step, starting from the first data in the dataset of the reduction operation; the calculation expression of the next reduction operation is converted into: the next recursive expression for calculating the reduction result of the current step by gradually relying on the local reduction result of the previous recursive expression in the current step, starting from the first data in the dataset of the reduction operation.

[0039] In step 108, based on the recursive expression obtained from the transformation, the implementation code is adjusted to target code. The target code is used to: synchronously read data from the dataset of the previous reduction operation and the dataset of the subsequent reduction operation, and progressively use the previous and subsequent recursive expressions to calculate the reduction result from the read data.

[0040] As an example, the code adjustment method in this embodiment can be applied to either the client or the server. The server can be a program installed on a background device to provide services to the user. For example, the background device can be a server, which can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks, and big data and artificial intelligence platforms. The client can be a program installed on the user's device to provide services to the user. Clients include, but are not limited to, applications, web pages, mini-programs, plugins, or components. User devices include, but are not limited to, smartphones, personal digital assistants, tablets, personal computers, laptops, virtual reality terminal devices, and augmented reality terminal devices.

[0041] As an example, the execution entity that performs the code adjustment method of this embodiment can be any program used to convert the implementation code of each of the multiple reduction operations into target code. Alternatively, it can be a compiler that can identify the implementation code of the multiple reduction operations contained in the input source code and convert it into target code.

[0042] In step 102, the implementation code can be obtained by the execution entity of this embodiment through various methods. For example, an input interface for the implementation code can be provided to the user, and the implementation code provided by the user can be obtained through this input interface. The implementation code may contain multiple (two or more) reduction operations. The multiple reduction operations include consecutively nested preceding and subsequent reduction operations. Consecutive nesting means that the input of the subsequent reduction operation depends on the reduction result of the preceding reduction operation.

[0043] In this system, the preceding reduction operation iterates through the data in the corresponding dataset to calculate the global reduction result. The subsequent reduction operation iterates through the data in the corresponding dataset and calculates its own reduction result based on the global reduction result. Therefore, the implementation code of the preceding reduction operation is typically a loop to iterate through each data point in the dataset and obtain the global reduction result. Because the subsequent reduction operation depends on the global reduction result of the preceding operation, it can only begin after the calculation of the preceding reduction operation is completed.

[0044] For example, the various mathematical symbols used in this embodiment are shown in Table 1 below:

[0045] Table 1 Symbol Table

[0046]

[0047] Suppose we have the following two consecutive nested reduction operations:

[0048]

[0049] Formula (1) is the computational expression for the previous reduction operation, which means: for the dataset Dataset1, from i=1 to N, traverse each data Dataset1(i) and perform calculations according to the function f to achieve a certain reduction operation. It can be seen that formula (1) requires one loop calculation. m is the global reduction result of the previous reduction operation.

[0050] Formula (2) is the computational expression for the latter reduction operation, where h(m) represents a certain calculation performed on m; g(i) represents the dataset; Formula (2) means that, in combination with h(m), for the dataset Dataset2, each Dataset2(i) is traversed from i=1 to N, and the calculation is performed according to the function g to achieve a certain reduction operation. Formula (2) also requires one loop calculation.

[0051] It is not difficult to see that the result of the computation expression d in the latter reduction operation depends on the reduction expression m. To solve for d, two reduction operations are required, which means two loops are required, thus doubling the number of memory accesses and computations.

[0052] Assuming it's implemented using a C-like language, the pseudocode is as follows:

[0053]

[0054] Here, "for i←1,N do" means a loop starting from i=1 to N, where the calculation performed for each i is the expression following "do". m0 is the initial value.

[0055] As can be seen, the first loop structure requires iterating through all the data to obtain the global reduction result. For example:

[0056] If it is an additive constraint, then m = f(1) + f(2) + ... + f(N);

[0057] If it is a maximum value reduction, then m = max(f(1),f(2),…,f(N)).

[0058] m is the result of the global reduction, which can only be obtained after all data has been traversed.

[0059] Since the second reduction operation depends on the global reduction result m, the computation must wait until the first loop ends before it can begin.

[0060] This embodiment can perform mathematical transformations on formulas (1) and (2), so that the reduction operation can be completed in only one loop to solve for the value of the reduction expression d.

[0061] Therefore, in this embodiment, the global reduction of the reduction operation is rewritten as a recursive expression m. i This makes the intermediate result m at each step... i Available in real time, m i That is, the local reduction result at each step. m i In the sequence, i starts from 1 and continues to N, m N This refers to the global reduction result of the reduction operation. Subsequent reduction operations can directly depend on the local reduction result m of the current step. i rather than the final global reduction result m N By eliminating the dependency on the global reduction result, the subsequent reduction operation can directly use the local reduction result of the previous reduction operation in each loop step, thus achieving single-pass computation for multiple reduction operations. Compared to two-pass computations, single-pass computation reduces two data loading operations to one. The original two-pass computations needed to be executed serially due to dependencies, while single-pass computation can be executed in parallel, resulting in higher computational efficiency.

[0062] In step 104, the computational expressions for the reduction operations can be obtained from the implementation code. This can be achieved by parsing the implementation code of each reduction operation and extracting its computational expression. For example, this can be implemented through static code analysis, which may include syntax parsing—analyzing the source code using an abstract syntax tree or intermediate representation to identify loop structures, reduction operators (such as sum, max, prod), and their dependencies. Symbol extraction can also be performed, such as extracting the representation of the dataset corresponding to the reduction operation (such as arrays f and g), operators (such as +, ×, max), and output variables (such as m and d). Dependency analysis can also be performed, such as determining the dependency path of a subsequent reduction operation on the result of a previous reduction operation (e.g., whether variable m is directly referenced by subsequent operations).

[0063] In step 106, the computational expressions of the reduction operations are transformed to obtain recursive expressions. Specifically, the computational expression of the previous reduction operation is transformed into a previous recursive expression that calculates the local reduction result of the current step by step, starting from the first data in the dataset of the reduction operation; the computational expression of the next reduction operation is transformed into a next recursive expression that calculates the reduction result of the current step by step, starting from the first data in the dataset of the reduction operation, and relying on the local reduction result of the previous recursive expression in the current step.

[0064] The number of consecutively nested reduction operations can be two or more:

[0065] ① It can include consecutively nested reduction operations L1 and L2, where reduction operation L2 depends on the reduction result of reduction operation L1. In this case, the preceding reduction operation in step 106 can be reduction operation L1, and the following reduction operation can be reduction operation L2.

[0066] ② It can include t reduction operations in succession, such as L1, L2, ..., Lt, where t > 2; and each reduction operation depends on the result of the previous reduction operation.

[0067] In this case, any pair or more adjacent pairs of reduction operations among these t reduction operations can be merged using the method of this embodiment; for example, the calculation expressions of an adjacent pair of reduction operations L1 and L2 are transformed using step 106, and the calculation expressions of an adjacent pair of reduction operations L5 and L6 are transformed using step 106.

[0068] Alternatively, two or more reduction operations can be merged using the method of this embodiment. For example, the calculation expressions of reduction operations L1, L2, and L3 can be transformed using step 106; or all reduction operations can be transformed using step 106. In practical applications, the method can be flexibly selected as needed, and this embodiment does not limit this.

[0069] It can be seen that in the case of multiple consecutive nested reduction operations, whether the solution of this embodiment is used on only one pair of reduction operations or on multiple reduction operations, it necessarily includes the transformation of the calculation expression of the previous reduction operation and the transformation of the calculation expression of the subsequent reduction operation. As long as the solution of this embodiment is used on one pair of reduction operations, the pair of reduction operations that originally required two traversals can be transformed into a single traversal calculation, which can reduce the number of data loadings and improve the calculation efficiency.

[0070] In step 108, since the previous recursive expression of the previous reduction operation and the next recursive expression of the next reduction operation were obtained in step 106, the implementation code for implementing multiple reduction operations can be converted into target code based on these two expressions. This allows the target code to: synchronously read the data in the dataset of the previous reduction operation and the dataset of the next reduction operation, and gradually use the previous and next recursive expressions to calculate the reduction result on the read data.

[0071] As an example, in actual implementation, code that implements a loop structure can be generated. In this loop structure, two recursive expressions are combined in a single loop. The data of the two reduction operations can be read sequentially step by step, and the previous and next recursive expressions can be calculated based on the data read in the current step, until the last data in the dataset is read, and the final reduction result is obtained.

[0072] As an example, we first transform formulas (1) and (2) as follows:

[0073]

[0074] d = d N (5)

[0075] Formula (3) means that partial results from j=1 to the current i are calculated step by step; that is, the local reduction result up to the current i is calculated at each step i.

[0076] For example:

[0077]

[0078] Here, ⊙ represents a reduction operator (such as addition, multiplication, etc.). The purpose of this conversion is to rewrite the calculation expression that yields the global reduction result from the previous reduction operation into a recursive expression, providing a basis for subsequent changes.

[0079] In traditional implementations, subsequent reduction operations depend on the global reduction result m, and computation can only begin after all data in the dataset of the previous reduction operation has been traversed.

[0080] In this embodiment, the global reduction is rewritten as a recursive form m. i This makes the local reduction results of each step available in real time. Subsequent operations can directly depend on the m of the current step. i rather than the final m N .

[0081] By using a recursive approach, subsequent reduction operations can utilize the local reduction result of the current step in real time, thus eliminating the need to wait for the global reduction result. For example:

[0082] When not overridden: The second reduction operation can only begin after the first reduction operation has completely finished.

[0083] Rewritten: The second reduction operation directly uses the current result of the first reduction operation in each loop step, thus achieving single-pass computation.

[0084] In formula (4), the expression for d in formula (2) is rewritten in recursive form, which means: for each step i, the result of the calculation up to the current i is calculated, h(m i It also depends on the current m i (rather than the final global reduction result m) N ).

[0085] In some examples, determining the computational expression for the reduction operation based on the implementation code includes:

[0086] Based on the preset algebraic group structure and distributive law, the computational expression of the reduction operation, which includes operators that satisfy the algebraic group structure and distributive law, is determined from the implementation code.

[0087] In other words, in this embodiment, the operators included in the computational expression of the reduction operation can satisfy the algebraic group structure and the distributive law. In the algebraic group structure, operators must satisfy the associative law, have an identity element, and an inverse element.

[0088] In this context, the identity element refers to a neutral element, such as 0 in addition (a + 0 = a) or 1 in multiplication (a · 1 = a). The inverse element is an element b for every element a such that a · b = the identity element. For example, the inverse of a in addition is -a, and the inverse of a non-zero number in multiplication is 1 / a. Specifically:

[0089] (1) Algebraic group structure, i.e., algebraic structure This forms a group on the set of real numbers, that is:

[0090] (a) Operations satisfy the associative law:

[0091] (a·b)·c=a·(b·c) (6)

[0092] (b) There exists a unit element:

[0093] e·a=a·e (7)

[0094] (c) All elements have inverses:

[0095] a·a -1 =a -1 ·a=e (8)

[0096] (2) The binary operators + and · satisfy the distributive law:

[0097] (a+b)·c=(a·c)+(b·c) (9)

[0098] (3) The binary operators × and · have the same associativity:

[0099] (a×b)·c=a×(b·c) (10)

[0100] Therefore, we can extract the last term from formula (4) to obtain:

[0101]

[0102] Using conditions (6)(7)(8)(10), transform (11) to obtain:

[0103]

[0104] Using condition (9), we can extract the invariant term from the ∑ in equation (12) to obtain:

[0105]

[0106] The result of ∑ in formula (13) is d. i-1 The replacement yields:

[0107] d i =d i-1 ·h(m i-1 ) -1 ·h(m i )+g(i)×h(m i (14)

[0108] In formula (14), di and d i-1 The recurrence relation no longer contains m N Then it can be calculated together with formula (3). That is, formula (14) is the recursive expression obtained by transforming formula (2).

[0109] Thus, the reduction expressions m and d can be calculated with only one reduction operation, as shown in the following pseudocode example:

[0110]

[0111] Here, "for i←1, N do" represents a loop starting from i=1 to N, where the calculation performed for each i is the recursive expression m following "do". i and d i .

[0112] As can be seen in the implementation code here, in each step, data f(i) from the dataset f of the previous reduction operation and data g(i) from the dataset g of the next reduction operation can be read to perform calculations; this embodiment only requires one loop to obtain the final result of the reduction expression d. Through algebraic transformation, the two expressions in the two loops are converted into recursive expressions, and the reduction operation that depends on the global reduction result is decomposed into local dependencies, realizing single-loop calculation.

[0113] In some examples, the method can be applied to a compiler; obtaining the implementation code that implements the multiple reduction operations may include:

[0114] Obtain the source code of the program to be compiled, and extract the implementation code for implementing multiple reduction operations from the source code.

[0115] The method of this embodiment can be applied to a compiler, for example, as a function in the compiler to adjust the implementation code of multiple reduction operations. This function can use the solution of this embodiment to adjust the implementation code of multiple reduction operations to the target code of this embodiment.

[0116] As an example, when applied to a compiler, the compiler can obtain the source code of the program to be compiled. In this embodiment, the implementation code for implementing multiple reduction operations can be obtained from the source code.

[0117] For example, the implementation code for multiple reduction operations can be obtained from the program source code through automatic identification. This can be achieved, for instance, using the aforementioned static code analysis; in other cases, the implementation code can be manually selected, such as obtaining the implementation code for multiple manually selected reduction operations.

[0118] Based on this, this embodiment is applied to a compiler, which can automatically adjust the implementation code of multiple reduction operations in the program source code, so that multiple reduction operations can achieve lower data access overhead and higher computational efficiency during implementation.

[0119] In some instances, the number of consecutively nested reduction operations is greater than two.

[0120] The transformation of the computational expression of the reduction operation to obtain the recursive expression may include:

[0121] The computational expression of each of the nested reduction operations is transformed; wherein, the computational expression of the first reduction operation in the plurality of reduction operations is transformed into a recursive expression that calculates the local reduction result of the current step step by step, starting from the first data in the dataset of the reduction operation; the computational expressions of each reduction operation after the first reduction operation in the plurality of reduction operations are transformed into a recursive expression that calculates the reduction result of the current step step by step, starting from the first data in the dataset of the reduction operation, depending on the local reduction result of the recursive expression of the previous reduction operation in the current step;

[0122] The target code is used to: synchronously read the dataset of each of the continuously nested reduction operations, and gradually use the various recursive expressions obtained from the transformation to calculate the reduction result.

[0123] This embodiment provides a transformation example for calculating expressions for multiple consecutive nested reduction operations; for example, assuming there are t consecutive nested reduction operations L1, L2, ..., Lt, in this embodiment:

[0124] For the first reduction operation L1, its computation expression can be transformed into: the recursive expression R1, which calculates the local reduction result of the current step step by step, starting from the first data in the dataset of the reduction operation L1;

[0125] For the second reduction operation L2, its computation expression can be transformed into: starting from the first data in the dataset of the reduction operation L2, gradually relying on the local reduction result of the recursive expression of the previous reduction operation L1 of the reduction operation L2 in the current step, to calculate the recursive expression R2 of the reduction result in the current step.

[0126] For the third reduction operation L3, its computation expression can be transformed into: starting from the first data in the dataset of the reduction operation L3, gradually relying on the local reduction result of the recursive expression of the previous reduction operation L2 of the reduction operation L3 in the current step, to calculate the recursive expression R3 of the reduction result of the current step.

[0127] And so on, until the last reduction operation Lt, whose computational expression can be transformed into: starting from the first data in the dataset of the reduction operation Lt, gradually relying on the local reduction result of the recursive expression of the previous reduction operation Lt-1 of the reduction operation Lt in the current step, to calculate the recursive expression Rt of the reduction result of the current step.

[0128] Thus, based on the recursive expressions R1 to Rt, the implementation code can be converted into target code. The target code is used to implement: synchronously reading the data in the dataset of the previous reduction operation and the data in the dataset of the next reduction operation, and gradually using the previous and next recursive expressions to calculate the reduction result on the read data.

[0129] In practical applications, the method of this embodiment can be applied to any computational scenario involving multiple reduction operations. In some examples, the reduction operations include: reduction operations in deep learning models, such as the Softmax operator and Attention operator in large-scale language models; and the calculation of variance and standard deviation in numerical computation. Based on this, the embodiments of this specification can improve the inference efficiency of deep learning models.

[0130] The following examples illustrate this further using the Softmax and Attention operators in deep learning models. First, we derive the fusion calculation formula for the Softmax operator, and from there, we can derive the calculation formula for Attention.

[0131] The Softmax (safe version) operator is m and s in the following formula; the Attention operator is y in the following formula. i :

[0132]

[0133] Each of m and s requires a reduction operation.

[0134] Rewrite the expressions m and s into the form of formulas (3) and (4):

[0135]

[0136] Next, following the principles of the aforementioned embodiments:

[0137] (1) Use Replace ∏ i

[0138] (2) Replace ∑ with the summation operation of real numbers j

[0139] (3) Replace × and · with real number multiplication operations.

[0140] (4) Take f(j) = x j

[0141] (5) Take

[0142] (6) Take Then s i The expression can be directly obtained from equation (14):

[0143]

[0144] It has been verified that the converted s i The calculation result is the same as the original calculation result.

[0145] The standard formula for calculating Attention is O = Softmax(QK). T V, pseudocode as follows:

[0146]

[0147] From the pseudocode, we can obtain o i The expression:

[0148]

[0149] o i Rewritten in the form of formula (4), we get:

[0150]

[0151] o i =o ′ N

[0152] The transformation yields:

[0153]

[0154] Next, following the principles of the aforementioned embodiments:

[0155] (1) Replace ∑ with the summation operation of real numbers. j

[0156] (2) Replace × and · with real number multiplication operations.

[0157] (3) Take

[0158] (4) Take

[0159] So o i ′ The expression can be directly obtained from equation (14):

[0160]

[0161] Thus, the expression m i d i and o i Both can be solved using a single reduction calculation. It has been verified that the derived o... i The calculation expression is consistent with the standard Attention calculation formula.

[0162] The effectiveness and correctness of the proposed solution are demonstrated above through the practical derivation of the Softmax and Attention operators in the large speech model.

[0163] For the two reduction operations that satisfy the aforementioned algebraic group structure and distributive law conditions, their expressions only need to be rewritten as equations (3) and (4), and substituted into the corresponding functional expressions. The computational expression for the optimized fusion-based first-order reduction can then be directly derived from equation (14). Therefore, the method in this embodiment is universal. For multiple nested reduction calculations that satisfy the conditions, this method can be used to convert them into implementation code that can be calculated through a single traversal. This embodiment utilizes the advantages of operator fusion to effectively improve computational efficiency. When applied to deep learning models, it can effectively improve the inference performance of deep learning models.

[0164] like Figure 2 The diagram shown is a flowchart illustrating a data processing method based on a specification operation according to an exemplary embodiment of this specification. The method may include:

[0165] Step 202: Load the dataset to be computed;

[0166] Step 204: Read the dataset and execute the target code obtained from the aforementioned code adjustment method embodiment to obtain the calculation result; wherein, the dataset includes: the dataset corresponding to the reduction operation in the target code.

[0167] In this embodiment, it can be applied to any data processing device, which can execute the target code of this embodiment to perform calculations of multiple consecutive nested reduction operations through a single traversal calculation.

[0168] As an example, the execution entity in this embodiment can be a processor, such as a central processing unit (CPU) or a graphics processing unit (GPU).

[0169] The process of loading the dataset in step 202 can involve loading the dataset stored in memory into the processor's cache. In this embodiment, multiple nested reduction operations are involved, and the loaded dataset can be the dataset corresponding to each of these nested reduction operations. It can be seen that in traditional implementations, since each reduction operation is independently computed, the datasets for each reduction operation need to be loaded multiple times, requiring multiple accesses to memory to read data from memory into the processor. However, this embodiment can load the datasets for multiple reduction operations at once.

[0170] In step 204, the execution of the target code can realize a single traversal calculation of multiple reduction operations. For example, in each step from i=1 to N, the i-th data of the dataset corresponding to each reduction operation is read in each step. If there are t reduction operations, a total of t data are read. Then, the calculation can be performed using each recursive expression in a loop.

[0171] As can be seen, this embodiment can eliminate the dependency on the global reduction result. The subsequent reduction operation can directly use the local reduction result of the previous reduction operation in the current step in each loop step, thereby achieving single-pass computation for multiple reduction operations. Compared with two-pass computation, single-pass computation reduces two data loadings to one. The original two-pass computation needed to be executed serially due to the dependency relationship, while single-pass computation can also be executed in parallel as needed, thus achieving higher computational efficiency.

[0172] As can be seen from the above embodiments, in traditional methods, consecutively nested reduction operations require multiple data loadings from memory to computational units (such as CPU / GPU caches), causing memory bandwidth to become a bottleneck. This solution, by merging multiple reduction operations into a single traversal computation, requires only one data load, reducing the number of memory accesses and significantly lowering data transfer overhead. For example, by combining two independent data loads into a single consecutive load, the scheduling overhead and latency of memory access are reduced, significantly optimizing the actual utilization of memory bandwidth.

[0173] Traditional implementations rely on the global reduction result of the previous reduction operation, requiring the sequential execution of multiple reduction operations. This solution transforms the operation into a recursive expression, allowing subsequent operations to directly depend on the local reduction result of the current step, rather than waiting for the global result. This enables multiple reduction operations to be executed synchronously in a single traversal, fully utilizing the parallel capabilities of computing units (such as multithreading or GPU parallel architectures), reducing idle cycles in the instruction pipeline, and improving hardware resource utilization.

[0174] By eliminating serial dependencies and reducing data loading, this solution reduces the time complexity of multiple reduction operations from O(kN) (where k is the number of operations and N is the amount of data) to O(N). For example, the computation time for two nested reduction operations is reduced from 2N steps to N steps. This optimization is particularly important in scenarios with high real-time requirements (such as online inference and streaming data processing), and can significantly improve system throughput.

[0175] Corresponding to the embodiments of the aforementioned code adjustment method / restriction-based data processing method, this specification also provides embodiments of the code adjustment apparatus / restriction-based data processing apparatus and the computer equipment on which it is applied.

[0176] The embodiments of the code adjustment device / protocol-based data processing device described in this specification can be applied to computer devices, such as servers or terminal devices. The device embodiments can be implemented in software, hardware, or a combination of both. Taking software implementation as an example, as a logical device, it is formed by its processor reading the corresponding computer program instructions from non-volatile memory into memory for execution. From a hardware perspective, such as... Figure 3 The diagram shown is a hardware structure diagram of a computer device containing the code adjustment device / protocol-based data processing device described in this specification, except... Figure 3 In addition to the processor 310, network interface 320, memory 330, and non-volatile memory 340 shown, the computer device in which the code adjustment device / protocol-based data processing device is located in the embodiment may also include other hardware depending on the actual function of the computer device, which will not be described in detail here.

[0177] like Figure 4 As shown, Figure 4 This is a block diagram illustrating a code adjustment apparatus according to an exemplary embodiment of this specification, the apparatus comprising:

[0178] The acquisition module 41 is used to: acquire implementation code for implementing multiple reduction operations; wherein, the multiple reduction operations include: a consecutively nested previous reduction operation and a subsequent reduction operation, wherein the previous reduction operation is used to traverse the data in the dataset corresponding to the reduction operation and calculate the global reduction result on the traversed data, and the subsequent reduction operation is used to traverse the data in the dataset corresponding to the reduction operation and calculate the reduction result based on the global reduction result and the traversed data;

[0179] The determining module 42 is used to: determine the computational expression of the reduction operation based on the implementation code;

[0180] The conversion module 43 is used to: convert the calculation expression of the reduction operation to obtain a recursive expression; wherein, the calculation expression of the previous reduction operation is converted into: the previous recursive expression that calculates the local reduction result of the current step step by step starting from the first data in the dataset of the reduction operation; the calculation expression of the next reduction operation is converted into: the next recursive expression that calculates the reduction result of the current step step by step depending on the local reduction result of the previous recursive expression in the current step, starting from the first data in the dataset of the reduction operation;

[0181] The adjustment module 44 is used to: adjust the implementation code into target code based on the recursive expression obtained by the transformation; wherein, the target code is used to: synchronously read the data in the dataset of the previous reduction operation and the data in the dataset of the next reduction operation, and gradually use the previous recursive expression and the next recursive expression to calculate the reduction result on the read data.

[0182] In some examples, the device is applied to a compiler; the acquisition module 41 acquires implementation code that implements multiple reduction operations, including:

[0183] Obtain the source code of the program to be compiled, and extract the implementation code for implementing multiple reduction operations from the source code.

[0184] In some instances, the number of consecutively nested reduction operations is greater than two.

[0185] In the conversion module 43, the calculation expression of the reduction operation is converted to obtain a recursive expression, including:

[0186] The computational expression of each of the nested reduction operations is transformed; wherein, the computational expression of the first reduction operation in the plurality of reduction operations is transformed into a recursive expression that calculates the local reduction result of the current step step by step, starting from the first data in the dataset of the reduction operation; the computational expressions of each reduction operation after the first reduction operation in the plurality of reduction operations are transformed into a recursive expression that calculates the reduction result of the current step step by step, starting from the first data in the dataset of the reduction operation, depending on the local reduction result of the recursive expression of the previous reduction operation in the current step;

[0187] The target code is used to: synchronously read data from the dataset of each of the nested reduction operations, and gradually use the various recursive expressions obtained from the transformation to calculate the reduction result from the read data.

[0188] In some examples, the determining module 42 determines the computational expression of the reduction operation based on the implementation code, including:

[0189] Based on the preset algebraic group structure and distributive law, the computational expression of the reduction operation, which includes operators that satisfy the algebraic group structure and distributive law, is determined from the implementation code.

[0190] In some examples, the reduction operation includes reduction operations in deep learning models.

[0191] like Figure 5 As shown, Figure 5 This is a block diagram illustrating a protocol-based data processing apparatus according to an exemplary embodiment, the apparatus comprising:

[0192] Loading module 51 is used to: load the dataset to be computed;

[0193] The execution module 52 is configured to: read the dataset and execute the target code obtained from the aforementioned code adjustment method embodiment to obtain the calculation result; wherein, the dataset includes: the dataset corresponding to the reduction operation in the target code.

[0194] The implementation process of the functions and roles of each module in the above-mentioned code adjustment device / reduction-based data processing device is detailed in the implementation process of the corresponding steps in the above-mentioned code adjustment method / reduction-based data processing method, and will not be repeated here.

[0195] Accordingly, embodiments of this specification also provide a data processing system, including a code adjustment device for performing code adjustment method embodiments and a data processing device for performing data processing method embodiments based on specification operations.

[0196] Accordingly, embodiments of this specification also provide a computer program product, including a computer program that, when executed by a processor, implements the steps of the aforementioned code adjustment method / data processing method embodiment based on specification operation.

[0197] Accordingly, embodiments of this specification also provide a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, it implements the steps of the code adjustment method / data processing method embodiment based on specification operation.

[0198] Accordingly, embodiments of this specification also provide a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the code adjustment method / data processing method embodiment based on reduction operation.

[0199] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The device embodiments described above are merely illustrative. The modules described as separate components may or may not be physically separate, and the components shown as modules may or may not be physical modules, that is, they may be located in one place or distributed across multiple network modules. Some or all of the modules can be selected to achieve the purpose of the solution in this specification according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0200] The above embodiments can be applied to one or more computer devices. The computer device is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions. The hardware of the computer device includes, but is not limited to, microprocessors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), digital signal processors (DSPs), embedded devices, etc.

[0201] The computer device can be any electronic product that can interact with the user, such as a personal computer, tablet computer, smartphone, personal digital assistant (PDA), game console, interactive network television (IPTV), smart wearable device, etc.

[0202] The computer equipment may also include network equipment and / or user equipment. The network equipment includes, but is not limited to, a single network server, a server group consisting of multiple network servers, or a cloud based on cloud computing consisting of a large number of hosts or network servers.

[0203] The network in which the computer device is located includes, but is not limited to, the Internet, wide area network, metropolitan area network, local area network, and virtual private network (VPN).

[0204] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0205] The steps of the various methods described above are only for clarity. In practice, they can be combined into one step or some steps can be split into multiple steps. As long as they include the same logical relationship, they are all within the scope of protection of this patent. Adding insignificant modifications or introducing insignificant designs to the algorithm or process, but without changing the core design of the algorithm and process, are also within the scope of protection of this application.

[0206] While this specification contains numerous specific implementation details, these should not be construed as limiting the scope of any invention or the scope of the claims, but rather are primarily intended to describe features of specific embodiments of a particular invention. Certain features described in the various embodiments herein may also be implemented in combination in a single embodiment. Conversely, various features described in a single embodiment may also be implemented separately in various embodiments or in any suitable sub-combination. Furthermore, while features may function in certain combinations as described above and even initially claimed in this way, one or more features from a claimed combination may be removed from that combination in some cases, and a claimed combination may refer to a sub-combination or a variation thereof.

[0207] The terms "specific example" or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with the embodiments or examples, which are included in at least one embodiment or example of this specification. In this specification, illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0208] Other embodiments of this specification will readily occur to those skilled in the art upon consideration of the specification and practice of the invention claimed herein. This specification is intended to cover any variations, uses, or adaptations that follow the general principles of this specification and include common knowledge or customary techniques in the art not claimed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this specification are indicated by the following claims.

[0209] It should be understood that this specification is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this specification is limited only by the appended claims.

[0210] The above description is merely a preferred embodiment of this specification and is not intended to limit this specification. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this specification should be included within the scope of protection of this specification.

Claims

1. A code adjustment method, the method comprising: Obtain implementation code for implementing multiple reduction operations; wherein, the multiple reduction operations include: a preceding reduction operation and a following reduction operation that are nested consecutively, wherein the preceding reduction operation is used to traverse the data in the dataset corresponding to the reduction operation and calculate the global reduction result on the traversed data, and the following reduction operation is used to traverse the data in the dataset corresponding to the reduction operation and calculate the reduction result based on the global reduction result and the traversed data; The computational expression for the reduction operation is determined based on the implementation code. The computational expression of the reduction operation is transformed to obtain a recursive expression; wherein, the computational expression of the previous reduction operation is transformed into: the previous recursive expression that calculates the local reduction result of the current step step by step starting from the first data in the dataset of the reduction operation; the computational expression of the next reduction operation is transformed into: the next recursive expression that calculates the reduction result of the current step step by step, depending on the local reduction result of the previous recursive expression in the current step, starting from the first data in the dataset of the reduction operation; Based on the recursive expression obtained from the transformation, the implementation code is adjusted into target code; wherein, the target code is used to implement: synchronously reading the data in the dataset of the previous reduction operation and the data in the dataset of the subsequent reduction operation, and gradually using the previous recursive expression and the subsequent recursive expression to calculate the reduction result on the read data.

2. The method according to claim 1, wherein the method is applied to a compiler; the step of obtaining the implementation code that implements the multiple reduction operations includes: Obtain the source code of the program to be compiled, and extract the implementation code for implementing multiple reduction operations from the source code.

3. The method according to claim 1, wherein the number of consecutively nested reduction operations in the plurality of reduction operations is greater than two; The transformation of the computational expression of the reduction operation to obtain the recursive expression includes: The computational expression of each of the nested reduction operations is transformed; wherein, the computational expression of the first reduction operation in the plurality of reduction operations is transformed into a recursive expression that calculates the local reduction result of the current step step by step, starting from the first data in the dataset of the reduction operation; the computational expressions of each reduction operation after the first reduction operation in the plurality of reduction operations are transformed into a recursive expression that calculates the reduction result of the current step step by step, starting from the first data in the dataset of the reduction operation, depending on the local reduction result of the recursive expression of the previous reduction operation in the current step; The target code is used to: synchronously read data from the dataset of each of the nested reduction operations, and gradually use the various recursive expressions obtained from the transformation to calculate the reduction result from the read data.

4. The method according to claim 1, wherein determining the computational expression of the reduction operation based on the implementation code comprises: Based on the preset algebraic group structure and distributive law, the computational expression of the reduction operation, which includes operators that satisfy the algebraic group structure and distributive law, is determined from the implementation code.

5. The method according to any one of claims 1 to 4, wherein the protocol operation comprises: Reduction operations in deep learning models.

6. A data processing method based on reduction operations, the method comprising: Load the dataset to be computed; Read the dataset and execute the target code obtained by the method of any one of claims 1 to 5 to obtain the calculation result; wherein, the dataset includes: the dataset corresponding to the reduction operation in the target code.

7. A data processing system comprising a code adjustment device for performing the method of any one of claims 1 to 5 and a data processing device for performing the method of claim 6.

8. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.

9. A computer program product comprising a computer program that, when executed by a processor, implements the steps of the method according to any one of claims 1 to 6.

10. A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method according to any one of claims 1 to 6.

Images