Operator acceleration method, device, equipment and storage medium

Abstract

The application discloses an operator acceleration method and device, equipment and storage medium, relates to the technical field of operator compilation optimization, and the operator acceleration method comprises the following steps: after buffer conversion is completed, all operators associated with each explicit buffer object under a current compilation scope are acquired, and all data flow dependency relationships are reconstructed; a plurality of fusion groups are constructed according to the data flow dependency relationships, and the data flow dependency relationship in each fusion group is determined; each eliminable intra-group data flow of the current fusion group is determined according to the intra-group data flow dependency relationship; the intermediate buffer read-write operation corresponding to each eliminable intra-group data flow is eliminated, each eliminable intra-group data flow is mapped to a register, and a mapping relationship is determined; target code is generated according to the mapping relationship, and the memory access logic under the current compilation scope is rewritten based on the target code, so that the operator under the current compilation scope is accelerated. Through implementation of the application scheme, the compilation operator calculation speed can be optimized.

Description

Technical Field

[0001] This application relates to the field of operator compilation optimization technology, and in particular to an operator acceleration method, apparatus, device and storage medium. Background Technology

[0002] Operator compilation optimization techniques for general vector matrix computation architectures are widely used in matrix operation compilation acceleration scenarios. Existing compilers typically perform operator acceleration analysis at the high-level tensor stage, where key information such as buffer layout, storage aliases, and block strategies are not yet determined. After the program completes buffer transformation, high-level tensor dependencies are transformed into buffer read-write relationships, and the high-level dependency chain can no longer accurately reflect the actual data flow at the storage layer.

[0003] Due to information gaps, traditional operator acceleration methods struggle to identify removable intra-group intermediates. These intermediates are generated and consumed solely within the buffer, meaning they are generated and read only by internal read / write operations and are not accessed by external read / write operations. For semantic safety reasons, compilers typically conservatively allocate independent intermediate buffers for all intra-group intermediates. However, many of these intermediates, which only circulate internally during computation, still need to be written to memory and read again, introducing high data transfer overhead. This repeated reading and writing of all intra-group intermediates within the buffer results in slow compilation speeds for vector and matrix computation scenarios.

[0004] Therefore, it is urgent to adjust the memory access method of intermediate quantities within the group during operator compilation in order to reduce the overhead of reading intermediate data in vector matrix calculation scenarios and improve the calculation speed of operator compilation. Summary of the Invention

[0005] The main objective of this application is to provide an operator acceleration method, apparatus, device, and storage medium, which aims to solve the technical problem that the existing technology allocates buffers for all intermediate quantities generated within a group during operator compilation, resulting in repeated reading and writing of intermediate quantities within the group in the buffer during compilation and calculation, thus causing slow compilation and calculation speed.

[0006] To achieve the above objectives, this application proposes an operator acceleration method, which includes: After the buffer transformation is completed, obtain all operators associated with each explicit buffer object in the current compilation scope, and reconstruct all data flow dependencies in the current compilation scope based on each operator. Based on the data flow dependencies, several fusion groups are constructed, and the data flow dependencies within each fusion group are determined. For each fusion group, the data streams that can be eliminated within the current fusion group are determined based on the data stream dependencies within the group. Eliminate the intermediate buffer read / write operations corresponding to the data streams within each of the eliminateable groups, and map each of the eliminateable groups' data streams to registers to determine the mapping relationship; Target code is generated based on the mapping relationship, and the memory access logic under the current compilation scope is rewritten based on the target code to accelerate the operators under the current compilation scope.

[0007] In a feasible embodiment, the step of constructing several fusion groups based on the data flow dependencies and determining the intra-group data flow dependencies of each fusion group includes: Construct a data flow dependency graph based on the data flow dependencies; Traverse the data flow dependency graph and select several data flow dependencies that can form a serial long connection from the data flow dependency graph as several intra-group data flow dependencies; For each data flow dependency within a group, a corresponding fusion group is constructed based on each operator associated with the data flow dependency within the group.

[0008] In a feasible embodiment, determining each anisodable intra-group data stream of the current fusion group based on the intra-group data stream dependency includes: Obtain the storage range information of all buffers under the current compilation scope, and determine the set of buffers with overlapping regions under the current compilation scope based on the storage range information; The first intermediate buffer set associated with the current fusion group is determined based on the data flow dependencies within the group; wherein, the first intermediate buffer set contains several intermediate buffers; Each intermediate buffer included in the buffer set is removed from the first intermediate buffer set to obtain a second intermediate buffer set; Traverse the second intermediate buffer set, remove the intermediate buffers containing only the association operators of the current fusion group, and obtain the third intermediate buffer set; Each initial eliminable group intra-data stream is determined based on each intermediate buffer in the third set of intermediate buffers; Obtain the iteration domain of each of the initial eliminable intra-group data flow association operators, and determine each eliminable intra-group data flow of the current fusion group based on each iteration domain and each of the initial eliminable intra-group data flows.

[0009] In one feasible embodiment, after determining the revocable intra-group data flows of the current fusion group based on the intra-group data flow dependencies, the method further includes: Based on the fusion group and the data flow dependency, determine the set of operators to be merged; Traverse the set of operators to be merged, merge the elements of the set of operators that meet the merging conditions into the fusion group, and update the data streams within each group that can be eliminated in the fusion group.

[0010] In a feasible embodiment, the step of traversing the set of operators to be merged and merging the elements of the set of operators that meet the merging conditions into the fusion group includes: During each iteration, a fusion group to be verified is constructed based on the elements of the set of operators to be merged corresponding to the current iteration round and the fusion group. Determine the data flow within each of the fusion groups to be verified; The iteration domain of the data flow association operator, the regional overlap relationship of the association intermediate buffer, the external operator access behavior of the association intermediate buffer, and the data type semantic features of each data flow association operator within the group to be verified are verified to obtain the verification result set of the data flow within each group to be verified; wherein, the verification result set includes: iteration domain verification results, regional overlap verification results, external access verification results, and data type semantic feature verification results; When all verification results in the verification result set pass, the elements of the operator set to be merged are merged into the fusion group.

[0011] In one feasible embodiment, mapping each of the revocable intra-group data streams to registers and determining the mapping relationship includes: Determine the intermediate operation values ​​contained in each of the data streams that can be eliminated within the group, and write each of the intermediate operation values ​​into several register blocks respectively; The mapping relationship is determined based on the correspondence between the block encoding information of each register block and each intermediate operation value.

[0012] In a feasible embodiment, the step of generating target code based on the mapping relationship and rewriting the memory access logic under the current compilation scope based on the target code includes: Based on the mapping relationship, generate target code containing the intermediate operation values ​​and the pointers to the register blocks; The target code is written into the current compiler so that the current compiler can rewrite the memory access logic in the current compilation scope based on the target code.

[0013] In addition, to achieve the above objectives, this application also proposes an operator acceleration device, which includes: a data stream reconstruction module, an intermediate quantity identification module, and a fusion mapping module; The data flow reconstruction module is used to obtain all operators associated with each explicit buffer object in the current compilation scope after the buffer conversion is completed, and to reconstruct all data flow dependencies in the current compilation scope based on each operator. The intermediate quantity identification module is used to construct several fusion groups based on the data flow dependencies and determine the intra-group data flow dependencies of each fusion group; for each fusion group, it determines the data flows that can be eliminated within the current fusion group based on the intra-group data flow dependencies. The fusion mapping module is used to eliminate the intermediate buffer read and write operations corresponding to the data streams within each of the eliminateable groups, and map the data streams within each of the eliminateable groups to registers to determine the mapping relationship; generate target code according to the mapping relationship, and rewrite the storage relationship under the current compilation scope based on the target code to achieve operator acceleration under the current compilation scope.

[0014] In addition, to achieve the above objectives, this application also proposes an operator acceleration device, the device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the operator acceleration method as described above.

[0015] In addition, to achieve the above objectives, this application also proposes a storage medium, which is a computer-readable storage medium, on which a computer program is stored, and which, when executed by a processor, implements the steps of the operator acceleration method described above.

[0016] In addition, to achieve the above objectives, this application also provides a computer program product, which includes a computer program that, when executed by a processor, implements the steps of the operator acceleration method described above.

[0017] One or more technical solutions proposed in this application have at least the following technical effects: This application proposes an operator acceleration method, apparatus, device, and storage medium. The method is applied after buffer conversion. Since the compiler's high-level tensor dependencies are transformed into buffer read / write relationships after buffer conversion, and these high-level tensor dependencies no longer accurately reflect the actual data flow at the storage level, this application obtains all operators associated with each explicit buffer object in the current compilation scope and reconstructs all data flow dependencies in the current compilation scope based on these operators. After restoring the data flow dependencies, several fusion groups are constructed based on these dependencies, and the intra-group data flow dependencies of each fusion group are determined, thereby identifying the eliminable intra-group data flows in each fusion group. The intermediate buffer read / write operations corresponding to the eliminable intra-group data flows are eliminated, and each eliminable intra-group data flow is mapped to a register to determine the mapping relationship. Target code is generated based on the mapping relationship, and the memory access logic in the current compilation scope is rewritten based on the target code to achieve operator acceleration in the current compilation scope.

[0018] The process of eliminating read / write operations on intermediate buffers corresponding to eliminateable data streams within a group does not directly delete the buffers allocated by the compiler for intermediate values ​​within the group. However, by eliminating these read / write operations, these buffers, which were allocated for intermediate values ​​within the group after buffer conversion, become buffers with no read / write operations. This triggers the compiler's dead code elimination (DCE) mechanism, causing the compiler to delete these buffers and reduce the compiler's memory allocation. Based on this, target code is generated through mapping relationships, converting intermediate values ​​within the group that require repeated read / write operations in intermediate buffers into register read / write operations. Leveraging the principle that registers are adjacent to arithmetic units and that computation results can be directly resided and reused, this eliminates the memory transfer operations of writing intermediate data back to the buffer and then reading it back from the buffer, thus eliminating read / write memory access overhead and improving operator execution speed. Attached Figure Description

[0019] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0020] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 This is a flowchart illustrating the first embodiment of the operator acceleration method of this application; Figure 2 This is a flowchart illustrating the second embodiment of the operator acceleration method of this application. Figure 3 This is a schematic diagram of the module structure of the operator acceleration device according to an embodiment of this application; Figure 4 This is a schematic diagram of the device structure of the hardware operating environment involved in the operator acceleration method in the embodiments of this application.

[0022] The purpose, features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0023] It should be understood that the specific embodiments described herein are merely illustrative of the technical solutions of this application and are not intended to limit this application.

[0024] To better understand the technical solution of this application, a detailed description will be provided below in conjunction with the accompanying drawings and specific implementation methods.

[0025] To better understand the technical solution of this application, it is necessary to clarify in advance that during the compilation operator processing, the compiler receives the description of the upper-level computational operators and performs compilation optimization based on the hardware characteristics, register capacity, memory, and buffer resources of the target processor. The processor is the hardware computational entity that executes machine instructions, including the central processing unit, graphics processing unit, neural network processor, etc., used to complete various numerical and logical operations; registers are located inside the processor and are high-speed on-chip storage resources, allowing the processor's arithmetic units to directly read and write temporary computation data. The compiler is a software processing tool that can receive high-level program code or neural network computation graphs, parse them to generate intermediate representations, and perform optimization processing based on the hardware characteristics of the target processor. The backend completes instruction selection, register allocation, and instruction scheduling, outputting machine instructions that the processor can recognize. Operator acceleration is a performance optimization technique implemented for fixed high-frequency computational operators such as matrix multiplication and convolution.

[0026] Because current compilers typically perform operator acceleration analysis at the high-level tensor stage, key information such as buffer layout, storage aliases, and block strategies are not yet determined at this stage. After the program completes buffer transformation, high-level tensor dependencies become buffer read-write relationships, and the high-level dependency chain can no longer accurately reflect the actual data flow at the storage level. Due to the lack of information, traditional operator acceleration methods struggle to identify removable intra-group intermediates. Intra-group intermediates are generated and consumed only within the buffer, such as buffer data that is generated and read only by internal read-write operations and is not accessed by external read-write operations. For semantic safety considerations, compilers typically conservatively allocate independent intermediate buffers for all intra-group intermediates. A large number of intra-group intermediates that only flow internally during computation still need to be written to memory and read again, introducing high data transfer overhead. This reading method, where all intra-group intermediates are repeatedly read and written in the buffer, results in slow compilation speeds for vector matrix computation scenarios.

[0027] In this embodiment, for ease of description, the compiler will be used as the execution subject in the following description.

[0028] It should be noted that the executing entity in this embodiment can be a computing service device with data processing, network communication, and program execution functions, such as a tablet computer, personal computer, or mobile phone, or an electronic device or compiler capable of performing the above functions. The following description uses a compiler as an example to illustrate this embodiment and the subsequent embodiments.

[0029] Based on this, embodiments of this application provide an operator acceleration method, referring to... Figure 1 , Figure 1 This is a flowchart illustrating the first embodiment of the operator acceleration method of this application.

[0030] In this embodiment, the operator acceleration method includes steps S10 to S50: Step S10: After the buffer conversion is completed, obtain all operators associated with each explicit buffer object in the current compilation scope, and reconstruct all data flow dependencies in the current compilation scope based on each operator. Specifically, the operator acceleration method of this application is applied after the buffer is completed. After the buffer conversion is completed, the high-level tensor dependency of the compiler is transformed into the buffer read-write relationship. The high-level tensor dependency can no longer accurately reflect the real data flow at the storage level. Therefore, it is necessary to first reconstruct the data flow dependency relationship under the current compilation scope of the current compiler.

[0031] The reconstruction of data flow dependencies is achieved by obtaining each explicit buffer object in the current compilation scope of the compiler, identifying all operators involved in the execution of each explicit buffer object, and then reconstructing the data flow dependencies between operators by analyzing the order of reading and writing buffers and the input-output relationships between operators. This process restores the logic of the order of operator execution and enables the reconstruction of all data flow dependencies in the current compilation scope.

[0032] It's important to note that explicit buffer objects refer to buffer instances explicitly declared and created by the compiler code within the current compilation scope. This differs from implicit temporary buffers automatically generated internally by the compiler. These explicit buffer objects possess an identifier that allows them to be directly referenced by operators, recording their input / output associated operators, storage location, and data attributes, and can be traversed and identified by the compiler. An operator is a computational unit that can be independently scheduled and executed during compilation, representing a specific set of data operation logic. Operators use buffers as both data input sources and output carriers, performing operations by reading data from the buffers and writing the results back to the corresponding buffer object. Data flow dependencies characterize the execution constraints between different operators based on data read / write operations. If a later operator needs to read data written to the buffer by a previous operator, then the later operator has a data dependency on the previous operator. This dependency limits the execution order between operators and is the foundation for the compiler to perform operator rearrangement, operator fusion, and scheduling optimizations.

[0033] For example, suppose there are buffers A, B, and C in the current compilation scope. After the buffer transformation is completed, the compiler traverses the compilation scope and identifies the associated operators Op1 and Op2 for buffer A, Op2 and Op3 for buffer B, and Op3 for buffer C. Based on the buffer read-write relationship, the output of Op1 is written to buffer A as the input of Op2, and the output of Op2 is written to buffer B as the input of Op3. In this way, the data flow dependency is reconstructed: Op1→Op2→Op3, and the data flow dependency relationship is obtained.

[0034] Step S20: Construct several fusion groups based on the data flow dependencies, and determine the data flow dependencies within each fusion group; Specifically, based on the data flow dependencies reconstructed in step S10, multiple fusion groups are divided under the current compilation scope, and the intra-group data flow dependencies of each fusion group are determined. It should be noted that a fusion group is a combination of multiple operators with data flow dependencies, and the intra-group data flow dependencies of a fusion group refer to the data flow dependencies between the operators contained in the group.

[0035] For example, the current compilation scope contains operators Op1, Op2, Op3, Op4, and Op5, and these operators construct two data flow dependencies, namely Op1→Op2→Op3 and Op4→Op5. At this time, the compiler groups and divides the data flow dependencies between the operators, and groups the operators with direct or indirect data flow dependencies into the same fusion group, thus obtaining the first fusion group and the second fusion group. The first fusion group contains operators Op1, Op2, and Op3, and the second fusion group contains operators Op4 and Op5. For the first fusion group, the data flow dependency within the group is that the data output by Op1 is read by Op2, and the data output by Op2 is read by Op3, forming the intra-group operator execution constraint Op1→Op2→Op3; for the second fusion group, the data flow dependency within the group is that the data output by Op4 is read by Op5, forming the intra-group operator execution constraint Op4→Op5. Each fusion group retains the original data read and write constraints of the operators, and there is no data flow dependency between different fusion groups, so intra-group operator fusion compilation processing can be carried out separately.

[0036] In one feasible implementation, step S20 may include steps A11 to A13: Step A11: Construct a data flow dependency graph based on the data flow dependency relationship; Specifically, a data flow dependency graph is first constructed based on the reconstructed operator data flow dependencies. This graph uses operators as nodes and data flow dependencies between operators as directed edges.

[0037] Step A12: Traverse the data flow dependency graph and select several data flow dependencies that can form a serial long connection from the data flow dependency graph as several intra-group data flow dependencies; Specifically, the data flow dependency graph is traversed to select data flow dependencies that can form a serial long connection, and the selected data flow dependencies are used as the intra-group data flow dependencies for each fusion group.

[0038] Step A13: For each data flow dependency within a group, construct a corresponding fusion group based on the operators associated with the data flow dependency within the group.

[0039] Specifically, for each selected group of intra-group data flow dependencies, all operators associated with that dependency are aggregated to generate a corresponding fusion group.

[0040] For example, suppose the data flow dependency graph constructed in step A11 contains the graph nodes V1, V2, V3, V4, V5, and V6 obtained when operators Op1, Op2, Op3, Op4, Op5, and Op6 are used as graph nodes, with directed edges: Op1→Op2 (directed edge between V1 and V2), Op2→Op3 (directed edge between V2 and V3), and Op5→Op6 (directed edge between V5 and V6). Graph node V4 (corresponding to operator Op4) has no input or output edges. Traversing the graph, two long serial connections are identified: the first is Op1→Op2→Op3 (corresponding to graph nodes V1→V2→V3), and the second is Op5→Op6 (corresponding to graph nodes V5→V6). These two long serial connections are treated as two groups of intra-group data flow dependencies. A first fusion group is constructed based on Op1, Op2, and Op3 (corresponding to V1, V2, and V3) associated with the first long serial connection, and a second fusion group is constructed based on Op5 and Op6 (corresponding to V5 and V6) associated with the second long serial connection. Operator Op4 (corresponding to V4) does not have a long serial connection and is not included in the above fusion groups.

[0041] It should be noted that a serial long connection refers to a chain-like path formed by multiple operators acting as graph nodes in a data flow dependency graph, connected sequentially through directed data flow dependency edges. The output of the preceding operator serves as the input of the following operator within the path, creating a serial data transfer relationship between the operators. For example, in the data flow dependency graph mentioned above, there are directed dependency edges Op1→Op2 and Op2→Op3 between graph nodes V1, V2, and V3. That is, the operator Op1 corresponding to graph node V1 outputs data into a buffer for the operator Op2 corresponding to graph node V2 to read, and the operator Op2 corresponding to graph node V2 outputs data into a buffer for the operator Op3 corresponding to graph node V3 to read. The operators sequentially transfer data without branches or parallel branches. Op1→Op2→Op3 together constitute a serial long connection, and this serial long connection has a data transfer relationship.

[0042] Step S30: For each fusion group, determine the data streams within the current fusion group that can be eliminated based on the data stream dependencies within the group; Specifically, for each completed fusion group, the intra-group data flows that can be eliminated within each fusion group are determined based on their intra-group data flow dependencies.

[0043] It should be noted that eliminating intra-group data flow means that after fusion, operators no longer need to rely on independent explicit buffers to transfer data. Data can be transferred directly through registers or on-chip caches, thus eliminating intra-group data read and write paths.

[0044] To avoid compiler operator errors after eliminating read / write operations on the corresponding intermediate buffers of the data streams within the eliminateable group, when determining the data streams within the eliminateable group, it is necessary to further consider the iteration domain of the associated operators of the data streams within the group, the regional overlap relationship of the associated intermediate buffers, and the external operator access behavior of the associated intermediate buffers. Based on the above considerations, this application proposes a preferred feasible implementation for step S30.

[0045] In one feasible implementation, step S30 may include steps B11 to B16: Step B11: Obtain the storage range information of all buffers under the current compilation scope, and determine the set of buffers with overlapping regions under the current compilation scope based on the storage range information. Specifically, the system obtains the storage range information for all buffers within the current compilation scope. Each storage range includes the buffer's starting memory address and its length, representing the memory space occupied by the buffer. Based on the storage range information for each buffer, it determines whether there is region overlap within the current compilation scope. Buffers with overlapping regions are written to the buffer set for subsequent auxiliary determination of whether there is region overlap in the associated intermediate buffers of the fusion group.

[0046] Step B12: Determine the first intermediate buffer set associated with the current fusion group based on the intra-group data flow dependency relationship; wherein, the first intermediate buffer set contains several intermediate buffers; Specifically, based on the data flow dependencies within the group, it is determined which intermediate buffers the operators associated with the data flow dependencies within the group are flowing through, thereby determining the intermediate buffers contained in the current fusion group. These intermediate buffers are then stored in the first intermediate buffer set to characterize which intermediate buffers are associated with the current fusion group.

[0047] Step B13: Remove each intermediate buffer contained in the buffer set from the first intermediate buffer set to obtain the second intermediate buffer set; Specifically, by comparing the set of buffers with overlapping regions in the current compilation scope with the first set of buffers, it is determined whether the first set of buffers contains buffers that have been determined to have overlapping regions. If so, the overlapping intermediate buffers are removed from the first set of intermediate buffers to obtain the second set of intermediate buffers. At this time, each intermediate buffer in the second set of intermediate buffers is an intermediate buffer without overlapping regions.

[0048] By determining the overlap of buffer regions, it is possible to prevent read and write operations on associated buffers from being affected after read and write operations on buffers with overlapping regions are eliminated and the buffer is deleted by the compiler's dead code mechanism.

[0049] Step B14: Traverse the second intermediate buffer set, remove the intermediate buffers containing only the current fusion group association operators, and obtain the third intermediate buffer set. Specifically, after determining buffer overlap, the second intermediate buffer set is traversed to check whether each intermediate buffer in the second intermediate buffer set has been read or written by operators outside the fusion group. If it has been used by an external operator, the intermediate buffer cannot be eliminated and is removed from the second intermediate buffer set, resulting in the third intermediate buffer set. The intermediate buffers in the third intermediate buffer set are those without buffer region overlap or access by external operators.

[0050] Step B15: Determine the data stream within each initial revocable group based on each intermediate buffer in the third intermediate buffer set; Specifically, based on the intermediate buffers included in the third intermediate buffer set and the data flow dependencies within the group, the initial eliminable intra-group data flows that are circulated in the third intermediate buffer set are deduced.

[0051] Step B16: Obtain the iteration domain of each of the initial eliminable intra-group data flow association operators, and determine the eliminable intra-group data flows of the current fusion group based on each iteration domain and each of the initial eliminable intra-group data flows.

[0052] Specifically, for each initial eliminable intra-group data stream, the iteration domain of the associated operator of the initial eliminable intra-group data stream is obtained, and the iteration domains of the operators at both ends of the initial eliminable intra-group data stream are verified. Only when the iteration domains match and are aligned can the buffer transfer be skipped after the operators are merged; if the iteration domains do not match, the initial eliminable intra-group data stream cannot be eliminated, and finally the real eliminable intra-group data stream is obtained by filtering.

[0053] For example, the initial eliminateable intra-group data streams Op2 write buffer B and Op3 read buffer B; group-level iterative domain descriptions are constructed for Op2 and Op3 respectively. The two-dimensional group-level iterative domain of Op2 is taken from the first two dimensions of buffer B, with upper bounds of [0,2) and [0,3) respectively, and step size of 1 for both dimensions; the iteration range, step size, and block mapping of Op3 and Op2 are checked dimension by dimension to see if they are compatible. Op3 reads buffer B according to the same block coordinates. The upper bounds of the corresponding dimensions of the two are taken from the corresponding dimensions of the same storage object buffer B, and the step size is consistent. If the group-level blocks have a definite one-to-one correspondence with no duplicate or missing elements, then the unified iterative domain constraint is satisfied, and the initial eliminateable intra-group data stream is determined to be an eliminateable intra-group data stream; if the upper bounds of the corresponding dimensions of the two are inconsistent, the static lengths are unequal, the dynamic lengths do not share the same size parameter, or the block mapping has duplicate or missing elements, and the unified iterative domain check is not satisfied, then the elimination of the initial eliminateable intra-group data stream is conservatively rejected.

[0054] By implementing the above feasible embodiments, when determining the data flow that can be eliminated within a group, by carrying out three types of verifications—operator iteration domain, intermediate buffer region overlap relationship, and external operator access behavior of the intermediate buffer—it is possible to avoid operator operation anomalies caused by blindly eliminating intermediate buffer read and write operations. On the one hand, it can filter out illegal elimination scenarios with memory alias conflicts, external operator read dependencies, and incompatible iteration domain-block mappings, ensuring the correctness and reliability of the compilation output results. On the other hand, it can identify truly eliminateable data flow within a group under the premise of safety, reduce redundant buffer memory access overhead, and balance compilation correctness with the running performance of operator fusion.

[0055] Step S40: Eliminate the intermediate buffer read / write operations corresponding to the data streams within each of the eliminateable groups, and map each of the data streams within the eliminateable groups to a register to determine the mapping relationship; Specifically, for the eliminated intra-group data streams, the corresponding intermediate buffer read / write instructions are deleted, and data is no longer transferred through the intermediate buffer; the data transfer path of the eliminated intra-group data stream is mapped to a register, and a mapping relationship is established between the eliminated data stream and the hardware register, relying on the register to complete the intermediate data transfer of the intra-group operator.

[0056] For example, the data path from Op2 to Op3 via buffer B within the group can be eliminated, eliminating buffer read / write operations where Op2 writes to buffer B and Op3 reads from buffer B. The intermediate calculation result output by Op2 is directly mapped and allocated to the target register, establishing the mapping relationship between the eliminated data stream within the group and this register. Op3 directly reads the calculation result of Op2 from this register, eliminating the buffer memory access step. Redundant intermediate buffer read / write operations are removed, and intermediate data is transferred using registers, reducing memory access overhead and improving the calculation speed after operator fusion.

[0057] It should be noted that the operation of eliminating read and write operations on intermediate buffers corresponding to the data streams within the eliminateable group does not directly delete the buffers allocated by the compiler for the intermediate values ​​within the group. However, by eliminating the read and write operations on intermediate buffers corresponding to the data streams within the eliminateable group, these buffers allocated for the intermediate values ​​within the group after buffer conversion become buffers with no read and write operations. At this time, the compiler's dead code elimination (DCE) mechanism will be triggered, and the compiler will delete these buffers, reducing the memory allocation of the compiler.

[0058] In one feasible implementation, the processing step of mapping the intra-group data streams of each of the eliminateable groups to registers in step S40, and determining the mapping relationship, may include steps C11-C12: Step C11: Determine the intermediate operation values ​​contained in each of the data streams within the eliminateable group, and write each of the intermediate operation values ​​into several register blocks respectively. Step C12: Determine the mapping relationship based on the correspondence between the block encoding information of each register block and each intermediate operation value.

[0059] Specifically, the intermediate operation values ​​corresponding to the data streams within each reversible group are determined, and each intermediate operation value is written into the corresponding register block. Then, the block encoding information of the register block and the corresponding association between each intermediate operation value are established. In this way, the mapping relationship between the data streams within the reversible group and the register is obtained, which is used to locate the register block and read the corresponding intermediate operation value when the operator is executed in the future.

[0060] By implementing the above embodiments, by writing the intermediate operation values ​​corresponding to the data stream within the group into register blocks and establishing a mapping relationship based on the register block encoding information, the data reading and writing of the original intermediate buffer can be replaced, saving memory access overhead. The high-speed data reading and writing capability of the register can be used to speed up the intermediate data transmission speed. At the same time, the orderly addressing of intermediate operation values ​​can be achieved through the correspondence of block encoding, ensuring that the fused operator can accurately read the required intermediate data and improve the overall compilation and execution efficiency.

[0061] Step S50: Generate target code based on the mapping relationship, and rewrite the memory access logic under the current compilation scope based on the target code to accelerate the operators under the current compilation scope.

[0062] Specifically, based on the established mapping relationship between intermediate operation values ​​and register blocks, hardware-executable target code is generated, the original memory access logic in the compilation scope is rewritten, the eliminated intermediate buffer read and write instructions are removed and replaced with register block read and write instructions, thereby completing operator acceleration in the compilation scope.

[0063] In one feasible implementation, step S40 may include steps D11-D12: Step D11: Based on the mapping relationship, generate target code containing the intermediate operation value and the pointers to the register block bindings; Step D12: Write the target code into the current compiler so that the current compiler can rewrite the memory access logic in the current compilation scope based on the target code.

[0064] Specifically, based on the mapping relationship between intermediate operation values ​​and register blocks, target code carrying the binding relationship between the two is generated. This target code is then imported into the compiler, which uses it to modify the original memory access logic in the compilation scope and replace the original intermediate buffer read / write operations.

[0065] By implementing this embodiment, target code can be generated based on the mapping relationship and memory access logic can be rewritten, replacing the original high-overhead buffer memory access with high-speed register access, reducing data transfer overhead, lowering memory access latency, accelerating operators while ensuring that the computational logic remains unchanged, and improving the overall program performance.

[0066] Based on the first embodiment described above, in order to further construct a more accurate fusion group, expand the judgment of the constructed fusion group, and further analyze the elements that can be incorporated into the fusion group, this application proposes a second embodiment.

[0067] Based on the first embodiment of this application, in the second embodiment of this application, the content that is the same as or similar to that in the first embodiment described above can be referred to the above description, and will not be repeated hereafter. Based on this, please refer to... Figure 2 After step S30, the operator acceleration method further includes steps S01~S02: Step S01: Determine the set of operators to be merged based on the fusion group and the data flow dependency relationship; Specifically, based on the constructed fusion groups and data flow dependencies, operators that meet the dependency association conditions are selected as elements of the operator set to be merged, forming the operator set to be merged. The merged objects can be either independent operators or other fusion groups that have already been constructed.

[0068] By performing this step, a set of operators to be merged can be obtained that simultaneously supports both independent operators and fusion groups as elements of the operator set to be merged. This enables flexible granular expansion and merging, supporting both the gradual merging of individual operators and the direct merging of existing fusion groups, reducing repetitive grouping processes and improving compilation optimization efficiency.

[0069] Step S02: Traverse the set of operators to be merged, merge the elements of the set of operators that meet the merging conditions into the fusion group, and update the data streams within each group that can be eliminated in the fusion group.

[0070] In one feasible implementation, step S02 may include steps E11 to E14: Step E11: In each traversal process, construct a fusion group to be verified based on the elements of the set of operators to be merged corresponding to the current traversal round and the fusion group. Specifically, the elements to be merged into the operator set in this round of traversal are taken out and merged with the original fusion group to obtain a temporary fusion group to be verified, which is used for subsequent verification and the formal merging is not performed for the time being.

[0071] Step E12: Determine the data flow within each verification group of the fusion group to be verified; Specifically, the data transmission paths between operators within the fusion group to be verified are analyzed, and all data streams within the group to be verified are identified by relying on intermediate buffers to complete the interaction between operators within the group.

[0072] Step E13: Verify the iteration domain of the data flow association operator within each group to be verified, the regional overlap relationship of the associated intermediate buffer, the external operator access behavior of the associated intermediate buffer, and the data type semantic features to obtain the verification result set of the data flow within each group to be verified; wherein, the verification result set includes: iteration domain verification results, regional overlap verification results, external access verification results, and data type semantic feature verification results; Specifically, the iterative domain of the data flow association operator within each group to be verified, the regional overlap relationship of the associated intermediate buffer, and the external operator access behavior of the associated intermediate buffer are verified. The verification logic here is the same as the association content in the first embodiment, and will not be repeated here.

[0073] To ensure that the data conversion logic and computational meaning remain consistent with the original compilation logic after the elements of the operator set to be merged into the fusion group, it is necessary to verify their data type semantic features. During data type semantic feature verification, the data flow within the group to be verified is traversed along the data flow direction, comparing the element type, bit width, and sign attribute of the upstream and downstream operator inputs and outputs sequentially. Simultaneously, the position and execution order of rounding, truncation, and type conversion operations are checked to determine if the type conversion semantics are compatible. If there are type mismatches, bit width misalignments, or offsets in rounding / truncation positions, the verification fails. Verification passes when all features are compatible.

[0074] Step E14: When all verification results in the verification result set pass, the elements of the operator set to be merged are merged into the fusion group.

[0075] Specifically, the formal merging will only proceed if all four checks within the verification result set pass; if any one check fails, the merging will be rejected, and the original merging group will remain unchanged.

[0076] By implementing the second embodiment of this application, the computational chain range of the fusion group can be expanded as much as possible, and more data flows within the intermediate group that can be eliminated can be mined, thereby reducing more buffer memory accesses. Based on the first embodiment described above, the compilation effect and compilation speed are further improved.

[0077] It should be noted that the above examples are only for understanding this application and do not constitute a limitation on the operator acceleration method of this application. Any simple transformations based on this technical concept are all within the protection scope of this application.

[0078] This application also provides an operator acceleration device; please refer to [reference needed]. Figure 3 The operator acceleration device includes: a data stream reconstruction module 10, an intermediate quantity identification module 20, and a fusion mapping module 30; The data flow reconstruction module 10 is used to obtain all operators associated with each explicit buffer object in the current compilation scope after the buffer conversion is completed, and to reconstruct all data flow dependencies in the current compilation scope based on each operator. The intermediate quantity identification module 20 is used to construct several fusion groups according to the data flow dependencies and determine the data flow dependencies within each fusion group; for each fusion group, it determines the data flows within the current fusion group that can be eliminated based on the data flow dependencies within the group. The fusion mapping module 30 is used to eliminate the intermediate buffer read and write operations corresponding to the data streams within each of the eliminateable groups, and map the data streams within each of the eliminateable groups to registers to determine the mapping relationship; generate target code according to the mapping relationship, and rewrite the storage relationship under the current compilation scope based on the target code to achieve operator acceleration under the current compilation scope.

[0079] The operator acceleration device provided in this application, employing the operator acceleration method in the above embodiments, can solve the technical problem of slow compilation speed caused by the prior art allocating buffers for all intermediate quantities generated within a group during operator compilation, resulting in repeated reading and writing of intermediate quantities within the group in the buffers during compilation. Compared with the prior art, the beneficial effects of the operator acceleration device provided in this application are the same as those of the operator acceleration method provided in the above embodiments, and other technical features in the operator acceleration device are the same as those disclosed in the methods of the above embodiments, and will not be repeated here.

[0080] This application provides an operator acceleration device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the operator acceleration method in the first embodiment described above.

[0081] The following is for reference. Figure 4 The diagram illustrates a structural schematic suitable for implementing the operator acceleration device in the embodiments of this application. The operator acceleration device in the embodiments of this application may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Description), PMPs (Portable Media Players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 4 The operator acceleration device shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.

[0082] like Figure 4 As shown, the operator acceleration device may include a processing unit 1001 (e.g., a central processing unit, a graphics processing unit, etc.) that can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1002 or a program loaded from a storage device 1003 into a random access memory (RAM) 1004. The RAM 1004 also stores various programs and data required for the operation of the operator acceleration device. The processing unit 1001, ROM 1002, and RAM 1004 are interconnected via a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems can be connected to the I / O interface 1006: input devices 1007 including, for example, touchscreens, touchpads, keyboards, mice, image sensors, microphones, accelerometers, gyroscopes, etc.; output devices 1008 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 1003 including, for example, magnetic tapes, hard disks, etc.; and communication devices 1009. The communication device 1009 allows the operator acceleration device to communicate wirelessly or wiredly with other devices to exchange data. Although the figure shows operator acceleration devices with various systems, it should be understood that it is not required to implement or possess all of the systems shown. More or fewer systems may be implemented alternatively.

[0083] Specifically, according to the embodiments disclosed in this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from storage device 1003, or installed from ROM 1002. When the computer program is executed by processing device 1001, it performs the functions defined in the methods of the embodiments disclosed in this application.

[0084] The operator acceleration device provided in this application, employing the operator acceleration method in the above embodiments, can solve the technical problem in the prior art where buffers are allocated for all intermediate quantities generated within a group during operator compilation, leading to repeated reading and writing of intermediate quantities within the group in the buffers during compilation, resulting in slow compilation speed. Compared with the prior art, the beneficial effects of the operator acceleration device provided in this application are the same as those of the operator acceleration method provided in the above embodiments, and other technical features in this operator acceleration device are the same as those disclosed in the method of the previous embodiment, and will not be repeated here.

[0085] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.

[0086] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0087] This application provides a computer-readable storage medium having computer-readable program instructions (i.e., a computer program) stored thereon, the computer-readable program instructions being used to execute the operator acceleration method in the above embodiments.

[0088] The computer-readable storage medium provided in this application may be, for example, a USB flash drive, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, system, or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.

[0089] The aforementioned computer-readable storage medium may be included in the operator acceleration device; or it may exist independently and not assembled into the operator acceleration device.

[0090] The aforementioned computer-readable storage medium carries one or more programs, which, when executed by the operator acceleration device, cause the operator acceleration device to execute an operator acceleration method.

[0091] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a Local Area Network (LAN) or a Wide Area Network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0092] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0093] The modules described in the embodiments of this application can be implemented in software or hardware. The names of the modules do not necessarily limit the functionality of the unit itself.

[0094] The readable storage medium provided in this application is a computer-readable storage medium that stores computer-readable program instructions (i.e., a computer program) for executing the above-described operator acceleration method. This solves the technical problem in existing technologies where buffers are allocated for all intermediate values ​​within a group generated during operator compilation, leading to repeated reading and writing of these intermediate values ​​in the buffers during compilation and computation, resulting in slow compilation speeds. Compared to the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as those of the operator acceleration method provided in the above embodiments, and will not be repeated here.

[0095] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the operator acceleration method described above.

[0096] The computer program product provided in this application can solve the technical problem of slow compilation speed caused by the prior art allocating buffers for all intermediate values ​​within a group generated during operator compilation, resulting in repeated reading and writing of intermediate values ​​within the group in the buffers during compilation. Compared with the prior art, the beneficial effects of the computer program product provided in this application are the same as the beneficial effects of the operator acceleration method provided in the above embodiments, and will not be repeated here.

[0097] The above description is only a part of the embodiments of this application and does not limit the patent scope of this application. All equivalent structural transformations made under the technical concept of this application and using the contents of the specification and drawings of this application, or direct / indirect applications in other related technical fields, are included in the patent protection scope of this application.

Claims

1. An operator acceleration method, characterized in that, The operator acceleration method includes: After the buffer transformation is completed, obtain all operators associated with each explicit buffer object in the current compilation scope, and reconstruct all data flow dependencies in the current compilation scope based on each operator. Based on the data flow dependencies, several fusion groups are constructed, and the data flow dependencies within each fusion group are determined. For each fusion group, the data streams that can be eliminated within the current fusion group are determined based on the data stream dependencies within the group. Eliminate the intermediate buffer read / write operations corresponding to the data streams within each of the eliminateable groups, and map each of the eliminateable groups' data streams to registers to determine the mapping relationship; Target code is generated based on the mapping relationship, and the memory access logic under the current compilation scope is rewritten based on the target code to accelerate the operators under the current compilation scope.

2. The operator acceleration method as described in claim 1, characterized in that, The step of constructing several fusion groups based on the data stream dependencies and determining the data stream dependencies within each fusion group includes: Construct a data flow dependency graph based on the data flow dependencies; Traverse the data flow dependency graph and select several data flow dependencies that can form a serial long connection from the data flow dependency graph as several intra-group data flow dependencies; For each data flow dependency within a group, a corresponding fusion group is constructed based on each operator associated with the data flow dependency within the group.

3. The operator acceleration method as described in claim 1, characterized in that, The step of determining the eliminateable intra-group data streams of the current fusion group based on the intra-group data stream dependencies includes: Obtain the storage range information of all buffers under the current compilation scope, and determine the set of buffers with overlapping regions under the current compilation scope based on the storage range information; The first intermediate buffer set associated with the current fusion group is determined based on the data flow dependencies within the group; wherein, the first intermediate buffer set contains several intermediate buffers; Each intermediate buffer included in the buffer set is removed from the first intermediate buffer set to obtain a second intermediate buffer set; Traverse the second intermediate buffer set, remove the intermediate buffers containing only the association operators of the current fusion group, and obtain the third intermediate buffer set; Each initial eliminable group intra-data stream is determined based on each intermediate buffer in the third set of intermediate buffers; Obtain the iteration domain of each of the initial eliminable intra-group data flow association operators, and determine each eliminable intra-group data flow of the current fusion group based on each iteration domain and each of the initial eliminable intra-group data flows.

4. The operator acceleration method as described in claim 1, characterized in that, After determining the revocable intra-group data flows of the current fusion group based on the intra-group data flow dependencies, the method further includes: Based on the fusion group and the data flow dependency, determine the set of operators to be merged; Traverse the set of operators to be merged, merge the elements of the set of operators that meet the merging conditions into the fusion group, and update the data streams within each group that can be eliminated in the fusion group.

5. The operator acceleration method as described in claim 4, characterized in that, The step of traversing the set of operators to be merged and merging elements of the set of operators that meet the merging conditions into the fusion group includes: During each iteration, a fusion group to be verified is constructed based on the elements of the set of operators to be merged corresponding to the current iteration round and the fusion group. Determine the data flow within each of the fusion groups to be verified; The iteration domain of the data flow association operator, the regional overlap relationship of the association intermediate buffer, the external operator access behavior of the association intermediate buffer, and the data type semantic features of each data flow association operator within the group to be verified are verified to obtain the verification result set of the data flow within each group to be verified; wherein, the verification result set includes: iteration domain verification results, regional overlap verification results, external access verification results, and data type semantic feature verification results; When all verification results in the verification result set pass, the elements of the operator set to be merged are merged into the fusion group.

6. The operator acceleration method as described in claim 1, characterized in that, The step of mapping the data streams within each of the eliminateable groups to registers and determining the mapping relationship includes: Determine the intermediate operation values ​​contained in each of the data streams that can be eliminated within the group, and write each of the intermediate operation values ​​into several register blocks respectively; The mapping relationship is determined based on the correspondence between the block encoding information of each register block and each intermediate operation value.

7. The operator acceleration method as described in claim 6, characterized in that, The step of generating target code based on the mapping relationship, and rewriting the memory access logic under the current compilation scope based on the target code, includes: Based on the mapping relationship, generate target code containing the intermediate operation values ​​and the pointers to the register blocks; The target code is written into the current compiler so that the current compiler can rewrite the memory access logic in the current compilation scope based on the target code.

8. An operator acceleration device, characterized in that, The operator acceleration device includes: a data stream reconstruction module, an intermediate quantity identification module, and a fusion mapping module; The data flow reconstruction module is used to obtain all operators associated with each explicit buffer object in the current compilation scope after the buffer conversion is completed, and to reconstruct all data flow dependencies in the current compilation scope based on each operator. The intermediate quantity identification module is used to construct several fusion groups based on the data flow dependencies and determine the intra-group data flow dependencies of each fusion group; for each fusion group, it determines the data flows that can be eliminated within the current fusion group based on the intra-group data flow dependencies. The fusion mapping module is used to eliminate the intermediate buffer read and write operations corresponding to the data streams within each of the eliminateable groups, and map the data streams within each of the eliminateable groups to registers to determine the mapping relationship; generate target code according to the mapping relationship, and rewrite the storage relationship under the current compilation scope based on the target code to achieve operator acceleration under the current compilation scope.

9. An operator acceleration device, characterized in that, The device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the operator acceleration method as described in any one of claims 1 to 7.

10. A storage medium, characterized in that, The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, it implements the steps of the operator acceleration method as described in any one of claims 1 to 7.

Images