52 results about "Fusion operator" patented technology

The invention relates to the technical field of artificial intelligence chips, and provides an operator fusion method and device, electronic equipment and a storage medium, and the method comprises the steps: determining an input tensor, an output tensor and a coverage range of a to-be-fused operator, and the to-be-fused operator comprises a plurality of view operators; on the basis of the metadata of the input tensor, the metadata of the output tensor and the logic relation between the view operators in the coverage range, determining the mapping relation between the dimensions of the input tensor and the output tensor; and performing data filling on the output tensor based on the mapping relationship and each data in the input tensor, and taking the filled output tensor as a fusion result of the operator to be fused. According to the method, multiple continuous view operators are efficiently and quickly fused, so that the problem of performance reduction caused by frequent data handling in the model is effectively solved.

A database query optimization method, comprising: acquiring a query statement, the query statement comprising a connection operation and a group aggregation operation; determining that a connection key and a group key in the query statement are the same, wherein an aggregation function in the query statement can be split, and an input parameter of the aggregation function comes from the same side as a connection condition in the query statement; creating a fusion operator, the fusion operator being used for completing the connection operation and the group aggregation operation; when a cost for executing the fusion operator is less than or equal to a target cost, on the basis of the fusion operator, generating an optimal execution plan related to the fusion operator, wherein the target cost is a cost for executing a connection operator and a group aggregation operator; and performing the operations step by step on the basis of an operation sequence in the optimal execution plan, so as to obtain a query result set. Therefore, when the query statement meets corresponding features, the connection operation and the group aggregation operation can be fused into one operator, thereby reducing the processing flow of an intermediate result, accelerating the execution of the query statement, and improving the query execution performance.

The database query optimization method comprises the steps that a query statement is obtained, and the query statement comprises a connection operation and a grouping aggregation operation; determining that a connection key and a grouping key in the query statement are the same, an aggregation function in the query statement is detachable, and input parameters of the aggregation function are from the same side of a connection condition in the query statement; creating a fusion operator, wherein the fusion operator is used for completing a connection operation and a grouping aggregation operation; under the condition that the cost of executing the fusion operator is smaller than or equal to a target cost, an optimal execution plan related to the fusion operator is generated according to the fusion operator, and the target cost is the cost of executing the connection operator and the grouping aggregation operator; and executing operations step by step according to the operation sequence in the optimal execution plan to obtain a query result set. Therefore, when the query statement meets the corresponding characteristics, the connection and grouping aggregation operations can be fused into one operator, so that the processing flow of an intermediate result can be reduced, the execution of the query statement is accelerated, and the execution performance of the query is improved.

The invention relates to the technical field of electronics, and discloses a deep learning operator optimization and fusion operator calculation instruction generation method and device.The deep learning operator optimization method comprises the steps that all nodes in a deep learning calculation graph are traversed to obtain at least one target node set, the target node set comprises multiple target nodes, and the target nodes in the target node set are extracted; the operator corresponding to each target node is a target operator of a target type, and different target operators have an association relationship; and fusing the operators corresponding to different target nodes in the target node set to obtain a fused operator. The target nodes which have the incidence relation in the computational graph and the corresponding operators are the target type operators are used as a set, the target operators corresponding to the target nodes in the same set are fused, the fused operators can be fully optimized in the aspect of specific instruction implementation, and the execution efficiency of the operators is improved.

The invention relates to the technical field of artificial intelligence, and provides a fusion operator execution method, electronic equipment, a storage medium and a program product, and the method comprises the steps: determining a plurality of to-be-fused target operators; according to the tensors of the multiple target operators, a pseudo address mapping table is generated, and the pseudo address mapping table is used for recording the mapping relation between the actual memory addresses of the tensors and pseudo addresses; operator fusion is carried out on the multiple target operators, a fusion operator is obtained, and the memory address of the tensor of the fusion operator is a continuous pseudo address; and executing the fusion operator, converting the pseudo address into an actual memory address according to the pseudo address mapping table in the execution process, and accessing data of the tensor according to the actual memory address. According to the method, a large amount of extra memory copy overhead caused by physical movement and rearrangement of the original tensor in a traditional operator fusion scheme is avoided, and the computing resource utilization rate and the execution efficiency are remarkably improved.

The invention provides a method and device for reasoning by using an open neural network switching ONNX model based on a fusion operator, electronic equipment and a medium. The method comprises the steps that shape information of an input tensor of the ONNX model is obtained, the shape information is a vector with the dimension being N, each element of the vector indicates each dimension of the input tensor, and N is larger than or equal to 1; generating an input shape vector based on the shape information, the input shape vector being a vector with a dimension of N + 1; the fusion operator is used for operating the input shape vector to obtain a changed shape vector, the fusion operator is fused with at least one ONNX operator, the fusion operator is based on a coefficient matrix, and one dimension of the coefficient matrix is N + 1; and based on the changed shape vector, reasoning is performed by using the ONNX model.

The invention discloses a modeling analysis and calculation optimization method for operator performance on an intelligent chip. The modeling analysis and calculation optimization method comprises the following steps: carrying out graph representation modeling on an intelligent chip hardware architecture and an interconnection mode; designing a link mechanism of operator loop tree nodes, so that operator loops are described and mapped in a hardware architecture model represented by a graph; and performing delay performance analysis on the single operators and the fusion operators in various combination forms by using a pipeline analysis method to obtain the performance of the operators under a hardware architecture, and further performing calculation optimization acceleration. By the adoption of the technical scheme, sufficient guidance can be provided for the initial stage of optimization of the hardware architecture, guidance is provided for optimization space exploration of operators, and therefore the calculation performance of intelligent chip hardware is improved.

The invention discloses an operator execution method and device, computer equipment and a storage medium, and belongs to the technical field of artificial intelligence, in the method, after an output result of first precision of a first operation in a fusion operator is obtained, the output result is stored in a TLR array according to a preset data arrangement mode, and the output result of the first precision of the first operation in the fusion operator is obtained; and converting the data in the TLR array from the first precision to a second precision based on a preset data arrangement mode, the second precision being an input precision corresponding to a second operation in the fusion operator, and after determining that the converted data meets a data arrangement requirement corresponding to the second operation, processing the data through each thread to obtain a calculation result of the fusion operator, the processing is determined according to the category of the second operation. Thus, the data arrangement mode of the output result of the previous operation in the fusion operator during output is changed, the data arrangement requirement corresponding to the next operation can be met after precision conversion, data exchange between threads is not needed, hardware overhead is small, and therefore the performance of the fusion operator can be improved.

A fast code generation device supporting the generation of fusion operators, belonging to the technical field of deep learning. The present invention includes: an LDM area division module for functionally partitioning the local storage space according to the network size parameters input by the upper-layer framework; a fusion operator address configuration module for defining the addresses of the input, output, and intermediate result data in the operator in the functional partition according to the type of fusion operator input by the upper-layer framework; a fusion operator data interaction module providing function interfaces for asynchronous memory access between the local and the main memory, and between the local and the local; a SIMD fusion operator calculation module for fusing the operator according to the addresses generated by the fusion operator address configuration module; and a systolic array instruction configuration module for configuring the instructions for driving the systolic array to perform calculations. The present invention can effectively reduce the code error rate, improve the code generation efficiency, and simplify the debugging process.

The invention provides a distributed computing method and system and related equipment. The method comprises the steps that a management node issues a fusion operator to a computing node; the fusion operator comprises two matrixes and an instruction for realizing matrix multiplication operation of the two matrixes; a plurality of matrix calculation units in the calculation node respectively execute part of the two matrix multiplication operations according to the fusion operator to obtain part of calculation results of the matrix multiplication operations; and executing a communication task to synchronize the partial calculation result. A management node only needs to send a fusion operator to a calculation node, the calculation node can complete calculation of a matrix multiplication according to the fusion operator, and the management node does not need to split the matrix multiplication operation into a plurality of subtasks for calculation and communication and then send the subtasks to the calculation node in sequence. The phenomenon that computing node resources are idle due to management node splitting and task issuing can be avoided, and the utilization rate of the resources and the efficiency of the distributed computing system are improved.

The present invention discloses an online automatic tuning and compilation system and method for large-scale model training, which belongs to the field of computing framework programming and compilers. The system includes an automatic tuning client, an automatic tuning server, a configuration manager, a Triton operator register, a communication manager, an automatic tuning scheduler, and a data collector. The present invention improves the computing performance of the fusion operator by searching online for the optimal hyperparameter configuration of the fusion operator during large-scale model training, thereby improving the efficiency of utilizing GPU hardware resources during large-scale model training. The online automatic tuning and compilation system of the present invention provides a search for a larger hyperparameter space, and quickly finds the optimal hyperparameter configuration of the fusion operator for end-to-end performance during large-scale model training.

The invention relates to a neural network with a fusion operator, which comprises convolution operation and element-by-element operation, and the convolution operation comprises convolution sub-operation and bias sub-operation; the neural network is deployed in the artificial intelligence accelerator and comprises a convolution unit used for executing convolution sub-operation and a single-point processing unit used for executing bias sub-operation and element-by-element operation; the single-point processing unit comprises a plurality of sub-module circuits which are connected in sequence and are configured by respective configuration parameters, and the configuration parameters of the sub-module circuits are determined according to the following modes: determining an operation relation of fusion operation of fusion bias sub-operation and element-by-element operation based on an operation relation of convolution operation and element-by-element operation; determining configuration parameters of the sub-module circuits based on the operation relation of the fusion operation, and configuring the sub-module circuits based on the configuration parameters; the convolution unit and the single-point processing unit are configured to enable the convolution unit and the single-point processing unit to perform convolution sub-operations and fusion operations in a pipelined mode.

The present invention discloses a method and device for end-cloud collaborative reasoning for neural network operator fusion, the method comprising converting a neural network into a directed acyclic graph; dividing the directed acyclic graph into a plurality of chain structures; fusing the network layers in each chain structure, and replacing the fused network layers with the obtained fusion blocks; predicting the inference time and output data size of each fusion block and each unfused network layer based on the data to be inferred, and calculating the intermediate data transmission time based on the output data size and the network bandwidth between the end and the cloud; segmenting the neural network based on the inference time and the intermediate data transmission time, and performing end-cloud collaborative reasoning based on the segmentation results. The present invention solves the minimum latency problem of network models with fusionable operators.

The database query optimization method comprises the following steps: obtaining a query statement containing a connection operation and a deduplication operation; determining that a column used by the connection operation contains a column used by a target operation in the query statement, wherein the target operation is an operation needing to be executed after the connection operation; creating a fusion operator, wherein the fusion operator is used for completing a connection operation and a deduplication operation; when the cost for executing the fusion operator is smaller than or equal to a target cost, an optimal execution plan related to the fusion operator is generated according to the fusion operator, the target cost is the cost for executing a connection operator and a deduplication operator, the connection operator is used for completing connection operation, and the deduplication operator is used for completing deduplication operation; and executing operations step by step according to the operation sequence in the optimal execution plan to obtain a query result set. Therefore, when the query statement meets the corresponding characteristics, the connection and deduplication operations can be fused into one operator, so that the processing flow of an intermediate result can be reduced, the execution of the query statement is accelerated, and the execution performance of the query is improved.

This application discloses a method, apparatus, device, medium, and product for multi-layer convolution operator fusion optimization, relating to the field of compiler optimization technology. The method includes: determining the original computation graph of the model to be deployed; determining multiple fusionable operator subgraphs in the original computation graph based on a hardware performance model; optimizing the original computation graph based on the fusionable operator subgraphs to obtain an optimized computation graph; obtaining parallel optimization code for the model to be deployed based on the optimized computation graph and the slice size; and running the parallel optimization code on the target machine to obtain the optimized performance of the model to be deployed. This application improves the performance of the target machine when executing a neural network model by optimizing the original computation graph based on fusionable operator subgraphs.

The invention discloses an online automatic tuning and compiling system and method for large model training, and belongs to the field of computing framework programming and compilers. The system comprises an automatic tuning client, an automatic tuning server, a configuration manager, a Triton operator register, a communication manager, an automatic tuning scheduler and a data collector. According to the method, the calculation performance of the fusion operator is improved by searching the optimal hyper-parameter configuration of the fusion operator in the large model training process on line, so that the efficiency of utilizing GPU hardware resources in the large model training process is improved; according to the online automatic tuning and compiling system, a larger hyper-parameter space is provided for searching, and the optimal hyper-parameter configuration of the fusion operator is quickly searched for aiming at the end-to-end performance in the large model training process.

This application provides a chip-based fusion operator processing method, apparatus, and related products, specifically applied in the field of data processing technology. The scheme includes: sequentially loading matrix units from the chip's storage device onto the chip's computing device, wherein the matrix units are obtained by splitting the target matrix according to a preset matrix unit dimension; iteratively calculating and summing the normalized denominators of each matrix unit using the computing device, determining the TopK value of each matrix unit, and iteratively processing until the sum of the normalized denominators of all matrix units in the target matrix and the corresponding TopK value of the target matrix are obtained; calculating the normalized numerator of each TopK value based on the corresponding TopK value of the target matrix using the computing device; and obtaining the processing result of the fusion operator for the target matrix based on the sum of the normalized denominators of the target matrix and the normalized numerator of each TopK value using the computing device.

The invention provides an internal combustion engine fault diagnosis method and system based on vibration characteristic white box fusion. The method comprises the steps that vibration signals and crank angle pulse signals of a to-be-tested diesel engine in normal states and different fault states under different rotating speeds and loads are synchronously collected; establishing a fault data set, wherein data collected in a normal state and a fault state are respectively marked; a plurality of time domain features and frequency domain features of the vibration signals are calculated, weighted fusion operators are constructed, and the weighted fusion operators are multiplied, combined, expanded and integrated into an integrated feature; a gradient descent algorithm is used, minimization of dichotomy cross entropy loss is taken as a target, the weight of each feature in the weighted fusion operator is optimized, and optimized fusion features strongly related to the fault state are fused; and determining a normal-fault distribution overlapping region, setting a classification threshold value in the overlapping region, and comparing the value of the optimized fusion feature with the classification threshold value to judge the working state of the engine.

The invention discloses an asymmetric filling and convolution operator fusion optimization method and system based on a mask mechanism, and belongs to the technical field of performance optimization of a universal graphics processor. The objective of the invention is to solve the problem of deep combination of an operator fusion technology in a high-level computational graph and a mask mechanism in a bottom kernel function. The method comprises the following steps: defining a Pytorch original model containing filling and convolution calculation modes to obtain a calculation graph; triggering a Pytorch-based graph optimization operator fusion method, namely traversing the computational graph to perform pattern matching, performing graph rewriting on the computational graph with a successfully matched pattern, and replacing original operator nodes in the computational graph with fusion operator nodes to obtain an optimized computational graph; and calling a fusion operator based on the asymmetric filling convolution of the mask mechanism, updating fusion nodes in the optimized calculation graph, obtaining the calculation graph fused and optimized by the asymmetric filling convolution operator based on the mask mechanism, and optimizing the reasoning performance under partial scenes of the GPGPU.

This application relates to a method for generating cross-architecture computationally intensive fusion operators based on hardware abstraction. It constrains and determines the operator subgraph according to preset legality rules, establishes a unified semantic abstraction, defines the fusion operator as a coordinate-driven pure function, and encapsulates auxiliary tensor memory access handles and layout geometry information through runtime context. A backend normalization layer is used to semantically reinterpret the accumulator layout and thread organization of different hardware vendors during compile time, and cross-platform vectorization primitives are invoked to mask the differences in backend instruction set architectures. Arithmetic logic is downgraded to a sequence of instructions executed within a scalar scope. The abstract vector view is first decomposed into independent scalar elements, arithmetic instructions are generated in the scalar dimension, the calculation results are then reorganized into a vector register, and finally, an address expression is constructed by deriving the effective step size to handle broadcast and indexing logic, generating fusion code that can access the underlying vendor's computationally intensive kernel, thus improving development efficiency.

The application discloses a hardware characteristic related deep model calculation graph automatic optimization method, comprising the following steps: converting a deep learning model into an intermediate representation according to a preset rule; performing first operator fusion on the intermediate representation according to a static operator fusion rule to form a composite subgraph; performing parallelization processing on the operators in the composite subgraph according to hardware resources, performing second operator fusion, and forming an aggregated subgraph; performing cross-boundary optimization processing on the aggregated subgraph; forming a list of original operators and fused operators according to the records of the first operator fusion and the second operator fusion; constructing a cost model on a target hardware platform based on the list of original operators and fused operators, and evaluating the fusion effect; and according to the evaluation result of the fusion effect, the fused operators are split, and the generated subgraph is used as the cut-in of the operator layer to generate code.

The present invention discloses a method, apparatus, device and medium for implementing a fusion operator. The method includes: performing semantic conversion on a high-level intermediate expression fusion operator on a multi-level storage chip to obtain a self-owned semantic fusion operator; performing algebraic optimization on the self-owned semantic fusion operator to obtain an optimized fusion operator; obtaining a cache size, a data flow loop polyhedron structure and a computation flow corresponding to the optimized fusion operator; combining the cache size, the data flow loop polyhedron structure and the computation flow to obtain underlying code, and implementing the optimized fusion operator by running the underlying code. By combining the obtained cache size, data flow loop polyhedron structure and computation flow through the information in the optimized fusion operator, the underlying code corresponding to the fusion operator is automatically determined, and the fusion operator is implemented by running the underlying code without manual writing by the user, thereby improving the efficiency and accuracy of implementing the fusion operator.

An operator processing method, an operator processing apparatus, an electronic device, and a computer-readable storage medium are disclosed. The operator processing method includes: acquiring a computation graph containing M operator nodes and the connection relationships between the M operator nodes, each operator node corresponding to at least one operator; splitting the computation graph into N subgraphs, each subgraph containing at least one operator node, the N subgraphs including a first subgraph, the first subgraph corresponding to K first fusion operators, the K first fusion operators being operators obtained by fusing multiple operators corresponding to multiple operator nodes included in the first subgraph; determining the operator codes corresponding to each of the N subgraphs, the operator codes corresponding to each of the N subgraphs including the operator codes of the K first fusion operators corresponding to the first subgraph; and obtaining code for executing the computation process based on the operator codes corresponding to each of the N subgraphs. This method is applicable to various scenarios, has a wide range of applications, and strong generalization ability.

The invention relates to a mercuric chloride NPU operator automatic tuning method and system based on fusion operator performance modeling, and the method comprises the steps: firstly constructing an end-to-end performance analysis model of a fusion operator for collaborative execution of a Cube core and a Vector core, and employing two modes to accurately predict execution delays under different parameter configurations through dynamic discrimination of a calculation speed relation between double cores; on the basis, a heuristic design space exploration algorithm with minimization of data movement amount as guidance is provided, and the algorithm is combined with the multilevel storage characteristic of the mercuric chloride NPU, adopts a two-stage strategy of block size optimization and buffer area allocation collaboration, and automatically searches for optimal parameter configuration meeting on-chip storage capacity constraints. According to the method, automatic performance adjustment and optimization of the fusion operator are realized, compared with traditional manual adjustment and optimization or compiler default configuration, near-optimal configuration can be found within a few minutes, the average performance prediction error is lower than 5%, the maximum performance speed-up ratio of part of operators can reach 1.46 times, and the efficiency of operator development and deployment on the mercuric chloride NPU is remarkably improved.

The embodiments of the present application provide a register resource allocation method, device, storage medium and program product, which relate to the field of artificial intelligence technology. The method includes: dividing the total register resources into multiple types of register buffers. Based on the operator combination corresponding to the fusion operator in each computing scenario, the resource amounts of the multiple types of register buffers allocated to the fusion operator are determined, so as to realize targeted register resource allocation for different computing scenarios of the fusion operator, thereby making full use of the high-speed on-chip storage resource of the register and greatly improving the performance of the fusion operator. Secondly, since the present application flexibly adjusts the resource ratios of the multiple types of register buffers for different computing scenarios of the fusion operator, the overall computing efficiency of the fusion operator in the generalized scenario is improved, and it has high performance and generalization versatility.

The present application relates to a fusion operator testing method, device, computer equipment, storage medium and computer program product. The method includes: obtaining a target graph structure of a deep learning model to be constructed, and obtaining a historical graph structure of a historical deep learning model corresponding to the deep learning model to be constructed; determining a fusion operator in the target graph structure, and a historical operator corresponding to the fusion operator in the historical graph structure; inputting a first input parameter associated with the deep learning model to be constructed into the fusion operator to obtain a first output result and a first output time corresponding to the first output result, and inputting the first input parameter into the historical operator to obtain a second output result and a second output time corresponding to the second output result; determining the acceleration ratio of the fusion operator based on the first output time and the second output time; and determining the first target test result of the fusion operator based on the acceleration ratio. The present method can improve the test efficiency of the fusion operator.

The application provides an operator fusion method for a deep learning inference task compiler, comprising the following steps: converting a standard model of a deep learning inference task model into an initial graph-level IR; traversing the initial graph-level IR, and identifying fusible operators in the initial graph-level IR as homogenous operators or heterogeneous operators; converting the initial graph-level IR into a tensor expression, and fusing the homogenous operators in the tensor expression; compiling the tensor expression into kernel source code, and fusing the heterogeneous operators in the kernel source code; and compiling the kernel source code into executable code and deploying and running the executable code. The application also provides an operator fusion system for a deep learning inference task compiler, and a data processing device for operator fusion of a deep learning inference task compiler.

The invention discloses a feature extraction method based on a 2T0C DRAM-1T1R RRAM fusion operator, and belongs to the field of novel in-memory computing technology and three-dimensional point cloud recognition. According to the method, a point cloud feature extraction array is formed based on a 1T1R RRAM array, matrix vector multiplication calculation is accelerated in situ, point cloud data is converted from a floating-point number digital domain to an analog domain through a 2T0CDRAM-1T1R RRAM fusion operator, matrix vector multiplication is accelerated through calculation in an analog domain memory, and all point cloud features are extracted. According to the method, high data conversion overhead does not need to be introduced, organic combination of two in-memory computing methods is realized, the system overhead is further reduced, and universal and efficient hardware acceleration basic configuration is provided for multiple point cloud neural network algorithms.