60 results about "Fusion operator" patented technology

The invention relates to the technical field of artificial intelligence chips, and provides an operator fusion method and device, electronic equipment and a storage medium, and the method comprises the steps: determining an input tensor, an output tensor and a coverage range of a to-be-fused operator, and the to-be-fused operator comprises a plurality of view operators; on the basis of the metadata of the input tensor, the metadata of the output tensor and the logic relation between the view operators in the coverage range, determining the mapping relation between the dimensions of the input tensor and the output tensor; and performing data filling on the output tensor based on the mapping relationship and each data in the input tensor, and taking the filled output tensor as a fusion result of the operator to be fused. According to the method, multiple continuous view operators are efficiently and quickly fused, so that the problem of performance reduction caused by frequent data handling in the model is effectively solved.

The invention belongs to the technical field of artificial intelligence and high-performance computing, and particularly relates to a method for improving the reasoning solution and training speed of a deep operator network model. According to the method, multi-level optimization is carried out on the PI-DeepONet model, and the reasoning solving speed and the training speed of the PI-DeepONet model are improved. According to the method, the SGEMM operator is customized, and compared with the best single-precision matrix multiplication in a basic linear algebraic library of a CUDA (Compute Unified Device Architecture) computing platform, the SGEMM operator can obtain a speed-up ratio of 1.5 times at most in a common model size range of PI-DeepONet. A fusion operator is designed according to the special structure of the modified MLP used in the model, the speed-up ratio of the end-to-end reasoning speed of the model is more than two times, and the calculation time of a graphics processor can be shortened by 5.7 times at most. According to the method, data parallel is used for carrying out distributed training on the model, a communication concealment strategy is used, and the expandability is kept at 83%-85%.

A database query optimization method, comprising: acquiring a query statement, the query statement comprising a connection operation and a group aggregation operation; determining that a connection key and a group key in the query statement are the same, wherein an aggregation function in the query statement can be split, and an input parameter of the aggregation function comes from the same side as a connection condition in the query statement; creating a fusion operator, the fusion operator being used for completing the connection operation and the group aggregation operation; when a cost for executing the fusion operator is less than or equal to a target cost, on the basis of the fusion operator, generating an optimal execution plan related to the fusion operator, wherein the target cost is a cost for executing a connection operator and a group aggregation operator; and performing the operations step by step on the basis of an operation sequence in the optimal execution plan, so as to obtain a query result set. Therefore, when the query statement meets corresponding features, the connection operation and the group aggregation operation can be fused into one operator, thereby reducing the processing flow of an intermediate result, accelerating the execution of the query statement, and improving the query execution performance.

The database query optimization method comprises the steps that a query statement is obtained, and the query statement comprises a connection operation and a grouping aggregation operation; determining that a connection key and a grouping key in the query statement are the same, an aggregation function in the query statement is detachable, and input parameters of the aggregation function are from the same side of a connection condition in the query statement; creating a fusion operator, wherein the fusion operator is used for completing a connection operation and a grouping aggregation operation; under the condition that the cost of executing the fusion operator is smaller than or equal to a target cost, an optimal execution plan related to the fusion operator is generated according to the fusion operator, and the target cost is the cost of executing the connection operator and the grouping aggregation operator; and executing operations step by step according to the operation sequence in the optimal execution plan to obtain a query result set. Therefore, when the query statement meets the corresponding characteristics, the connection and grouping aggregation operations can be fused into one operator, so that the processing flow of an intermediate result can be reduced, the execution of the query statement is accelerated, and the execution performance of the query is improved.

The invention relates to the technical field of electronics, and discloses a deep learning operator optimization and fusion operator calculation instruction generation method and device.The deep learning operator optimization method comprises the steps that all nodes in a deep learning calculation graph are traversed to obtain at least one target node set, the target node set comprises multiple target nodes, and the target nodes in the target node set are extracted; the operator corresponding to each target node is a target operator of a target type, and different target operators have an association relationship; and fusing the operators corresponding to different target nodes in the target node set to obtain a fused operator. The target nodes which have the incidence relation in the computational graph and the corresponding operators are the target type operators are used as a set, the target operators corresponding to the target nodes in the same set are fused, the fused operators can be fully optimized in the aspect of specific instruction implementation, and the execution efficiency of the operators is improved.

The invention relates to the technical field of artificial intelligence, and provides a fusion operator execution method, electronic equipment, a storage medium and a program product, and the method comprises the steps: determining a plurality of to-be-fused target operators; according to the tensors of the multiple target operators, a pseudo address mapping table is generated, and the pseudo address mapping table is used for recording the mapping relation between the actual memory addresses of the tensors and pseudo addresses; operator fusion is carried out on the multiple target operators, a fusion operator is obtained, and the memory address of the tensor of the fusion operator is a continuous pseudo address; and executing the fusion operator, converting the pseudo address into an actual memory address according to the pseudo address mapping table in the execution process, and accessing data of the tensor according to the actual memory address. According to the method, a large amount of extra memory copy overhead caused by physical movement and rearrangement of the original tensor in a traditional operator fusion scheme is avoided, and the computing resource utilization rate and the execution efficiency are remarkably improved.

The invention provides a method and device for reasoning by using an open neural network switching ONNX model based on a fusion operator, electronic equipment and a medium. The method comprises the steps that shape information of an input tensor of the ONNX model is obtained, the shape information is a vector with the dimension being N, each element of the vector indicates each dimension of the input tensor, and N is larger than or equal to 1; generating an input shape vector based on the shape information, the input shape vector being a vector with a dimension of N + 1; the fusion operator is used for operating the input shape vector to obtain a changed shape vector, the fusion operator is fused with at least one ONNX operator, the fusion operator is based on a coefficient matrix, and one dimension of the coefficient matrix is N + 1; and based on the changed shape vector, reasoning is performed by using the ONNX model.

The invention discloses a modeling analysis and calculation optimization method for operator performance on an intelligent chip. The modeling analysis and calculation optimization method comprises the following steps: carrying out graph representation modeling on an intelligent chip hardware architecture and an interconnection mode; designing a link mechanism of operator loop tree nodes, so that operator loops are described and mapped in a hardware architecture model represented by a graph; and performing delay performance analysis on the single operators and the fusion operators in various combination forms by using a pipeline analysis method to obtain the performance of the operators under a hardware architecture, and further performing calculation optimization acceleration. By the adoption of the technical scheme, sufficient guidance can be provided for the initial stage of optimization of the hardware architecture, guidance is provided for optimization space exploration of operators, and therefore the calculation performance of intelligent chip hardware is improved.

The invention discloses an operator execution method and device, computer equipment and a storage medium, and belongs to the technical field of artificial intelligence, in the method, after an output result of first precision of a first operation in a fusion operator is obtained, the output result is stored in a TLR array according to a preset data arrangement mode, and the output result of the first precision of the first operation in the fusion operator is obtained; and converting the data in the TLR array from the first precision to a second precision based on a preset data arrangement mode, the second precision being an input precision corresponding to a second operation in the fusion operator, and after determining that the converted data meets a data arrangement requirement corresponding to the second operation, processing the data through each thread to obtain a calculation result of the fusion operator, the processing is determined according to the category of the second operation. Thus, the data arrangement mode of the output result of the previous operation in the fusion operator during output is changed, the data arrangement requirement corresponding to the next operation can be met after precision conversion, data exchange between threads is not needed, hardware overhead is small, and therefore the performance of the fusion operator can be improved.

The invention relates to an operator fusion method based on a Shenwei processor, and belongs to the technical field of operator fusion, and the method comprises the steps: obtaining an operator of a computational graph in a deep learning model, analyzing a data access mode of the operator, determining a to-be-fused operator meeting a fusion condition according to a preset fusion judgment mechanism, and obtaining a feature graph of the to-be-fused operator, performing data processing on the feature map by using an operator to be fused, pre-loading parameters of the operator to be fused into a local data memory of a Shenwei processor, judging the total data flow of the feature map and the capacity of the local data memory, dividing data blocks of the feature map according to a judgment result, and storing the divided data blocks in the Shenwei processor; the data blocks are used for completing operation of the operators to be fused, and the operated data blocks are output, so that frequent transmission of data between the processor and the external memory is reduced, the computing resources of the Shenwei processor can be fully utilized, the parallel computing efficiency is improved, the memory access mode is optimized, and the bottleneck of memory bandwidth is reduced.

The invention discloses a multilayer convolution operator fusion optimization method and device, equipment, a medium and a product, and relates to the technical field of compilation optimization, and the method comprises the steps: determining an original calculation graph of a to-be-deployed model, and determining a plurality of fusible operator sub-graphs in the original calculation graph based on a hardware performance model; optimizing the original calculation graph based on the fusible operator sub-graph to obtain an optimized calculation graph; obtaining a parallel optimization code of the to-be-deployed model based on the optimization calculation graph and the fragment size; the target machine runs the parallel optimization code to obtain the optimization performance of the to-be-deployed model; according to the method, the original calculation graph is optimized based on the fusible operator sub-graph, so that the performance of the target machine during execution of the neural network model can be improved.

A fast code generation device supporting the generation of fusion operators, belonging to the technical field of deep learning. The present invention includes: an LDM area division module for functionally partitioning the local storage space according to the network size parameters input by the upper-layer framework; a fusion operator address configuration module for defining the addresses of the input, output, and intermediate result data in the operator in the functional partition according to the type of fusion operator input by the upper-layer framework; a fusion operator data interaction module providing function interfaces for asynchronous memory access between the local and the main memory, and between the local and the local; a SIMD fusion operator calculation module for fusing the operator according to the addresses generated by the fusion operator address configuration module; and a systolic array instruction configuration module for configuring the instructions for driving the systolic array to perform calculations. The present invention can effectively reduce the code error rate, improve the code generation efficiency, and simplify the debugging process.

The invention provides a distributed computing method and system and related equipment. The method comprises the steps that a management node issues a fusion operator to a computing node; the fusion operator comprises two matrixes and an instruction for realizing matrix multiplication operation of the two matrixes; a plurality of matrix calculation units in the calculation node respectively execute part of the two matrix multiplication operations according to the fusion operator to obtain part of calculation results of the matrix multiplication operations; and executing a communication task to synchronize the partial calculation result. A management node only needs to send a fusion operator to a calculation node, the calculation node can complete calculation of a matrix multiplication according to the fusion operator, and the management node does not need to split the matrix multiplication operation into a plurality of subtasks for calculation and communication and then send the subtasks to the calculation node in sequence. The phenomenon that computing node resources are idle due to management node splitting and task issuing can be avoided, and the utilization rate of the resources and the efficiency of the distributed computing system are improved.

The present invention discloses an online automatic tuning and compilation system and method for large-scale model training, which belongs to the field of computing framework programming and compilers. The system includes an automatic tuning client, an automatic tuning server, a configuration manager, a Triton operator register, a communication manager, an automatic tuning scheduler, and a data collector. The present invention improves the computing performance of the fusion operator by searching online for the optimal hyperparameter configuration of the fusion operator during large-scale model training, thereby improving the efficiency of utilizing GPU hardware resources during large-scale model training. The online automatic tuning and compilation system of the present invention provides a search for a larger hyperparameter space, and quickly finds the optimal hyperparameter configuration of the fusion operator for end-to-end performance during large-scale model training.

The invention relates to a neural network with a fusion operator, which comprises convolution operation and element-by-element operation, and the convolution operation comprises convolution sub-operation and bias sub-operation; the neural network is deployed in the artificial intelligence accelerator and comprises a convolution unit used for executing convolution sub-operation and a single-point processing unit used for executing bias sub-operation and element-by-element operation; the single-point processing unit comprises a plurality of sub-module circuits which are connected in sequence and are configured by respective configuration parameters, and the configuration parameters of the sub-module circuits are determined according to the following modes: determining an operation relation of fusion operation of fusion bias sub-operation and element-by-element operation based on an operation relation of convolution operation and element-by-element operation; determining configuration parameters of the sub-module circuits based on the operation relation of the fusion operation, and configuring the sub-module circuits based on the configuration parameters; the convolution unit and the single-point processing unit are configured to enable the convolution unit and the single-point processing unit to perform convolution sub-operations and fusion operations in a pipelined mode.

The present invention discloses a method and device for end-cloud collaborative reasoning for neural network operator fusion, the method comprising converting a neural network into a directed acyclic graph; dividing the directed acyclic graph into a plurality of chain structures; fusing the network layers in each chain structure, and replacing the fused network layers with the obtained fusion blocks; predicting the inference time and output data size of each fusion block and each unfused network layer based on the data to be inferred, and calculating the intermediate data transmission time based on the output data size and the network bandwidth between the end and the cloud; segmenting the neural network based on the inference time and the intermediate data transmission time, and performing end-cloud collaborative reasoning based on the segmentation results. The present invention solves the minimum latency problem of network models with fusionable operators.

The database query optimization method comprises the following steps: obtaining a query statement containing a connection operation and a deduplication operation; determining that a column used by the connection operation contains a column used by a target operation in the query statement, wherein the target operation is an operation needing to be executed after the connection operation; creating a fusion operator, wherein the fusion operator is used for completing a connection operation and a deduplication operation; when the cost for executing the fusion operator is smaller than or equal to a target cost, an optimal execution plan related to the fusion operator is generated according to the fusion operator, the target cost is the cost for executing a connection operator and a deduplication operator, the connection operator is used for completing connection operation, and the deduplication operator is used for completing deduplication operation; and executing operations step by step according to the operation sequence in the optimal execution plan to obtain a query result set. Therefore, when the query statement meets the corresponding characteristics, the connection and deduplication operations can be fused into one operator, so that the processing flow of an intermediate result can be reduced, the execution of the query statement is accelerated, and the execution performance of the query is improved.

This application discloses a method, apparatus, device, medium, and product for multi-layer convolution operator fusion optimization, relating to the field of compiler optimization technology. The method includes: determining the original computation graph of the model to be deployed; determining multiple fusionable operator subgraphs in the original computation graph based on a hardware performance model; optimizing the original computation graph based on the fusionable operator subgraphs to obtain an optimized computation graph; obtaining parallel optimization code for the model to be deployed based on the optimized computation graph and the slice size; and running the parallel optimization code on the target machine to obtain the optimized performance of the model to be deployed. This application improves the performance of the target machine when executing a neural network model by optimizing the original computation graph based on fusionable operator subgraphs.

The invention discloses an online automatic tuning and compiling system and method for large model training, and belongs to the field of computing framework programming and compilers. The system comprises an automatic tuning client, an automatic tuning server, a configuration manager, a Triton operator register, a communication manager, an automatic tuning scheduler and a data collector. According to the method, the calculation performance of the fusion operator is improved by searching the optimal hyper-parameter configuration of the fusion operator in the large model training process on line, so that the efficiency of utilizing GPU hardware resources in the large model training process is improved; according to the online automatic tuning and compiling system, a larger hyper-parameter space is provided for searching, and the optimal hyper-parameter configuration of the fusion operator is quickly searched for aiming at the end-to-end performance in the large model training process.

The invention relates to the technical field of artificial intelligence, particularly provides a data processing method for a neural network model, a server, equipment and a medium, and aims to solve the problem of how to carry out attention calculation on the neural network model based on a CPU of mobile equipment. The method provided by the invention comprises the steps of obtaining a first calculation graph according to a model file of a neural network model, wherein the first calculation graph is used for describing all operators of the neural network model and a connection relationship among the operators; obtaining a plurality of target operators in the first calculation graph, wherein the plurality of target operators are a plurality of operators used for carrying out attention calculation by a Transform module; the calculation operations corresponding to the multiple target operators are fused to form a fusion operator, and the fusion operator is used for a CPU of the mobile device to execute the calculation operations; and replacing the plurality of target operators with the fusion operator to obtain a second calculation graph. Through the method, the calculation operation corresponding to the fusion operator can be executed based on the CPU, the attention calculation speed is increased, and therefore the model reasoning speed is increased.

The invention relates to the technical field of artificial intelligence, and provides an operator fusion method and system, electronic equipment and a storage medium, and the method comprises the steps: receiving the operator number of to-be-fused operators and an equipment function identifier sent by a host, and the equipment function identifier is determined by the host based on the operator identifier of the to-be-fused operators; based on the device function identifier, loading a device function corresponding to the to-be-fused operator, the device function being developed in advance based on the to-be-fused operator, and being compiled and loaded into a memory of the device; and fusing the operators to be fused based on the number of the operators and the equipment function. According to the invention, automatic fusion during operator operation can be realized, the development amount of operator fusion is greatly reduced, and the efficiency of operator fusion is improved.

This application provides a chip-based fusion operator processing method, apparatus, and related products, specifically applied in the field of data processing technology. The scheme includes: sequentially loading matrix units from the chip's storage device onto the chip's computing device, wherein the matrix units are obtained by splitting the target matrix according to a preset matrix unit dimension; iteratively calculating and summing the normalized denominators of each matrix unit using the computing device, determining the TopK value of each matrix unit, and iteratively processing until the sum of the normalized denominators of all matrix units in the target matrix and the corresponding TopK value of the target matrix are obtained; calculating the normalized numerator of each TopK value based on the corresponding TopK value of the target matrix using the computing device; and obtaining the processing result of the fusion operator for the target matrix based on the sum of the normalized denominators of the target matrix and the normalized numerator of each TopK value using the computing device.

The invention provides an internal combustion engine fault diagnosis method and system based on vibration characteristic white box fusion. The method comprises the steps that vibration signals and crank angle pulse signals of a to-be-tested diesel engine in normal states and different fault states under different rotating speeds and loads are synchronously collected; establishing a fault data set, wherein data collected in a normal state and a fault state are respectively marked; a plurality of time domain features and frequency domain features of the vibration signals are calculated, weighted fusion operators are constructed, and the weighted fusion operators are multiplied, combined, expanded and integrated into an integrated feature; a gradient descent algorithm is used, minimization of dichotomy cross entropy loss is taken as a target, the weight of each feature in the weighted fusion operator is optimized, and optimized fusion features strongly related to the fault state are fused; and determining a normal-fault distribution overlapping region, setting a classification threshold value in the overlapping region, and comparing the value of the optimized fusion feature with the classification threshold value to judge the working state of the engine.

The invention discloses a compilation optimization method and device of a computational graph based on a multi-level storage chip. The method comprises the following steps: screening calculation graphs running on a multi-level storage chip to obtain candidate operator nodes; obtaining the dynamic type of each current candidate operator node; performing forward search and backward search on candidate operator nodes in the computational graph according to the dynamic types to obtain fusion operator nodes, updating the computational graph according to the fusion operator nodes to obtain an optimized computational graph, and screening out the candidate operator nodes meeting fusion conditions from the computational graph and the dynamic types of the candidate operator nodes to obtain a fusion result. According to the method, the candidate operator nodes are selected, the candidate operator nodes are searched according to the dynamic types to obtain the fusion operator nodes, and the candidate operator nodes located in the same fusion operator node only carry out tensor carrying in the shared storage layer, so that the carrying frequency of tensor data among different storage layers is avoided, and the data processing efficiency and the calculation performance of the computational graph are improved.

The invention discloses an asymmetric filling and convolution operator fusion optimization method and system based on a mask mechanism, and belongs to the technical field of performance optimization of a universal graphics processor. The objective of the invention is to solve the problem of deep combination of an operator fusion technology in a high-level computational graph and a mask mechanism in a bottom kernel function. The method comprises the following steps: defining a Pytorch original model containing filling and convolution calculation modes to obtain a calculation graph; triggering a Pytorch-based graph optimization operator fusion method, namely traversing the computational graph to perform pattern matching, performing graph rewriting on the computational graph with a successfully matched pattern, and replacing original operator nodes in the computational graph with fusion operator nodes to obtain an optimized computational graph; and calling a fusion operator based on the asymmetric filling convolution of the mask mechanism, updating fusion nodes in the optimized calculation graph, obtaining the calculation graph fused and optimized by the asymmetric filling convolution operator based on the mask mechanism, and optimizing the reasoning performance under partial scenes of the GPGPU.

This application relates to a method for generating cross-architecture computationally intensive fusion operators based on hardware abstraction. It constrains and determines the operator subgraph according to preset legality rules, establishes a unified semantic abstraction, defines the fusion operator as a coordinate-driven pure function, and encapsulates auxiliary tensor memory access handles and layout geometry information through runtime context. A backend normalization layer is used to semantically reinterpret the accumulator layout and thread organization of different hardware vendors during compile time, and cross-platform vectorization primitives are invoked to mask the differences in backend instruction set architectures. Arithmetic logic is downgraded to a sequence of instructions executed within a scalar scope. The abstract vector view is first decomposed into independent scalar elements, arithmetic instructions are generated in the scalar dimension, the calculation results are then reorganized into a vector register, and finally, an address expression is constructed by deriving the effective step size to handle broadcast and indexing logic, generating fusion code that can access the underlying vendor's computationally intensive kernel, thus improving development efficiency.

The application discloses a hardware characteristic related deep model calculation graph automatic optimization method, comprising the following steps: converting a deep learning model into an intermediate representation according to a preset rule; performing first operator fusion on the intermediate representation according to a static operator fusion rule to form a composite subgraph; performing parallelization processing on the operators in the composite subgraph according to hardware resources, performing second operator fusion, and forming an aggregated subgraph; performing cross-boundary optimization processing on the aggregated subgraph; forming a list of original operators and fused operators according to the records of the first operator fusion and the second operator fusion; constructing a cost model on a target hardware platform based on the list of original operators and fused operators, and evaluating the fusion effect; and according to the evaluation result of the fusion effect, the fused operators are split, and the generated subgraph is used as the cut-in of the operator layer to generate code.

The present invention discloses a method, apparatus, device and medium for implementing a fusion operator. The method includes: performing semantic conversion on a high-level intermediate expression fusion operator on a multi-level storage chip to obtain a self-owned semantic fusion operator; performing algebraic optimization on the self-owned semantic fusion operator to obtain an optimized fusion operator; obtaining a cache size, a data flow loop polyhedron structure and a computation flow corresponding to the optimized fusion operator; combining the cache size, the data flow loop polyhedron structure and the computation flow to obtain underlying code, and implementing the optimized fusion operator by running the underlying code. By combining the obtained cache size, data flow loop polyhedron structure and computation flow through the information in the optimized fusion operator, the underlying code corresponding to the fusion operator is automatically determined, and the fusion operator is implemented by running the underlying code without manual writing by the user, thereby improving the efficiency and accuracy of implementing the fusion operator.

An operator processing method, an operator processing apparatus, an electronic device, and a computer-readable storage medium are disclosed. The operator processing method includes: acquiring a computation graph containing M operator nodes and the connection relationships between the M operator nodes, each operator node corresponding to at least one operator; splitting the computation graph into N subgraphs, each subgraph containing at least one operator node, the N subgraphs including a first subgraph, the first subgraph corresponding to K first fusion operators, the K first fusion operators being operators obtained by fusing multiple operators corresponding to multiple operator nodes included in the first subgraph; determining the operator codes corresponding to each of the N subgraphs, the operator codes corresponding to each of the N subgraphs including the operator codes of the K first fusion operators corresponding to the first subgraph; and obtaining code for executing the computation process based on the operator codes corresponding to each of the N subgraphs. This method is applicable to various scenarios, has a wide range of applications, and strong generalization ability.