An operator processing method and device based on MindSpore

Abstract

The application provides an operator processing method and device based on MindSpore, and relates to the technical field of data processing. The method comprises the following steps: extracting graph nodes with explicit calculation semantics in a deep learning model, uniformly abstracting them into an independent operator type set, constructing a minimum executable network structure, verifying whether the structure can be successfully identified and intermediate representation generated in the MindSpore framework, further calling a graph compiler to determine whether the back-end graph optimization can be completed, loading the structure to a running environment for execution verification if it can pass, and calculating an error based on forward output and gradient backpropagation results, and identifying the structure as a functionally effective operator if the error is lower than a set tolerance. The application can solve the problem of incomplete or insufficient support of operators in the MindSpore framework during large model training and inference, and guarantee the functional effectiveness and execution consistency of target operators in the graph construction, compilation and running stages.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data processing, and in particular to an operator processing method and device based on MindSpore. BACKGROUND

[0002] MindSpore is a full-scene artificial intelligence computing framework developed by Huawei, which is oriented to cloud, edge and end heterogeneous environments, has the capabilities of automatic differentiation, dynamic graph and static graph fusion, distributed training scheduling, operator fusion optimization, supports multiple hardware platforms such as GPU and CPU, has the characteristics of high performance, high scalability and modular design, can cover large model training, inference deployment and AI engineering process, and is particularly suitable for building a self-contained deep learning application system.

[0003] The operator processing of MindSpore refers to the whole process of system identification, adaptation, completion and optimization of the basic computing unit relied on in the large model inference and training process, mainly including extracting all operator types required by the model, verifying the support of the MindSpore framework for these operators, identifying missing or performance bottleneck operators, and realizing the functional completeness and performance enhancement of the operators through means such as constructing equivalent logic, custom registration and execution optimization, so as to guarantee the efficient operation and stable deployment of large models on MindSpore.

[0004] In MindSpore, the support of an operator not only means that the API interface can be called by the front end, but also requires that the operator can be correctly identified and converted into a unified intermediate representation MindIR in the static graph construction stage, and maintain functional integrity and consistent execution in the subsequent graph compilation, graph optimization and underlying device scheduling process, therefore the verification of the operator must cover the whole process from model graphing to final running, and cannot rely only on the correctness judgment during dynamic execution, otherwise problems such as functional loss, performance degradation or running failure may occur in the static graph compilation or deployment process. SUMMARY

[0005] The present application provides an operator processing method and device based on MindSpore, which can solve the problem of incomplete or insufficient performance of the MindSpore framework in the large model training and inference process, and guarantee the functional effectiveness and execution consistency of the target operator in the graphing, compiling and running stages.

[0006] In a first aspect of the present application, an operator processing method based on MindSpore is provided, which comprises:

[0007] obtaining the complete computation graph of a target deep learning model in the inference and training stages, uniformly abstracting all graph nodes with explicit computing semantics in the computation graph, and extracting an independent operator type set;

[0008] For each target operator in the set of independent operator types, a minimum executable network structure containing the target operator is constructed in the MindSpore framework, it is verified whether the target operator can be successfully recognized by the computation graph construction mechanism of the MindSpore framework and converted into an intermediate representation;

[0009] The intermediate representation is converted into a bottom-layer execution graph by calling the graph compiler of the MindSpore framework, and it is determined whether the target operator can be parsed by the backend compiler and completed graph-level optimization;

[0010] If it is determined that the target operator can be parsed by the backend compiler and completed graph-level optimization, the target operator is loaded into the MindSpore running environment and executed on a specified hardware backend, and the functional output and gradient backpropagation of the target operator are verified by inputting a tensor, and if the output error is lower than a set tolerance, the target operator is identified as a functionally valid operator.

[0011] On the basis of the above technical solutions, preferably, after the functional output and gradient backpropagation of the target operator are verified by inputting a tensor, the method further comprises:

[0012] The execution time of each target operator on different devices is collected;

[0013] The algorithmic utilization rate and performance sensitivity of the target operator are calculated based on the execution time;

[0014] The performance bottleneck map of the target operator is constructed based on the algorithmic utilization rate and performance sensitivity;

[0015] The performance bottleneck map is associated with the MindSpore backend execution log for correlation analysis to identify bottleneck factors affecting the performance of the target operator, including memory bandwidth limitation, operator fusion failure, scheduling delay, or thread concurrency conflict;

[0016] For the target operator identified as a performance anomaly in the performance bottleneck map, an operator-level optimization interface provided by the MindSpore graph optimizer is called to perform optimization operations to generate an optimized execution path of the target operator, the optimization operations including operator fusion reconstruction, memory access mode rearrangement, instruction-level parallelism adjustment, and scheduling priority reordering;

[0017] The optimized execution path is written into the position of the target operator in the bottom-layer execution graph to form an updated bottom-layer execution graph structure, and the updated bottom-layer execution graph structure is reloaded into the MindSpore running environment to obtain the optimized output tensor and optimized gradient tensor of the optimized target operator for verification.

[0018] Based on the above technical scheme, preferably, the calculation of the computing power utilization rate and the performance sensitivity of the target operator based on the execution time specifically includes:

[0019] The computing power utilization rate of the target operator is calculated based on the execution time and the theoretical calculation complexity of the target operator, and the theoretical calculation complexity is calculated according to the input tensor dimension and operation type of the target operator.

[0020] Based on the change trend of the execution time proportion of the target operator under different input tensor scales and concurrent running conditions, a performance response model of the target operator is fitted and constructed, a response coefficient of the change trend to the overall graph execution time change is calculated, and the response coefficient is defined as the performance sensitivity.

[0021] Based on the above technical scheme, preferably, for each target operator in the independent operator type set, a minimum executable network structure containing the target operator is constructed in the MindSpore framework, it is verified whether the target operator can be successfully recognized by the computation graph construction mechanism of the MindSpore framework and converted into an intermediate representation, specifically including:

[0022] For the target operator, a test tensor satisfying the input and output dimension constraints is constructed based on the input tensor type and the output tensor type, and the tensor dimension and data type of the test tensor match the operator input requirements of the MindSpore framework;

[0023] A single-operator network structure is constructed with the target operator as the core, the single-operator network structure only contains one execution node, the input of the execution node is the test tensor, and the output is the tensor transformation result after executing the target operator;

[0024] The single-operator network structure is embedded into the front-end network construction interface of the MindSpore framework, and a computation graph is generated using a symbolic graphing mechanism;

[0025] The MindSpore graph builder is called to perform graph-level analysis on the computation graph, and it is judged whether the target operator can be successfully recognized by the computation graph construction mechanism;

[0026] If the target operator can be successfully recognized by the computation graph construction mechanism, the front-end network construction interface is called to convert the computation graph into the intermediate representation.

[0027] Based on the above technical scheme, preferably, the graph compiler of the MindSpore framework is called to convert the intermediate representation into a bottom-layer execution graph, and it is judged whether the target operator can be parsed by the back-end compiler and complete graph-level optimization, specifically including:

[0028] an entry function of the graph compiler is called, the intermediate representation is loaded, and back-end environment information of target hardware is bound, the back-end environment information including device identification, execution memory parameters, parallel granularity parameters, and scheduler strategy identification of the target hardware;

[0029] a parsing process of the intermediate representation is triggered in the graph compiler, and the structure graph nodes of the intermediate representation are traversed layer by layer;

[0030] According to the operator type, input and output tensor, parameter configuration, and data dependency relationship of the parsed structure graph node, the target operator is mapped to a back-end supported operator kernel library, and if it is determined that the target operator is successfully mapped to the operator kernel library, a graph optimization operation is performed on the graph structure containing the target operator, the graph optimization operation including operator fusion, redundant calculation elimination, memory rearrangement, and constant folding.

[0031] During the execution of the graph optimization operation, it is determined whether the target operator participates in fusion, whether scheduling replacement or kernel selection adjustment is triggered, and if the target operator causes inconsistent graph structure during optimization, the target operator is identified as an optimization success operator.

[0032] On the basis of the above technical solutions, preferably, the target operator is loaded into a MindSpore running environment and executed on a specified hardware back-end, the functional output and gradient back-propagation of the target operator are verified through input tensors, and if the output error is lower than a set tolerance, the target operator is identified as a functional effective operator, specifically including:

[0033] The bottom-layer execution graph containing the target operator is loaded into a MindSpore runtime execution engine, the graph node scheduling order, kernel mapping information, and memory allocation structure in the bottom-layer execution graph are used to complete initialization of resources required by the target operator and loading of execution instructions on the specified hardware back-end;

[0034] An input tensor matching the input requirements of the target operator is constructed, the dimensions, data types, and numerical distribution of the input tensor are consistent with the operator input requirements of the MindSpore framework, and the input tensor is injected into an input buffer of the MindSpore runtime execution engine;

[0035] The MindSpore runtime scheduler is called to execute the bottom-layer execution graph, forward calculation of the target operator is completed through the specified hardware back-end, forward output tensors of the target operator are obtained, and based on an automatic differentiation mechanism, backward calculation is completed, and gradient tensors of the target operator are obtained;

[0036] The forward output tensor is compared with the corresponding reference output to calculate the maximum error value, and the gradient tensor is compared with the reference gradient to calculate the mean square error value.

[0037] If both the maximum error value and the mean square error value meet the tolerance threshold, then the target operator is identified as the functionally effective operator.

[0038] Based on the above technical solutions, preferably, the step of uniformly abstracting all graph nodes with explicit computational semantics in the computation graph and extracting an independent operator type set specifically includes:

[0039] The model structure of the target deep learning model is loaded, and the model structure is converted into a static computation graph based on the symbolic graph construction interface provided by the MindSpore framework. The static computation graph consists of a set of graph nodes and edges, and each graph node corresponds to a function module with tensor transformation logic.

[0040] By traversing all graph nodes in the static computation graph, non-computation graph nodes used for control flow or auxiliary attribute definition are excluded, and target graph nodes with explicit computation semantics are selected. The target graph nodes are operation operators registered in MindSpore corresponding to the operation type or combination function modules with tensor input and output characteristics.

[0041] Parse the operator information called by each target graph node, extract the operator type and parameter configuration corresponding to the target graph node, and record them as independent operator instances;

[0042] For multiple target graph nodes that reference the same operator type but have different parameter configurations, they are categorized according to the operator type to remove duplicate statistics caused by parameter differences;

[0043] After extracting operator information from all the target graph nodes, deduplication is performed based on operator type to form an independent set of operator types that do not contain duplicate entries.

[0044] A second aspect of this application provides a MindSpore-based operator processing apparatus for executing a MindSpore-based operator processing method as described in any of the above embodiments. The apparatus includes an acquisition module, a processing module, and an output module, wherein:

[0045] The acquisition module is used to acquire the complete computation graph of the target deep learning model during the inference and training phases, and to uniformly abstract all graph nodes with explicit computational semantics in the computation graph to extract a set of independent operator types.

[0046] The processing module is configured to, for each target operator in the set of independent operator types, construct a minimum executable network structure containing the target operator in a MindSpore framework, verify whether the target operator can be successfully identified by a computation graph construction mechanism of the MindSpore framework and converted into an intermediate representation;

[0047] The processing module is configured to invoke a graph compiler of the MindSpore framework, convert the intermediate representation into a bottom-layer execution graph, and determine whether the target operator can be parsed by a back-end compiler and completed with graph-level optimization.

[0048] The output module is configured to, if it is determined that the target operator can be parsed by the back-end compiler and completed with graph-level optimization, load the target operator into a MindSpore running environment and execute on a designated hardware back-end, verify a functional output and gradient back-propagation of the target operator by inputting a tensor, and if an output error is lower than a set tolerance, identify the target operator as a functionally valid operator.

[0049] In a third aspect of the present application, an electronic device is provided, which includes a processor, a memory, a user interface, and a network interface. The memory is configured to store instructions. The user interface and the network interface are both configured to communicate with other devices. The processor is configured to execute the instructions stored in the memory, so that the electronic device performs the method according to any one of the preceding aspects.

[0050] In a fourth aspect of the present application, a computer-readable storage medium is provided, which stores instructions. When the instructions are executed, the method according to any one of the preceding aspects is performed.

[0051] In summary, the one or more technical solutions provided in the embodiments of the present application have at least the following technical effects or advantages:

[0052] 1. The present application constructs a full-process operator processing mechanism from computation graph parsing, minimum network structure verification, intermediate representation conversion, graph compilation optimization to runtime precision verification, realizes systematic verification and completion of support of target operators in the MindSpore framework, can accurately identify and solve problems that operators cannot be identified in the graph construction phase, kernel mapping fails in the compilation phase, or performance does not meet the standard in the running phase, ensures that each target operator has functional consistency and running stability in model construction, graph optimization, and execution scheduling, and thus effectively improves the operator support completeness and execution reliability of the MindSpore framework in large model training and inference tasks.

[0053] 2. By collecting the execution time of each target operator on different devices and calculating its performance sensitivity and performance sensitivity, the performance characteristics of the operator in various hardware environments can be constructed. The causes of bottlenecks are analyzed combined with the backend execution log, and the performance of the MindSpore framework in large models is optimized and adjusted for performance abnormal operator execution structure, thereby improving the operator execution efficiency and resource utilization of the MindSpore framework in large models, and enhancing the controllability and adjustability of the overall training and inference performance.

[0054] 3. By constructing the minimum executable network structure for each target operator and verifying whether it can be successfully graphed and converted into an intermediate representation in MindSpore, it is ensured that each target operator has structural legality and semantic recognizability in the static graph construction phase, which excludes the problems of graphing failure or operator missing in advance, establishes the correct structural foundation for subsequent graph compilation and execution, and enhances the front-end integrity of operator support.

[0055] 4. By loading the intermediate representation and calling the graph compiler to parse the graph nodes and scheduling paths, mapping the target operator to the backend operator kernel library, and performing graph optimization operations, it is ensured that each target operator can be successfully registered, kernel bound and structure optimized in the graph compilation phase, and the graph fusion failure or scheduling abnormality is identified, which improves the graph compilation robustness of MindSpore and the optimization space of operator-level execution performance. BRIEF DESCRIPTION OF DRAWINGS

[0056] Figure 1 is a flowchart of an operator processing method based on MindSpore disclosed by an embodiment of the present application;

[0057] Figure 2 is a module schematic diagram of an operator processing device based on MindSpore disclosed by an embodiment of the present application;

[0058] Figure 3 is a structural schematic diagram of an electronic device disclosed by an embodiment of the present application.

[0059] MARKED DESCRIPTION: 201, acquisition module; 202, processing module; 203, output module; 301, processor; 302, communication bus; 303, user interface; 304, network interface; 305, memory. DETAILED DESCRIPTION

[0060] In order for those skilled in the art to better understand the technical solutions in the specification, the technical solutions in the specification will be described clearly and completely in conjunction with the drawings in the embodiments of the specification. Obviously, the described embodiments are only part of the embodiments of the present application, not all embodiments.

[0061] In the description of the embodiments of this application, the words "for example" or "for instance" are used to indicate examples, illustrations, or explanations. Any embodiment or design that is described as "for example" or "for instance" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design options. Rather, the use of the words "for example" or "for instance" is intended to present the relevant concepts in a specific manner.

[0062] In the description of the embodiments of this application, the term "multiple" means two or more. For example, multiple systems means two or more systems, and multiple screen terminals means two or more screen terminals. Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the indicated technical features. Thus, a feature defined with "first" or "second" may explicitly or implicitly include one or more of that feature. The terms "comprising," "including," "having," and variations thereof all mean "including but not limited to," unless otherwise specifically emphasized.

[0063] MindSpore is a high-performance artificial intelligence computing framework developed by Huawei for heterogeneous cloud, edge, and device environments. It has core capabilities such as automatic differentiation, graph fusion, and distributed scheduling, supports multiple hardware platforms, and is suitable for training and deploying large models. Its operator processing mechanism covers the entire process from operator type extraction, support verification, function completion to performance optimization. It requires operators to not only have API-level callability, but also to maintain semantic consistency and effective execution in all stages, such as static graph construction, unified intermediate representation MindIR generation, graph compilation optimization, and device scheduling, to ensure that the model can run efficiently with complete functions and stable performance under the MindSpore framework.

[0064] This embodiment discloses an operator processing method based on MindSpore, referring to... Figure 1 This includes the following steps S110-S140:

[0065] S110: Obtain the complete computation graph of the target deep learning model during the inference and training phases, and abstract all graph nodes with explicit computational semantics in the computation graph in a unified manner to extract the set of independent operator types.

[0066] This application discloses a MindSpore-based operator processing method applied to a server. The server includes, but is not limited to, electronic devices such as mobile phones, tablets, wearable devices, and PCs (Personal Computers), and can also be a backend server running a MindSpore-based operator processing method. The server can be implemented using a standalone server or a server cluster composed of multiple servers.

[0067] In a possible implementation, the uniform abstraction of all graph nodes with explicit computing semantics in the computing graph extracts a set of independent operator types, specifically including: loading a model structure of a target deep learning model, and converting the model structure into a static computing graph based on a symbolic graphing interface provided by the MindSpore framework, where the static computing graph is composed of a set of graph nodes and edges, and each graph node corresponds to a function module with tensor transformation logic; by traversing all graph nodes in the static computing graph, excluding non-computing graph nodes for control flow or auxiliary attribute definition, filtering out target graph nodes with explicit computing semantics, the target graph nodes being operation type corresponding MindSpore registered operation operators or combined function modules with tensor input and output characteristics; analyzing operator information called by each target graph node, extracting operator types and parameter configurations corresponding to the target graph nodes as independent operator instance records; for multiple target graph nodes referring to the same operator type but with different parameter configurations, classifying according to the operator type, and removing the repeated statistics caused by parameter differences; after completing the operator information extraction of all target graph nodes, deduplicating based on the operator type to form a set of independent operator types without duplicate entries.

[0068] Specifically, first, after loading the model structure of the target deep learning model, a symbolic graphing interface provided by the MindSpore framework is called to perform model structure analysis. The symbolic graphing interface can automatically convert the dynamic graph structure defined by the user in the Python front end into a static computing graph form. The static computing graph is a directed acyclic graph composed of a set of graph nodes and edges, where the graph nodes represent specific computing operations, and the edges represent the transmission path of tensors between nodes. Taking a Transformer encoder containing a residual structure as an example, the graph nodes in the static computing graph will represent matrix multiplication, addition, normalization, activation function, and other operations, and each node records the information of operation type, input tensor, and output tensor.

[0069] Subsequently, the static computation graph is traversed, and the graph nodes therein are accessed one by one and their function attributes are identified, and the MindSpore graph analysis tool or the IR parsing interface is used to exclude control nodes that do not have actual tensor operation behaviors, such as If nodes, While nodes, or auxiliary nodes for marking learning rate attributes, and only graph nodes with explicit calculation semantics are retained. Here, the explicit calculation semantics refer to the operation behavior of the operation performed by the node that can produce a specific numerical transformation on the input tensor, such as convolution, full connection, activation function, Softmax, normalization, etc. For example, for a LayerNorm node in a residual structure, it has explicit input tensors, mean calculation, standard deviation calculation, and normalization output, and belongs to the target graph node with explicit calculation semantics.

[0070] Next, for each target graph node selected, the operator information called therein is parsed. Specifically, the operator type identifier (such as MatMul, ReLU, LayerNorm, etc.) bound to the graph node and its execution configuration parameters (such as whether to use Bias, activation function type, axis direction, etc.) are obtained through the MindSpore intermediate representation (MindIR) or abstract syntax tree interface, and these parameters collectively determine the behavior logic of the operator. The operator type and parameter configuration parsed from each graph node are combined to form an independent operator instance, which is used for subsequent verification.

[0071] If multiple graph nodes call the same type of operator but their parameter configurations differ, such as two MatMul nodes set to transA=1 and transA=0 respectively, which are inconsistent at the parameter level but belong to the same type of operator, they can be classified into the same type of operator at this time, and only the operator type identifier is retained for subsequent processing. This classification operation can be implemented with a hash deduplication mechanism, that is, a mapping table is established with the operator type as the key value, and repeated entries are merged.

[0072] After completing the extraction of operator instance information for all target graph nodes, the extracted list of all operator types is processed to remove duplicates, ensuring that the final operator type set contains only non-duplicate operator type names. This independent operator type set is the set of basic operators that must be supported by the deep learning model during MindSpore execution, and serves as the input basis for operator support verification, compilation testing, and optimization processing. Taking the Transformer model as an example, the final independent operator type set typically includes MatMul, Add, ReLU, LayerNorm, Softmax, and several core operator types. This set is model-independent and migratable, and can be used for operator capability verification of other large model structures.

[0073] S120, for each target operator in the independent operator type set, a minimum executable network structure containing the target operator is constructed in the MindSpore framework respectively, it is verified whether the target operator can be successfully recognized by the computation graph construction mechanism of the MindSpore framework, and is converted into an intermediate representation.

[0074] In a possible implementation, the constructing, for each target operator in the independent operator type set, a minimum executable network structure containing the target operator in the MindSpore framework respectively, verifying whether the target operator can be successfully recognized by the computation graph construction mechanism of the MindSpore framework, and converting into an intermediate representation, specifically includes: for the target operator, based on the input tensor type and the output tensor type, a test tensor satisfying the input-output dimension constraint is constructed, the tensor dimension and data type of the test tensor match the operator input requirements of the MindSpore framework; a single-operator network structure is constructed with the target operator as the core, the single-operator network structure only contains one execution node, the input of the execution node is the test tensor, and the output is the tensor transformation result after executing the target operator; the single-operator network structure is embedded into the front-end network construction interface of the MindSpore framework, and a computation graph is generated using a symbolic graphing mechanism; the MindSpore graph constructor is called to perform graph-level analysis on the computation graph, and it is judged whether the target operator can be successfully recognized by the computation graph construction mechanism; if the target operator can be successfully recognized by the computation graph construction mechanism, the front-end network construction interface is called to convert the computation graph into the intermediate representation.

[0075] Specifically, first, for the target operator, the corresponding input tensor type and output tensor type of the target operator are obtained based on the registration information of the target operator in the MindSpore framework, the input tensor type includes tensor dimension (such as two-dimensional, three-dimensional or four-dimensional tensor), shape of each dimension (such as [64, 128] representing batch as 64 and feature dimension as 128), and data type of the tensor (such as float32, int32, etc.). The test tensor is constructed according to the above information, which is used as the input data carrier of the operator and must meet the strict requirements of the MindSpore on the tensor structure and data type of the execution interface of the target operator. For example, if the target operator is MatMul, the input tensor must be a two-dimensional or three-dimensional tensor, and the shape must meet the matrix multiplication rule, such as the input shape of [32, 64] and [64, 128] tensors, and the output shape should be [32, 128], and the data type should be unified as float32.

[0076] After the test tensor is constructed, a minimized single-operator network structure is constructed with the target operator as the core. The single-operator network structure only includes an execution node whose operation type corresponds to the target operator type. The input of the execution node is the test tensor constructed above, and the output is the result tensor after the target operator is applied to the test tensor, without other intermediate calculation logic or auxiliary modules. The single-operator network structure is used to isolate the calculation path on which the target operator depends, so as to facilitate accurate verification of whether the graph construction and execution logic of the target operator are correct. For example, for a Softmax operator, a tensor with a shape of [32, 100] can be constructed, and the Softmax is applied to the last dimension to output a tensor with the same dimension but the data is normalized to the interval of 0 to 1.

[0077] The single-operator network structure is embedded into the MindSpore front-end network construction interface, and a computation graph is generated by calling the symbolic graph construction mechanism. The symbolic graph construction mechanism refers to the process of converting the network structure defined by a user in the form of Python code into an internal graph structure in MindSpore. The generated computation graph is a static graph structure, the graph nodes are operator call instances, and the graph edges are tensor flow paths.

[0078] Subsequently, the graph constructor of MindSpore is called to perform graph-level analysis on the computation graph, to determine whether the execution node can be successfully identified as the target operator type that has been registered in MindSpore. The process includes node type verification, tensor dimension reasoning, and parameter configuration legality check, to ensure that the target operator is not marked as an illegal node or an unknown node in the graph construction phase. If the execution node is successfully parsed in the graph construction process, it indicates that the target operator has front-end construction support capability in the current version of the MindSpore framework.

[0079] If it is confirmed that the target operator can be successfully identified by the MindSpore graph construction mechanism, the computation graph is converted into a unified intermediate representation MindIR by calling the MindSpore front-end interface. MindIR is an intermediate format used to uniformly describe the model structure in the MindSpore framework. It records the operator type, input and output tensor structure, and parameter configuration of each graph node, and is the core representation structure of MindSpore in graph compilation, graph optimization, and back-end execution. The finally generated MindIR file will be used as the input for subsequent graph compilation and execution verification, so as to realize complete parsing and semantic expression verification of the target operator in the static graph construction phase.

[0080] In S130, the graph compiler of the MindSpore framework is called to convert the intermediate representation into a bottom-layer execution graph, to determine whether the target operator can be parsed by the back-end compiler and complete graph-level optimization.

[0081] In one possible implementation, the step of calling the MindSpore framework's graph compiler to convert the intermediate representation into a low-level execution graph and determining whether the target operator can be parsed and optimized at the graph level by the backend compiler specifically includes: calling the graph compiler's entry function, loading the intermediate representation and binding the target hardware's backend environment information, including the target hardware's device identifier, execution memory parameters, parallel granularity parameters, and scheduler policy identifier; triggering the parsing process of the intermediate representation in the graph compiler, traversing the structure graph nodes of the intermediate representation layer by layer; and optimizing the structure graph based on the parsed results. The operator type, input / output tensors, parameter configuration, and data dependencies corresponding to the graph nodes are used to map the target operator to the backend supported operator kernel library. If it is determined that the target operator is successfully mapped to the operator kernel library, then graph optimization operations are performed on the graph structure containing the target operator. The graph optimization operations include operator fusion, redundant computation elimination, memory rearrangement, and constant folding. During the execution of the graph optimization operations, it is determined whether the target operator participates in fusion, whether it triggers scheduling replacement, or kernel selection adjustment. If the target operator causes graph structure inconsistency during the optimization process, then the target operator is marked as a successfully optimized operator.

[0082] Specifically, first, the entry function of the graph compiler in the MindSpore framework is called to load the model structure that has been constructed and transformed into a unified intermediate representation. This unified intermediate representation is a static graph structure in MindIR format, containing complete graph node topology, operator types, tensor edge information, and parameter configuration data. Simultaneously with loading the MindIR file, the backend environment information of the target hardware needs to be bound. This backend environment information guides the graph compiler in selecting the appropriate device execution path. The backend environment information includes device identifiers (e.g., Ascend 910, GPU, or CPU), execution memory parameters (e.g., memory upper and lower limits, cache page size), parallel granularity parameters (e.g., number of threads, tensor partitioning strategy), and scheduler policy identifiers (e.g., priority scheduling, streaming scheduling, or tensor fusion strategy), ensuring that the graph compiler's optimization behavior conforms to the resource constraints and computational characteristics of the target hardware platform. For example, on the Ascend device, the Ascend operator scheduling mode and L1 cache strategy need to be explicitly configured.

[0083] The parsing process of the intermediate representation is triggered in the graph compiler, that is, each graph node is read from the MindIR structure graph and traversed layer by layer to extract the operator type identifier, input and output tensor dimensions, data type, parameter configuration, and edge dependency information between graph nodes. This process is the syntax parsing and structure analysis phase of the graph structure, which is used to establish the positional relationship of each operator in the graph and initialize the data structures required for subsequent kernel mapping and scheduling path construction. For example, for a graph structure containing MatMul and ReLU, the traversal order will first parse the MatMul node, record its input and output tensor information, then parse the ReLU node, and establish the dependency edge of its input tensor depending on the MatMul output tensor.

[0084] According to the above parsing result, the target operator type bound to each graph node is searched in the operator kernel library registered by MindSpore backend, which contains the operator implementation supported by a specific device, including kernel call instructions, tensor layout rules, and concurrent scheduling mechanisms. If the target operator has a corresponding implementation in the backend operator kernel library, and the input and output tensors and parameter configurations can be correctly matched, it is determined that the target operator has been successfully mapped to the backend execution path. If the mapping fails (for example, the input tensor dimension does not meet the requirements of the backend operator, or the data type is not supported), the process is exited and the target operator is identified as an uncompileable operator.

[0085] After confirming the successful mapping of the target operator, graph optimization operations are performed on the graph structure containing the target operator. Graph optimization is a core phase in the MindSpore graph compiler, which contains multiple optimization sub-modules, mainly including operator fusion, redundant computation elimination, memory rearrangement, and constant folding. Operator fusion refers to merging multiple adjacent graph nodes (such as MatMul and BiasAdd) into a composite kernel to reduce memory access overhead and scheduling delay; redundant computation elimination is to analyze the usage path of the output tensor of the graph node, identify intermediate calculations that are not called by subsequent nodes, and perform pruning; memory rearrangement refers to optimizing the layout of tensors in memory according to the tensor read-write mode; constant folding is to calculate the constant expressions in the graph in advance to reduce runtime load.

[0086] In the process of performing the above graph optimization operation, it is determined in real time whether the target operator participates in the operator fusion process, whether the scheduling path reconstruction or the reselection of the kernel calling instruction is triggered. For example, when the target operator is ReLU and the predecessor node thereof is Conv2D, the compiler can fuse the two into a FusedConvRelu kernel, and if the fusion is successful, it is recorded that the target operator participates in the fusion. If the fusion process causes the number of input and output edges of the graph node to change or the operator execution logic to change, it is further determined whether the target operator causes the graph structure to be inconsistent (for example, the data dependency chain is broken or the topology closed loop is incorrect). If no structural inconsistency is caused and the fusion or scheduling replacement operation is successfully applied to the graph structure, the target operator is identified as an optimization successful operator, and the execution path information and the kernel instruction template corresponding to the target operator before and after optimization are recorded for subsequent scheduling verification stage calling.

[0087] In S140, if it is determined that the target operator can be parsed by the back-end compiler and the graph-level optimization is completed, the target operator is loaded into the MindSpore running environment and executed on the specified hardware back-end, the functional output and gradient back-propagation of the target operator are verified through the input tensor, and if the output error is lower than the set tolerance, the target operator is identified as a functionally valid operator.

[0088] In a possible implementation, the loading of the target operator into the MindSpore running environment and the execution on the specified hardware back-end, the verification of the functional output and gradient back-propagation of the target operator through the input tensor, and the identification of the target operator as a functionally valid operator if the output error is lower than the set tolerance specifically include: loading a bottom-layer execution graph containing the target operator into a MindSpore runtime execution engine, initializing the resources required by the target operator and loading the execution instructions according to the graph node scheduling order, kernel mapping information and memory allocation structure in the bottom-layer execution graph on the specified hardware back-end; constructing an input tensor matched with the input requirement of the target operator, the dimension, data type and numerical distribution of the input tensor being consistent with the operator input requirement of the MindSpore framework, and injecting the input tensor into the input buffer of the MindSpore runtime execution engine; calling the MindSpore runtime scheduler to execute the bottom-layer execution graph, completing the forward calculation of the target operator through the specified hardware back-end, obtaining the forward output tensor of the target operator, and completing the reverse calculation based on the automatic differentiation mechanism to obtain the gradient tensor of the target operator; comparing the forward output tensor with the corresponding reference output, calculating the maximum error value, and comparing the gradient tensor with the reference gradient, calculating the mean square error value; if the maximum error value and the mean square error value both satisfy the tolerance threshold, the target operator is identified as the functionally valid operator.

[0089] Specifically, a bottom-layer execution graph containing a target operator is loaded to a MindSpore runtime execution engine, and the bottom-layer execution graph is an executable graph structure generated through a graph construction and graph compilation stage, containing a graph node scheduling order, kernel mapping information, and memory allocation structure. The runtime execution engine determines the execution order of each graph node in turn according to the graph node scheduling order, that is, the topological sorting result; selects a specific backend operator implementation template according to the kernel mapping information; and completes the static allocation or dynamic allocation of the tensor buffer according to the memory allocation structure. On a specified hardware backend, such as an Ascend chip or a GPU device, the runtime engine loads the kernel function, tensor buffer space, and intermediate variable structure required by the target operator into the target device through a driver interface, and completes the initialization of the execution context, such as instruction cache preheating, L2 data preloading, and the like.

[0090] Subsequently, based on the interface definition of the target operator, an input tensor matching the input requirements thereof is constructed. The input tensor must meet the input dimension structure, data type, and data arrangement requirements declared by the target operator. For example, for a Conv2D operator, the input tensor should be a four-dimensional tensor, and the shape format is usually [batch_size, channel, height, width], and the data type is float32. If the target operator has multiple input ports (such as an AddN operator), an independent input tensor needs to be constructed for each input port. At the same time, in order to ensure numerical stability, the numerical distribution should be set to conform to the value range that the operator may encounter in normal model training, such as a normal distribution tensor with a mean of 0 and a standard deviation of 1. After the construction is completed, the input tensor is injected into the input buffer of the MindSpore runtime execution engine, which is a tensor storage area maintained in the running engine for graph node execution, to ensure the readability and alignment of the input data when the backend device is scheduled.

[0091] Next, the MindSpore runtime scheduler is called to start the execution flow of the bottom-layer execution graph. The scheduler calls the kernel instructions bound to the target operator in turn according to the graph node scheduling order, and completes the forward calculation of the tensor through the backend hardware to output the corresponding forward output tensor. After the forward calculation is completed, the MindSpore automatic differentiation mechanism (AutoGrad) is called to perform the backpropagation operation on the target operator, automatically build a gradient flow graph, and call the corresponding gradient operator (such as GradReLU, GradMatMul, etc.) to perform reverse calculation to generate an output gradient tensor. The automatic differentiation mechanism relies on the operation track and tensor dependency relationship in the graph structure to ensure that the generated gradient tensor has consistent dimension structure and mathematical relationship with the forward output.

[0092] After the forward output tensor and the gradient tensor calculation are completed, error comparison is performed with the preset reference output respectively. The forward output error comparison uses the maximum error value (Max Error) as an index, that is, the difference between the predicted output and the reference output is calculated element by element and the maximum absolute value is taken, which is used to evaluate the output stability; the gradient error comparison uses the mean square error value (MSE) as an index, that is, the square difference mean value between the predicted gradient and the reference gradient is calculated, which is used to evaluate the accuracy of the back propagation result. The two error indicators together constitute the numerical precision verification criterion of the target operator.

[0093] If the maximum error value and the mean square error value are both less than the preset tolerance threshold, the maximum error is less than the preset first tolerance threshold, the mean square error is less than the preset second tolerance threshold, and no abnormality (such as segment error, kernel overflow or data out-of-bound) occurs during the graph execution process, the target operator is identified as a functional effective operator, indicating that it has triple guarantee capability of graph construction, graph execution and numerical correctness in the current MindSpore version. If any index does not meet the requirements, it is determined that the target operator function does not meet the verification standard, and further analysis of the operator implementation logic, kernel precision or numerical stability problem is required. The verification process ensures that the functional support of each target operator in MindSpore not only reflects in the API interface level, but also is confirmed through the actual execution path and the precision result.

[0094] In a possible implementation, after the functionality output and gradient back of the target operator are verified by the input tensor, the method further includes: collecting the execution time of each target operator on different devices; calculating the algorithm utilization rate and performance sensitivity of the target operator based on the execution time; constructing a performance bottleneck map of the target operator based on the algorithm utilization rate and performance sensitivity; correlating and analyzing the performance bottleneck map with the MindSpore backend execution log to identify bottleneck factors affecting the performance of the target operator, the bottleneck factors including memory bandwidth limitation, operator fusion failure, scheduling delay or thread concurrency conflict; for the target operator identified as a performance anomaly in the performance bottleneck map, calling the operator-level optimization interface provided by the MindSpore graph optimizer to perform optimization operations to generate an optimized execution path of the target operator, the optimization operations including operator fusion reconstruction, memory access mode rearrangement, instruction-level parallelism adjustment and scheduling priority reordering; writing the optimized execution path to the position of the target operator in the bottom layer execution graph to form an updated bottom layer execution graph structure, and reloading to the MindSpore running environment to obtain the optimized output tensor and the optimized gradient tensor of the optimized target operator for verification.

[0095] Specifically, firstly, after verifying the functional output and gradient backpropagation of the target operator, the runtime performance monitoring module integrated in the MindSpore framework is invoked to collect the actual execution time of the target operator on different hardware devices. The execution time refers to the total number of clock cycles experienced by the target operator from kernel startup to result output in a single graph scheduling, typically measured in milliseconds (ms). The collection method includes inserting profiling markers into the underlying execution graph and capturing the start and end timestamps of the target operator's execution phase through hardware-level performance counters. Optionally, NVIDIA CUPTI is used on the GPU, and the Ascend Profiling API is used on the Ascend device, in conjunction with MindSpore's runtime profiler interface to complete data aggregation. This process can be repeated multiple times to obtain the average execution time, eliminating the interference of environmental fluctuations on the measurement.

[0096] Subsequently, based on the collected execution time data, the computational utilization rate and performance sensitivity of the target operator are calculated. Computational utilization rate refers to the ratio of the theoretical computational load of the target operator to its actual execution time. The theoretical computational load is derived based on the dimension and data type of the operator's input tensor and the operator's arithmetic complexity. Performance sensitivity is calculated by constructing the elasticity coefficient of the operator's execution time to the overall model execution time. This involves measuring the response relationship between changes in the target operator's execution time and changes in the overall graph runtime under different input sizes or scheduling densities. Regression fitting or principal component analysis methods are commonly used for modeling to derive the performance impact weight of the target operator throughout the entire model execution path.

[0097] In one possible implementation, calculating the computational utilization rate and performance sensitivity of the target operator based on the execution time specifically includes: calculating the computational utilization rate of the target operator based on the execution time and the theoretical computational complexity of the target operator, wherein the theoretical computational complexity is calculated according to the input tensor dimension and operation type of the target operator, and the computational utilization rate is the ratio of the theoretical computational complexity to the actual execution time; based on the changing trend of the execution time ratio of the target operator under different input tensor sizes and concurrent operation conditions, fitting and constructing a performance response model of the target operator, calculating the response coefficient of the changing trend to the change in the overall graph execution time, and defining the response coefficient as the performance sensitivity.

[0098] In particular, the computing power utilization rate is calculated based on the execution time of the target operator and its theoretical calculation complexity. The theoretical calculation complexity refers to the total amount of basic mathematical operations required to be completed by the target operator under ideal conditions without considering hardware bandwidth, memory access delay, and thread scheduling cost, which is usually measured by floating point operation (FLOPs). The theoretical calculation complexity is determined by the dimension information of the input tensor and the operation type. For a two-dimensional matrix multiplication operator MatMul, the input tensors are matrices with dimensions [M x K] and [K x N], respectively. The basic multiplication and addition operation count is extended to a joint expression that includes tensor access cost, operation parallelism influence, and memory access rearrangement overhead:

[0099]

[0100] wherein: a represents an operation-intensive weighting factor, which is converted for hardware FMA (Fused Multiply-Add) capability, β represents a tensor access cost coefficient, which is used to describe the operation overhead caused by intermediate storage and loading, γ is a memory path rearrangement factor, which reflects the scheduling cost in the cache hierarchy, M, K, N are input tensor dimensions, corresponding to the row and column dimensions in the matrix multiplication scenario. The coefficient 2 represents that each multiplication and addition operation includes one multiplication and one addition.

[0101] The formula is used to quantify the total operation cost required by the target operator in an ideal execution environment, considering various sources of overhead such as multiplication and addition operations, tensor access, and memory scheduling. The first term 2•M•K•N represents the basic floating point calculation count required to complete all multiplications and additions in the matrix multiplication operation, with the coefficient 2 representing that each multiplication and addition includes one multiplication and one addition. This term multiplied by a reflects the utilization efficiency of the FMA (Fused Multiply-Add) operation unit in different hardware, for example, on devices supporting the FMA instruction set, a < 1, indicating that there is a combined acceleration effect. The second term β•(M•N) represents the tensor storage and loading burden caused by the output tensor writing operation, with the coefficient β measuring the cost of memory access in the overall calculation. The third term γ•log_2(K) is used to represent the memory path rearrangement cost, especially when the input dimension is large, the tensor clipping, arrangement, and alignment in the cache hierarchy will cause additional overhead, and γ is the mapping weight of this memory overhead in different architectures. Overall, this complex expression extends the traditional FLOPs model by introducing structured factors, providing a more realistic calculation load evaluation.

[0102] The actual execution time of the target operator is provided by the MindSpore runtime performance monitoring module, and further, the computing power utilization rate complex expression is as follows:

[0103]

[0104] wherein: is the actual execution time of the target operator, denotes the device idle time caused by memory access conflict or load waiting, denotes the scheduling overhead of thread or device inter-synchronization, denotes the idle delay caused by unbalanced thread load, are the corresponding weighted scheduling coefficients, which are set according to the device architecture and thread model, respectively.

[0105] The formula measures the utilization efficiency of the target operator on the computing resources in the actual running environment, which is the ratio between the theoretical calculation load and the total cost of the real execution time. The denominator is the main time axis of actually running the operator on the hardware. However, due to many non-ideal factors of modern computing platforms, such as memory waiting, thread blocking, and the like, three time compensation items need to be introduced: denotes the device pause time caused by memory access conflict or insufficient bandwidth, which is common in deep models with a large amount of random memory access; denotes the synchronization waiting time between operators, threads or devices, which is particularly significant in multi-threaded or distributed scenarios; is the overhead of some threads being forced to wait due to unbalanced thread allocation. The three compensation items are multiplied by the weighted coefficients related to the architecture to reflect their proportion on different platforms to form the weighted total execution time. The physical meaning of the entire fraction structure is that the numerator is the computing power requirement, and the denominator is the actual running cost. The larger the ratio, the more fully the resources are utilized, and the closer the operator's execution in MindSpore is to the hardware upper limit.

[0106] The linear sensitivity is extended to a curvature index based on the high-order derivative response function, and multi-dimensional input perturbation and local extreme characteristics are introduced:

[0107]

[0108] wherein S represents an input tensor scale perturbation variable (for example, the number of elements, the number of channels or the sequence length), P represents a concurrency perturbation variable (for example, the number of threads or the parallel batch), the first term is the second-order derivative of the execution time proportional change under the input perturbation, reflecting the curvature or nonlinear acceleration characteristics, the second term is the first-order response square under the concurrency perturbation, measuring the influence of the parallel strategy on the performance fluctuation, and η is the concurrency influence weight, adjusting the sensitivity measurement of the parallel structure. Through high-order derivative and combined perturbation analysis, this sensitivity expression can identify potential bottleneck operators with nonlinear expansion characteristics and reveal the performance unstable regions in large-scale running.

[0109] ​The formula is used to measure the response strength of the running time of the target operator to the input size disturbance and the concurrent parameter disturbance, that is, the fluctuation degree of its performance with the change of the input and the scheduling condition. The first term is the performance response curvature under the input disturbance, which captures the nonlinear growth characteristics by taking the second-order derivative of the execution time ratio function with respect to the input tensor size variable S. For example, in multi-dimensional convolution and long sequence Transformer, when the input dimension reaches a certain critical point, the operator time consumption presents exponential growth, and this term can reveal this high-order expansion behavior. The second term is the sensitivity under the concurrent strategy disturbance, which represents how the execution time ratio of the target operator changes when adjusting the thread number, parallel batch, and tensor splitting mode P. The square of the first derivative represents the absolute strength of the change amplitude, which amplifies the mutation trend. η is the concurrent influence weight, which adjusts the proportion of this term in the overall sensitivity, and is used to adapt to the differences of different platform scheduling models. After integrating the two terms, the performance sensitivity can be used to identify those highly sensitive bottleneck operators to input size or thread configuration, guiding subsequent local graph structure reconstruction, scheduling strategy optimization, or operator kernel replacement.

[0110] After obtaining the above two performance indicators, a performance bottleneck graph is constructed with the target operator as the node. The performance bottleneck graph is a graph structure representing the performance influence path of each target operator in model execution. The graph node represents the target operator, and the graph edge represents the dependency relationship between operators. Each graph node is attached with the computing power utilization rate and the performance sensitivity attribute. The bottleneck operators are highlighted in the graph (such as utilization rate less than 30% or sensitivity greater than 0.7). The bottleneck graph structure can be used for visual analysis to assist in locating the operator optimization priority and forming the basis for performance improvement strategies.

[0111] Subsequently, the performance bottleneck graph is analyzed in association with the MindSpore backend execution log. The backend execution log contains runtime behavior information such as scheduling state, memory read-write log, scheduler delay, and thread blocking of each graph node. By comparing the target operators identified as performance anomalies in the graph with their specific running behaviors in the execution log, the performance bottleneck reasons are extracted. Common bottleneck factors include: ① memory bandwidth limitation — the data accessed by the operator is large, and the memory access mode is discontinuous, causing bandwidth bottleneck; ② operator fusion failure — adjacent operators are not fused due to inconsistent data types or boundary conditions, increasing the scheduling load; ③ scheduling delay — the target operator has a long queuing time in the scheduler and is not prioritized; ④ thread concurrency conflict — the operator binds thread resources with intense competition, causing frequent context switching.

[0112] For the target operator identified as performance anomaly in the above bottleneck graph, the operator-level optimization interface provided by the MindSpore graph optimizer is called to perform optimization operations. Specifically, it includes: ① operator fusion reconstruction - redefine the data boundary and fusion strategy between the target operator and the preceding and following operators, and combine multiple graph nodes into a unified kernel, such as reconstructing MatMul and BiasAdd into FusedMatMul; ② memory access mode rearrangement - optimize data alignment by changing the tensor storage order or adjusting the tile strategy to improve memory access efficiency; ③ instruction-level parallelism adjustment - change the thread granularity configuration, vectorization strategy or pipeline depth in the kernel to improve execution concurrency; ④ scheduling priority reordering - modify the scheduling order in the execution graph to improve the scheduling response speed of the target operator in the critical path.

[0113] After the optimization operation is completed, the optimization execution path of the target operator is written to the original position of the underlying execution graph, replacing the old graph structure, and an updated underlying execution graph structure is generated. This structure still maintains the original graph node number and dependency relationship, ensuring that the optimized graph structure is functionally equivalent to the original graph structure. Then the updated underlying execution graph is reloaded into the MindSpore running environment, and the operators in the optimization path are executed to obtain the optimized output tensor and gradient tensor.

[0114] Finally, the error verification of the optimized output tensor and gradient tensor with the original reference value is performed, and the maximum error and mean square error comparison process is repeated to ensure that the optimization operation improves the execution efficiency without introducing numerical bias. If the error still meets the set tolerance threshold, the target operator optimization is successful, the state identifier is updated, and the performance comparison data before and after optimization is recorded to form a complete optimization feedback link. This process realizes a closed-loop optimization path from performance bottleneck identification, optimization scheme execution to functional consistency verification, improving the running efficiency and system stability of the target operator under the MindSpore framework.

[0115] The embodiment also discloses an operator processing device based on MindSpore, referring to Figure 2 , comprising an acquisition module 201, a processing module 202 and an output module 203, and the device is used for executing any one of the above-mentioned operator processing methods based on MindSpore, wherein:

[0116] The acquisition module 201 is used for acquiring the complete computation graph of the target deep learning model in the inference and training stages, uniformly abstracting all graph nodes with explicit computation semantics in the computation graph, and extracting an independent operator type set.

[0117] The processing module 202 is configured to, for each target operator in the set of independent operator types, construct a minimum executable network structure containing the target operator in the MindSpore framework, verify whether the target operator can be successfully recognized by a computation graph construction mechanism of the MindSpore framework and converted into an intermediate representation.

[0118] The processing module 202 is configured to call a graph compiler of the MindSpore framework to convert the intermediate representation into a bottom-layer execution graph, and determine whether the target operator can be parsed by a back-end compiler and completed with graph-level optimization.

[0119] The output module 203 is configured to, if it is determined that the target operator can be parsed by the back-end compiler and completed with graph-level optimization, load the target operator into a MindSpore running environment and execute on a specified hardware back-end, verify the functional output and gradient back-propagation of the target operator by inputting a tensor, and if the output error is lower than a set tolerance, identify the target operator as a functionally valid operator.

[0120] In a possible implementation, the acquisition module 201 is configured to collect the execution time of each target operator on different devices.

[0121] The processing module 202 is configured to calculate the computing power utilization rate and performance sensitivity of the target operator based on the execution time.

[0122] The output module 203 is configured to construct a performance bottleneck map of the target operator based on the computing power utilization rate and the performance sensitivity.

[0123] The processing module 202 is configured to perform correlation analysis on the performance bottleneck map and a MindSpore back-end execution log, identify bottleneck factors affecting the performance of the target operator, and the bottleneck factors include memory bandwidth limitation, operator fusion failure, scheduling delay, or thread concurrency conflict.

[0124] The processing module 202 is configured to, for the target operator identified as a performance anomaly in the performance bottleneck map, call an operator-level optimization interface provided by a MindSpore graph optimizer, perform an optimization operation to generate an optimized execution path of the target operator, and the optimization operation includes operator fusion reconstruction, memory access mode rearrangement, instruction-level parallelism adjustment, and scheduling priority reordering.

[0125] The output module 203 is configured to write the optimized execution path into a position of the target operator in the bottom-layer execution graph, form an updated bottom-layer execution graph structure, and reload the updated bottom-layer execution graph structure into the MindSpore running environment, and verify the optimized output tensor and the optimized gradient tensor of the optimized target operator.

[0126] In a possible implementation, the processing module 202 is configured to calculate the computing power utilization rate of the target operator based on the execution time and the theoretical calculation complexity of the target operator, and the theoretical calculation complexity is calculated according to the input tensor dimension and the operation type of the target operator.

[0127] The processing module 202 is configured to fit and construct a performance response model of the target operator based on the change trend of the execution time proportion of the target operator under different input tensor scales and concurrent running conditions, and calculate a response coefficient of the change trend to the overall graph execution time change, and the response coefficient is defined as a performance sensitivity.

[0128] In a possible implementation, the output module 203 is configured to, for the target operator, construct a test tensor that satisfies an input-output dimension constraint based on an input tensor type and an output tensor type, and the tensor dimension and data type of the test tensor match the operator input requirements of the MindSpore framework.

[0129] The processing module 202 is configured to construct a single-operator network structure with the target operator as the core, and the single-operator network structure only includes one execution node, and the input of the execution node is the test tensor and the output is the tensor transformation result after the target operator is executed.

[0130] The processing module 202 is configured to embed the single-operator network structure into a front-end network construction interface of the MindSpore framework, and generate a computation graph using a symbolic mapping mechanism.

[0131] The processing module 202 is configured to call a MindSpore graph constructor to perform graph-level analysis on the computation graph, and determine whether the target operator can be successfully identified by a computation graph construction mechanism.

[0132] The processing module 202 is configured to, if the target operator can be successfully identified by the computation graph construction mechanism, call the front-end network construction interface to convert the computation graph into an intermediate representation.

[0133] In a possible implementation, an entry function of a graph compiler is called, the intermediate representation is loaded, and back-end environment information of a target hardware is bound, and the back-end environment information includes a device identifier, an execution memory parameter, a parallel granularity parameter, and a scheduler policy identifier of the target hardware.

[0134] The processing module 202 is configured to trigger an analysis process of the intermediate representation in the graph compiler, and iteratively traverse structure graph nodes of the intermediate representation.

[0135] The processing module 202 is configured to map the target operator to an operator kernel library supported by the backend according to the operator type, input and output tensor, parameter configuration, and data dependency relationship corresponding to the parsed structure diagram node, and perform a graph optimization operation on the graph structure containing the target operator if it is determined that the target operator is successfully mapped to the operator kernel library. The graph optimization operation includes operator fusion, redundant computation elimination, memory rearrangement, and constant folding.

[0136] The processing module 202 is configured to determine whether the target operator participates in fusion and triggers scheduling replacement or kernel selection adjustment during the execution of the graph optimization operation, and identify the target operator as an optimization success operator if the target operator causes inconsistency in the graph structure during the optimization process.

[0137] In a possible implementation, the processing module 202 is configured to load a bottom-layer execution graph containing the target operator to a MindSpore runtime execution engine, and complete initialization of resources required by the target operator and loading of execution instructions on a specified hardware backend according to a graph node scheduling order, kernel mapping information, and memory allocation structure in the bottom-layer execution graph.

[0138] The processing module 202 is configured to construct an input tensor matched with the input requirement of the target operator, and inject the input tensor into an input buffer of the MindSpore runtime execution engine, where the dimensions, data type, and numerical distribution of the input tensor are consistent with the operator input requirement of the MindSpore framework.

[0139] The processing module 202 is configured to call a MindSpore runtime scheduler to execute the bottom-layer execution graph, complete forward calculation of the target operator through the specified hardware backend, obtain a forward output tensor of the target operator, and complete reverse calculation based on an automatic differentiation mechanism to obtain a gradient tensor of the target operator.

[0140] The processing module 202 is configured to perform error comparison on the forward output tensor and a corresponding reference output, calculate a maximum error value, and perform error comparison on the gradient tensor and a reference gradient, and calculate a mean square error value.

[0141] The processing module 202 is configured to identify the target operator as a functionally valid operator if both the maximum error value and the mean square error value meet a tolerance threshold.

[0142] In a possible implementation, the processing module 202 is configured to load a model structure of a target deep learning model, and convert the model structure into a static computation graph based on a symbolic graphing interface provided by the MindSpore framework, where the static computation graph is composed of a group of graph nodes and edges, and each graph node corresponds to a function module with tensor transformation logic.

[0143] The processing module 202 is configured to filter out target graph nodes with explicit computing semantics by traversing all graph nodes in the static computing graph, excluding no-computing graph nodes for control flow or auxiliary attribute definition.

[0144] The processing module 202 is configured to parse operator information called by each target graph node, and extract an operator type and parameter configuration corresponding to the target graph node as an independent operator instance record.

[0145] The processing module 202 is configured to classify target graph nodes referring to the same operator type but having different parameter configurations according to the operator type, and remove repeated statistics caused by parameter differences.

[0146] The processing module 202 is configured to remove duplicates based on the operator type after extracting operator information of all target graph nodes, and form an independent operator type set that does not contain duplicate entries.

[0147] It should be noted that the apparatus provided in the above embodiments is only used as an example to divide the above functional modules in realizing its functions, and in actual applications, the above functions can be completed by different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules to complete all or part of the above described functions. In addition, the apparatus and method embodiments provided in the above embodiments belong to the same concept, and the specific implementation process is detailed in the method embodiments, which will not be described here.

[0148] The embodiment also discloses an electronic device, which refers to Figure 3 The electronic device can include at least one processor 301, at least one communication bus 302, a user interface 303, a network interface 304, and at least one memory 305.

[0149] The communication bus 302 is configured to realize the connection and communication between the components.

[0150] The user interface 303 can include a display screen (Display) and a camera (Camera), and the optional user interface 303 can further include a standard wired interface and a wireless interface.

[0151] The network interface 304 can optionally include a standard wired interface and a wireless interface (such as a WI-FI interface).

[0152] The processor 301 can include one or more processing cores. The processor 301 connects various parts within the server through various interfaces and lines, performs various functions of the server and processes data by running or executing instructions, programs, code sets or instruction sets stored in the memory 305, and calling data stored in the memory 305. Alternatively, the processor 301 can be implemented in at least one of a hardware form of a digital signal processing (DSP), a field-programmable gate array (FPGA), and a programmable logic array (PLA). The processor 301 can integrate a combination of one or more of a central processing unit (CPU), a graphics processing unit (GPU), and a modem. Among them, the CPU mainly processes the operating system, user interface, and application programs. The GPU is responsible for rendering and drawing the content to be displayed on the display screen. The modem is used to process wireless communication. It can be understood that the above-mentioned modem can also not be integrated into the processor 301, but can be realized by a separate chip.

[0153] The memory 305 can include a random access memory (RAM) and a read-only memory (ROM). Optionally, the memory includes a non-transitory computer-readable storage medium. The memory 305 can be used to store instructions, programs, codes, code sets or instruction sets. The memory 305 can include a program storage area and a data storage area, wherein the program storage area can store instructions for implementing an operating system, instructions for at least one function (such as a touch function, a sound playing function, an image playing function, etc.), instructions for implementing the above-mentioned various method embodiments, etc. The data storage area can store data related to the above-mentioned various method embodiments, etc. The memory 305 can also be at least one storage device located away from the aforementioned processor 301. As a computer storage medium, the memory 305 can include an operating system, a network communication module, a user interface 303 module, and an application program of an operator processing method based on MindSpore.

[0154] In Figure 3In the electronic device shown, the user interface 303 is mainly used to provide an interface for the user to input, and obtain data input by the user. The processor 301 can be used to call an application program stored in the memory 305 and storing an operator processing method based on MindSpore, which, when executed by one or more processors 301, causes the electronic device to perform the method of one or more of the above embodiments.

[0155] It should be noted that, for the foregoing method embodiments, in order to simply describe, they are all described as a series of action combinations, but those skilled in the art should know that the present application is not limited to the order of the actions described, because according to the present application, certain steps can be performed in other order or at the same time. Secondly, those skilled in the art should know that the embodiments described in the specification all belong to preferred embodiments, and the actions and modules involved are not necessarily required by the present application.

[0156] In the above embodiments, the description of each embodiment has its own focus, and the parts not described in detail in a certain embodiment can be referred to the related description of other embodiments.

[0157] In several embodiments provided in the present application, it should be understood that the disclosed device can be implemented by other means. For example, the device embodiments described above are only schematic, and the division of units is only a logical function division, and actual implementation can have another division manner, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the displayed or discussed units can be indirect coupling or communication connection through some service interface, device or unit, which can be electrical or other forms.

[0158] The units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, that is, they can be located in one place, or they can be distributed on multiple network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiment scheme.

[0159] In addition, each functional unit in each embodiment of the present application can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0160] The integrated unit, if implemented in the form of a software function unit and sold or used as an independent product, can be stored in a computer readable memory. Based on such understanding, the technical solutions of the present application essentially or say the part that contributes to the prior art or the whole or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a memory 305 and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server or a network device, etc.) to execute all or part of the steps of the embodiments of the present application. The aforementioned memory 305 includes: a U disk, a mobile hard disk, a magnetic or optical disk and various program code storage media.

[0161] The present application also discloses a computer readable storage medium, which stores instructions. When executed by one or more processors 301, the instructions cause an electronic device to perform one or more methods as described in the above embodiments.

[0162] The above are only exemplary embodiments of the present disclosure, and cannot limit the scope of the present disclosure. That is, any equivalent changes and modifications made in accordance with the teachings of the present disclosure are still within the scope of the present disclosure. Other embodiments of the present disclosure will be readily apparent to those skilled in the art upon considering the specification and practicing the true principles of the present disclosure. The present application is intended to cover any variations, uses or adaptive changes of the present disclosure that follow the general principles of the present disclosure and include common knowledge or conventional technical means in the art that are not described in the present disclosure. The specification and examples are only considered as exemplary, and the scope and spirit of the present disclosure are defined by the claims.

Claims

1. A MindSpore-based operator processing method, characterized in that, The method includes: Obtain the complete computation graph of the target deep learning model during the inference and training phases, and abstract all graph nodes with explicit computational semantics in the computation graph in a unified manner to extract a set of independent operator types. For each target operator in the set of independent operator types, a minimum executable network structure containing the target operator is constructed in the MindSpore framework to verify whether the target operator can be successfully identified by the computation graph construction mechanism of the MindSpore framework and transformed into an intermediate representation; The MindSpore framework's graph compiler is invoked to convert the intermediate representation into a low-level execution graph, and it is determined whether the target operator can be parsed by the backend compiler and completed at the graph level. If it is determined that the target operator can be parsed by the backend compiler and completed at the graph level, the target operator is loaded into the MindSpore runtime environment and executed on the specified hardware backend. The functional output and gradient backpropagation of the target operator are verified by input tensors. If the output error is lower than the set tolerance, the target operator is marked as a functionally valid operator. The process of uniformly abstracting all graph nodes with explicit computational semantics in the computation graph and extracting an independent set of operator types specifically includes: The model structure of the target deep learning model is loaded, and the model structure is converted into a static computation graph based on the symbolic graph construction interface provided by the MindSpore framework. The static computation graph consists of a set of graph nodes and edges, and each graph node corresponds to a function module with tensor transformation logic. By traversing all graph nodes in the static computation graph, non-computation graph nodes used for control flow or auxiliary attribute definition are excluded, and target graph nodes with explicit computation semantics are selected. The target graph nodes are operation operators registered in MindSpore corresponding to the operation type or combination function modules with tensor input and output characteristics. Parse the operator information called by each target graph node, extract the operator type and parameter configuration corresponding to the target graph node, and record them as independent operator instances; For multiple target graph nodes that reference the same operator type but have different parameter configurations, they are categorized according to the operator type to remove duplicate statistics caused by parameter differences; After extracting operator information from all the target graph nodes, deduplication is performed based on operator type to form an independent set of operator types that do not contain duplicate entries.

2. The operator processing method based on MindSpore according to claim 1, characterized in that, After verifying the functional output and gradient backpropagation of the target operator using the input tensor, the method further includes: The execution time of each target operator is collected on different devices; The computational efficiency and performance sensitivity of the target operator are calculated based on the execution time. A performance bottleneck map of the target operator is constructed based on computing power utilization and performance sensitivity; The performance bottleneck map is correlated with the MindSpore backend execution logs to identify bottleneck factors affecting the performance of the target operator. These bottleneck factors include memory bandwidth limitations, operator fusion failures, scheduling delays, or thread concurrency conflicts. For the target operators identified as having abnormal performance in the performance bottleneck graph, the operator-level optimization interface provided by the MindSpore graph optimizer is called to perform optimization operations to generate the optimized execution path of the target operator. The optimization operations include operator fusion and reconstruction, memory access mode rearrangement, instruction-level parallelism adjustment and scheduling priority reordering. The optimized execution path is written into the location of the target operator in the underlying execution graph to form an updated underlying execution graph structure. It is then reloaded into the MindSpore runtime environment to obtain the optimized output tensor and optimized gradient tensor of the optimized target operator for verification.

3. The operator processing method based on MindSpore according to claim 2, characterized in that, The calculation of the computational efficiency and performance sensitivity of the target operator based on the execution time specifically includes: The computational efficiency of the target operator is calculated based on the execution time and the theoretical computational complexity of the target operator. The theoretical computational complexity is calculated according to the input tensor dimension and operation type of the target operator. Based on the changing trend of the execution time ratio of the target operator under different input tensor sizes and concurrent operation conditions, a performance response model of the target operator is fitted and constructed. The response coefficient of the changing trend to the overall graph execution time change is calculated, and the response coefficient is defined as the performance sensitivity.

4. The operator processing method based on MindSpore according to claim 1, characterized in that, For each target operator in the set of independent operator types, a minimal executable network structure containing the target operator is constructed in the MindSpore framework. This verifies whether the target operator can be successfully recognized by the computation graph construction mechanism of the MindSpore framework and transformed into an intermediate representation. Specifically, this includes: For the target operator, based on the input tensor type and the output tensor type, a test tensor that satisfies the input and output dimension constraints is constructed. The tensor dimension and data type of the test tensor match the operator input requirements of the MindSpore framework. A single-operator network structure is constructed with the target operator as the core. The single-operator network structure contains only one execution node. The input of the execution node is the test tensor, and the output is the tensor transformation result after executing the target operator. The single-operator network structure is embedded into the front-end network construction interface of the MindSpore framework, and a computational graph is generated using a symbolic graphing mechanism. The MindSpore graph builder is invoked to perform graph-level parsing on the computation graph to determine whether the target operator can be successfully identified by the computation graph construction mechanism. If the target operator can be successfully identified by the computation graph construction mechanism, then the front-end network construction interface is called to convert the computation graph into the intermediate representation.

5. The operator processing method based on MindSpore according to claim 1, characterized in that, The graph compiler, which invokes the MindSpore framework, converts the intermediate representation into a low-level execution graph and determines whether the target operator can be parsed by the backend compiler and optimized at the graph level. Specifically, this includes: The entry function of the graph compiler is called to load the intermediate representation and bind the backend environment information of the target hardware. The backend environment information includes the device identifier, execution memory parameters, parallel granularity parameters and scheduler policy identifier of the target hardware. The intermediate representation parsing process is triggered in the graph compiler, and the structural graph nodes of the intermediate representation are traversed layer by layer; Based on the operator type, input / output tensor, parameter configuration and data dependency relationship corresponding to the parsed structural graph node, the target operator is mapped to the backend supported operator kernel library. If it is determined that the target operator is successfully mapped to the operator kernel library, then graph optimization operation is performed on the graph structure containing the target operator. The graph optimization operation includes operator fusion, redundant calculation elimination, memory rearrangement and constant folding. During the graph optimization operation, it is determined whether the target operator participates in fusion, whether it triggers scheduling replacement or kernel selection adjustment. If the target operator causes graph structure inconsistency during the optimization process, the target operator is marked as a successfully optimized operator.

6. The operator processing method based on MindSpore according to claim 1, characterized in that, The process involves loading the target operator into the MindSpore runtime environment and executing it on a designated hardware backend. The functional output and gradient backpropagation of the target operator are verified using input tensors. If the output error is below a set tolerance, the target operator is identified as a functionally valid operator. Specifically, this includes: The underlying execution graph containing the target operator is loaded into the MindSpore runtime execution engine. Based on the graph node scheduling order, kernel mapping information and memory allocation structure in the underlying execution graph, the initialization of the resources required by the target operator and the loading of execution instructions are completed on the specified hardware backend. Construct an input tensor that matches the input requirements of the target operator. The dimension, data type, and numerical distribution of the input tensor are consistent with the operator input requirements of the MindSpore framework. Then, inject the input tensor into the input buffer of the MindSpore runtime execution engine. The MindSpore runtime scheduler is invoked to execute the underlying execution graph. The forward computation of the target operator is completed through the specified hardware backend to obtain the forward output tensor of the target operator. The backward computation is completed based on the automatic differentiation mechanism to obtain the gradient tensor of the target operator. The forward output tensor is compared with the corresponding reference output to calculate the maximum error value, and the gradient tensor is compared with the reference gradient to calculate the mean square error value. If both the maximum error value and the mean square error value meet the tolerance threshold, then the target operator is identified as the functionally effective operator.

7. An operator processing device based on MindSpore, characterized in that, The apparatus is used to execute a MindSpore-based operator processing method as described in any one of claims 1-6, the apparatus comprising an acquisition module (201), a processing module (202), and an output module (203), wherein: The acquisition module (201) is used to acquire the complete computation graph of the target deep learning model in the inference and training stages, and to uniformly abstract all graph nodes with explicit computational semantics in the computation graph to extract a set of independent operator types. The processing module (202) is used to construct a minimum executable network structure containing the target operator in the MindSpore framework for each target operator in the set of independent operator types, verify whether the target operator can be successfully identified by the computation graph construction mechanism of the MindSpore framework, and convert it into an intermediate representation; The processing module (202) is used to call the graph compiler of the MindSpore framework to convert the intermediate representation into the underlying execution graph and determine whether the target operator can be parsed by the backend compiler and complete graph-level optimization. The output module (203) is used to load the target operator into the MindSpore runtime environment and execute it on a specified hardware backend if it is determined that the target operator can be parsed by the backend compiler and complete graph-level optimization. The output module verifies the functional output and gradient backpropagation of the target operator by input tensors. If the output error is lower than the set tolerance, the target operator is marked as a functionally valid operator.

8. An electronic device, characterized in that, The device includes a processor (301), a communication bus (302), a user interface (303), a network interface (304), and a memory (305). The memory (305) is used to store instructions. The user interface (303) and the network interface (304) are both used to communicate with other devices. The communication bus (302) is used to realize the connection and communication between the components within the electronic device. The processor (301) is used to execute the instructions stored in the memory (305) so that the electronic device performs the method as described in any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores instructions that, when executed, perform the method as described in any one of claims 1-6.

Images