A model optimization method, an electronic device, and a computer-readable storage medium

Abstract

This application discloses a model optimization method, an electronic device, and a computer-readable storage medium. The method includes: during the graph optimization process of the model to be optimized, detecting whether there is a failure in the derivation of the output tensor shape of the target operator in the model to be optimized; if so, obtaining the manually derived output tensor shape of the target operator; and performing graph optimization processing based on the manually derived output tensor shape of the target operator to obtain the target model. This ensures the smooth completion of model optimization, thereby significantly improving the performance and stability of the target model.

Description

Technical Field

[0001] This invention relates to the field of model technology, and in particular to a model optimization method, electronic device, and computer-readable storage medium. Background Technology

[0002] With the rapid development of artificial intelligence technology, models have been widely used in tasks such as image recognition, video analysis, and object detection. To meet the requirements of real-time performance and low power consumption, more and more models are being deployed in edge computing devices.

[0003] In edge computing devices, deep learning inference typically employs a heterogeneous computing architecture, adapting to different processing units based on the computational characteristics of each stage of model processing. Different processing units support limitations in their supported operators. When encountering custom or non-standard operators, graph optimization is often interrupted because the output tensor shape cannot be derived, thus affecting critical optimization processes such as operator fusion and constant folding. Summary of the Invention

[0004] The main technical problem addressed by this application is to provide a model optimization method, electronic device, and computer-readable storage medium that can ensure the smooth execution of model optimization.

[0005] To address the aforementioned technical problems, this application provides a model optimization method, comprising: during the graph optimization process of the model to be optimized, detecting whether there is a failure in the derivation of the output tensor shape of the target operator in the model to be optimized; if so, obtaining the manually derived output tensor shape of the target operator; and performing graph optimization processing based on the manually derived output tensor shape of the target operator to obtain the target model.

[0006] To solve the above-mentioned technical problems, another technical solution adopted in this application is: to provide an electronic device, including a memory and a processor, wherein the memory stores program instructions, and the processor retrieves the program instructions from the memory to execute the above-mentioned model optimization method.

[0007] To solve the above-mentioned technical problems, another technical solution adopted in this application is to provide a computer-readable storage medium including program data, which is used to implement the above-mentioned model optimization method when executed by a processor.

[0008] The model optimization method of this application detects whether there are any targets in the model whose output tensor shape derivation has failed during the graph optimization process. If so, the manually derived output tensor shape of the target operator is obtained. Graph optimization processing is then performed based on the manually derived output tensor shape of the target operator to obtain the target model. Thus, by detecting a target operator whose output tensor shape derivation has failed, the fully derived shape of the target operator is obtained, ensuring the smooth completion of model optimization and significantly improving the performance and stability of the target model. Attached Figure Description

[0009] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly described below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort, wherein: Figure 1 This is a flowchart illustrating an exemplary embodiment of the model optimization method shown in this application; Figure 2 yes Figure 1 A schematic flowchart of an exemplary embodiment of the model optimization method prior to step S110 is shown. Figure 3 This is a schematic diagram of a framework of an exemplary embodiment of the operator merging process shown in this application; Figure 4 This is a schematic diagram of the framework of an exemplary embodiment of the task execution process shown in this application; Figure 5 This is a schematic diagram of an exemplary embodiment of the task execution process shown in this application; Figure 6 This is a flowchart illustrating another exemplary embodiment of the model optimization method shown in this application; Figure 7 This is a schematic diagram of an exemplary embodiment of the model optimization apparatus shown in this application; Figure 8 This is a schematic diagram of the structure of an embodiment of the electronic device provided in this application; Figure 9 This is a schematic diagram of an embodiment of the computer-readable storage medium provided in this application. Detailed Implementation

[0010] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. It is understood that the specific embodiments described herein are only for explaining this application and not for limiting it. Furthermore, it should be noted that, for ease of description, only the parts related to this application are shown in the accompanying drawings, not all structures. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.

[0011] First, it's important to note that before deploying a model, it can be optimized to make it more concise and efficient. For example, model optimization tools can be used, such as ONNX Simplifier (OpenNeural Network Exchange Simplifier) ​​and TensorRT Graph Optimization. However, when encountering custom or non-standard operators, these tools often fail to derive the output tensor shape. If the output tensor shape of custom operators supported by the NPU is not explicitly derived, the optimization tool will be unable to complete forward shape analysis, causing critical optimizations such as operator fusion and constant folding to be interrupted or skipped. This ultimately results in a fragmented computation graph, affecting the effectiveness of model optimization.

[0012] Based on this, embodiments of this application propose a model optimization method, an electronic device, and a computer-readable storage medium. When the derivation of the output tensor shape of the target operator fails, the manually derived output tensor shape is obtained, ensuring the successful execution of graph optimization. For details, please refer to [reference needed]. Figure 1 , Figure 1 This is a flowchart illustrating an exemplary embodiment of the model optimization method shown in this application.

[0013] The execution entity of the model optimization method can be a terminal device, a server, or other processing device. The terminal device can be a user equipment (UE), computer, mobile device, user terminal, terminal, cellular phone, cordless phone, personal digital assistant (PDA), handheld device, computing device, in-vehicle device, wearable device, etc. The execution entity of the model optimization method can also be a model optimization device. In some possible implementations, the model optimization method can be implemented by a processor calling computer-readable instructions stored in memory.

[0014] Specifically, the model optimization method in this embodiment includes the following steps: S110: During the graph optimization process of the model to be optimized, detect whether there is a failure in the derivation of the output tensor shape of the target operator in the model to be optimized.

[0015] A model to be optimized refers to a model awaiting optimization. For example, the equipment required by the currently deployed edge devices can be considered the model to be optimized. Different edge devices may utilize different hardware, inference engines, etc., and different hardware and inference engines may offer varying degrees of support for the model. Therefore, when optimizing the model, it is necessary to perform model optimization based on the equipment information of the currently deployed edge devices. The model to be optimized can be a deep learning inference model to be deployed. The format of the model to be optimized can be ONNX format.

[0016] Graph optimization refers to a series of structural transformations of the computation graph without changing the model's input and output, thereby making the model more efficient. In graph optimization, the model can be represented by a computation graph, where nodes represent operations such as matrix multiplication, convolution, and addition, and edges represent data, i.e., tensors flowing between nodes. For example, the graph optimization process includes loading the model and resolving it into a computation graph in memory; a model optimization tool traversing the computation graph to find patterns to which optimization rules can be applied; reconstructing the computation graph based on the matched patterns to ensure that the optimized computation graph produces the same output as the original graph given the input; and finally, outputting the optimized model. It should be noted that the same output can have fluctuations within the allowable range of numerical error.

[0017] Operators are nodes in a model that perform computations, such as convolution, matrix multiplication, and addition. Input / output tensors are the edges connecting these operators, possessing a specific shape that describes the tensor's dimensions and the size of each dimension, typically represented as tuples like (N, C, H, W). The shape of an operator's output tensor is determined by the shape of its input tensor and its corresponding mathematical logic. For example, the type of operator determines the shape of its output tensor. When the operator type is a one-to-one mapping, the output tensor shape is the same as the input tensor shape; when the operator type is a mathematical formula, the output tensor shape is determined by the input tensor shape and the mathematical formula. For instance, for the operator GroupNorm (group normalization), both its output and input tensor shapes are [N, C, H, W].

[0018] The target operator refers to the operator in the model to be optimized that fails to derive the output tensor shape during the graph optimization process. For example, during the graph optimization of the model to be optimized, it is possible to detect in real time whether the output tensor shape derivation fails or the graph optimization is interrupted. If so, the operator causing this situation is obtained as the target operator. In practical applications, operators that commonly fail to derive the output tensor shape may be custom operators or non-standard operators in the model to be optimized.

[0019] S120: If so, obtain the manually derived output tensor shape of the target operator.

[0020] Manually derived output tensor shape refers to the shape of the output tensor of the target operator derived manually. When the model optimization device detects a failure in deriving the output tensor shape of the target operator, it can pause the optimization process and notify the user of this issue. The user can manually derive the output tensor shape of the target operator based on its input tensor shape and other information; after manually deriving the output tensor shape, the user inputs it into the model optimization device.

[0021] S130: Based on the manually derived tensor shape of the target operator, graph optimization processing is performed to obtain the target model.

[0022] The target model is the optimized model. For example, the target model can be a model that meets preset requirements, such as having advantages like a regular structure and hardware friendliness. For example, after obtaining the manually derived output tensor shape of the target operator, the model optimization device injects the manually derived output tensor shape as a ValueInfoProto node into the ONNX computation graph, restores the shape integrity of the node, and restarts the graph optimization process of the model. The ValueInfoProto node is a data structure in the ONNX model format used to describe tensor data information, including the tensor's name, data type, and tensor shape. During the restarted graph optimization process, other target operators in the model to be optimized that failed to derive are continuously detected, and the above shape completion process is repeated until the entire model has completed all optimization operations.

[0023] As can be seen, the model optimization method in this embodiment detects whether there are any failed derivations of the output tensor shape of the target operator in the model to be optimized during the graph optimization process. If so, the manually derived output tensor shape of the target operator is obtained. Based on the manually derived output tensor shape of the target operator, graph optimization processing is performed to obtain the target model. Thus, after detecting a target operator whose output tensor shape derivation has failed, the fully derived output tensor shape is obtained to achieve a complete shape analysis of the target operator, ensuring the smooth completion of model optimization, thereby significantly improving the performance and stability of the target model.

[0024] Based on the above embodiments, the embodiments of this application adopt... Figure 2 The flowchart details how to obtain the model to be optimized; please refer to [link / reference]. Figure 2 , Figure 2 yes Figure 1 The illustrated flowchart shows an exemplary embodiment of the model optimization method prior to step S110. Specifically, prior to step S110, the method further includes the following steps: S210: In response to the existence of a mismatched operator of the target processing unit in the initial model, the mismatched operator is modified to obtain the logical equivalent operator corresponding to the mismatched operator. The mismatched operator is an operator that is not supported by the target processing unit.

[0025] The initial model can be a model that has not yet undergone operator replacement. For example, a trained model can be used as the initial model.

[0026] The target processing unit refers to the processing unit in the device where the initial model is deployed. For example, the target processing unit may be an NPU (Neural Network Processing Unit). In other embodiments, the target processing unit may also be a CPU, a DSP (Digital Signal Processor) / IVE (Intelligent Video Engine), etc.

[0027] A mismatched operator refers to an operator that exists in the initial model but is not supported by the target processing unit. For example, after determining the device to which the initial model is to be deployed, the model optimization device identifies each operator in the initial model according to the list of operators supported by the target processing unit in the device, and determines the operators that exist in the initial model but are not supported by the target processing unit.

[0028] A logical equivalent operator is an operator that achieves the same effect as a mismatched operator. In some embodiments, the model optimization apparatus can rewrite the model based on the desired effect of the mismatched operator to obtain a logical equivalent operator. In other embodiments, the model optimization apparatus can also obtain at least one preset operator from a preset operator library of the target processing unit based on the mismatched operator; and generate a logical equivalent operator corresponding to the mismatched operator based on each preset operator. This ensures that the generated logical equivalent operator is supported by the target processing unit.

[0029] The preset operator library includes at least one preset operator. For example, the model optimization device can obtain a list of operators supported by the target processing unit and assemble the operators supported by the target processing unit into a preset operator library. Then, the model optimization device retrieves preset operators from the preset operator library that have the same processing logic as the mismatched operators. The number of preset operators may be one or more; for example, a single preset operator can be selected to form a logically equivalent operator, or multiple preset operators can be arranged sequentially according to the processing logic of the mismatched operators to obtain a logically equivalent operator.

[0030] Specifically, the steps described above for generating logical equivalent operators corresponding to mismatched operators based on each preset operator can further include: assembling each preset operator to obtain an initial operator; declaring the operator name, operator tensor, and attribute information of the initial operator to obtain the logical equivalent operators corresponding to the mismatched operators. This simplifies the declaration requirements, requiring only the operator name, operator tensor, and attribute information to achieve model optimization.

[0031] When there are multiple preset operators, they are assembled according to the processing logic to obtain the initial operator. The initial operator can be an operator that has not yet been declared. During model optimization, the operator declaration does not need to include the specific kernel implementation; only the operator name, operator tensor, and attribute information of the initial operator need to be declared, which greatly improves development speed. The operator tensor can include the operator's input tensor and output tensor.

[0032] S220: Replace the mismatch operators in the model to be optimized with logical equivalent operators to obtain the model to be optimized.

[0033] It should be noted that if mismatched operators in the model to be optimized are not replaced with logically equivalent operators, the conversion of mismatched operators may fail due to lack of support from the target processing unit in the device, resulting in the inability to operate normally. Alternatively, mismatched operators may fall into other processing units for execution, leading to a fragmented model execution path, increased data migration overhead, and disruption of subsequent optimization processes. As an example, taking the NPU as the target processing unit, if there are mismatched operators in the model to be optimized that are not supported by the target processing unit, and if these mismatched operators are not adapted, they may fall back to CPU execution during actual runtime, preempting CPU resources and disrupting the integrity of the NPU's main inference process. Based on this, the model optimization device can obtain the preset operator library supported by the NPU, match each operator in the initial model with each preset operator in the preset operator library to obtain mismatched operators; obtain at least one preset operator from the preset operator library based on the mismatched operators, and generate logical equivalent operators corresponding to the mismatched operators based on each preset operator; replace the mismatched operators in the initial model with the corresponding logical equivalent operators to obtain the model to be optimized.

[0034] As can be seen, the model optimization method in this application replaces mismatched operators not supported by the NPU with supported logical peer operators, ensuring that operators that should be executed by the NPU will not fall back to the CPU. This avoids frequent switching of the model execution path between the CPU and NPU, reducing data migration and synchronization overhead across processing units. Simultaneously, the adapted model can form a continuous computational subgraph on the NPU, which is beneficial for generating a compact pipeline scheduling plan. When multiple tasks are executed concurrently, the NPU can be exclusively used for the model inference stage, avoiding resource contention and execution interruptions caused by operator fragmentation, ensuring the performance stability of each inference task, and reducing end-to-end latency fluctuations.

[0035] When a replaced logical equivalent operator exists in the model to be optimized, the model optimization tool often fails to derive the output tensor shape for such operators, leading to graph optimization interruption and affecting key optimization processes such as operator fusion and constant folding. Therefore, this embodiment detects the problem of logical equivalent operators causing output tensor shape derivation failure or optimization interruption during the graph optimization process of the model to be optimized. Specifically, during the graph optimization process of the model to be optimized, the model optimization device determines whether there is a model optimization interruption phenomenon; if so, it determines from the reasons for the optimization interruption of the model to be optimized whether there is a failure to derive the output tensor shape of the target operator. If the reason for the optimization interruption is that there is a failure to derive the output tensor shape of the target operator, the optimization process is paused, the target operator currently blocking optimization is identified, the output tensor shape is manually derived based on the input tensor shape and mathematical logic of the target operator, and graph optimization processing is performed on the model to be optimized based on the manually derived output tensor shape to obtain the target model.

[0036] In other application scenarios, when the derivation of the output tensor shape fails, the graph optimization process of the model will not be interrupted, but the corresponding optimization process will be skipped. Therefore, the model optimization device can detect the derivation of the output tensor shape in the graph optimization process in real time. When it is found that the derivation of the output tensor shape of a target operator has failed, the optimization process is paused, the target operator that is currently blocking the optimization is identified, the output tensor shape is manually derived based on the input tensor shape and mathematical logic of the target operator, and the graph optimization process is performed on the model to be optimized based on the manually derived output tensor shape to obtain the target model.

[0037] The graph optimization process includes operator fusion, constant folding, and redundant node elimination. Operator fusion combines multiple operators in the computation graph into a single node, reducing the number of operators and improving computational efficiency. In the computation graph, if the values ​​of some nodes are determined at compile time, these nodes can be called constant nodes. Constant folding involves calculating these constant nodes at compile time, storing the results, and replacing the original constant operators, thus saving computational resources at runtime. Redundant node elimination identifies and removes redundant nodes from the computation graph, simplifying its structure and improving runtime efficiency.

[0038] In other embodiments, the model optimization device can inject the manually derived output tensor shape of the target operator into the output node of the target operator, obtain the adjacent and next-nearest operators of the target operator in the model to be optimized, and the adjacent and next-nearest operators are adjacent; in response to the execution process of the next-nearest operator including the execution process of the adjacent operator, the adjacent and next-nearest operators are merged to obtain the merged operator; the target model is determined based on the merged operator. This method of merging operators with repetitive logical operations reduces the number of operators and improves computational efficiency.

[0039] After obtaining the manually derived output tensor shape of the target operator, the model optimization device can inject this shape into the output node of the target operator to restore its shape integrity. The model optimization process is then restarted to continue. When it is detected that the execution process of the second-next neighbor operator of the target operator includes the execution process of an adjacent operator, the adjacent and second-next neighbor operators are merged, and the target model is determined based on the merged operator. It is understood that when the execution process of the second-next neighbor operator includes the execution process of an adjacent operator, it indicates that the second-next neighbor operator has duplicate execution actions of the adjacent operator. To reduce computational complexity, the model optimization device can merge the adjacent and second-next neighbor operators to reduce operator redundancy.

[0040] In this process, following the execution order of the model to be optimized, an adjacent operator can be the next operator of the target operator, and the next adjacent operator can be the next operator of the adjacent operator. In other embodiments, the merging process can also be any operator in the model to be optimized, and it may not be an adjacent or next adjacent operator of the target operator. The number of operators to be merged can be two or more, for example, two or more operators can be merged simultaneously.

[0041] In other embodiments, when the model optimization device detects that the execution process of an adjacent operator includes the execution process of the next adjacent operator, it can also merge the adjacent operator and the next adjacent operator to obtain the merged operator, and determine the target model based on the merged operator.

[0042] As an example, please refer to Figure 3 , Figure 3 This is a schematic diagram of an exemplary embodiment of the operator merging process shown in this application. The operators in the model to be optimized sequentially include convolution, logical equivalence operators, padding, and convolution. During model optimization, a problem occurs where the output tensor shape derivation fails during the logical equivalence operator, pausing the graph optimization process. The output tensor shape of the logical equivalence operator is manually injected, restarting the graph optimization. Subsequently, padding is detected before the convolution in the model to be optimized. During graph optimization, the convolution and padding are merged to obtain the merged convolution, ultimately yielding the target model. However, when the logical equivalence operator's output tensor shape is not manually derived and there is no complete graph optimization process, the subsequent padding and convolution of that logical equivalence operator will not be merged. Instead, the padding and convolution processes will be executed during the inference process, increasing the model execution steps.

[0043] After obtaining the target model, it can be deployed to the corresponding device, and the tasks to be executed for the target model can be allocated to different processing units of the device based on the hardware architecture characteristics of the device. For example, the tasks to be executed for the target model are obtained; the tasks to be executed for the target model are split into subtasks according to the attribute information of each processing unit, resulting in at least one subtask; each subtask is then allocated to its corresponding processing unit for execution, yielding the target execution result. This allows each subtask to be executed independently in different processing units, avoiding resource contention and cross-processing unit execution.

[0044] The task to be executed can be the complete inference task to be performed by the target model. After obtaining the task to be executed from the target model, it can be split into subtasks based on the attribute information of each processing unit, and the resulting subtasks can be assigned to the corresponding processing units for execution to obtain the target execution result. The attribute information of the processing unit includes its computing power, throughput, memory bandwidth, and resource availability.

[0045] A common task decomposition approach is to divide the task to be executed into three sub-tasks: input data preprocessing, model inference, and output post-processing, and then assign these three sub-tasks to corresponding processing units. As an example, the device can employ a heterogeneous computing architecture, including CPUs, DSPs, IVEs, and NPUs. Input data preprocessing can be assigned to DSPs, CPUs, or IVEs for execution; model inference is performed by the NPU; and post-processing can be assigned to CPUs, DSPs, or other hardware resources. Input data preprocessing can be further divided into operations such as image filtering, image format conversion, normalization, scaling, and padding; and output post-processing can be further divided into operations such as image decoding and image denormalization.

[0046] For details, please refer to [link / reference]. Figure 4 , Figure 4 This is a schematic diagram illustrating an exemplary embodiment of the task execution process shown in this application. Firstly, within the node's internal configuration, the node's runtime resources can be configured based on information from the device. For example, the input data preprocessing process can be handled by the IVE (Entity Execution Environment), with a dump node added after the IVE to export the node's post-processing information and verify the node's processing results. Of course, dump nodes can also be added after other nodes. The model inference process can be handled by the NPU (Natural Processing Unit), and the output result post-processing can be executed by the CPU, ultimately outputting the final target execution result. In other application scenarios, resizing may be allocated to DSP (Digital Signal Processing), etc.

[0047] Specifically, when the processing unit includes a first processing unit and a second processing unit, the first processing unit executes subtasks earlier than the second processing unit. The model optimization device responds to the first processing unit by executing the corresponding subtasks and obtaining the first result output by the first processing unit. The second processing unit receives the first result from the first processing unit and executes the subtasks based on the first result to obtain the target execution result. In this way, it is ensured that there is only a data dependency between the subtasks, and there is no resource preemption or scheduling conflict on the same processing unit.

[0048] In other embodiments, at least two processing units include a first processing unit, a second processing unit, and a third processing unit. The first processing unit performs processing on a corresponding subtask to obtain a first result output by the first processing unit. The second processing unit receives the first result from the first processing unit and executes the corresponding subtask based on the first result to obtain a second result. The third processing unit receives the second result from the second processing unit and executes the corresponding subtask based on the second result to obtain a target execution result. And so on.

[0049] As an example, please refer to Figure 5Taking a DSP as the first processing unit, an NPU as the second processing unit, and a CPU as the third processing unit as the third processing unit, while the first frame of the image is being post-processed by the CPU, the second frame of the image is being used for model inference on the NPU, and the third frame of the image is being used for input data preprocessing on the DSP. This enables multiple inference tasks or consecutive video frames to be executed in a pipelined parallel manner across different processing units, improving the resource utilization of each processing unit. Simultaneously, each processing unit works independently, only transmitting intermediate results synchronously through data in memory, avoiding runtime resource contention and context overhead.

[0050] To elaborate on the model optimization method applied in this application, Figure 6 The flowchart shown below provides further explanation, as detailed below: First, taking the NPU as the target processing unit as an example, the model optimization device obtains the initial model to be deployed, performs matching processing on each operator in the initial model according to the preset operator library supported by the target processing unit, identifies mismatched operators not supported by the NPU, obtains preset operators from the preset operator library based on the mismatched operators, and adapts the preset operators to logical equivalent operators supported by the NPU, and replaces the mismatched operators in the initial model with the corresponding logical equivalent operators to obtain the model to be optimized. Graph optimization is performed on the model to be optimized. During the optimization process, if the output tensor shape derivation of the target operator fails, the optimization process is paused. The output tensor shape of the target operator is then manually derived and injected into the corresponding node of the target operator to restore the shape integrity of that node. The model optimization process is then restarted to continue optimizing other target operators with failed output tensor shape derivation to complete the shape information until the entire model is graph optimized. It should be noted that in practical applications, due to the limitations of model optimization tools, the replaced logical equivalent operators are more likely to be target operators with failed output tensor shape derivation. After obtaining the target model through graph optimization, the target model can be used to execute the corresponding tasks to be executed. Different operators of the target model are deployed in different processing units of the device, so it is necessary to configure the tasks to be executed. Specifically, the tasks are split according to the attribute information of the processing unit to obtain at least one sub-task; each sub-task is assigned to each processing unit for execution to obtain the final target execution result.

[0051] Please see Figure 7 , Figure 7This is a schematic diagram of an exemplary embodiment of the model optimization device shown in this application. The vehicle type recognition device 700 includes a detection module 710, an acquisition module 720, and an optimization module 730. The detection module 710 is used to detect whether there is a failure in the derivation of the output tensor shape of the target operator in the model to be optimized during the graph optimization process of the model to be optimized. The acquisition module 720 is used to acquire the manually derivation output tensor shape of the target operator. The optimization module 730 is used to perform graph optimization processing based on the manually derivation output tensor shape of the target operator to obtain the target model.

[0052] In the above scheme, during the graph optimization process of the model to be optimized, the model optimization device detects whether there are any targets whose output tensor shape derivation has failed. If so, it obtains the manually derived output tensor shape of the target operator. Based on the manually derived output tensor shape of the target operator, graph optimization processing is performed to obtain the target model. Thus, after detecting a target operator whose output tensor shape derivation has failed, obtaining the manually derived output tensor shape enables a complete shape analysis of the target operator, ensuring the smooth completion of model optimization and significantly improving the performance and stability of the target model.

[0053] The functions of each module can be found in the implementation examples of the model optimization method, and will not be repeated here.

[0054] To implement the model optimization method of the above embodiments, this application proposes another electronic device, please refer to [link / reference needed]. Figure 8 , Figure 8 This is a schematic diagram of the structure of an embodiment of the electronic device provided in this application.

[0055] Electronic device 800 includes memory 810 and processor 820, wherein memory 810 and processor 820 are coupled together.

[0056] The memory 810 is used to store program data, and the processor 820 is used to execute the program data to implement the model optimization method of the above embodiment.

[0057] In this embodiment, processor 820 can also be referred to as a CPU (Central Processing Unit). Processor 820 may be an integrated circuit chip with signal processing capabilities. Processor 820 can also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. The general-purpose processor can be a microprocessor, or processor 820 can be any conventional processor.

[0058] This application also provides a computer-readable storage medium, such as Figure 9 As shown, the computer-readable storage medium 900 is used to store program data 910, which, when executed by the processor, is used to implement the model optimization method as described in the method embodiments of this application.

[0059] The methods involved in the model optimization method embodiments of this application, when implemented as software functional units and sold or used as independent products, can be stored in a device, such as a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0060] The above description is merely an embodiment of this application and does not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

Claims

1. A model optimization method, characterized in that, The model optimization method includes: During the graph optimization process of the model to be optimized, it is detected whether there is a failure in the derivation of the output tensor shape of the target operator in the model to be optimized; If so, then obtain the manually derived output tensor shape of the target operator; The target model is obtained by performing graph optimization on the manually derived tensor shape of the target operator.

2. The model optimization method according to claim 1, characterized in that, Before the step of detecting whether the derivation of the output tensor shape of the target operator fails in the graph optimization process of the model to be optimized, the method further includes: In response to the existence of a mismatched operator of the target processing unit in the initial model, the mismatched operator is modified to obtain the logical equivalent operator corresponding to the mismatched operator, wherein the mismatched operator is an operator not supported by the target processing unit. The mismatch operator in the model to be optimized is replaced with the logical equality operator to obtain the model to be optimized.

3. The model optimization method according to claim 2, characterized in that, The step of modifying the mismatch operator to obtain the logical equivalent operator corresponding to the mismatch operator includes: At least one preset operator is obtained from the preset operator library of the target processing unit according to the mismatch operator; Generate logical equivalent operators corresponding to the mismatched operators based on each preset operator.

4. The model optimization method according to claim 3, characterized in that, The step of generating logical equivalent operators corresponding to the mismatched operators based on each preset operator includes: The preset operators are combined to obtain the initial operators; Declare the operator name, operator tensor, and attribute information of the initial operator to obtain the logical equivalent operator corresponding to the mismatched operator.

5. The model optimization method according to claim 1, characterized in that, The model optimization apparatus includes at least two processing units. After the step of performing graph optimization processing on the manually derived output tensor shape based on the target operator to obtain the target model, the method further includes: Obtain the task to be executed for the target model; The task to be executed in the target model is split into subtasks based on the attribute information of each processing unit, resulting in at least one subtask. Each subtask is assigned to its corresponding processing unit for execution, resulting in the target execution result.

6. The model optimization method according to claim 5, characterized in that, The at least two processing units include a first processing unit and a second processing unit, wherein the first processing unit executes the sub-tasks in a sequence earlier than the second processing unit. The step of deploying each sub-task to its corresponding processing unit for execution to obtain the target execution result includes: In response to the first processing unit performing processing on the corresponding subtask, a first result output by the first processing unit is obtained; The second processing unit receives the first result from the first processing unit and performs sub-tasks based on the first result to obtain the target execution result.

7. The model optimization method according to claim 1, characterized in that, The step of performing graph optimization processing on the manually derived tensor shape based on the target operator to obtain the target model includes: The manually derived tensor shape of the target operator is injected into the output node of the target operator, and the adjacent operators and the second adjacent operators of the target operator in the model to be optimized are obtained, wherein the adjacent operators and the second adjacent operators are adjacent; In response to the fact that the execution process of the next adjacent operator includes the execution process of the adjacent operator, the adjacent operator and the next adjacent operator are merged to obtain the merged operator; The target model is determined based on the merged operator.

8. The model optimization method according to claim 1, characterized in that, The step of detecting whether there is a failure in the derivation of the output tensor shape of the target operator in the graph optimization process of the model to be optimized includes: During the graph optimization process of the model to be optimized, it is determined whether there is a model optimization interruption phenomenon; If so, determine from the reasons for the interruption of the optimization of the model to be optimized whether there is a failure in the derivation of the output tensor shape of the target operator.

9. An electronic device, characterized in that, include: A memory and a processor, wherein the memory stores program instructions, and the processor retrieves the program instructions from the memory to perform the method as claimed in any one of claims 1-8.

10. A computer-readable storage medium, characterized in that, include: The system stores program data, which, when executed by a processor, is used to implement the method as described in any one of claims 1-8.

Images