A processing method of a neural network model, an electronic device, and a readable medium

Abstract

The application relates to the technical field of machine learning, and discloses a neural network model processing method, an electronic device and a readable medium. The neural network model processing method comprises the following steps: determining a conversion node in a first neural network model; determining attribute values of parent nodes and child nodes of each conversion node in the first neural network model; determining each conversion node group matched in attributes based on the attribute values of the parent nodes and the child nodes of each conversion node in the first neural network model; wherein each conversion node group matched in attributes comprises two adjacent conversion nodes, and the conversion modes of the two adjacent conversion nodes are opposite; and deleting each conversion node group matched in attributes in the first neural network model to obtain a second neural network model. The above scheme can reduce the memory space occupied by the neural network model and improve the running efficiency of the neural network model by deleting the conversion nodes matched in attributes.

Description

Technical Field

[0001] This application relates to the field of machine learning technology, and in particular to a method for processing neural network models, an electronic device, and a readable medium. Background Technology

[0002] In the engineering deployment of neural network models, there is a problem of inconsistent interfaces between the platform for training the neural network model (such as a server) and the electronic devices that apply the neural network model, which makes it impossible to directly apply the neural network model to the terminal device.

[0003] To address these issues, the trained neural network model can first be converted to a format supported by the interface of the terminal device where it will be deployed. Then, the converted neural network model can be deployed on the terminal device. However, this format conversion process introduces numerous memory-intensive conversion operations, such as layout operations, which in turn introduce transpose operators like transpose or permute operators. This increases the memory footprint of the converted neural network model and reduces its overall performance. Summary of the Invention

[0004] To reduce the memory space occupied by neural network models, this application provides a method for processing neural network models, an electronic device, and a readable medium.

[0005] In a first aspect, embodiments of this application provide a method for processing a neural network model for use in an electronic device. The method includes: determining transformation nodes in a first neural network model, wherein the transformation nodes include nodes introduced by performing transformation operations on the first neural network model; determining the attribute values ​​of the parent and child nodes of each transformation node in the first neural network model; determining groups of transformation nodes with matching attributes based on the attribute values ​​of the parent and child nodes of each transformation node in the first neural network model; wherein each group of transformation nodes with matching attributes includes two adjacent transformation nodes, and the transformation modes of the two adjacent transformation nodes are opposite; and deleting the groups of transformation nodes with matching attributes in the first neural network model to obtain a second neural network model.

[0006] This application embodiment reduces the memory space occupied by the neural network model and improves the running efficiency of the neural network model by deleting attribute-matching transformation nodes, thus achieving global optimization of the neural network model.

[0007] In one possible implementation, based on the attribute values ​​of the parent and child nodes of each transformation node in the first neural network model, the transformation node groups that match the attributes are determined, including: obtaining two adjacent transformation nodes in the first neural network model; determining that the attribute value of the parent node of the first transformation node is the same as the attribute value of the child node of the second transformation node, and the attribute value of the child node of the first transformation node is the same as the attribute value of the parent node of the second transformation node, and taking the first transformation node and the second transformation node as the transformation node group that matches the attributes.

[0008] In one possible implementation, determining the transition nodes in the first neural network model includes: determining the transition nodes in the first neural network model based on a block-level formatting context algorithm.

[0009] In one possible implementation, determining the attribute values ​​of the parent and child nodes of each transformation node in the first neural network model includes: determining the parent and child nodes of each transformation node in the first neural network model based on a depth-first search algorithm; and determining the attribute values ​​of the parent and child nodes of each transformation node in the first neural network model according to the data format of the parent and child nodes of each transformation node in the first neural network model.

[0010] In one possible implementation, the method further includes: labeling each transformation node in the first neural network model with a first identifier; labeling the parent and child nodes of each transformation node in the first neural network model with a second identifier; determining the non-transformation nodes in the first neural network model based on a block-level formatting context algorithm, and labeling each non-transformation node in the first neural network model with a third identifier.

[0011] In one possible implementation, deleting each group of transition nodes in the first neural network model that matches the attribute includes: determining that each node in the first neural network model has an identifier, and deleting each group of transition nodes in the first neural network model that matches the attribute, wherein the identifier includes a first identifier, a second identifier, or a third identifier.

[0012] In one possible implementation, after the step of deleting the attribute-matching transformation node groups in the first neural network model, the method further includes: determining the attribute value of the parent node of the first transformation node in the deleted transformation node group of the first neural network model; or determining the attribute value of the child node of the second transformation node in the deleted transformation node group of the first neural network model; updating the parameters of the nodes of the first neural network model based on the attribute value of the parent node of the first transformation node; or updating the parameters of the nodes of the first neural network model based on the attribute value of the child node of the second transformation node.

[0013] It's understandable that attribute values ​​can be assigned based on the node's attributes. For example, a node converting from NHWC (where N represents batch, H represents height, W represents width, and C represents channel) format to NCHW format can be assigned a layout identifier (e.g., a-layout), and a node converting from NCHW format to NHWC format can be assigned a layout identifier (e.g., b-layout). For instance, based on the attribute values ​​of the parent node of conversion node M or the child nodes of conversion node N in the conversion node group, the parameters of the nodes between the original conversion nodes M and N are updated. These parameters include the attribute values, data format, shape, and dimensions of the conversion nodes. For example, the data format of the NCHW format nodes between the original conversion nodes M and N can be updated to NHWC format, the attribute values ​​can be updated to the b-layout identifier, and the shape and dimensions of the NCHW format nodes between the original conversion nodes M and N can also be updated.

[0014] In one possible implementation, the transformation node includes a transpose operator or a permutation operator.

[0015] In a second aspect, embodiments of this application provide an electronic device, including: a memory for storing instructions executed by one or more processors of the electronic device, and a processor, which is one of the one or more processors of the electronic device, for implementing the processing method of any neural network model provided by the first aspect and various possible implementations of the first aspect.

[0016] Thirdly, embodiments of this application provide a readable medium storing instructions that, when executed on an electronic device, cause the electronic device to implement the processing method of any neural network model provided by the first aspect and various possible implementations of the first aspect. Attached Figure Description

[0017] Figure 1 According to some embodiments of this application, a schematic diagram of a scenario in which a neural network model 11 is deployed on a mobile phone 20 is shown;

[0018] Figure 2 According to some embodiments of this application, a flowchart of a method for processing a neural network model is shown;

[0019] Figure 3 According to some embodiments of this application, a flowchart of a method for processing a neural network model is shown;

[0020] Figure 4 According to some embodiments of this application, a flowchart of a method for processing a neural network model is shown;

[0021] Figure 5According to some embodiments of this application, a schematic diagram of the structure of an electronic device 100 is shown. Detailed Implementation

[0022] The illustrative embodiments of this application include, but are not limited to, a method for processing a neural network model, an electronic device, and a readable medium.

[0023] It is understood that the neural network model mentioned in the technical solution of this application can be a neural network model used in tasks such as image recognition, object detection, reinforcement learning, and semantic analysis. The types of neural network models are not limited to convolutional neural networks, Transformer (a model based on multi-head attention mechanism), recurrent neural networks (RNN), long short-term memory neural networks (LSTM), and any other neural network model.

[0024] like Figure 1 As shown, in one specific implementation, server 10 can perform format conversion on the NHWC format neural network model 11 used for image detection based on a conversion node, obtaining an NCHW format neural network model 11 supported by the interface of mobile phone 20. Then, the format-converted neural network model 11 is sent to mobile phone 20 so that mobile phone 20 can perform image recognition based on the format-converted NCHW format neural network model. In some embodiments, the mobile phone can also be replaced by any implementable terminal device such as a tablet computer or computer. However, during the format conversion process of neural network model 11 by server 10, transpose or permutation operators are introduced, causing the format-converted neural network model 11 to occupy a large amount of memory space, reducing the overall performance of the neural network model.

[0025] To address the aforementioned issues, this application provides a method for processing a neural network model. The method includes: after converting the neural network model to a format supported by the terminal device's interface, traversing the neural network model across multiple dimensions (such as breadth and depth) to determine transformation nodes (such as transpose or permutation operators) within the model. These transformation nodes are used to convert data formats, such as converting NHWC format data to NCHW format data. When any two adjacent transformation nodes are determined to have opposite transformation modes, or attribute matching, the attribute-matching transformation nodes are deleted. It can be understood that two adjacent attribute-matching transformation nodes ensure that the format of the converted data is the same as the format before conversion, and the data between two adjacent attribute-matching transformation nodes is intermediate data generated during the format conversion process. Therefore, deleting two adjacent attribute-matching transformation nodes does not affect the normal operation of the neural network model. Thus, by deleting attribute-matching transformation nodes, the memory space occupied by the neural network model can be reduced, the operating efficiency of the neural network model can be improved, and global optimization of the neural network model can be achieved.

[0026] In some embodiments, after deleting the attribute-matching transformation node groups in the initial neural network model (i.e., the first neural network model mentioned in this application), a second neural network model can be obtained, and the input data can be processed based on the second neural network model. The input data of the second neural network model can be image data, audio data, etc., and can realize tasks such as image recognition and target detection. For example, identifying whether there is a target person in an image.

[0027] In some embodiments, determining whether the attributes of any two adjacent transformation nodes match can be done as follows: Transformation node M and transformation node N are considered to match if the data format of the parent node of transformation node M is the same as the data format of the child node of transformation node N, and the data format of the child node of transformation node M is the same as the data format of the parent node of transformation node N. For example, when it is determined that the data format of the parent node of transformation node M is NHWC and the data format of its child node is NCHW, and the data format of the parent node of transformation node N is NCHW and the data format of its child node is NHWC, then the attributes of transformation node M and transformation node N are considered to match.

[0028] In some embodiments, the method of traversing the neural network model based on multiple dimensions (such as breadth and depth) to determine the transformation nodes in the neural network model and delete the transformation nodes with matching attributes may include: traversing each layer of the neural network model breadth-wise using a block formatting context (BFC) to determine the transformation nodes introduced by the transformation operation and assigning a first identifier to each transformation node; traversing the parent and child nodes of each transformation node in the neural network model sequentially using a depth-first search (DFS) algorithm and assigning attribute values ​​to the parent and child nodes of each transformation node in the neural network model; determining the transformation nodes with matching attributes among the transformation nodes based on the attribute values ​​of the parent and child nodes of each transformation node in the neural network model to obtain a transformation node group, and deleting each node in the transformation node group.

[0029] In this embodiment, the neural network model is traversed across multiple dimensions, avoiding the omission of transformation nodes. Transformation nodes with matching attributes are determined based on attribute values, preventing the accidental deletion of nodes that determine the performance of the neural network model and achieving global optimization of the neural network model.

[0030] The method for assigning attribute values ​​to the parent and child nodes of each transformation node in the neural network model can include: sequentially traversing the parent and child nodes of each transformation node in the neural network model; determining whether the current node has a first identifier or a second identifier; if the result is no, assigning attribute values ​​to the current node according to its attributes and marking it with the second identifier; if the result is yes, traversing to the next node until there are no more nodes, then stopping the traversal.

[0031] The above-mentioned method of assigning attribute values ​​to the current node based on its attributes may include: assigning an a layout identifier (e.g., a-layout) to a node converting from NHWC format to NCHW format, and assigning a b layout identifier (e.g., b-layout) to a node converting from NCHW format to NHWC format.

[0032] The above method of determining the attribute-matching transformation nodes in the transformation nodes based on the attribute values ​​of the parent and child nodes of each transformation node in the neural network model, obtaining the transformation node group, and deleting each node in the transformation node group can be as follows: when it is determined that the parent node of transformation node M has the layout identifier 'a' and the child node has the layout identifier 'b', and the parent node of transformation node N has the layout identifier 'b' and the child node has the layout identifier 'a', then it is determined that the attributes of transformation node M and transformation node N match, that is, transformation node M and transformation node N are a pair of transformation node groups, and transformation node M and transformation node N are deleted.

[0033] In some embodiments, after deleting each node in the transformation node group, the parameters of the nodes in the neural network model can be updated based on the attribute values ​​of the parent node of the first transformation node or the child node of the second transformation node in the transformation node group, so as to unify the parameters of each node in the neural network model. For example, based on the attribute values ​​of the parent node of transformation node M or the child node of transformation node N in the transformation node group, the parameters of the nodes between the original transformation node M and the original transformation node N are updated, where the parameters include the attribute values ​​of the transformation nodes, data format, shape, dimensions, etc. For example, the data format of the NCHW format nodes between the original transformation nodes M and N is updated to NHWC format, the attribute values ​​are updated to b layout identifier, and the shape, dimensions, etc. of the NCHW format nodes between the original transformation nodes M and N are updated.

[0034] The processing method of the neural network model provided in the embodiments of this application will be described in detail below. The processing method of the neural network model in the embodiments of this application is applied to a terminal device. Figure 2 The diagram illustrates a processing method for a neural network model according to an embodiment of this application. The processing method for the neural network model includes:

[0035] 201: After the first neural network model is converted into a format supported by the terminal device's interface, the first neural network model is traversed by BFS.

[0036] In this embodiment, the first neural network model can be first searched for all transformation nodes, such as transpose operators or permutation operators, by using algorithms such as BFS to perform a breadth-first traversal of the first neural network model, so as to avoid missing any transformation nodes in the first neural network model.

[0037] The conversion node is used to convert the data format, such as converting NHWC format data to NCHW format data, and NCHW format data to NHWC format data.

[0038] 202: Assign an identifier to each node based on the parameters of each node in the first neural network model, wherein the identifier includes a first identifier and a third identifier.

[0039] The processing method in step 202 is as follows: Figure 3 The example shown. Figure 3 As shown, step 202 includes:

[0040] 301: Determine whether the i-th node is a transition node. If the result is yes, proceed to 302: Mark the i-th node with the first identifier; if the result is no, proceed to 303: Mark the i-th node with the third identifier.

[0041] In some embodiments, it can be determined whether the i-th node is a memory-intensive transformation node, such as a layout transformation node, based on the parameters of the i-th node. When the i-th node contains transformation parameters, such as a transpose operator with transformation parameters, the i-th node is determined to be a transformation node and is marked with a first identifier. When the i-th node does not contain transformation parameters, such as a convolution node or an analysis node, the i-th node is determined not to be a transformation node and is marked with a third identifier. The first identifier can be gray, and the third identifier can be white. It is understood that the first and third identifiers can be set according to actual conditions.

[0042] Therefore, by labeling different types of nodes with different identifiers, the transformation nodes can be filtered out.

[0043] 302: Mark the first identifier for the i-th node.

[0044] 303: Mark the i-th node with a third identifier.

[0045] 304: i = i + 1

[0046] 305: Determine if the i-th node exists. If the result is yes, proceed to 301: Determine if the i-th node is a transformation node. If the result is no, end.

[0047] 203: Traverse the parent and child nodes of the node with the first identifier in the first neural network model using DFS.

[0048] To avoid deleting transformation nodes with algorithmic semantics and affecting the performance of the first neural network model, in this embodiment, the parent and child nodes of the nodes with the first identifier in the first neural network model are traversed by depth-first search (DFS) to identify transformation nodes with the first identifier but without algorithmic semantics, and then these transformation nodes are deleted.

[0049] In this embodiment of the application, the nodes with the first identifier can be saved to array T[N], and then the parent and child nodes of each node in array T[N] can be traversed by DFS.

[0050] 204: Based on the parameters of the parent and child nodes of each node with a first identifier in the first neural network model, determine the attribute values ​​of the parent and child nodes of each node with a first identifier, and mark the parent and child nodes of each node with a first identifier with a second identifier.

[0051] The processing method in step 204 is as follows: Figure 4 The example shown. Figure 4 As shown, step 204 includes:

[0052] 401: Determine whether the parent and child nodes of the i-th node with the first identifier in the first neural network model have either the first or second identifier. If the result is no, proceed to 402: Assign attribute values ​​to the corresponding nodes based on the parameters of the parent and child nodes of the i-th node, and mark the corresponding nodes with the second identifier. If the result is yes, proceed to 403: Take the next node with the first identifier as the starting point for traversal, i = i + 1.

[0053] In this embodiment, the parent node of the i-th node with the first identifier can be traversed first, and then the child nodes of the i-th node with the first identifier can be traversed. In some embodiments, the child nodes of the i-th node with the first identifier can be traversed first, and then the parent node of the i-th node with the first identifier can be traversed.

[0054] It's understandable that when the parent or child node of the i-th node has a first identifier, it's determined to be a transformation node, so there's no need to further determine its attribute value; the (i+1)-th node with the first identifier can be traversed. When the i-th node has a second identifier, it's determined that its parent or child node has already been marked with an attribute value, so there's no need to assign an attribute value; the (i+1)-th node with the first identifier can be traversed. When the parent or child node of the i-th node has neither a first nor a second identifier, an attribute value can be assigned to the parent node using the layout function [ni][Parent], or an attribute value can be assigned to the child node using the layout function [ni][Child]. The second identifier can be a black identifier.

[0055] For example, first, iterate through the parent nodes of the first node. If the first parent node of the first node has neither a first identifier nor a second identifier, assign the layout identifier 'a' to the first parent node using the layout function [ni][parent node] according to the format of the first parent node (e.g., NHWC format). Continue this process, assigning attribute values ​​to each parent node of the first node using the layout function [ni][parent node]. Then, iterate through the child nodes of the first node. If the first child node of the first node has neither a first identifier nor a second identifier, assign the layout identifier 'b' to the first child node using the layout function [ni][child node] according to the format of the first child node (e.g., NCHW format). Continue this process, assigning attribute values ​​to each child node of the first node using the layout function [ni][child node].

[0056] It is understandable that attribute values ​​can include layout identifiers such as 'a' and 'b'. Attribute values ​​are identifiers without logical information and will not affect the attributes of the marked node.

[0057] 402: Assign attribute values ​​to the corresponding node based on the parameters of the parent and child nodes of the i-th node, and mark the corresponding node with a second identifier.

[0058] 403: Take the next node with the first identifier as the starting point for traversal, i = i + 1.

[0059] 404: Determine if there exists an i-th node with a first identifier. If the result is yes, proceed to 401: Determine if the parent and child nodes of the i-th node with the first identifier in the first neural network model have a first identifier or a second identifier. If the result is no, end.

[0060] It is understandable that the traversal stops after all nodes with the first identifier in the first neural network model have been labeled.

[0061] 205: Determine if all nodes in the first neural network model have an identifier. If the result is yes, proceed to 206: Based on the attribute values ​​of each node in the first neural network model, determine the matching nodes among the nodes with the first identifier in the first neural network model. If the result is no, end.

[0062] It is understood that when there are nodes in the first neural network model that do not have an identifier (such as a first identifier, a second identifier, or a third identifier), the first neural network model is not suitable for the processing method of the first neural network model provided in the embodiments of this application, and the processing flow ends.

[0063] 206: Based on the attribute values ​​of each node in the first neural network model, determine the transformation node group in the first neural network model.

[0064] In this embodiment, when it is determined that the parent node of node M with the first identifier has a layout identifier 'a' and the child node has a layout identifier 'b', and the parent node of node N with the first identifier has a layout identifier 'b' and the child node has a layout identifier 'a', it is determined that the transformation modes of node M with the first identifier and node N with the first identifier are opposite, or in other words, attribute matching. When transformation node M and transformation node N are adjacent, it is determined that node M with the first identifier and node N with the first identifier form a pair of transformation node groups.

[0065] It is understood that the nodes with the first identifier in the first neural network model include not only the transformation nodes introduced by the format conversion operation, but also the transformation nodes with algorithmic semantics in the first neural network model. In this embodiment, based on the attribute values ​​of the parent and child nodes of the nodes with the first identifier, the matching nodes among the nodes with the first identifier in the first neural network model are determined, that is, the transformation nodes without algorithmic semantics are identified.

[0066] 207: Delete each transformation node group in the first neural network model.

[0067] This application embodiment solves the problem of redundant conversion nodes in the first neural network model by deleting each conversion node group in the first neural network model, thereby simplifying the first neural network model and improving its performance.

[0068] 208: Update the parameters of the nodes with the second identifier in the first neural network model according to the attribute value of the parent node of the first transformation node in each transformation node group in the first neural network model, or the attribute value of the child node of the second transformation node, to obtain the second neural network model.

[0069] It is understood that the second neural network model obtained in the embodiments of this application can be a neural network model used in tasks such as image recognition, object detection, reinforcement learning, and semantic analysis, and the input data can be image, audio, video data, etc.

[0070] In this embodiment, the attribute values ​​of the input or output data of each transformation node group can be determined based on the attribute values ​​of the parent node of the first transformation node or the child node of the second transformation node in each transformation node group of the first neural network model. This allows for the adjustment of the parameters of each node in the first neural network model based on the attribute values ​​of the input or output data of each transformation node group. The parameters include the attribute values ​​of the transformation nodes, data format, shape, dimension, etc.

[0071] For example, based on the attribute values ​​of the parent node of transformation node M in the transformation node group, or the attribute values ​​of the child nodes of transformation node N, update the parameters of the nodes between the original transformation node M and the original transformation node N. For instance, update the data format of the NCHW format nodes between the original transformation node M and the original transformation node N to NHWC format, update the attribute values ​​to the b layout identifier, and update the shape, dimensions, etc. of the NCHW format nodes between the original transformation node M and the original transformation node N.

[0072] To address the issue that transpose or permutation operators introduced by transformation operations consume significant memory and degrade the overall performance of neural network models, some embodiments utilize pattern matching to remove these operators. However, this method can only identify a subset of transpose or permutation operators in the neural network model and cannot optimize the entire model globally.

[0073] The neural network model processing method provided in this application traverses the neural network model based on multiple dimensions (such as breadth and depth), which can determine all transpose operators or permutation operators in the neural network model, and realize global optimization of the neural network model.

[0074] Figure 5 According to some embodiments of this application, a schematic diagram of the structure of an electronic device 100 is shown. For example... Figure 5 As shown, the electronic device 100 includes one or more processors 101, system memory 102, non-volatile memory (NVM) 103, communication interface 104, input / output (I / O) devices 105, and system control logic 106 for coupling the processor 101, system memory 102, non-volatile memory 103, communication interface 104, and input / output (I / O) devices 105. Wherein:

[0075] Processor 101 can be used to control an electronic device to execute the processing method of the neural network model of this application. Processor 101 may include one or more processing units, such as a central processing unit (CPU), graphics processing unit (GPU), digital signal processor (DSP), microprocessor (MCU), artificial intelligence (AI) processor, or a processing module or circuit of a field programmable gate array (FPGA). It may include one or more single-core or multi-core processors. System memory 102 is volatile memory, such as random-access memory (RAM), double data rate synchronous dynamic random-access memory (DDR SDRAM), etc. System memory is used for temporary storage of data and / or instructions.

[0076] The non-volatile memory 103 may include one or more tangible, non-transitory computer-readable media for storing data and / or instructions. In some embodiments, the non-volatile memory 103 may include any suitable non-volatile memory such as flash memory and / or any suitable non-volatile storage device, such as a hard disk drive (HDD), a compact disc (CD), a digital versatile disc (DVD), a solid-state drive (SSD), etc. In some embodiments, the non-volatile memory 103 may also be a removable storage medium, such as a Secure Digital (SD) memory card, etc.

[0077] Specifically, system memory 102 and non-volatile memory 103 may each include a temporary copy and a permanent copy of instruction 107. Instruction 107 may include, when executed by processor 101, a processing method that causes electronic device 100 to implement the neural network model provided in the embodiments of this application.

[0078] The communication interface 104 may include a transceiver for providing a wired or wireless communication interface for the electronic device 100, thereby enabling communication with any other suitable device via one or more networks. In some embodiments, the communication interface 104 may be integrated into other components of the electronic device 100, for example, the communication interface 104 may be integrated into the processor 101. In some embodiments, the electronic device 100 may communicate with other devices through the communication interface 104; for example, the electronic device 100 may obtain the processing method of a neural network model to be run from other electronic devices through the communication interface 104.

[0079] Input / output (I / O) device 105 may include input devices such as keyboard, mouse, etc., and output devices such as monitor, etc. Users can interact with electronic device 100 through input / output (I / O) device 105.

[0080] System control logic 106 may include any suitable interface controller to provide any suitable interface to other modules of electronic device 100. For example, in some embodiments, system control logic 106 may include one or more memory controllers to provide an interface to system memory 102 and non-volatile memory 103.

[0081] In some embodiments, at least one of the processors 101 may be packaged together with the logic of one or more controllers for system control logic 106 to form a system in package (SiP). In other embodiments, at least one of the processors 101 may also be integrated on the same chip with the logic of one or more controllers for system control logic 106 to form a SoC.

[0082] It is understood that electronic device 100 can be any electronic device or terminal device capable of running neural networks, including but not limited to mobile phones, wearable devices (such as smartwatches), tablets, desktops, laptops, handheld computers, laptops, ultra-mobile personal computers (UMPCs), netbooks, as well as cellular phones, personal digital assistants (PDAs), augmented reality (AR) / virtual reality (VR) devices, etc., which are not limited in the embodiments of this application.

[0083] Understandable. Figure 5 The structure of the electronic device 100 shown is merely an example. In other embodiments, the electronic device 100 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0084] The various embodiments of the mechanisms disclosed in this application can be implemented in hardware, software, firmware, or a combination of these implementation methods. Embodiments of this application can be implemented as computer programs or program code executable on a programmable system, the programmable system including at least one processor, a storage system (including volatile and non-volatile memory and / or storage elements), at least one input device, and at least one output device.

[0085] Program code can be applied to input instructions to execute the functions described in this application and generate output information. The output information can be applied to one or more output devices in a known manner. For the purposes of this application, the processing system includes any system having a processor such as, for example, a Digital Signal Processor (DSP), a microcontroller, an Application Specific Integrated Circuit (ASIC), or a microprocessor.

[0086] The program code can be implemented using a high-level procedural language or an object-oriented programming language to communicate with the processing system. Assembly language or machine language can also be used when needed. In fact, the mechanisms described in this application are not limited to any particular programming language. In either case, the language can be a compiled language or an interpreted language.

[0087] In some cases, the disclosed embodiments may be implemented in hardware, firmware, software, or any combination thereof. The disclosed embodiments may also be implemented as instructions carried or stored thereon on one or more temporary or non-temporary machine-readable (e.g., computer-readable) storage media, which may be read and executed by one or more processors. For example, the instructions may be distributed via a network or through other computer-readable media. Therefore, machine-readable media may include any mechanism for storing or transmitting information in a machine-readable (e.g., computer-readable) form, including but not limited to floppy disks, optical disks, CD-ROMs, magneto-optical disks, read-only memory (ROM), random access memory (RAM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), magnetic cards or optical cards, flash memory, or tangible machine-readable storage for transmitting information (e.g., carrier waves, infrared signals, digital signals, etc.) using the Internet in the form of electrical, optical, acoustic, or other forms of propagated signals. Therefore, machine-readable media include any type of machine-readable medium suitable for storing or transmitting electronic instructions or information in a machine-readable (e.g., computer-readable) form.

[0088] In the accompanying drawings, some structural or methodological features may be shown in a specific arrangement and / or order. However, it should be understood that such a specific arrangement and / or order may not be necessary. Rather, in some embodiments, these features may be arranged in a manner and / or order different from that shown in the illustrative drawings. Furthermore, the inclusion of structural or methodological features in a particular figure does not imply that such features are required in all embodiments, and in some embodiments, these features may be omitted or may be combined with other features.

[0089] It should be noted that all units / modules mentioned in the device embodiments of this application are logical units / modules. Physically, a logical unit / module can be a physical unit / module, a part of a physical unit / module, or a combination of multiple physical units / modules. The physical implementation of these logical units / modules themselves is not the most important factor; the combination of functions implemented by these logical units / modules is the key to solving the technical problems proposed in this application. Furthermore, to highlight the innovative aspects of this application, the above-described device embodiments of this application have not introduced units / modules that are not closely related to solving the technical problems proposed in this application. This does not mean that the above-described device embodiments do not contain other units / modules.

[0090] It should be noted that in the examples and description of this patent, the terms "comprising," "including," or any other variations thereof are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one" does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element. Although this application has been illustrated and described with reference to certain preferred embodiments, those skilled in the art will understand that various changes in form and detail may be made without departing from the spirit and scope of this application.

Claims

1. A method for processing neural network models, applied to electronic devices, characterized in that, include: A transformation node is identified in the first neural network model, wherein the transformation node includes a node introduced to perform a transformation operation on the first neural network model; the transformation node is used to convert the format of the first neural network model into a format supported by the interface of the device that needs to deploy the neural network model; Determine the attribute values ​​of the parent and child nodes of each transformation node in the first neural network model; Based on the attribute values ​​of the parent and child nodes of each transformation node in the first neural network model, groups of transformation nodes with matching attributes are determined; wherein, each group of transformation nodes with matching attributes includes two adjacent transformation nodes, and the transformation modes of the two adjacent transformation nodes are opposite. Delete the transformation node groups with matching attributes in the first neural network model to obtain a second neural network model, which is used to process image data or audio data.

2. The method for processing neural network models according to claim 1, characterized in that, The step of determining the attribute-matching groups of transformation nodes based on the attribute values ​​of the parent and child nodes of each transformation node in the first neural network model includes: Obtain two adjacent transformation nodes in the first neural network model; If the attribute value of the parent node of the first conversion node is the same as the attribute value of the child node of the second conversion node, and the attribute value of the child node of the first conversion node is the same as the attribute value of the parent node of the second conversion node, then the first conversion node and the second conversion node are identified as a conversion node group with attribute matching.

3. The method for processing a neural network model according to any one of claims 1-2, characterized in that, The step of determining the transformation nodes in the first neural network model includes: determining the transformation nodes in the first neural network model based on a block-level formatting context algorithm.

4. The method for processing neural network models according to claim 1, characterized in that, Determining the attribute values ​​of the parent and child nodes of each transformation node in the first neural network model includes: The parent and child nodes of each transformation node in the first neural network model are determined based on the depth-first search algorithm; Based on the data format of the parent and child nodes of each transformation node in the first neural network model, determine the attribute values ​​of the parent and child nodes of each transformation node in the first neural network model.

5. The method for processing a neural network model according to any one of claims 1-2, characterized in that, The method further includes: Each transformation node in the first neural network model is labeled with a first identifier; The parent and child nodes of each transformation node in the first neural network model are labeled with a second identifier; The non-transformation nodes in the first neural network model are determined based on the block-level formatting context algorithm, and a third identifier is assigned to each non-transformation node in the first neural network model.

6. The method for processing neural network models according to claim 5, characterized in that, The step of deleting each transformation node group with matching attributes in the first neural network model includes: Each node in the first neural network model is determined to have an identifier, and each group of transformation nodes in the first neural network model that matches the attribute is deleted, wherein the identifier includes the first identifier, the second identifier, or the third identifier.

7. The method for processing neural network models according to claim 1 or 2, characterized in that, After the step of deleting the attribute-matching transformation node groups in the first neural network model, the method further includes: Determine the attribute value of the parent node of the first transformation node in each transformation node group of the first neural network model to be deleted; or determine the attribute value of the child node of the second transformation node in each transformation node group of the first neural network model to be deleted. Update the parameters of the nodes in the first neural network model based on the attribute values ​​of the parent node of the first transformation node; or The parameters of the nodes in the first neural network model are updated based on the attribute values ​​of the child nodes of the second transformation node.

8. The method for processing neural network models according to claim 1, characterized in that, The transformation node includes a transpose operator or a permutation operator.

9. An electronic device, characterized in that, It includes: a memory for storing instructions executed by one or more processors of the electronic device, and the processor being one of the one or more processors of the electronic device for executing a processing method of the neural network model according to any one of claims 1 to 8.

10. A readable medium, characterized in that, The readable medium stores instructions that, when executed on an electronic device, cause the electronic device to perform the processing method of the neural network model according to any one of claims 1 to 8.

Images