Quantization node determination method for neural network model and related apparatus

Abstract

The application provides a method for determining a quantization node of a neural network model and related devices. First, node information of a target model is obtained, the target model representing a neural network model loaded on a preset framework. Then, a first model graph is determined according to the node information, the first model graph being used to indicate a directed acyclic graph of the target model. Next, a first model subgraph is determined according to the first model graph, the first model subgraph being used to indicate a feature flow graph of the target model. Finally, a target node is determined according to the first model subgraph, the target node being used to indicate a quantization node of the target model. The quantization node can be automatically determined based on the characteristics of the directed acyclic graph, without manual selection, greatly improving the determination efficiency of the quantization node.

Description

Technical Field

[0001] This application relates to the field of machine learning technology, and in particular to a method and apparatus for determining quantized nodes in a neural network model. Background Technology

[0002] With the development of machine learning algorithms, the performance of neural network models has become increasingly powerful, but they also require a lot of computing resources and memory. In order to make neural network models more efficient and lightweight, quantization processing can be performed on the neural network models.

[0003] Existing quantization methods include quantization during training to improve accuracy (Quantization Aware Training, QAT) or post-training quantization (PTQ). However, regardless of the method, the quantization nodes of the neural network model need to be determined first. Current quantization nodes are generally selected manually, which lacks flexibility and is inefficient. Summary of the Invention

[0004] In view of this, this application provides a method and related apparatus for determining quantization nodes in a neural network model, which can be applied to neural network models under various frameworks, quickly determine quantization nodes, and greatly improve the efficiency of determining quantization nodes.

[0005] In a first aspect, embodiments of this application provide a method for determining quantization nodes in a neural network model, the method comprising:

[0006] Obtain node information of the target model, where the target model represents a neural network model mounted on a preset framework;

[0007] A first model graph is determined based on the node information, and the first model graph is used to indicate the directed acyclic graph of the target model;

[0008] A first model subgraph is determined based on the first model graph, and the first model subgraph is used to indicate the feature flow graph of the target model;

[0009] The target node is determined based on the first model subgraph, and the target node is used to indicate the quantization node of the target model.

[0010] Secondly, embodiments of this application provide a data processing apparatus, the apparatus comprising:

[0011] A node acquisition unit is used to acquire node information of a target model, wherein the target model represents a neural network model mounted on a preset framework;

[0012] The model graph determination unit is used to determine a first model graph based on the node information, wherein the first model graph is used to indicate the directed acyclic graph of the target model;

[0013] A feature flow graph determination unit is configured to determine a first model subgraph based on the first model graph, wherein the first model subgraph is used to indicate the feature flow graph of the target model.

[0014] A quantization node determination unit is used to determine a target node based on the first model subgraph, wherein the target node is used to indicate the quantization node of the target model.

[0015] Thirdly, embodiments of this application provide an electronic device, including a processor, a memory, a communication interface, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the processor, and the programs include instructions for performing steps in any method of the first aspect of this application.

[0016] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program for electronic data interchange, wherein the computer program causes a computer to perform some or all of the steps described in any method of the first aspect of this application.

[0017] Fifthly, embodiments of this application provide a computer program product, wherein the computer program product includes a non-transitory computer-readable storage medium storing a computer program, the computer program being operable to cause a computer to perform some or all of the steps described in any method of the first aspect of this application. The computer program product may be a software installation package.

[0018] As can be seen, the above-described method and apparatus for determining quantization nodes in a neural network model first obtains node information of the target model, which represents a neural network model mounted on a preset framework; then, a first model graph is determined based on the node information, which indicates the directed acyclic graph (DAG) of the target model; next, a first model subgraph is determined based on the first model graph, which indicates the feature flow graph of the target model; finally, target nodes are determined based on the first model subgraph, which indicates the quantization nodes of the target model. Quantization nodes can be automatically determined based on the characteristics of DAGs, eliminating the need for manual selection and significantly improving the efficiency of quantization node determination. Attached Figure Description

[0019] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 A system architecture diagram of a method for determining quantized nodes in a neural network model provided in this application embodiment;

[0021] Figure 2 A flowchart illustrating a method for determining quantization nodes in a neural network model, provided in an embodiment of this application;

[0022] Figure 3A A first model diagram of a target model provided in an embodiment of this application;

[0023] Figure 3B A first model subgraph of a target model provided in an embodiment of this application;

[0024] Figure 3C A schematic diagram illustrating the process of determining the excitation quantization node of a target model, as provided in an embodiment of this application;

[0025] Figure 4 A flowchart illustrating another method for determining quantization nodes in a neural network model provided in this application embodiment;

[0026] Figure 5 A schematic diagram illustrating a feature flow graph visualization provided in an embodiment of this application;

[0027] Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application;

[0028] Figure 7 A functional unit block diagram of a quantization node determination device for a neural network model provided in this application embodiment;

[0029] Figure 8 A block diagram of the functional units of a quantization node determination device for another neural network model provided in an embodiment of this application. Detailed Implementation

[0030] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present application.

[0031] The terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish different objects, not to describe a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or apparatuses.

[0032] It should be understood that the term "and / or" in this document is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Additionally, the character " / " in this document indicates that the preceding and following related objects are in an "or" relationship. In the embodiments of this application, "multiple" refers to two or more.

[0033] In this application, the term "connection" refers to various connection methods, such as direct connection or indirect connection, to achieve communication between devices. This application does not impose any limitations on this.

[0034] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0035] The background technology and related terms of this application are explained below.

[0036] Background technology related:

[0037] A Directed Acyclic Graph (DAG), in graph theory, refers to a directed graph without any loops. For example, if there is a non-directed acyclic graph where a path from point A to B via point C returns to A, forming a cycle, changing the direction of the edge from C to A to A to C transforms it into a directed acyclic graph. The graph structure of neural network models conforms to the characteristics of directed acyclic graphs in graph theory.

[0038] Quantization is the process of approximating weights and activation values ​​represented by high-bit widths (such as high-precision floating-point numbers, approximately 1E-38 to 1E38) using lower-bit widths (such as low-precision 8-bit fixed-point numbers, 0 to 255, -128 to 127). For example, a 32-bit floating-point number is represented using int8 or uint8. Quantization is performed because fixed-point arithmetic can process more data per unit time compared to floating-point arithmetic, and it can compress the size of neural network models; for example, quantizing 32-bit weights to 8 bits can reduce storage space by a factor of four. Existing quantization algorithms include symmetric quantization and asymmetric quantization. Since quantizing a neural network model requires determining quantization nodes, these nodes are currently often determined manually or rely on the inherent quantization strategies of a specific platform. However, manually setting quantization nodes is often inefficient, and the inherent quantization strategies of a specific platform are only applicable to the neural network model trained on that platform, resulting in poor flexibility.

[0039] To address the aforementioned issues, this application provides a method for determining quantization nodes in a neural network model. This method leverages the characteristics of directed acyclic graphs to perform relevant processing on the quantization of the neural network model, automatically determining the quantization nodes of the neural network model under any framework. This significantly improves the efficiency of determining quantization nodes in neural network models, is applicable to any platform, and exhibits strong decoupling.

[0040] The following is combined Figure 1 The system architecture of a method for determining quantized nodes in a neural network model, as described in this application, is explained. Figure 1The system architecture diagram of a method for determining quantization nodes of a neural network model provided in this application embodiment includes an external framework 110 and a quantization framework 120. The external framework 110 represents existing mainstream frameworks for building neural network models, such as TensorFlow, tf.keras, PyTorch, ONNX, etc., which are not specifically limited here. The quantization framework 120 includes a node unit 121, a model graph unit 122, a feature flow graph unit 123, and a quantization node unit 124. It can be understood that the node unit 121 of the quantization framework 120 can extract node information from the neural network model of the external framework 110. Then, the model graph unit 122 determines the model graph based on the node information. The model graph conforms to the characteristics of a directed acyclic graph. The feature flow graph unit 123 further determines the feature flow graph related to the feature tensor flow propagation. Finally, the quantization node unit 124 can determine the quantization nodes according to the feature flow graph and the preset quantization node selection rules.

[0041] It is understood that the above-mentioned quantization nodes refer to the node layers (SelectedQuantizers) that need to be quantized. In the embodiments of this application, "node" refers to a layer in a neural network model, or it can refer to a "glue node" related to data transmission, such as a node responsible for extracting and transmitting the shape parameters of the target layer.

[0042] Understandably, the aforementioned quantization nodes can be used to quantize neural network models. Through the above system architecture, decoupling from existing mainstream frameworks can be achieved, making it applicable to multiple mainstream frameworks. This improves the flexibility of the quantization node determination process for neural network models while also increasing the efficiency of quantization node determination.

[0043] The following is combined Figure 2 This application describes a method for determining quantization nodes in a neural network model, as described in an embodiment. Figure 2 A method for determining quantization nodes in a neural network model provided in this application embodiment specifically includes the following steps:

[0044] Step 201: Obtain the node information of the target model.

[0045] The target model refers to a neural network model mounted on a preset framework. The preset framework can be an existing framework for building neural network models, such as TensorFlow, tf.keras, PyTorch, ONNX, etc. After loading the structural information and weight information of the target model mounted on the preset framework, the node information of the target model can be determined according to the structural information and the weight information. The node information includes node number, node name, list of parent node names for layer type, child node names, and list of input / output layer names.

[0046] As can be seen, by using the interface of the existing framework, the quantization framework of this application can be applied regardless of the framework in which the neural network model is built, and the node information can be automatically determined, which greatly improves the decoupling of the process of determining the quantization nodes of the neural network model.

[0047] Step 202: Determine the first model diagram based on the node information.

[0048] The first model graph is used to indicate the directed acyclic graph of the target model. It can be understood that the graph of the target model conforms to the characteristics of a directed acyclic graph, so the graph of the target model can also be called a directed acyclic graph. Once the node information of the target model is determined, its corresponding first model graph can be constructed.

[0049] Specifically, such as Figure 3A As shown, Figure 3A The first model diagram of a target model provided in this application embodiment should be noted. It should be noted that the diagram is only a schematic diagram of some nodes and does not show all nodes. It can be seen that when the feature vector flow is transmitted in the neural network model, it often has multiple branch paths. The branch paths may include node 1-node 3-node 8-node 9, node 1-node 3-node 6-node 8-node 9, node 1-node 2-node 5-node 7-node 8-node 9, node 1-node 2-node 4-node 7-node 8-node 9, and node 1-node 2-node 4-node 7-node 8-node 9. The nodes may include the layers of the neural network model and parameters used for data processing, etc.

[0050] It should be noted that, Figure 3A Nodes 1-9 in the model can include nodes such as Conv2D, ReLU, Shape, StridedSlice, Mean, Pack, ResizeBilinear, and ConcatV2. The connection between different nodes follows the existing inherent connection methods. The functions of each node are not described in detail here, but can be found in the description of node functions in existing neural network architectures.

[0051] Understandably, since the graph structure of the neural network model conforms to the characteristics of a directed acyclic graph in graph theory, many commonly used graph theory algorithms can be integrated into the extracted first model graph, including depth-first search (DFS), topological sorting, shortest path search (Dijkstra's algorithm), path traversal, etc., which will not be elaborated here. Users can execute the above graph theory algorithms in the first model graph.

[0052] It is evident that determining the first model graph based on the node information can provide data support for the subsequent determination of the feature flow graph.

[0053] Step 203: Determine the first model subgraph based on the first model graph.

[0054] The first model subgraph is used to indicate the feature flow graph of the target model. A first node belonging to a preset node type can be determined in the first model graph. The preset node type is used to indicate the node type related to feature flow propagation in the target model. The first model subgraph is then determined based on the first node.

[0055] It should be noted that quantization of neural network models mainly targets two types of objects: firstly, node layers with floating-point parameters, such as convolutional layers (Conv) and fully connected layers (Fc); and secondly, the feature vector flow propagated between layers. Many nodes in the first model graph are not related to the propagation of the feature flow, but rather act as "glue nodes," responsible for extracting and transmitting the shape parameters of the target layer. After excluding these nodes, the node layers involved in feature flow processing can be extracted, further forming a subgraph, which is the first model subgraph. The quantization nodes to be determined are a subset of the nodes included in this first model subgraph.

[0056] Specifically, the first directed edge formed by the first node can be determined. Any first directed edge includes the first node as the starting node and the first node as the ending node. The part outside the first node and the first directed edge in the first model graph is deleted to obtain the first model subgraph.

[0057] For example, refer to Figure 3AIn the first model graph, the node types for feature flow propagation are actually quite general. A list of node types related to feature flow propagation can be pre-constructed, such as node type list = {InputLayer, Conv2D, ReLU, ResizeBilinear, ConcatV2, DepthToSpace, Reshape, Mean, AveragePooling2D, GlobalAveragePooling2D, Mul, Div, Sub, Add, ZeroPadding2D, MaxPooling2D, BatchNormalization, Activation, Dense, Softmax}, etc. After comparing this node type list, if... Figure 3A In the first model diagram, nodes 1, 3, 6, 8, and 9 have node types that are a subset of the list of node types mentioned above. Therefore, it can be determined that the first nodes belonging to this preset node type include nodes 1, 3, 6, 8, and 9, while nodes not belonging to this preset node type include nodes 2, 4, 5, and 7. It can be seen that... Figure 3B As shown, after excluding nodes 2, 4, 5, 7, and related branches, we can obtain... Figure 3B The first model subgraph shown is shown.

[0058] For example, node types that do not belong to the preset node types, such as Shape, StridedSlice, and Pack, do not actually participate in feature flow propagation. Therefore, nodes of the Shape, StridedSlice, and Pack node types and their related branches can be deleted, and the remaining first nodes and the directed edges formed by the remaining nodes can be saved to construct the first model subgraph.

[0059] As can be seen, determining the first model subgraph based on the first model graph can eliminate the interference of some irrelevant nodes and improve the efficiency of determining quantization nodes.

[0060] Step 204: Determine the target node based on the first model subgraph.

[0061] The target node is used to indicate the quantization node of the target model. Quantization nodes mainly include two types: weight quantizer (WQ) nodes and activation quantizer (AQ) nodes. The former mainly quantizes the weight values ​​in the node, such as Conv layers and Fc layers, while the latter mainly quantizes the feature vector values ​​output by each node.

[0062] In one possible embodiment, the first nodes belonging to a first preset quantization type can be selected as the weight quantization nodes; the first nodes belonging to a second preset quantization type can be selected as the incentive quantization nodes. The first preset quantization type can be a pre-defined general node type for weight quantization nodes, such as {Conv2D, Conv1D, Dense}, etc., and the second preset quantization type can be a pre-defined general node type for incentive quantization nodes, such as {ReLU, Softmax, ConcatV2}, etc. Only by selecting nodes belonging to the first preset quantization type as weight quantization nodes and nodes belonging to the second preset quantization type as incentive quantization nodes, the target nodes to be quantized can be determined.

[0063] In one possible embodiment, the first nodes belonging to a first preset quantization type among the first nodes can be selected as the weight quantization nodes; the first nodes belonging to a second preset quantization type among the first nodes can be selected, and the quantization weight value of the first nodes belonging to the second preset quantization type can be determined; the first nodes belonging to the second preset quantization type whose quantization weight value is greater than a preset weight threshold can be selected as the incentive quantization nodes.

[0064] Specifically, custom rules can be defined for determining the incentive quantization nodes. This involves first determining a common first node that conforms to the second preset quantization type, then calculating the quantization weight value, and retaining the portion of the quantization weight value greater than a preset threshold as the incentive quantization node. For example, such as... Figure 3C As shown, Figure 3C This is a schematic diagram illustrating the process of determining an excitation quantization node, as provided in an embodiment of this application. Figure 3C The left side is Figure 3B The first model subgraph includes nodes 1, 3, 6, 8, and 9. The first nodes that conform to the second preset quantization type include nodes 3, 6, and 8. We can first find the nodes that are directly connected to the specific node type in the graph and whose node type is in the general node type of the incentive quantization node. According to the order in the topology sort, the node with the smaller ranking can be used as the incentive quantization node. Here, we take node 6 as a specific node type for explanation. It can be seen that the nodes directly connected to node 6 and whose node type is in the general node type of the incentive quantization node include nodes 3 and 8. If the topology ranking of node 3 is greater than that of node 8, then only nodes 6 and 8 can be used as incentive quantization nodes. At the same time, we can filter the nodes that belong to the first preset quantization type as the weight quantization nodes. Here, we take nodes 1 and 9 belonging to the first preset quantization type as an example for explanation. As shown in the right figure, finally, nodes 1 and 9 are weight quantization nodes, and nodes 6 and 8 are incentive quantization nodes.

[0065] As can be seen, the method for determining quantization nodes in the aforementioned neural network model first obtains the node information of the target model, which represents a neural network model mounted on a preset framework; then, a first model graph is determined based on the node information, which indicates the directed acyclic graph (DAG) of the target model; next, a first model subgraph is determined based on the first model graph, which indicates the feature flow graph of the target model; finally, target nodes are determined based on the first model subgraph, which indicates the quantization nodes of the target model. This method can automatically determine quantization nodes based on the characteristics of DAGs, eliminating the need for manual selection and significantly improving the efficiency of quantization node determination.

[0066] The following is combined Figure 4 Another method for determining quantization nodes in a neural network, as described in the embodiments of this application, will be explained. Figure 4 A flowchart illustrating another method for determining quantization nodes in a neural network model provided in this application embodiment is shown, specifically including the following steps:

[0067] Step 401: Obtain the node information of the target model.

[0068] Step 402: Determine the first model diagram based on the node information.

[0069] Step 403: Determine the first model sub-graph based on the first model graph.

[0070] Step 404: Determine the target node based on the first model subgraph.

[0071] Step 405: Obtain feedback data.

[0072] The feedback data is used to indicate the accuracy of the target node. It is understood that users can view and review the identified target node. If the user's review indicates that the identified target node is normal, the feedback data indicates that the identified target node meets the requirements, and subsequent model quantization steps can be performed. If the user's review indicates that the identified target node does not meet the requirements, the feedback data may include incorrect target nodes, missing target nodes, etc.

[0073] As can be seen, by acquiring feedback data, an error correction mechanism can be added to the quantization node determination method of the neural network model in this application embodiment, thereby improving the accuracy of the determined target nodes.

[0074] Step 406: Adjust the first preset quantization type and / or the second preset quantization type according to the feedback data.

[0075] If the feedback data indicates that the identified target node is normal, the first preset quantization type and / or the second preset quantization type can be maintained. If the feedback data indicates that the identified target node is incorrect or missing, relevant node types can be added or removed from the first preset quantization type and / or the second preset quantization type based on the feedback data. This will not be elaborated here.

[0076] In one possible embodiment, the first model subgraph in this application embodiment can be visualized, that is, each node can be labeled with its identifier and type, and the first model subgraph can be displayed according to topological sorting, which can facilitate user viewing. Figure 5 As shown, Figure 5 This is a schematic diagram illustrating a feature flow graph visualization provided in an embodiment of this application.

[0077] As can be seen, the above-described method for determining quantization nodes in a neural network model first obtains node information of the target model, which represents a neural network model mounted on a preset framework; then, a first model graph is determined based on the node information, which indicates the directed acyclic graph (DAG) of the target model; next, a first model subgraph is determined based on the first model graph, which indicates the feature flow graph of the target model; finally, target nodes are determined based on the first model subgraph, which indicates the quantization nodes of the target model. This method can automatically determine quantization nodes based on the characteristics of DAGs, eliminating the need for manual selection and significantly improving the efficiency of quantization node determination. Furthermore, it adds an error correction mechanism to the method for determining quantization nodes in the neural network model of this application embodiment, improving the accuracy of the determined target nodes.

[0078] The following is combined Figure 6 An electronic device according to an embodiment of this application will be described. Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application, such as... Figure 6 As shown, the electronic device 600 includes a processor 601, a communication interface 602, and a memory 603, which are interconnected. The electronic device 600 may also include a bus 604, through which the processor 601, communication interface 602, and memory 603 are interconnected. The bus 604 can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The bus 604 can be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, Figure 6The bus is represented by a single thick line, but this does not indicate that there is only one bus or one type of bus. The memory 603 is used to store a computer program, which includes program instructions. The processor is configured to call the program instructions and execute the above-mentioned... Figure 2 , Figure 4 All or part of the methods described herein.

[0079] The above primarily describes the solutions of the embodiments of this application from the perspective of the method execution process. It is understood that, in order to achieve the above functions, the electronic device includes corresponding hardware structures and / or software modules for executing each function. Those skilled in the art should readily recognize that, in conjunction with the units and algorithm steps of the various examples described in the embodiments provided herein, this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed by hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0080] This application embodiment can divide the electronic device into functional units according to the above method example. For example, each function can be divided into a separate functional unit, or two or more functions can be integrated into one processing unit. The integrated unit can be implemented in hardware or as a software functional unit. It should be noted that the unit division in this application embodiment is illustrative and only represents one logical functional division. In actual implementation, there may be other division methods.

[0081] When dividing functional modules according to their respective functions, the following is combined with... Figure 7 This application provides a detailed description of a quantization node determination device for a neural network model, as described in an embodiment. Figure 7 A functional unit block diagram of a quantization node determination device for a neural network model provided in this application embodiment. The quantization node determination device 700 for the neural network model includes:

[0082] The node acquisition unit 710 is used to acquire node information of the target model, wherein the target model represents a neural network model mounted on a preset framework;

[0083] The model graph determination unit 720 is used to determine a first model graph based on the node information, wherein the first model graph is used to indicate the directed acyclic graph of the target model;

[0084] The feature flow graph determination unit 730 is configured to determine a first model subgraph based on the first model graph, wherein the first model subgraph is used to indicate the feature flow graph of the target model.

[0085] The quantization node determination unit 740 is used to determine a target node based on the first model subgraph, wherein the target node is used to indicate the quantization node of the target model.

[0086] As can be seen, the above-described method and apparatus for determining quantization nodes in a neural network model first obtains node information of the target model, which represents a neural network model mounted on a preset framework; then, a first model graph is determined based on the node information, which indicates the directed acyclic graph (DAG) of the target model; next, a first model subgraph is determined based on the first model graph, which indicates the feature flow graph of the target model; finally, target nodes are determined based on the first model subgraph, which indicates the quantization nodes of the target model. Quantization nodes can be automatically determined based on the characteristics of DAGs, eliminating the need for manual selection and significantly improving the efficiency of quantization node determination.

[0087] When using integrated units, the following is combined with Figure 8 A detailed description of another quantization node determination device 800 for a neural network model in the embodiments of this application is provided. The quantization node determination device 800 for the neural network model includes a processing unit 801 and a communication unit 802. The processing unit 801 is used to perform any step as described in the above method embodiments, and when performing data transmission such as sending, the communication unit 802 can be selectively invoked to complete the corresponding operation.

[0088] The quantization node determination device 800 for the neural network model may further include a storage unit 803 for storing program code and data. The processing unit 801 may be a processor, the communication unit 802 may be a wireless communication module, and the storage unit 803 may be a memory.

[0089] The processing unit 801 is specifically used for:

[0090] Obtain node information of the target model, where the target model represents a neural network model mounted on a preset framework;

[0091] A first model graph is determined based on the node information, and the first model graph is used to indicate the directed acyclic graph of the target model;

[0092] A first model subgraph is determined based on the first model graph, and the first model subgraph is used to indicate the feature flow graph of the target model;

[0093] The target node is determined based on the first model subgraph, and the target node is used to indicate the quantization node of the target model.

[0094] As can be seen, the above-described method and apparatus for determining quantization nodes in a neural network model first obtains node information of the target model, which represents a neural network model mounted on a preset framework; then, a first model graph is determined based on the node information, which indicates the directed acyclic graph (DAG) of the target model; next, a first model subgraph is determined based on the first model graph, which indicates the feature flow graph of the target model; finally, target nodes are determined based on the first model subgraph, which indicates the quantization nodes of the target model. Quantization nodes can be automatically determined based on the characteristics of DAGs, eliminating the need for manual selection and significantly improving the efficiency of quantization node determination.

[0095] This application also provides a computer storage medium storing a computer program for electronic data interchange, which causes a computer to perform some or all of the steps of any of the methods described in the above method embodiments.

[0096] This application also provides a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program operable to cause a computer to perform some or all of the steps of any of the methods described in the above method embodiments. The computer program product may be a software installation package, and the computer may include an electronic device.

[0097] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0098] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0099] In the several embodiments provided in this application, it should be understood that the disclosed apparatus can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of the units described above is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical or other forms.

[0100] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0101] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0102] If the integrated units described above are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage device (CMD). Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a memory and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned memory includes various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0103] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, which may include: flash drive, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0104] The embodiments of this application have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The description of the above embodiments is only for the purpose of helping to understand the method and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A method for determining a quantization node of a neural network model, comprising: The method includes: Obtain node information of the target model, where the target model represents a neural network model mounted on a preset framework; A first model graph is determined based on the node information, and the first model graph is used to indicate the directed acyclic graph of the target model; Determine a first node in the first model graph that belongs to a preset node type, wherein the preset node type is used to indicate the node type in the target model that is related to feature flow transmission; The first model subgraph is determined based on the first node; The target node is determined based on the first model subgraph. The target node is used to indicate the quantization node of the target model. The first node belonging to the first preset quantization type is selected as the weight quantization node. The first node belonging to the second preset quantization type is selected as the incentive quantization node.

2. The method of claim 1, wherein, Determining the first model subgraph based on the first node includes: Determine the first directed edge formed by the first node, where any first directed edge includes the first node as the starting node and the first node as the ending node; The first model subgraph is obtained by deleting the part outside the first node and the first directed edge in the first model graph.

3. The method of claim 2, wherein, After selecting the first node belonging to the first preset quantization type from the first nodes as the weighted quantization node, the method further includes: First nodes belonging to the second preset quantization type are selected from the first nodes, and the quantization weight value of the first nodes belonging to the second preset quantization type is determined. First nodes belonging to the second preset quantization type whose quantization weight value is greater than the preset weight threshold are selected as the incentive quantization nodes.

4. The method according to claim 1 or 3, characterized in that, After determining the target node based on the first model subgraph, the method further includes: Obtain feedback data, which is used to indicate the accuracy of the target node; Adjust the first preset quantization type and / or the second preset quantization type based on the feedback data.

5. The method of claim 1, wherein, The acquisition of node information of the target model includes: Load the structural and weight information of the target model mounted on the preset framework; The node information of the target model is determined based on the structural information and the weight information. The node information includes node number, node name, list of parent node names for layer type, child node names, and list of input / output layer names. 6.A quantization node determination apparatus of a neural network model, characterized in that, The device includes: A node acquisition unit is used to acquire node information of a target model, wherein the target model represents a neural network model mounted on a preset framework; The model graph determination unit is used to determine a first model graph based on the node information, wherein the first model graph is used to indicate the directed acyclic graph of the target model; A feature flow graph determination unit is used to determine a first node in the first model graph that belongs to a preset node type, wherein the preset node type is used to indicate the node type in the target model that is related to feature flow propagation; and to determine a first model subgraph based on the first node. A quantization node determination unit is used to determine target nodes based on the first model subgraph. The target nodes are used to indicate the quantization nodes of the target model. Specifically, the first nodes belonging to a first preset quantization type are selected as weight quantization nodes, and the first nodes belonging to a second preset quantization type are selected as incentive quantization nodes.

7. An electronic device, comprising: It includes a processor, a memory, and one or more programs, said one or more programs being stored in the memory and configured to be executed by the processor, said programs including instructions for performing the steps of the method as described in any one of claims 1-5.

8. A computer storage medium, characterized in that, The computer storage medium stores a computer program, the computer program including program instructions, which, when executed by the baseband chip, cause the baseband chip to perform the method as described in any one of claims 1-5.

Images