Model optimization method and related apparatus

Abstract

The embodiment of the application discloses a model optimization method, the method comprises the following steps: a data processing device acquires model information, and generates a model node according to the model information, the model node is a calculation unit; the data processing device determines a connection relationship between the model nodes, and determines a quantifiable structure according to the connection relationship; the quantifiable structure is quantified, the quantifiable structure comprises two or more nodes, that is, comprises multiple operators, and the optimization efficiency is improved.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence, and in particular to a model optimization method and related apparatus. Background Technology

[0002] In recent years, artificial intelligence has experienced explosive growth, with deep learning being its main driving force. It has been widely applied in computer vision, such as image recognition, semantic segmentation, object detection and tracking, and image retrieval. Various deep learning models have been expanded to varying degrees in terms of width and depth, leading to a decrease in performance. For example, in camera scene recognition systems (such as the recognition of portraits, food, and night scenes), in pursuit of scene segmentation and accuracy, the models of camera scene recognition systems are constantly expanded, which may lead to stuttering and other phenomena.

[0003] To solve the above problems, it is necessary to quantize the model structure. In the existing technology, the quantization of the model structure is usually done on a single operator, which is inefficient. Summary of the Invention

[0004] The first aspect of this application provides a model optimization method, including:

[0005] The data processing device acquires model information and generates model nodes based on the model information. The model nodes are computing units. The number of model nodes generated based on the model information is two or more. The data processing device determines the connection relationship between the model nodes and determines the quantifiable structure based on the connection relationship. The quantifiable structure includes two or more model nodes.

[0006] This application provides a model optimization method that quantizes quantizable structures, which include two or more nodes, i.e., multiple operators, thereby improving optimization efficiency.

[0007] Based on the first aspect of the embodiments of this application, in a first embodiment of the first aspect of the embodiments of this application, the method of the data processing device to determine a quantifiable structure according to the connection relationship includes: the data processing device determining the shortest path node according to the connection relationship, wherein the shortest path node is the model node included in the shortest path between the start node and the end node, wherein the start node and the end node are both model nodes; the data processing device selecting quantifiable nodes from the shortest path nodes, wherein the quantifiable nodes include nodes connected to three or more model nodes among the start node, the end node, and the shortest path nodes; and the data processing device determining the quantifiable structure according to the quantifiable nodes.

[0008] In this application embodiment, a specific method for determining quantifiable structures is provided.

[0009] Based on the first implementation of the first aspect of the present application, in the second implementation of the first aspect of the present application, the quantifiable node may further include: a single node, wherein the single node is a shortest path node that is connected to only two nodes other than the starting node and the ending node, and the two nodes connected to the single node have no other connection method other than through the single node.

[0010] In this embodiment, a single node can be added to the quantifiable node. The pre-set quantifiable structure includes not only structures with multiple nodes, such as 1-7, but also single nodes such as 8, 9, and 10. When the model is divided according to the structure, nodes 8, 9, and 10 are nodes of separate links and are analyzed separately, thereby improving the effect of model optimization.

[0011] Based on any one of the first to second embodiments of the present application, in the third embodiment of the first aspect of the present application, after the data processing device determines the quantizable structure according to the connection relationship, the data processing device determines the quantization method of the quantizable structure.

[0012] Based on the third implementation of the first aspect of the present application, in the fourth implementation of the first aspect of the present application, the quantization method includes 1-bit, 4-bit, or 8-bit quantization methods, and / or a mixture of multiple quantization methods among 1-bit, 4-bit, or 8-bit.

[0013] A second aspect of this application provides a data processing apparatus that performs the methods described in the first aspect and any embodiment thereof.

[0014] A third aspect of this application provides a computer storage medium storing instructions that, when executed on a computer, cause the computer to perform the methods described in the first aspect and any embodiment of the first aspect.

[0015] The fourth aspect of this application provides a computer software product that, when executed on a computer, causes the computer to perform the methods described in the first aspect and any implementation thereof. Attached Figure Description

[0016] Figure 1 This is a schematic diagram of the optimizable structure of the model in the embodiments of this application;

[0017] Figure 2 This is a schematic diagram of a model optimization method in an embodiment of this application;

[0018] Figure 3 This is a schematic diagram of model information in an embodiment of this application;

[0019] Figure 4 This is a schematic diagram of a node network in one embodiment of this application;

[0020] Figure 5 This is a schematic diagram of another node network in an embodiment of this application;

[0021] Figure 6 This is a schematic diagram of a quantifiable structure in an embodiment of this application;

[0022] Figure 7 This is a schematic diagram of another quantifiable structure in an embodiment of this application;

[0023] Figure 8 This is a schematic diagram of the structure of a data processing device in an embodiment of this application;

[0024] Figure 9 This is another structural schematic diagram of the data processing device in the embodiments of this application. Detailed Implementation

[0025] The technical solutions in the embodiments of this application will be described below with reference to the accompanying drawings. In this application, "at least one" means one or more, and "more than one" means two or more. "And / or" describes the relationship between related objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist simultaneously, or B exists alone, where A and B can be singular or plural. The character " / " generally indicates that the related objects before and after are in an "or" relationship. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can mean: a, b, c, a and b, a and c, b and c, or a and b and c, where a, b, and c can be single or multiple. It is worth noting that "at least one" can also be interpreted as "one or more".

[0026] It should be noted that, in this application, the terms "exemplary" or "for example" are used to indicate that something is being described as an example, illustration, or illustration. Any embodiment or design described as "exemplary" or "for example" in this application should not be construed as being more preferred or advantageous than other embodiments or design solutions. Specifically, the use of terms such as "exemplary" or "for example" is intended to present the relevant concepts in a concrete manner.

[0027] The descriptions of "first," "second," etc., appearing in the embodiments of this application are for illustrative purposes and to distinguish the objects being described. They have no order and do not indicate any special limitation on the number of devices in the embodiments of this application, nor do they constitute any limitation on the embodiments of this application.

[0028] This application provides a model optimization method that quantizes quantizable structures, which include two or more nodes, i.e., multiple operators, thereby improving optimization efficiency.

[0029] With the development of deep learning, neural networks have been widely applied in various fields. However, the improvement in model performance has also introduced a huge increase in the number of parameters and computational demands. Model quantization is a technique that converts floating-point calculations into low-ratio, point-to-point calculations, effectively reducing the computational intensity, parameter size, and memory consumption of the model.

[0030] In the embodiments of this application, the operator is the smallest computational unit, such as (2.5+3), (7 / 2), etc.

[0031] Based on the optimizable structure of the model, the template can be divided into four levels: node, cell, block, and network. A node is the unit quantized in the model, currently represented by a single operator. A cell consists of multiple nodes, a block consists of multiple cells, and a network consists of multiple blocks. For details, please refer to [reference needed]. Figure 1 .

[0032] To improve optimization efficiency, this application provides a model optimization method that quantizes quantizable structures (i.e., newly defined nodes), where the quantizable structure includes two or more nodes.

[0033] For details, please refer to [link / reference]. Figure 2 This application provides a model optimization method, comprising:

[0034] 201. Data processing equipment acquires model information;

[0035] In this embodiment, the data processing device can be a terminal device, also known as a user equipment (UE), mobile station (MS), mobile terminal (MT), etc., which refers to a device that provides voice and / or data connectivity to a user. Examples include handheld devices with wireless connectivity and in-vehicle devices. Currently, some examples of terminals include: mobile phones, tablets, laptops, PDAs, mobile internet devices (MIDs), wearable devices, virtual reality (VR) devices, augmented reality (AR) devices, wireless terminals in industrial control, wireless terminals in self-driving, wireless terminals in remote medical surgery, wireless terminals in smart grids, wireless terminals in transportation safety, wireless terminals in smart cities, and wireless terminals in smart homes.

[0036] The data processing device loads the model data package, parses the file to obtain model information, see reference. Figure 3 This is an example of model information.

[0037] 202. The data processing equipment generates model nodes based on the model information;

[0038] See Figure 3 The model information shown has 5 model nodes, which can be numbered from 301 to 305.

[0039] See Figure 4 It is a node network generated based on model information, including 17 model nodes, namely nodes 401 to 417, where node 401 is the starting node and node 417 is the ending node.

[0040] 203. The data processing equipment determines the connection relationships between model nodes;

[0041] by Figure 4 For example, the connection relationships between model nodes include: node 401 is connected to nodes 402 and 403, node 402 is connected to nodes 401 and 407, and so on. Other nodes are similar, and will not be described in detail here.

[0042] 204. The data processing equipment determines the shortest path node;

[0043] In this embodiment of the application, the distance between two connected nodes can be considered as one unit distance. The shortest path node is determined according to the minimum path algorithm, such as Dijkstra's algorithm, the single-source shortest path (SPFA) algorithm, the Bellman-Ford algorithm, or the Floyd algorithm. The specific algorithm is not limited here.

[0044] This embodiment uses a traversal approach. If there is more than one shortest path between the starting node and the ending node, the shortest path nodes included in each of the multiple shortest paths can be recorded separately, such as... Figure 5 As shown, when the starting node is node 501 and the ending node is node 505, the shortest path nodes between the starting node and the ending node include node group {node 501, node 502, node 504, node 505} and node group {node 501, node 503, node 504, node 505}.

[0045] See Figure 4 The shortest path nodes between the starting node and the ending node include the node group {node 401, node 402, node 407, node 408, node 409, node 410, node 411, node 412, node 417}.

[0046] 205. Data processing equipment determines quantifiable nodes;

[0047] Quantifiable nodes include start nodes, end nodes, shortest path nodes, and nodes connected by three or more model nodes. See also... Figure 4 The quantifiable nodes include the node group {node 401, node 407, node 411, node 417}.

[0048] In one possible implementation, quantifiable nodes may also include single nodes. A single node is a shortest path node that, besides the start and end nodes, is connected to only two other nodes, and these two nodes have no other connection than through that single node. See also... Figure 4 The quantifiable nodes include the node group {node 401, node 407, node 408, node 409, node 410, node 411, node 417}.

[0049] 206. The data processing device determines the quantizable structure based on the quantizable nodes;

[0050] The quantizable structure is determined based on the quantizable nodes. See [link / reference]. Figure 6If the quantizable nodes include the node group {node 401, node 407, node 411, node 417}, the quantizable structures include: node 401->node 407 structure, node 407->node 411 structure, and node 411->node 417 structure.

[0051] The node 401->node 407 structure includes node 401, node 402, node 403, node 404, node 405, node 406, and node 407, as well as the connection relationships between these seven nodes.

[0052] The node 407->node 411 structure includes node 407, node 408, node 409, node 410, node 411, and the connection relationship between the above 5 nodes.

[0053] The node 411->node 417 structure includes nodes 411, 412, 413, 414, 415, 416, and 417, as well as the connection relationships between these seven nodes.

[0054] See Figure 7 For example, when the quantifiable nodes include the node group {node 401, node 407, node 408, node 409, node 410, node 411, node 417}, the quantifiable structures include: node 401->node 407 structure, node 407->node 408 structure, node 408->node 409 structure, node 409->node 410 structure, node 410->node 411 structure, and node 411->node 417 structure. The specific structure content and... Figure 6 Similar to that, it will not be elaborated here.

[0055] 207. The data processing equipment determines the quantization method of the quantifiable structure.

[0056] The quantizable methods include 1-bit, 4-bit, or 8-bit quantization methods, and / or a combination of multiple 1-bit, 4-bit, or 8-bit quantization methods. For example, if the quantizable structure is 1+2.5, the quantizable method can be 4-bit quantization or a combination of 1-bit and 4-bit quantization methods. The data processing device iterates through the data to find the optimal quantization method and records the optimal quantization method for each quantizable structure. The optimized model is obtained by quantizing according to the optimal quantization method.

[0057] The data transmission method in the embodiments of this application has been described above. The apparatus in the embodiments of this application is described below. Please refer to [link / reference]. Figure 8 One embodiment of the data processing device in this application includes:

[0058] Acquisition unit 801 is used to acquire model information.

[0059] Generation unit 802 generates model nodes based on model information. The model nodes are computational units, and the number of model nodes is two or more.

[0060] The determining unit 803 is used to determine the connection relationships between model nodes and to determine the quantizable structure based on the connection relationships. The quantizable structure includes two or more nodes. Specifically, the determining unit is used to determine the shortest path node based on the connection relationships. The shortest path node is the model node included in the shortest path between the start node and the end node. Both the start node and the end node are model nodes. The determining unit is also used to determine the quantizable node, which includes the start node, the end node, the shortest path node, and the node connected to three or more model nodes. The determining unit is used to determine the quantizable structure based on the quantizable node and to determine the quantization method of the quantizable structure.

[0061] In this embodiment, the operations performed by each unit are the same as before. Figure 2 In the illustrated embodiments, the operations performed by the data processing device are described similarly and will not be repeated here.

[0062] Figure 9 This is a schematic diagram of the structure of a data processing device provided in an embodiment of this application. The data processing device 900 may include one or more processors 901 and a memory 905, in which one or more applications or data are stored.

[0063] The memory 905 can be volatile or persistent storage. The program stored in the memory 905 can include one or more modules, each module including a series of instruction operations on the data processing device 900. Furthermore, the processor 901 can be configured to communicate with the memory 905 and execute the series of instruction operations in the memory 905 on the data processing device 900.

[0064] The data processing device 900 may also include one or more power supplies 902, one or more wired or wireless network interfaces 903, one or more input / output interfaces 904, and / or one or more operating systems, such as any one of Microsoft Windows, Android, Mac OS, Unix, and Linux.

[0065] The processor 901 can perform the operations performed by the data processing device in any of the foregoing embodiments, which will not be described in detail here.

[0066] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0067] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection between apparatuses or units through some interfaces, and may be electrical, mechanical, or other forms.

[0068] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0069] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0070] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

Claims

1. A model optimization method, characterized in that, include: Data processing equipment acquires model information; The data processing device generates model nodes based on the model information. Each model node is a computing unit, and the number of model nodes is two or more. The data processing device determines the connection relationships between the model nodes; The data processing device determines a quantifiable structure based on the connection relationship, and the quantifiable structure includes two or more nodes. The data processing device iterates through the quantizable methods of the quantizable structure to determine the optimal quantization method; The data processing device obtains an optimized model based on the optimal quantization method, and the optimized model is used to achieve at least one of the following: image recognition, target detection and tracking, or image retrieval; The data processing device determines the quantifiable structure based on the connection relationship, including: The data processing device determines the shortest path node based on the connection relationship. The shortest path node is the model node included in the shortest path between the start node and the end node. Both the start node and the end node belong to the model node. The data processing device determines quantifiable nodes, which include the starting node, the ending node, the shortest path node, and nodes connected to three or more model nodes. The data processing device determines the quantizable structure based on the quantizable nodes.

2. The method according to claim 1, characterized in that, The quantifiable node further includes: a single node, wherein the single node is the shortest path node that is connected to only two nodes other than the starting node and the ending node, and the two nodes connected to the single node have no other connection method except through the single node.

3. The method according to claim 1, characterized in that, The quantization methods include 1-bit, 4-bit, or 8-bit quantization methods, and / or a combination of multiple 1-bit, 4-bit, or 8-bit quantization methods.

4. A data processing device, characterized in that, include: Acquisition unit, used to acquire model information; A generation unit generates model nodes based on the model information. Each model node is a computation unit, and the number of model nodes is two or more. A determining unit is used to determine the connection relationships between the model nodes; The determining unit is further configured to determine a quantifiable structure based on the connection relationship, wherein the quantifiable structure includes two or more nodes; The determining unit is also used to traverse the quantizable methods of the quantizable structure and determine the optimal quantization method; The determining unit is further configured to obtain an optimized model based on the optimal quantization method, wherein the optimized model is configured to perform at least one of the following: image recognition, target detection and tracking, or image retrieval; The determining unit is specifically used for: The shortest path node is determined based on the connection relationship. The shortest path node is the model node included in the shortest path between the start node and the end node. Both the start node and the end node belong to the model node. Determine quantifiable nodes, which include nodes connected to three or more model nodes among the start node, the end node, and the shortest path node; The quantizable structure is determined based on the quantizable nodes.

5. The device according to claim 4, characterized in that, The quantifiable node further includes: a single node, wherein the single node is the shortest path node that is connected to only two nodes other than the starting node and the ending node, and the two nodes connected to the single node have no other connection method except through the single node.

6. The device according to claim 4, characterized in that, The quantization methods include 1-bit, 4-bit, or 8-bit quantization methods, and / or a combination of multiple 1-bit, 4-bit, or 8-bit quantization methods.

7. A computer storage medium, characterized in that, The computer storage medium stores instructions that, when executed on the computer, cause the computer to perform the method as described in any one of claims 1 to 3.

8. A computer program product, characterized in that, When the computer program product is executed on a computer, it causes the computer to perform the method as described in any one of claims 1 to 3.

Images