A network precision quantification method and system

Abstract

The present invention discloses a method for quantizing network accuracy, which is applied to a multi-core chip. The method comprises: determining a reference accuracy according to the total amount of core resources of the multi-core chip and each network to be quantized, wherein the total amount of core resources required for quantizing each network to be quantized according to the reference accuracy is less than or equal to the total amount of core resources of the multi-core chip; determining the target accuracy corresponding to each network to be quantized according to the reference accuracy and the total amount of core resources of the multi-core chip. The present invention also discloses a system for quantizing network accuracy. The beneficial effects of the present invention are: improving the chip resource utilization rate when multiple networks coexist, while reducing the accuracy loss caused by over-quantization.

Description

Technical Field

[0001] The present invention relates to the field of neural network technology, and in particular to a method and system for quantifying network accuracy. Background Art

[0002] In related technologies, when performing precision quantization on neural networks, the precision selection is single, and only the reduction in resource utilization caused by the quantization of a single network is considered. When multiple networks coexist on a multi-core chip, the problem of how to select the precision of multiple networks and achieve reasonable allocation of on-chip resources is not solved, resulting in insufficient utilization of chip resources or precision loss caused by over-quantization. Summary of the invention

[0003] To solve the above problems, the present invention provides a network accuracy quantification method and system, which can improve chip resource utilization when multiple networks coexist, and reduce the accuracy loss caused by network accuracy quantization.

[0004] The present invention provides a method for quantifying network accuracy, which is applied to a multi-core chip and includes:

[0005] Determine a reference accuracy according to the total amount of core resources of the many-core chip and each network to be quantized, wherein the total amount of core resources required for quantizing each network to be quantized according to the reference accuracy is less than or equal to the total amount of core resources of the many-core chip;

[0006] According to the reference accuracy and the total amount of core resources of the many-core chip, a target accuracy corresponding to each network to be quantized is determined.

[0007] As a further improvement of the present invention, the reference accuracy is determined according to the total amount of core resources of the multi-core chip and each network to be quantized, including:

[0008] Determine the total amount of core resources S1 required for quantization of each network to be quantized according to the first precision;

[0009] Determine whether the total amount of core resources S1 is less than or equal to the total amount of core resources Z of the many-core chip;

[0010] If the total amount of core resources S1 is greater than the total amount of core resources Z of the many-core chip, then determine the total amount of core resources S2 required for quantization of each network to be quantized according to the second precision, and judge whether the total amount of core resources S2 is less than or equal to the total amount of core resources Z of the many-core chip, wherein the second precision is lower than the first precision;

[0011] The quantization precision decreases step by step, and so on, until the total amount of core resources S required for quantization of each network to be quantized according to the jth precision is determined. jis less than or equal to the total amount of core resources Z of the many-core chip, and the jth precision is determined as the reference precision, where j is an integer greater than or equal to 2.

[0012] As a further improvement of the present invention, according to the reference accuracy and the total amount of core resources of the many-core chip, the target accuracy corresponding to each network to be quantized is determined, including:

[0013] According to the total amount of core resources S required for quantization of each network to be quantized according to the reference accuracy j j and the total amount of core resources Z of the multi-core chip, and determine the remaining amount of core resources Y j , where Y j =ZS j , j is an integer greater than or equal to 2;

[0014] Determine at least one difference in the number of core resources of each network to be quantized according to each precision quantization and according to the j-th precision quantization W[i]={M[i][1]-M[i][j],M[i][2]-M[i][j],...}, where i is used to represent the number of the network to be quantized, and i is an integer greater than or equal to 1;

[0015] According to the remaining nuclear resources Y j The target accuracy corresponding to each network to be quantized is determined by the difference in the number of core resources of each network to be quantized step by step.

[0016] As a further improvement of the present invention, according to the remaining nuclear resource amount Y j and the difference in the number of core resources of each network to be quantized step by step, and determining the target accuracy corresponding to each network to be quantized, including:

[0017] For each network to be quantified, a core resource quantity difference is determined from the at least one core resource quantity difference W[i]={M[i][1]-M[i][j], M[i][2]-M[i][j], ...}, so that the sum of the core resource quantity differences in each network to be quantified is less than or equal to the remaining core resource quantity Y j When , the sum of the core resource differences of the networks to be quantized is the largest;

[0018] When the sum of the core resource differences of the networks to be quantized is the largest, the target accuracy corresponding to the networks to be quantized is determined.

[0019] As a further improvement of the present invention, each network to be quantized includes a first type of network to be quantized and a second type of network to be quantized, and the target accuracy corresponding to the second type of network to be quantized is the kth accuracy.

[0020] Determining the target accuracy corresponding to each network to be quantized according to the reference accuracy and the total amount of core resources of the many-core chip includes:

[0021] Determine the first type of network to be quantized according to the reference accuracy j / Quantify the total amount of nuclear resources required S j / And the total amount of core resources S required for the second type of network to be quantized according to the specified precision k k , where j / is an integer greater than or equal to 1, k is an integer greater than or equal to 1;

[0022] According to the total amount of nuclear resources S j / The total amount of nuclear resources S k and the total amount of core resources Z of the multi-core chip, and determine the remaining amount of core resources Y j / , where Y j / =ZS j / -S k ;

[0023] Determine the first type of network to be quantized according to each precision and according to the jth / The difference in the number of core resources of at least one precision quantization W[i′]={M[i′][1]-M[i′][j / ],M[i′][2]-M[i′][j / ],...}, where i / The number used to represent the first type of network to be quantized, i / is an integer greater than or equal to 1;

[0024] According to the remaining nuclear resources Y j / As a further improvement of the present invention, according to the remaining core resource quantity Y j / and the difference in the number of core resources of the first type of networks to be quantized step by step, and determining the target accuracy corresponding to the first type of networks to be quantized, including:

[0025] For each network in the first type of networks to be quantified, from the at least one core resource quantity difference W[i′]={M[i′][1]-M[i′][j / ],M[i′][2]-M[i′][j / ],...} to determine a core resource quantity difference so that the sum of the core resource quantity differences of the first type of networks to be quantified is less than or equal to the remaining core resource quantity Y j / When , the sum of the resource differences of each core of the first type of network to be quantized is the largest;

[0026] When the sum of the differences in the core resources of the first type of networks to be quantized is the largest, the target accuracy corresponding to the first type of networks to be quantized is determined.

[0027] As a further improvement of the present invention, the target accuracies corresponding to the networks to be quantized are not completely the same.

[0028] As a further improvement of the present invention, the determining of the total amount of core resources S1 required for quantization of each network to be quantized according to the first precision includes:

[0029] Calculate the number of core resources M[i][1] required when the i-th network is quantized to the first precision;

[0030] Determine the total amount of core resources S1 required for quantization of each network to be quantized according to the first precision;

[0031] in, The number of networks is N, i is used to represent the number of the network to be quantized, i is an integer greater than or equal to 1, and N is an integer greater than or equal to 1.

[0032] As a further improvement of the present invention, the reference accuracy is determined according to the total amount of core resources of the multi-core chip and each network to be quantized, including:

[0033] Determine the total amount of core resources S1 required for quantization of each network to be quantized according to the first precision;

[0034] Determine whether the total amount of core resources S1 is less than or equal to the total amount of core resources Z of the many-core chip;

[0035] If the total amount of core resources S1 is less than or equal to the total amount of core resources Z of the many-core chip, the first precision is determined as the reference precision;

[0036] Wherein, determining the target accuracy corresponding to each network to be quantized according to the reference accuracy and the total amount of core resources of the many-core chip includes:

[0037] The first accuracy is determined as the target accuracy corresponding to each network to be quantized.

[0038] As a further improvement of the present invention, the quantization accuracy includes one or more of fp32, fp16, int8, and int4.

[0039] The present invention also provides a network accuracy quantification system, which is applied to a multi-core chip and includes:

[0040] A reference accuracy determination module, used to determine a reference accuracy according to the total amount of core resources of the many-core chip and each network to be quantized, wherein the total amount of core resources required for quantization of each network to be quantized according to the reference accuracy is less than or equal to the total amount of core resources of the many-core chip;

[0041] The target accuracy determination module is used to determine the target accuracy corresponding to each network to be quantized according to the reference accuracy and the total amount of core resources of the many-core chip.

[0042] As a further improvement of the present invention, the reference accuracy determination module is used for:

[0043] Determine the total amount of core resources S1 required for quantization of each network to be quantized according to the first precision;

[0044] Determine whether the total amount of core resources S1 is less than or equal to the total amount of core resources Z of the many-core chip;

[0045] If the total amount of core resources S1 is greater than the total amount of core resources Z of the many-core chip, then determine the total amount of core resources S2 required for quantization of each network to be quantized according to the second precision, and judge whether the total amount of core resources S2 is less than or equal to the total amount of core resources Z of the many-core chip, wherein the second precision is lower than the first precision;

[0046] The quantization precision decreases step by step, and so on, until the total amount of core resources S required for quantization of each network to be quantized according to the jth precision is determined. j is less than or equal to the total amount of core resources Z of the many-core chip, and the jth precision is determined as the reference precision, where j is an integer greater than or equal to 2.

[0047] As a further improvement of the present invention, the target accuracy determination module is used to:

[0048] According to the total amount of core resources S required for quantization of each network to be quantized according to the reference accuracy j j and the total amount of core resources Z of the multi-core chip, and determine the remaining amount of core resources Y j , where Y j =ZS j , j is an integer greater than or equal to 2;

[0049] Determine at least one difference in the number of core resources of each network to be quantized according to each precision quantization and according to the j-th precision quantization W[i]={M[i][1]-M[i][j],M[i][2]-M[i][j],...}, where i is used to represent the number of the network to be quantized, and i is an integer greater than or equal to 1;

[0050] According to the remaining nuclear resources Y j The target accuracy corresponding to each network to be quantized is determined by the difference in the number of core resources of each network to be quantized step by step.

[0051] As a further improvement of the present invention, according to the remaining nuclear resource amount Y j and the difference in the number of core resources of each network to be quantized step by step, and determining the target accuracy corresponding to each network to be quantized, including:

[0052] For each network to be quantified, a core resource quantity difference is determined from the at least one core resource quantity difference W[i]={M[i][1]-M[i][j], M[i][2]-M[i][j], ...}, so that the sum of the core resource quantity differences in each network to be quantified is less than or equal to the remaining core resource quantity Y j When , the sum of the core resource differences of the networks to be quantized is the largest;

[0053] When the sum of the core resource differences of the networks to be quantized is the largest, the target accuracy corresponding to the networks to be quantized is determined.

[0054] As a further improvement of the present invention, each network to be quantized includes a first type of network to be quantized and a second type of network to be quantized, the target accuracy corresponding to the second type of network to be quantized is the kth accuracy, and the target accuracy determination module is used to:

[0055] Determining the target accuracy corresponding to each network to be quantized according to the reference accuracy and the total amount of core resources of the many-core chip includes:

[0056] Determine the first type of network to be quantized according to the reference accuracy j / Quantify the total amount of nuclear resources required S j / And the total amount of core resources S required for the second type of network to be quantized according to the specified precision k k , where j / is an integer greater than or equal to 1, k is an integer greater than or equal to 1;

[0057] According to the total amount of nuclear resources S j / The total amount of nuclear resources S k and the total amount of core resources Z of the multi-core chip, and determine the remaining amount of core resources Y j / , where Y j / =ZS j / -S k ;

[0058] Determine the first type of network to be quantized according to each precision and according to the jth / The difference in the number of core resources of at least one precision quantization W[i′]={M[i′][1]-M[i′][j / ],M[i′][2]-M[i′][j / ],...}, where i / The number used to represent the first type of network to be quantized, i / is an integer greater than or equal to 1;

[0059] According to the remaining nuclear resources Y j / and the difference in the number of core resources of each level of quantization of the first type of network to be quantized, to determine the target accuracy corresponding to the first type of network to be quantized.

[0060] As a further improvement of the present invention, according to the remaining nuclear resource amount Y j / and the difference in the number of core resources of the first type of networks to be quantized step by step, and determining the target accuracy corresponding to the first type of networks to be quantized, including:

[0061] For each network in the first type of networks to be quantified, from the at least one core resource quantity difference W[i′]={M[i′][1]-M[i′][j / ],M[i′][2]-M[i′][j / ],...} to determine a core resource quantity difference so that the sum of the core resource quantity differences of the first type of networks to be quantified is less than or equal to the remaining core resource quantity Y j / When , the sum of the resource differences of each core of the first type of network to be quantized is the largest;

[0062] When the sum of the differences in the core resources of the first type of networks to be quantized is the largest, the target accuracy corresponding to the first type of networks to be quantized is determined.

[0063] As a further improvement of the present invention, the target accuracies corresponding to the networks to be quantized are not completely the same.

[0064] As a further improvement of the present invention, the determining of the total amount of core resources S1 required for quantization of each network to be quantized according to the first precision includes:

[0065] Calculate the number of core resources M[i][1] required when the i-th network is quantized to the first precision;

[0066] Determine the total amount of core resources S1 required for quantization of each network to be quantized according to the first precision;

[0067] in, The number of networks is N, i represents the number of the network to be quantized, i is an integer greater than or equal to 1, and N is an integer greater than or equal to 1.

[0068] As a further improvement of the present invention, the reference accuracy determination module is used for:

[0069] Determine the total amount of core resources S1 required for quantization of each network to be quantized according to the first precision;

[0070] Determine whether the total amount of core resources S1 is less than or equal to the total amount of core resources Z of the many-core chip;

[0071] If the total amount of core resources S1 is less than or equal to the total amount of core resources Z of the many-core chip, the first precision is determined as the reference precision;

[0072] Wherein, determining the target accuracy corresponding to each network to be quantized according to the reference accuracy and the total amount of core resources of the many-core chip includes:

[0073] The first accuracy is determined as the target accuracy corresponding to each network to be quantized.

[0074] As a further improvement of the present invention, the quantization accuracy includes one or more of fp32, fp16, int8, and int4.

[0075] The present invention also provides an electronic device, comprising a memory and a processor, wherein the memory is used to store one or more computer instructions, wherein the one or more computer instructions are executed by the processor to implement the described method.

[0076] The present invention also provides a computer-readable storage medium on which a computer program is stored. The computer program is executed by a processor to implement the method.

[0077] The beneficial effects of the present invention are: solving the on-chip resource allocation problem of neural network multi-network models by quantization, solving the optimal selection problem of quantization accuracy by a multi-level dynamic programming algorithm, optimizing the allocation and use of chip on-chip memory, and achieving full utilization of limited chip memory resources in the memory allocation process of on-chip multi-network models, while reducing the accuracy loss caused by over-quantization. BRIEF DESCRIPTION OF THE DRAWINGS

[0078] In order to more clearly illustrate the technical solutions in the embodiments of the present disclosure or the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present disclosure, and for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative labor.

[0079] Figure 1 A schematic diagram of a flow chart of a method for quantifying network accuracy according to an exemplary embodiment of the present disclosure;

[0080] Figure 2 A schematic diagram of a process of step-by-step quantization of multiple networks according to an exemplary embodiment of the present disclosure;

[0081] Figure 3 A schematic diagram of a process of step-by-step quantization of multiple networks according to another exemplary embodiment of the present disclosure;

[0082] Figure 4 A schematic diagram of resource allocation for multiple network cores of a many-core chip according to an exemplary embodiment of the present disclosure;

[0083] In the figure,

[0084] 1. The first network; 2. The second network; 3. The third network; 4. The fourth network; 5. The fifth network; 6. The sixth network; 7. The seventh network. DETAILED DESCRIPTION

[0085] The following will be combined with the drawings in the embodiments of the present disclosure to clearly and completely describe the technical solutions in the embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, not all of the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present disclosure.

[0086] It should be noted that if the embodiments of the present disclosure involve directional indications (such as up, down, left, right, front, back, etc.), such directional indications are only used to explain the relative position relationship, movement status, etc. between the components under a certain specific posture (as shown in the accompanying drawings). If the specific posture changes, the directional indication will also change accordingly.

[0087] In addition, in the description of the present disclosure, the terms used are only for illustrative purposes and are not intended to limit the scope of the present disclosure. The terms "include" and / or "comprise" are used to specify the existence of the elements, steps, operations and / or components, but do not exclude the existence or addition of one or more other elements, steps, operations and / or components. The terms "first", "second" and the like may be used to describe various elements, do not represent the order, and do not limit these elements. In addition, in the description of the present disclosure, unless otherwise specified, the meaning of "multiple" is two and more than two. These terms are only used to distinguish one element from another element. In conjunction with the following drawings, these and / or other aspects become apparent, and it is easier for those of ordinary skill in the art to understand the description of the embodiments described in the present disclosure. The accompanying drawings are used to describe the embodiments described in the present disclosure for illustrative purposes only. Those skilled in the art will easily recognize from the following description that, without departing from the principles described in the present disclosure, alternative embodiments of the structures and methods shown in the present disclosure can be adopted.

[0088] A network accuracy quantification method according to an embodiment of the present disclosure is applied to a multi-core chip, such as Figure 1 As shown, the method includes:

[0089] In step S1, a reference accuracy is determined according to the total amount of core resources of the many-core chip and each network to be quantized, wherein the total amount of core resources required for quantizing each network to be quantized according to the reference accuracy is less than or equal to the total amount of core resources of the many-core chip;

[0090] In step S2, the target accuracy corresponding to each network to be quantized is determined according to the reference accuracy and the total amount of core resources of the many-core chip.

[0091] Among them, the network to be quantized can be various neural network models that need to be quantized. The total amount of core resources of the many-core chip can refer to the number of cores of the many-core chip or the core storage resources of the many-core chip. The reference accuracy and the target accuracy can be determined from various types of quantization accuracy. For example, the quantization accuracy may include one or more of fp32 (32-bit data type), fp16 (16-bit data type), int8 (8-bit data type), and int4 (4-bit data type). For example, the quantization accuracy includes fp32, fp16, int8, and int4, and the reference accuracy and the target accuracy corresponding to each network to be quantized can be selected from fp32, fp16, int8, and int4. Among them, the target accuracy corresponding to each network to be quantized can be the same, for example, the target accuracy corresponding to each network to be quantized is the reference accuracy, and the target accuracy corresponding to each network to be quantized may not be exactly the same.

[0092] In artificial intelligence chips, chip storage resources are limited, and as neural networks consume more chip storage resources, it becomes particularly important to quantify the accuracy of neural network models. Many-core chips include multiple cores, which have great advantages in neural network applications, and many-core chips have a large number of cores, so quantization accuracy is not just a simple accuracy selection. When multiple neural networks coexist, it is necessary to reasonably allocate resources for multiple networks to achieve high utilization of core resources and reduce the loss of accuracy caused by excessive quantization. The method described in the present disclosure is aimed at the situation where multiple networks coexist in many-core chips. Based on the number of networks and core resource perception, according to the on-chip resources of the many-core chip and the specific resource requirements of multiple networks, multi-level quantization of the accuracy of multiple networks is performed through a multi-level dynamic programming method, and the resources of each network are dynamically planned and allocated to achieve reasonable resource allocation and full utilization of chip on-chip resources, and reduce the loss of accuracy caused by excessive quantization.

[0093] In an optional implementation manner, determining the reference accuracy according to the total amount of core resources of the many-core chip and each network to be quantized includes:

[0094] Determine the total amount of core resources S1 required for quantization of each network to be quantized according to the first precision;

[0095] Determine whether the total amount of core resources S1 is less than or equal to the total amount of core resources Z of the many-core chip;

[0096] If the total amount of core resources S1 is less than or equal to the total amount of core resources Z of the many-core chip, the first precision is determined as the reference precision;

[0097] Wherein, determining the target accuracy corresponding to each network to be quantized according to the reference accuracy and the total amount of core resources of the many-core chip includes:

[0098] The first accuracy is determined as the target accuracy corresponding to each network to be quantized.

[0099] For example, the first precision is the highest precision, and each network to be quantized can be quantized according to the highest precision. When the total amount of core resources required for quantizing each network to be quantized according to the highest precision is less than or equal to the total amount of core resources of the multi-core chip, each network to be quantized can be directly allocated core resources according to the highest quantization precision to make the most effective use of core resources and ensure that each network to be quantized has high precision.

[0100] In an optional implementation, determining the total amount of core resources S1 required for quantization of each network to be quantized according to the first precision includes:

[0101] Calculate the number of core resources M[i][1] required when the i-th network is quantized to the first precision;

[0102] Determine the total amount of core resources S1 required for quantization of each network to be quantized according to the first precision;

[0103] in, The number of networks to be quantized is N, i represents the number of the network to be quantized, i is an integer greater than or equal to 1, and N is an integer greater than or equal to 1.

[0104] In an optional implementation manner, determining the reference accuracy according to the total amount of core resources of the many-core chip and each network to be quantized includes:

[0105] Determine the total amount of core resources S1 required for quantization of each network to be quantized according to the first precision;

[0106] Determine whether the total amount of core resources S1 is less than or equal to the total amount of core resources Z of the many-core chip;

[0107] If the total amount of core resources S1 is greater than the total amount of core resources Z of the many-core chip, then determine the total amount of core resources S2 required for quantization of each network to be quantized according to the second precision, and judge whether the total amount of core resources S2 is less than or equal to the total amount of core resources Z of the many-core chip, wherein the second precision is lower than the first precision;

[0108] The quantization precision decreases step by step, and so on, until the total amount of core resources S required for quantization of each network to be quantized according to the jth precision is determined. j is less than or equal to the total amount of core resources Z of the many-core chip, and the jth precision is determined as the reference precision, where j is an integer greater than or equal to 2.

[0109] Among them, when determining the total amount of core resources required for quantization of each network to be quantized according to the quantization accuracy, it can be done by Calculate, the number of networks to be quantized is N, i represents the number of the network to be quantized, i is an integer greater than or equal to 1, and N is an integer greater than or equal to 1.

[0110] For example, the quantization precision includes fp32, fp16, int8, and int4. The first precision can represent fp32 precision, the second precision represents fp16, the third precision represents int8, and the fourth precision represents int4. When the total amount of core resources S1 required for quantization of each network to be quantized according to the first precision is greater than the total amount of core resources Z of the multi-core chip, the quantization precision is gradually reduced. For example, the total amount of core resources S2 required for quantization of each network to be quantized according to the second precision (j=2) can be determined, and the second precision is lower than the first precision. If S2 is less than or equal to the total amount of core resources Z of the multi-core chip, the second precision (j=2) is determined as the reference precision.

[0111] If S2 is greater than the total amount of core resources Z of the multi-core chip, the quantization accuracy is continued to be reduced step by step, and the total amount of core resources S3 required for quantization of each network to be quantized according to the third accuracy (j=3) is determined. If S3 is less than or equal to the total amount of core resources Z of the multi-core chip, the third accuracy (j=3) is determined as the reference accuracy. If S3 is greater than the total amount of core resources Z of the multi-core chip, the quantization accuracy is reduced step by step, and so on. For example, when it is determined that the total amount of core resources S4 required for quantization of each network to be quantized according to the fourth accuracy is less than or equal to the total amount of core resources Z of the multi-core chip, the fourth accuracy (j=4) is determined as the reference accuracy.

[0112] It should be understood that the reference accuracy can be determined in various ways, as long as the total amount of core resources required for quantizing each network to be quantized according to the reference accuracy is less than or equal to the total amount of core resources of the multi-core chip. The present disclosure does not limit the method of determining the reference accuracy.

[0113] In an optional implementation, determining a target accuracy corresponding to each network to be quantized according to the reference accuracy and the total amount of core resources of the many-core chip includes:

[0114] According to the total amount of core resources S required for quantization of each network to be quantized according to the reference accuracy j j and the total amount of core resources Z of the multi-core chip, and determine the remaining amount of core resources Y j , where Y j =ZS j , j is an integer greater than or equal to 2;

[0115] Determine at least one difference in the number of core resources of each network to be quantized according to each precision quantization and according to the j-th precision quantization W[i]={M[i][1]-M[i][j],M[i][2]-M[i][j],...}, where i is used to represent the number of the network to be quantized, and i is an integer greater than or equal to 1;

[0116] According to the remaining nuclear resources Y j The target accuracy corresponding to each network to be quantized is determined by the difference in the number of core resources of each network to be quantized step by step.

[0117] In another optional implementation, according to the remaining nuclear resource amount Y j and the difference in the number of core resources of each network to be quantized step by step, and determining the target accuracy corresponding to each network to be quantized, including:

[0118] For each network to be quantized, the difference in the number of core resources is

[0119] Determine a core resource quantity difference in W[i]={M[i][1]-M[i][j], M[i][2]-M[i][j], ...} so that the sum of the core resource quantity differences of the networks to be quantified is less than or equal to the remaining core resource quantity Y j When , the sum of the core resource differences of the networks to be quantized is the largest;

[0120] When the sum of the core resource differences of the networks to be quantized is the largest, the target accuracy corresponding to the networks to be quantized is determined.

[0121] For example, Figure 2 As shown, multiple networks are quantized step by step from fp32, int8 to int4, and the quantization accuracy can be selected from fp32, int8, and int4. Among them, the first accuracy is fp32, the second accuracy is int8, and the third accuracy is int4.

[0122] Step 1, determine the total amount of core resources S1 required for quantization of each network to be quantized according to fp32 precision, and determine whether the total amount of core resources S1 is less than or equal to the total amount of core resources Z of the many-core chip;

[0123] If the total amount of core resources S1 is less than or equal to the total amount of core resources Z of the multi-core chip, the fp32 precision is determined as the target precision corresponding to each network to be quantized, and each network to be quantized is allocated core resources according to the fp32 precision;

[0124] If the total amount of core resources S1 is greater than the total amount of core resources Z of the many-core chip, proceed to Step 2;

[0125] Step 2, determine the total amount of core resources S2 required for quantization of each network to be quantized according to int8 precision, and determine whether the total amount of core resources S2 is less than or equal to the total amount of core resources Z of the many-core chip;

[0126] If the total amount of core resources S2 is less than or equal to the total amount of core resources Z of the many-core chip, the int8 precision is determined as the reference precision, and the target precision corresponding to each network to be quantized is determined respectively. The precision of each network to be quantized can be selected from fp32 precision or int8 precision, and the total amount of core resources required for each network to be quantized after quantization according to its own target precision is ensured to be less than or equal to the total amount of core resources Z of the many-core chip;

[0127] If the total amount of core resources S2 is greater than the total amount of core resources Z of the many-core chip, proceed to Step 3;

[0128] Step 3, determine the total amount of core resources S3 required for quantization of each network to be quantized according to int4 precision, and determine whether the total amount of core resources S3 is less than or equal to the total amount of core resources Z of the multi-core chip;

[0129] If the total amount of core resources S3 is less than or equal to the total amount of core resources Z of the many-core chip, the int4 precision is determined as the reference precision, and the target precision corresponding to each network to be quantized is determined respectively. The precision of each network to be quantized can be selected from fp32 precision, int8 precision or int4 precision, so as to ensure that the total amount of core resources required for each network to be quantized according to its own target precision is less than or equal to the total amount of core resources Z of the many-core chip;

[0130] If the total amount of core resources S3 is greater than the total amount of core resources Z of the many-core chip, it is determined that all networks cannot be stored simultaneously.

[0131] In some optional embodiments, it is determined in Step 1 that the total amount of core resources S1 is greater than the total amount of core resources Z of the many-core chip, and it is determined in Step 2 that the total amount of core resources S2 is less than or equal to the total amount of core resources Z of the many-core chip, int8 precision is determined as the reference precision, and the remaining amount of core resources Y2=Z-S2 is determined. When calculating the difference in the number of core resources of each network to be quantized from fp32 precision to int8 precision, the difference in the number of core resources of each network to be quantized is W[i]=M[i][1]-M[i][2]. According to the remaining amount of core resources Y2 and the difference in the number of core resources of each network to be quantized step by step, the target precision corresponding to each network to be quantized is determined.

[0132] In some optional embodiments, it is determined in Step 1 that the total amount of core resources S1 is greater than the total amount of core resources Z of the many-core chip, it is determined in Step 2 that the total amount of core resources S2 is greater than the total amount of core resources Z of the many-core chip, it is determined in Step 3 that the total amount of core resources S3 is less than or equal to the total amount of core resources Z of the many-core chip, the int4 precision is determined as the reference precision, and the remaining amount of core resources Y3=Z-S3 is determined. When calculating the difference in the number of core resources of each network to be quantized from fp32 precision to int4 precision, the difference in the number of core resources of each network to be quantized can be W[i]=M[i][1]-M[i][3], or it can be W[i]=M[i][2]-M[i][3]. According to the remaining amount of core resources Y3=Z-S3 and the difference in the number of core resources of each network to be quantized step by step, the target precision corresponding to each network to be quantized is determined.

[0133] When determining the target accuracy of each network to be quantified, the present disclosure can quantize each network to be quantified according to the reference accuracy, and use the remaining core resource amount after quantification as the capacity of the new backpack. The difference in the number of core resources of each network to be quantified represents the value of the new backpack items. The 0-1 backpack dynamic programming algorithm is used to obtain the maximum sum of the values ​​of each network to be quantified. At this time, the target accuracy corresponding to each network to be quantified when the sum of the values ​​is the maximum can be determined. For multiple networks on a chip, the present disclosure uses the maximum sum of the core resource differences of each network as the optimal solution for the target accuracy of each network, so that the core resources of the chip are most fully utilized, and the reasonable allocation of core resources is achieved. At the same time, it also reduces the accuracy loss caused by selecting a single quantization accuracy for multiple networks.

[0134] In an optional implementation, the target accuracies corresponding to the networks to be quantized are not completely the same.

[0135] For example, Figure 4 As shown, after the dynamic programming algorithm, the first network is quantized to fp32 precision, the second network, the fourth network and the fifth network are quantized to int8 precision, and the third network, the sixth network and the seventh network are quantized to int4 precision. The sum of the values ​​of all networks is the largest, that is, the target precision of the first network is determined to be fp32 precision, the target precision of the second network, the fourth network and the fifth network is determined to be int8 precision, and the target precision of the third network, the sixth network and the seventh network is determined to be int4 precision.

[0136] In an optional implementation, the networks to be quantized include a first type of networks to be quantized and a second type of networks to be quantized, and the target accuracy corresponding to the second type of networks to be quantized is the kth accuracy.

[0137] Determining the target accuracy corresponding to each network to be quantized according to the reference accuracy and the total amount of core resources of the many-core chip includes:

[0138] Determine the first type of network to be quantized according to the reference accuracy j / Quantify the total amount of nuclear resources required S j / And the total amount of core resources S required for the second type of network to be quantized according to the specified precision k k , where j / is an integer greater than or equal to 1, k is an integer greater than or equal to 1;

[0139] According to the total amount of nuclear resources S j / The total amount of nuclear resources S k and the total amount of core resources Z of the multi-core chip, and determine the remaining amount of core resources Y j / , where Y j / =ZS j / -S k ;

[0140] Determine the first type of network to be quantized according to each precision and according to the jth / The difference in the number of core resources of at least one precision quantization W[i′]={M[i′][1]-M[i′][j / ],M[i′][2]-M[i′][j / ],...}, where i / Indicates the number of the first type of network to be quantized, i / is an integer greater than or equal to 1;

[0141] According to the remaining nuclear resources Y j / and the difference in the number of core resources of each level of quantization of the first type of network to be quantized, to determine the target accuracy corresponding to the first type of network to be quantized.

[0142] In another optional implementation, according to the remaining nuclear resource amount Y j / and the difference in the number of core resources of the first type of networks to be quantized step by step, and determining the target accuracy corresponding to the first type of networks to be quantized, including:

[0143] For each network in the first type of networks to be quantified, from the at least one core resource quantity difference W[i′]={M[i′][1]-M[i′][j / ],M[i′][2]-M[i′][j / ],...} to determine a core resource quantity difference so that the sum of the core resource quantity differences of the first type of networks to be quantified is less than or equal to the remaining core resource quantity Y j / When , the sum of the resource differences of each core of the first type of network to be quantized is the largest;

[0144] When the sum of the differences in the core resources of the first type of networks to be quantized is the largest, the target accuracy corresponding to the first type of networks to be quantized is determined.

[0145] When the present disclosure performs precision quantization on each network to be quantized, precision quantization can also be performed according to demand. For example, a certain network to be quantized or several networks to be quantized are quantized according to a specified precision, and then corresponding target precisions are determined for other networks to be quantized.

[0146] For example, Figure 3As shown, multiple networks are quantized step by step from fp32, fp16, int8 to int4. The quantization accuracy can be selected from four types: fp32, fp16, int8, and int4. / =1 means fp32 precision, j / =2 means fp16 precision, j / =3 means int8 precision, j / =4 indicates int4 precision:

[0147] Step 1, determine the total amount of core resources S1 required for quantization of each network to be quantized according to fp32 precision, and determine whether the total amount of core resources S1 is less than or equal to the total amount of core resources Z of the many-core chip;

[0148] If the total amount of core resources S1 is less than or equal to the total amount of core resources Z of the multi-core chip, the fp32 precision is determined as the target precision corresponding to each network to be quantized, and each network to be quantized is allocated core resources according to the fp32 precision;

[0149] If the total amount of core resources S1 is greater than the total amount of core resources Z of the many-core chip, proceed to Step 2;

[0150] Step 2, determine the total amount of core resources S2 required for quantization of each network to be quantized according to fp16 precision, and determine whether the total amount of core resources S2 is less than or equal to the total amount of core resources Z of the many-core chip;

[0151] If the total amount of core resources S2 is less than or equal to the total amount of core resources Z of the many-core chip, the int8 precision is determined as the reference precision, and the target precision corresponding to each network to be quantized is determined respectively. The precision of each network to be quantized can be selected from fp32 precision or fp16 precision, so as to ensure that the total amount of core resources required for each network to be quantized according to its own target precision is less than or equal to the total amount of core resources Z of the many-core chip;

[0152] If the total amount of core resources S2 is greater than the total amount of core resources Z of the many-core chip, proceed to Step 3;

[0153] Step 3, determine the total amount of core resources S3 required for quantization of each network to be quantized according to int8 precision, and determine whether the total amount of core resources S3 is less than or equal to the total amount of core resources Z of the many-core chip;

[0154] If the total amount of core resources S3 is less than or equal to the total amount of core resources Z of the many-core chip, the int8 precision is determined as the reference precision, and the target precision corresponding to each network to be quantized is determined respectively. The precision of each network to be quantized can be selected from fp32 precision, fp16 precision or int8 precision, so as to ensure that the total amount of core resources required for each network to be quantized according to its own target precision is less than or equal to the total amount of core resources Z of the many-core chip;

[0155] If the total amount of core resources S3 is greater than the total amount of core resources Z of the many-core chip, proceed to Step 4;

[0156] Step 4, determining the total amount of core resources S4 required for quantization of each network to be quantized according to int4 precision, and judging whether the total amount of core resources S4 is less than or equal to the total amount of core resources Z of the multi-core chip;

[0157] If the total amount of core resources S4 is less than or equal to the total amount of core resources Z of the many-core chip, the int4 precision is determined as the reference precision, and the target precision corresponding to each network to be quantized is determined respectively. The precision of each network to be quantized can be selected from fp32 precision, fp16 precision, int8 precision or int4 precision, so as to ensure that the total amount of core resources required for each network to be quantized according to its own target precision is less than or equal to the total amount of core resources Z of the many-core chip;

[0158] If the total amount of core resources S4 is greater than the total amount of core resources Z of the many-core chip, it is determined that all networks cannot be stored simultaneously.

[0159] For example, in Step 1, it is determined that the total amount of core resources S1 is greater than the total amount of core resources Z of the many-core chip, in Step 2, it is determined that the total amount of core resources S2 is greater than the total amount of core resources Z of the many-core chip, in Step 3, it is determined that the total amount of core resources S3 is greater than the total amount of core resources Z of the many-core chip, and in Step 4, it is determined that the total amount of core resources S4 is less than or equal to the total amount of core resources Z of the many-core chip. At this time, the int4 precision is determined as the reference precision of the first type of network to be quantized, and the total amount of core resources required for quantization of the first type of network to be quantized according to the int4 precision is S4. / The total amount of core resources required for the second type of network to be quantized according to the specified precision fp32 is S k At this time, determine the remaining nuclear resources Y4 / =Z-S4 / -S kWhen calculating the difference in the number of core resources of the first type of network to be quantized from fp32 precision to int4 precision, the difference in the number of core resources of each network in the first type of network to be quantized can be W[i′]=M[i′][1]-M[i′][4], or W[i′]=M[i′][2]-M[i′][4], or W[i′]=M[i′][3]-M[i′][4], according to the remaining core resource amount Y4 / =Z-S4 / -S k As well as the difference in the number of core resources of the first type of networks to be quantized step by step, the target accuracy corresponding to each network in the first type of networks to be quantized is determined.

[0160] When determining the target accuracy of each network to be quantified, the present invention can quantify the first type of network to be quantified according to the reference accuracy, and quantify the second type of network to be quantified according to the specified accuracy. The remaining core resource amount of the two types of networks to be quantified after quantification is used as the capacity of the new backpack. The difference in the number of core resources of each network in the first type of network to be quantified represents the value of the new backpack items. The maximum total value of the first type of network to be quantified is obtained through the 0-1 backpack dynamic programming algorithm. At this time, the target accuracy corresponding to the first type of network to be quantified when the total value is maximized can be determined.

[0161] The method disclosed in the present invention solves the problem of on-chip resource allocation of neural network multi-network models by means of quantization, solves the problem of optimal selection of quantization accuracy by means of a multi-level dynamic programming algorithm, optimizes the allocation and use of on-chip memory of the chip, and fully utilizes the limited chip memory resources in the memory allocation process of the on-chip multi-network models, while reducing the accuracy loss caused by over-quantization.

[0162] A network accuracy quantization system according to an embodiment of the present disclosure is applied to a multi-core chip, and the system includes:

[0163] A reference accuracy determination module, used to determine a reference accuracy according to the total amount of core resources of the many-core chip and each network to be quantized, wherein the total amount of core resources required for quantization of each network to be quantized according to the reference accuracy is less than or equal to the total amount of core resources of the many-core chip;

[0164] The target accuracy determination module is used to determine the target accuracy corresponding to each network to be quantized according to the reference accuracy and the total amount of core resources of the many-core chip.

[0165] In an optional implementation, the reference accuracy determination module is further configured to:

[0166] Determine the total amount of core resources S1 required for quantization of each network to be quantized according to the first precision;

[0167] Determine whether the total amount of core resources S1 is less than or equal to the total amount of core resources Z of the many-core chip;

[0168] If the total amount of core resources S1 is less than or equal to the total amount of core resources Z of the many-core chip, the first precision is determined as the reference precision;

[0169] Wherein, determining the target accuracy corresponding to each network to be quantized according to the reference accuracy and the total amount of core resources of the many-core chip includes:

[0170] The first accuracy is determined as the target accuracy corresponding to each network to be quantized.

[0171] The above can be understood as that when each network to be quantized is quantized according to the highest precision and the total amount of core resources required is less than or equal to the total amount of core resources of the multi-core chip, each network to be quantized can be directly allocated core resources according to the highest quantization precision to make the most effective use of core resources.

[0172] In an optional implementation, determining the total amount of core resources S1 required for quantization of each network to be quantized according to the first precision includes:

[0173] Calculate the number of core resources M[i][1] required when the i-th network is quantized to the first precision;

[0174] Determine the total amount of core resources S1 required for quantization of each network to be quantized according to the first precision;

[0175] in, The number of networks is N, i represents the number of the network to be quantized, i is an integer greater than or equal to 1, and N is an integer greater than or equal to 1.

[0176] In an optional implementation manner, determining the reference accuracy according to the total amount of core resources of the many-core chip and each network to be quantized includes:

[0177] Determine the total amount of core resources S1 required for quantization of each network to be quantized according to the first precision;

[0178] Determine whether the total amount of core resources S1 is less than or equal to the total amount of core resources Z of the many-core chip;

[0179] If the total amount of core resources S1 is greater than the total amount of core resources Z of the many-core chip, then determine the total amount of core resources S2 required for quantization of each network to be quantized according to the second precision, and judge whether the total amount of core resources S2 is less than or equal to the total amount of core resources Z of the many-core chip, wherein the second precision is lower than the first precision;

[0180] The quantization precision decreases step by step, and so on, until the total amount of core resources S required for quantization of each network to be quantized according to the jth precision is determined. jis less than or equal to the total amount of core resources Z of the many-core chip, and the jth precision is determined as the reference precision, where j is an integer greater than or equal to 2.

[0181] Among them, when determining the total amount of core resources required for quantization of each network to be quantized according to the quantization accuracy, it can be done by Calculate, the number of networks to be quantized is N, i represents the number of the network to be quantized, i is an integer greater than or equal to 1, and N is an integer greater than or equal to 1.

[0182] In an optional implementation, the quantization accuracy includes one or more of fp32 (32-bit data type), fp16 (16-bit data type), int8 (8-bit data type), and int4 (4-bit data type). For example, the quantization accuracy of each network to be quantized is selected from fp32, fp16, int8, and int4. At this time, j=1 represents fp32 accuracy, j=2 represents fp16 accuracy, j=3 represents int8 accuracy, and j=4 represents int4 accuracy. For example, the quantization accuracy of each network to be quantized is selected from fp32, int8, and int4. j=1 represents fp32 accuracy, j=2 represents int8 accuracy, and j=3 represents int4 accuracy.

[0183] In an optional implementation, the target accuracy determination module is further configured to:

[0184] According to the total amount of core resources S required for quantization of each network to be quantized according to the reference accuracy j j and the total amount of core resources Z of the multi-core chip, and determine the remaining amount of core resources Y j , where Y j =ZS j , j is an integer greater than or equal to 2;

[0185] Determine at least one difference in the number of core resources of each network to be quantized according to each precision quantization and according to the j-th precision quantization W[i]={M[i][1]-M[i][j],M[i][2]-M[i][j],...}, where i is used to represent the number of the network to be quantized, and i is an integer greater than or equal to 1;

[0186] According to the remaining nuclear resources Y j The target accuracy corresponding to each network to be quantized is determined by the difference in the number of core resources of each network to be quantized step by step.

[0187] In an optional implementation, according to the remaining nuclear resource amount Y j and the difference in the number of core resources of each network to be quantized step by step, and determining the target accuracy corresponding to each network to be quantized, including:

[0188] For each network to be quantified, a core resource quantity difference is determined from the at least one core resource quantity difference W[i]={M[i][1]-M[i][j], M[i][2]-M[i][j], ...}, so that the sum of the core resource quantity differences in each network to be quantified is less than or equal to the remaining core resource quantity Y j When , the sum of the core resource differences of the networks to be quantized is the largest;

[0189] When the sum of the core resource differences of the networks to be quantized is the largest, the target accuracy corresponding to the networks to be quantized is determined.

[0190] When determining the target accuracy of each network to be quantified, the system described in the present disclosure can quantize each network to be quantified according to the reference accuracy, and use the remaining core resource amount after quantification as the capacity of the new backpack. The difference in the number of core resources of each network to be quantified represents the value of the new backpack items. The sum of the values ​​of each network to be quantified is maximized through the 0-1 backpack dynamic programming algorithm. At this time, the target accuracy corresponding to each network to be quantified when the sum of the values ​​is maximized can be determined. For multiple networks on a chip, the present disclosure uses the maximum sum of the core resource differences of each network as the optimal solution for the target accuracy of each network, so that the core resources of the chip are most fully utilized, and the reasonable allocation of core resources is achieved. At the same time, it also reduces the accuracy loss caused by selecting a single quantization accuracy for multiple networks.

[0191] In an optional implementation, the target accuracies corresponding to the networks to be quantized are not completely the same.

[0192] In an optional implementation, the networks to be quantized include a first type of networks to be quantized and a second type of networks to be quantized, the target accuracy corresponding to the second type of networks to be quantized is the kth accuracy, and the target accuracy determination module is used to:

[0193] Determining the target accuracy corresponding to each network to be quantized according to the reference accuracy and the total amount of core resources of the many-core chip includes:

[0194] Determine the first type of network to be quantized according to the reference accuracy j / Quantify the total amount of nuclear resources required S j / And the total amount of core resources S required for the second type of network to be quantized according to the specified precision k k , where j / is an integer greater than or equal to 1, k is an integer greater than or equal to 1;

[0195] According to the total amount of nuclear resources S j / The total amount of nuclear resources S kand the total amount of core resources Z of the multi-core chip, and determine the remaining amount of core resources Y j / , where Y j / =ZS j / -S k ;

[0196] Determine the first type of network to be quantized according to each precision and according to the jth / The difference in the number of core resources of at least one precision quantization W[i′]={M[i′][1]-M[i′][j / ],M[i′][2]-M[i′][j / ],...}, where i / Indicates the number of the first type of network to be quantized, i / is an integer greater than or equal to 1;

[0197] According to the remaining nuclear resources Y j / and the difference in the number of core resources of each level of quantization of the first type of network to be quantized, to determine the target accuracy corresponding to the first type of network to be quantized.

[0198] In an optional implementation, according to the remaining nuclear resource amount Y j / and the difference in the number of core resources of the first type of networks to be quantized step by step, and determining the target accuracy corresponding to the first type of networks to be quantized, including:

[0199] For each network in the first type of networks to be quantified, from the at least one core resource quantity difference W[i′]={M[i′][1]-M[i′][j / ],M[i′][2]-M[i′][j / ],...} to determine a core resource quantity difference so that the sum of the core resource quantity differences of the first type of networks to be quantified is less than or equal to the remaining core resource quantity Y j / When , the sum of the resource differences of each core of the first type of network to be quantized is the largest;

[0200] When the sum of the differences in the core resources of the first type of networks to be quantized is the largest, the target accuracy corresponding to the first type of networks to be quantized is determined.

[0201] When the present disclosure performs precision quantization on each network to be quantified, precision quantization can also be performed according to demand. For example, a certain network to be quantified or several networks to be quantified can be quantified according to a specified precision, and then the corresponding target precision can be determined for other networks to be quantified. When determining the target precision of each network to be quantified, the first type of network to be quantified can be quantified according to the reference precision, and the second type of network to be quantified can be quantified according to the specified precision. The remaining amount of core resources after the quantification of the two types of networks to be quantified is used as the capacity of the new backpack. The difference in the number of core resources of each network in the first type of network to be quantified represents the value of the new backpack items. The maximum sum of the values ​​of the first type of networks to be quantified is obtained through the 0-1 backpack dynamic programming algorithm. At this time, the target precision corresponding to the first type of networks to be quantified when the sum of values ​​is maximized can be determined.

[0202] The system disclosed in the present invention solves the problem of on-chip resource allocation for neural network multi-network models by means of quantization, solves the problem of optimal selection of quantization accuracy by means of a multi-level dynamic programming algorithm, optimizes the allocation and use of on-chip memory, and fully utilizes limited chip memory resources in the memory allocation process for on-chip multi-network models, while reducing the accuracy loss caused by over-quantization.

[0203] The present disclosure also relates to an electronic device, including a server, a terminal, etc. The electronic device includes: at least one processor; a memory connected to the at least one processor; and a communication component connected to the storage medium, wherein the communication component receives and sends data under the control of the processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to implement the network accuracy quantization method in the above-mentioned embodiment.

[0204] In an optional embodiment, the memory is a non-volatile computer-readable storage medium that can be used to store non-volatile software programs, non-volatile computer executable programs and modules. The processor executes various functional applications and data processing of the device by running the non-volatile software programs, instructions and modules stored in the memory, that is, implementing the method.

[0205] The memory may include a program storage area and a data storage area, wherein the program storage area may store an operating system and applications required for at least one function; the data storage area may store a list of options, etc. In addition, the memory may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some embodiments, the memory may optionally include a memory remotely arranged relative to the processor, and these remote memories may be connected to an external device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0206] One or more modules are stored in the memory, and when executed by one or more processors, the network accuracy quantification method in any of the above method embodiments is executed.

[0207] The above-mentioned product can execute the method provided in the embodiment of the present application, and has the functional modules and beneficial effects corresponding to the execution method. For technical details not fully described in this embodiment, please refer to the network accuracy quantification method provided in the embodiment of the present application.

[0208] The present disclosure also relates to a computer-readable storage medium for storing a computer-readable program, wherein the computer-readable program is used for a computer to execute part or all of the above-mentioned embodiments of the method for quantizing network accuracy.

[0209] That is, those skilled in the art can understand that all or part of the steps in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a program, and the program is stored in a storage medium, including a number of instructions to enable a device (which can be a single-chip microcomputer, chip, etc.) or a processor to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0210] In the description provided herein, a large number of specific details are described. However, it is understood that the embodiments of the present disclosure can be practiced without these specific details. In some instances, well-known methods, structures and techniques are not shown in detail so as not to obscure the understanding of this description.

[0211] In addition, it will be understood by those skilled in the art that, although some embodiments described herein include certain features included in other embodiments but not other features, the combination of features of different embodiments is meant to be within the scope of the present disclosure and form different embodiments. For example, in the claims, any one of the claimed embodiments may be used in any combination.

[0212] It will be appreciated by those skilled in the art that, although the present disclosure has been described with reference to exemplary embodiments, various changes may be made and elements thereof may be replaced with equivalents without departing from the scope of the present disclosure. In addition, many modifications may be made to adapt specific circumstances or materials to the teachings of the present disclosure without departing from the substantive scope of the present disclosure. Therefore, the present disclosure is not limited to the specific embodiments disclosed, but the present disclosure will include all embodiments falling within the scope of the appended claims.

Claims

1. A method for quantifying network accuracy, characterized in that: The method is applied to a many-core chip, and the method includes: Determine a reference accuracy according to the total amount of core resources of the many-core chip and each network to be quantized, wherein the total amount of core resources required for quantizing each network to be quantized according to the reference accuracy is less than or equal to the total amount of core resources of the many-core chip; According to the reference accuracy and the total amount of core resources of the many-core chip, a target accuracy corresponding to each network to be quantized is determined when the sum of the core resource differences of each network to be quantized is maximum.

2. The method of claim 1, wherein: According to the total amount of core resources of the many-core chip and each network to be quantized, the reference accuracy is determined, including: Determine the total amount of core resources S1 required for quantization of each network to be quantized according to the first precision; Determine whether the total amount of core resources S1 is less than or equal to the total amount of core resources Z of the many-core chip; If the total amount of core resources S1 is greater than the total amount of core resources Z of the many-core chip, then determine the total amount of core resources S2 required for quantization of each network to be quantized according to the second precision, and judge whether the total amount of core resources S2 is less than or equal to the total amount of core resources Z of the many-core chip, wherein the second precision is lower than the first precision; The quantization accuracy is gradually reduced, and so on, until the total amount of core resources S required for each network to be quantized according to the jth accuracy is determined. j is less than or equal to the total amount of core resources Z of the many-core chip, and the jth precision is determined as the reference precision, where j is an integer greater than or equal to 2.

3. The method of claim 1, wherein: Determining the target accuracy corresponding to each network to be quantized according to the reference accuracy and the total amount of core resources of the many-core chip includes: According to the total amount of core resources S required for quantization of each network to be quantized according to the reference accuracy j j and the total amount of core resources Z of the multi-core chip, and determine the remaining amount of core resources Y j , where Y j =ZS j , j is an integer greater than or equal to 2; Determine at least one difference in the number of core resources of each network to be quantized according to each precision quantization and according to the j-th precision quantization W[i]={M[i][1]-M[i][j],M[i][2]-M[i][j],...}, where i is used to represent the number of the network to be quantized, and i is an integer greater than or equal to 1; According to the remaining nuclear resources Y j The target accuracy corresponding to each network to be quantized is determined by the difference in the number of core resources of each network to be quantized step by step.

4. The method of claim 3, wherein: According to the remaining nuclear resources Y j and the difference in the number of core resources of each network to be quantized step by step, and determining the target accuracy corresponding to each network to be quantized, including: For each network to be quantified, a core resource quantity difference is determined from the at least one core resource quantity difference W[i]={M[i][1]-M[i][j], M[i][2]-M[i][j], ...}, so that the sum of the core resource quantity differences in each network to be quantified is less than or equal to the remaining core resource quantity Y j When , the sum of the core resource differences of the networks to be quantized is the largest; When the sum of the core resource differences of the networks to be quantized is the largest, the target accuracy corresponding to the networks to be quantized is determined.

5. The method of claim 1, wherein: The networks to be quantized include a first type of network to be quantized and a second type of network to be quantized, and the target accuracy corresponding to the second type of network to be quantized is the kth accuracy. Determining the target accuracy corresponding to each network to be quantized according to the reference accuracy and the total amount of core resources of the many-core chip includes: Determine the first type of network to be quantized according to the reference accuracy j / Quantify the total amount of nuclear resources required S j / And the total amount of core resources S required for the second type of network to be quantized according to the specified precision k k , where j / is an integer greater than or equal to 1, and k is an integer greater than or equal to 1; According to the total amount of nuclear resources S j / 、The total amount of nuclear resources S k and the total amount of core resources Z of the multi-core chip, and determine the remaining amount of core resources Y j / , where Y j / =ZS j / -S k ; Determine the first type of network to be quantized according to each precision and according to the jth / At least one core resource quantity difference W[i′]={M[i′][1]-M[i′][j / ],M[i′][2]-M[i′][j / ],...}, where i / The number used to represent the first type of network to be quantized, i / is an integer greater than or equal to 1; According to the remaining nuclear resources Y j / and the difference in the number of core resources of each level quantized by the first type of network to be quantized, to determine the target accuracy corresponding to the first type of network to be quantized.

6. The method of claim 5, wherein: According to the remaining nuclear resources Y j / and the difference in the number of core resources of the first type of network to be quantized step by step, and determining the target accuracy corresponding to the first type of network to be quantized, including: For each network in the first type of networks to be quantified, from the at least one core resource quantity difference W[i′]={M[i′][1]-M[i′][j / ],M[i′][2]-M[i′][j / ],...} to determine a core resource quantity difference so that the sum of the core resource quantity differences of the first type of networks to be quantified is less than or equal to the remaining core resource quantity Y j / When , the sum of the resource differences of each core of the first type of network to be quantized is the largest; When the sum of the differences in the core resources of the first type of networks to be quantized is the largest, a target accuracy corresponding to the first type of networks to be quantized is determined.

7. The method of claim 1, wherein: The target accuracies corresponding to the networks to be quantized are not completely the same.

8. The method of claim 2, wherein: The determining of the total amount of core resources S1 required for each network to be quantized after quantization according to the first precision includes: Calculate the number of core resources M[i][1] required when the i-th network is quantized to the first precision; Determine the total amount of core resources S1 required for each network to be quantized after quantization according to the first precision; in, The number of networks is N, i is used to represent the number of the network to be quantized, i is an integer greater than or equal to 1, and N is an integer greater than or equal to 1.

9. The method of claim 1, wherein: According to the total amount of core resources of the many-core chip and each network to be quantized, the reference accuracy is determined, including: Determine the total amount of core resources S1 required for quantization of each network to be quantized according to the first precision; Determine whether the total amount of core resources S1 is less than or equal to the total amount of core resources Z of the many-core chip; If the total amount of core resources S1 is less than or equal to the total amount of core resources Z of the many-core chip, the first precision is determined as the reference precision; Wherein, determining the target accuracy corresponding to each network to be quantized according to the reference accuracy and the total amount of core resources of the many-core chip includes: The first accuracy is determined as the target accuracy corresponding to each network to be quantized.

10. The method according to any one of claims 1 to 9, wherein: The quantization precision includes one or more of fp32, fp16, int8, and int4.

11. A network accuracy quantification system, characterized in that: The system is applied to a multi-core chip, and the system includes: A reference accuracy determination module, used to determine a reference accuracy according to the total amount of core resources of the many-core chip and each network to be quantized, wherein the total amount of core resources required for quantization of each network to be quantized according to the reference accuracy is less than or equal to the total amount of core resources of the many-core chip; The target accuracy determination module is used to determine the target accuracy corresponding to each network to be quantized when the sum of the core resource differences of each network to be quantized is the largest according to the reference accuracy and the total amount of core resources of the multi-core chip.

12. The system of claim 11, wherein: The reference accuracy determination module is used for: Determine the total amount of core resources S1 required for quantization of each network to be quantized according to the first precision; Determine whether the total amount of core resources S1 is less than or equal to the total amount of core resources Z of the many-core chip; If the total amount of core resources S1 is greater than the total amount of core resources Z of the many-core chip, then determine the total amount of core resources S2 required for quantization of each network to be quantized according to the second precision, and judge whether the total amount of core resources S2 is less than or equal to the total amount of core resources Z of the many-core chip, wherein the second precision is lower than the first precision; The quantization precision decreases step by step, and so on, until the total amount of core resources S required for quantization of each network to be quantized according to the jth precision is determined. j is less than or equal to the total amount of core resources Z of the many-core chip, and the jth precision is determined as the reference precision, where j is an integer greater than or equal to 2.

13. The system of claim 11, wherein: The target accuracy determination module is used for: According to the total amount of core resources S required for quantization of each network to be quantized according to the reference accuracy j j and the total amount of core resources Z of the multi-core chip, and determine the remaining amount of core resources Y j , where Y j =ZS j , j is an integer greater than or equal to 2; Determine at least one difference in the number of core resources of each network to be quantized according to each precision quantization and according to the j-th precision quantization W[i]={M[i][1]-M[i][j],M[i][2]-M[i][j],...}, where i is used to represent the number of the network to be quantized, and i is an integer greater than or equal to 1; According to the remaining nuclear resources Y j The target accuracy corresponding to each network to be quantized is determined by the difference in the number of core resources of each network to be quantized step by step.

14. The system of claim 13, wherein: According to the remaining nuclear resources Y j and the difference in the number of core resources of each network to be quantized step by step, and determining the target accuracy corresponding to each network to be quantized, including: For each network to be quantified, a core resource quantity difference is determined from the at least one core resource quantity difference W[i]={M[i][1]-M[i][j], M[i][2]-M[i][j], ...}, so that the sum of the core resource quantity differences in each network to be quantified is less than or equal to the remaining core resource quantity Y j When , the sum of the core resource differences of the networks to be quantized is the largest; When the sum of the core resource differences of the networks to be quantized is the largest, the target accuracy corresponding to the networks to be quantized is determined.

15. The system of claim 11, wherein: The networks to be quantized include a first type of networks to be quantized and a second type of networks to be quantized, the target accuracy corresponding to the second type of networks to be quantized is the kth accuracy, and the target accuracy determination module is used to: Determining the target accuracy corresponding to each network to be quantized according to the reference accuracy and the total amount of core resources of the many-core chip includes: Determine the first type of network to be quantized according to the reference accuracy j / Quantify the total amount of nuclear resources required S j / And the total amount of core resources S required for the second type of network to be quantized according to the specified precision k k , where j / is an integer greater than or equal to 1, k is an integer greater than or equal to 1; According to the total amount of nuclear resources S j / The total amount of nuclear resources S k and the total amount of core resources Z of the multi-core chip, and determine the remaining amount of core resources Y j / , where Y j / =ZS j / -S k ; Determine the first type of network to be quantized according to each precision and according to the jth / At least one core resource quantity difference W[i′]={M[i′][1]-M[i′][j / ],M[i′][2]-M[i′][j / ],...}, where i / The number used to represent the first type of network to be quantized, i / is an integer greater than or equal to 1; According to the remaining nuclear resources Y j / and the difference in the number of core resources of each level of quantization of the first type of network to be quantized, to determine the target accuracy corresponding to the first type of network to be quantized.

16. The system of claim 15, wherein: According to the remaining nuclear resources Y j / and the difference in the number of core resources of the first type of network to be quantized step by step, and determining the target accuracy corresponding to the first type of network to be quantized, including: For each network in the first type of networks to be quantified, from the at least one core resource quantity difference W[i′]={M[i′][1]-M[i′][j / ],M[i′][2]-M[i′][j / ],...} to determine a core resource quantity difference so that the sum of the core resource quantity differences of the first type of networks to be quantified is less than or equal to the remaining core resource quantity Y j / When , the sum of the resource differences of each core of the first type of network to be quantized is the largest; When the sum of the differences in the core resources of the first type of networks to be quantized is the largest, a target accuracy corresponding to the first type of networks to be quantized is determined.

17. The system of claim 11, wherein: The target accuracies corresponding to the networks to be quantized are not completely the same.

18. The system of claim 12, wherein: The determining of the total amount of core resources S1 required for quantizing each network to be quantized according to the first precision includes: Calculate the number of core resources M[i][j] required when the i-th network is quantized to the first precision; Determine the total amount of core resources S1 required for quantization of each network to be quantized according to the first precision; in, The number of networks is N, i is used to represent the number of the network to be quantized, i is an integer greater than or equal to 1, and N is an integer greater than or equal to 1.

19. The system of claim 11, wherein: The reference accuracy determination module is used for: Determine the total amount of core resources S1 required for quantization of each network to be quantized according to the first precision; Determine whether the total amount of core resources S1 is less than or equal to the total amount of core resources Z of the many-core chip; If the total amount of core resources S1 is less than or equal to the total amount of core resources Z of the many-core chip, the first precision is determined as the reference precision; Wherein, determining the target accuracy corresponding to each network to be quantized according to the reference accuracy and the total amount of core resources of the many-core chip includes: The first accuracy is determined as the target accuracy corresponding to each network to be quantized.

20. The system of any one of claims 11 to 19, wherein: The quantization precision includes one or more of fp32, fp16, int8, and int4.

21. An electronic device comprising a memory and a processor, characterized in that: The memory is used to store one or more computer instructions, wherein the one or more computer instructions are executed by the processor to implement the method according to any one of claims 1-10.

22. A computer-readable storage medium having a computer program stored thereon, characterized in that: The computer program is executed by a processor to implement the method according to any one of claims 1 to 10.

Images