Quantization method and device of neural network

Abstract

The application discloses a quantification method and device of a neural network, wherein the method comprises the following steps: performing gene coding processing on a plurality of initial quantification models randomly generated based on a first neural network model to obtain an initial population comprising a plurality of gene groups; performing diversity processing on the gene groups in the initial population to obtain a processed population, wherein the number of gene groups in the processed population is greater than that in the initial population; determining whether there is a gene group in the processed population whose fitness value meets a termination condition; if not, selecting a plurality of gene groups from the processed population based on the fitness values of the gene groups in the processed population to construct a new initial population, and returning to the diversity processing step; and if yes, determining quantification data of the first neural network model based on the gene group meeting the termination condition. In the implementation, a neural evolution strategy is adopted to automatically search for the best combination of a plurality of quantification values to construct a high-efficiency and accurate mixed-precision quantification model.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and more specifically, to a method and apparatus for quantizing neural networks. Background Technology

[0002] The basic process of embedded deployment of neural network models is as follows: Figure 1 As shown. Combined with Figure 1 The two main factors affecting the performance of neural network models in embedded inference are network structure and fixed-point quantization method. Therefore, a reasonable quantization method can effectively improve inference performance.

[0003] Quantization enables the mapping from floating-point to fixed-point values ​​within neural network layers; this mapping is essentially a re-encoding process. Currently, embedded neural network models primarily use a single bit depth to map floating-point numbers, making this simplistic quantization method insufficient for performance requirements. For example, using only 8-bit binary quantization is faster than 16-bit binary quantization but results in lower inference accuracy. Therefore, how to construct efficient and accurate mixed-precision quantization models through quantization processing has become a crucial research topic for professionals in the field. Summary of the Invention

[0004] In view of the above, this application provides the following technical solution:

[0005] A method for quantizing neural networks, comprising:

[0006] Gene encoding processing is performed on multiple initial quantization models generated by random quantization based on the first neural network model to obtain an initial population including multiple genomes, wherein the number of genomes is the same as the number of initial quantization models;

[0007] Diversity processing is performed on the genomes in the initial population to obtain a processed population, wherein the number of genomes in the processed population is greater than the number of genomes in the initial population;

[0008] Determine whether there are any genomes in the treated population whose fitness values ​​satisfy the termination condition;

[0009] If none exist, select multiple genomes from the treated population based on the fitness values ​​of each genome in the treated population to construct a new initial population, and return to the above-described diversity treatment steps;

[0010] If present, the quantification data for the first neural network model is determined based on the genome that meets the termination condition.

[0011] Optionally, before determining whether there are genomes in the treated population with fitness values ​​that satisfy the termination condition, the method further includes:

[0012] The genomes of the treated population were classified to obtain several different species.

[0013] Optionally, the genomes of the treated population are classified to obtain multiple different species, including:

[0014] The genomes of the treated population were clustered based on similarity to obtain multiple different species.

[0015] Optionally, multiple initial quantization models generated based on the random quantization of the first neural network model include:

[0016] The floating-point objects in each layer of the first neural network are represented by random fixed-point representations, and the bit width of the fractional part of the output vector of each layer is randomly configured to obtain multiple initial quantization models.

[0017] Optionally, the gene encoding process performed on multiple initial quantized models generated randomly based on the first neural network model includes:

[0018] The target network layer structure in the initial quantization model is fused and encoded. The target network layer includes at least two network layers. The fusion encoding is to fuse at least two network layers in the first neural network and represent them as a single line of code.

[0019] Optionally, performing diversity processing on the genomes of the initial population to obtain a processed population includes:

[0020] Random crossover and mutation processes are performed on the genomes of the initial population to obtain the processed population.

[0021] Optionally, the population evolves once for each diversity treatment performed, and the method further includes:

[0022] If, after the first number of population evolutions, no genome still meets the termination criteria, a catastrophic process is performed on the latest treated population.

[0023] Optionally, determining the fitness value of the genome includes:

[0024] Determine the difference between the final output of the genome in the smart chip inference and the floating-point output of the first neural network;

[0025] The fitness value of the genome is determined based on the differential data and the first parameter of the chip inference, wherein the first parameter represents the number of frames of images processed per second.

[0026] Optionally, the step of selecting multiple genomes from the treated population to construct a new initial population based on the fitness values ​​of each genome in the treated population includes:

[0027] The average fitness value of the second number of genomes selected within each species is determined as the standard fitness value for the corresponding species;

[0028] A portion of the genomes of the third number of species with the highest standard fitness values ​​was selected to construct a new initial population.

[0029] This application also discloses a quantization device for a neural network, comprising:

[0030] The encoding processing module is used to perform gene encoding processing on multiple initial quantization models generated by random quantization based on the first neural network model to obtain an initial population including multiple genomes, wherein the number of genomes is the same as the number of initial quantization models;

[0031] A diversity processing module is used to perform diversity processing on the genomes in the initial population to obtain a processed population, wherein the number of genomes in the processed population is greater than the number of genomes in the initial population;

[0032] The termination determination module is used to determine whether there are genomes in the post-processed population whose fitness values ​​meet the termination conditions.

[0033] The population update module is used to select multiple genomes from the processed population to construct a new initial population based on the fitness values ​​of each genome in the processed population when the determination result of the termination determination module is that no genome exists.

[0034] A quantization determination module is used to determine the quantization data of the first neural network model based on the genome that satisfies the termination condition when the determination result of the termination determination module is that it exists.

[0035] As can be seen from the above technical solutions, the embodiments of this application disclose a method and apparatus for quantizing a neural network, wherein the method includes: quantizing p generated randomly based on a first neural network model. q An initial quantization model was used for gene encoding processing to obtain p qAn initial population of genomes is established; diversity processing is performed on the genomes in the initial population to obtain a processed population, wherein the number of genomes in the processed population is greater than the number of genomes in the initial population; it is determined whether there are genomes in the processed population whose fitness values ​​satisfy the termination condition; if not, multiple genomes are selected from the processed population based on the fitness values ​​of each genome to construct a new initial population, and the above diversity processing steps are returned; if so, the quantization data of the first neural network model is determined based on the genomes that satisfy the termination condition. The above scheme employs a neuroevolutionary strategy to automatically search for the optimal combination of multiple quantization values ​​to construct an efficient and accurate mixed-precision quantization model, expands the quantization mapping method, defines the feasibility of specific network porting configurations, and has good versatility. Attached Figure Description

[0036] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of this application. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0037] Figure 1 This is a schematic diagram illustrating the basic process of embedded deployment of the neural network model disclosed in the embodiments of this application;

[0038] Figure 2 This is a flowchart of a neural network quantization method disclosed in an embodiment of this application;

[0039] Figure 3 This is an example diagram of a fusion encoding disclosed in an embodiment of this application;

[0040] Figure 4 This is a schematic diagram illustrating the specific implementation process of the neural network quantization method disclosed in the embodiments of this application;

[0041] Figure 5 This is a schematic diagram illustrating genomic crossover and variation as disclosed in the embodiments of this application;

[0042] Figure 6 This is a schematic diagram of the structure of a quantization device for a neural network disclosed in an embodiment of this application. Detailed Implementation

[0043] For the sake of clarity and citation, the explanations, abbreviations, or acronyms used in the following text are summarized below:

[0044] Quantization: Quantization is a process that uses the maximum and minimum values ​​of the current parameters to represent floating-point numbers using fixed-point numbers and scaling factors, while preserving the original information.

[0045] Mixed-precision quantization: assigns different bit widths to the parameters of each layer of the neural network, taking advantage of the inconsistent quantization sensitivity of the parameters in each layer.

[0046] Floating-point numbers: The position of the decimal point in the calculations changes, and they can be represented using scientific notation.

[0047] Fixed-point numbers: Binary data in a fixed-point format, with a defined word length, number of decimal places, number of integer places, and sign bit.

[0048] Population: A group of people composed of multiple genomes.

[0049] Gene encoding: Each quantized network in the population is represented directly by a describable multidimensional vector matrix.

[0050] Genome: A quantified network is a genome represented by a multidimensional matrix of gene-encoded representations.

[0051] Crossover: exchanging several layers of coding values ​​between two genomes.

[0052] Variation: Random changes in the coding values ​​of a certain layer of the genome.

[0053] Cataclysmic events: adding new genomes to a population or randomly removing existing genomes.

[0054] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0055] Figure 2 This is a flowchart illustrating a neural network quantization method disclosed in an embodiment of this application. See also... Figure 2 As shown, quantization methods for neural networks can include:

[0056] Step 201: Gene encoding processing is performed on multiple initial quantization models generated by random quantization based on the first neural network model to obtain an initial population including multiple genomes, wherein the number of genomes is the same as the number of initial quantization models.

[0057] In this embodiment, for the first neural network that needs to be quantized, the p generated by its random quantization will be first used. qThe initial quantization model can be generated by: randomly representing floating-point objects in each layer of the first neural network as fixed-point objects, and randomly configuring the bit width of the fractional part of the output vector of each layer to obtain multiple initial quantization models.

[0058] The initial quantization model is processed by gene encoding, that is, the initial quantization model is represented by a describable multidimensional vector matrix to facilitate subsequent related data processing.

[0059] The specific implementation of generating the initial quantization model and gene encoding will be described in detail in the following examples, and will not be elaborated on here.

[0060] Step 202: Perform diversity processing on the genomes in the initial population to obtain a processed population, wherein the number of genomes in the processed population is greater than the number of genomes in the initial population.

[0061] The diversity processing may include, but is not limited to, genomic crossover and genomic variation. By performing genomic crossover and genomic variation processing based on the genome in the initial population, more genomes can be obtained, thus ensuring genomic diversity.

[0062] Step 203: Determine whether there are genomes in the treated population whose fitness values ​​meet the termination condition, and proceed to step 204 or step 205.

[0063] In this embodiment, the termination condition may be that the fitness value is less than a set fitness value threshold. This fitness value threshold may be determined by experience or obtained through calibration. This application does not impose any restrictions on this.

[0064] Step 204: If none exist, select multiple genomes from the treated population based on the fitness values ​​of each genome in the treated population to construct a new initial population, and return to step 202.

[0065] If no genome has a fitness value that meets the termination condition, it means that the reasoning effect of the genome in the current processed population is not ideal and has not reached sufficient accuracy. Therefore, it is necessary to construct a new initial population and perform diversity processing again based on the new initial population to obtain a new genome and its corresponding fitness value.

[0066] The process of constructing a new initial population and repeating diversity treatments can be called neural evolution. Each neural evolution results in a corresponding generation of treated populations, and the genomes included in different generations of treated populations are different.

[0067] Step 205: If it exists, determine the quantification data of the first neural network model based on the genome that meets the termination condition.

[0068] If a genome exists whose fitness value satisfies the termination condition, it indicates that the inference performance of the genome in the currently processed population has reached a certain level and possesses sufficient accuracy. Therefore, the quantized data of the first neural network model can be determined based on the genome that satisfies the termination condition. The quantization method is to set a reasonable fixed-point form for each layer of the network: intN.M. Assume that N bits represent floating-point numbers in binary. M is the number of bits occupied by the fractional part, and the integer part is (NM-1). The larger M is, the more accurate the fixed-point representation and the better the inference performance.

[0069] The quantization method of the neural network described in this embodiment adopts a neuroevolutionary strategy to automatically search for the best combination of multiple quantization values ​​to construct an efficient and accurate mixed-precision quantization model. Compared with the quantization singularity of traditional neural network embedded deployment, it expands the quantization mapping method and has good versatility.

[0070] In the above embodiments, before determining whether there are genomes in the treated population whose fitness values ​​meet the termination condition, the method may further include: classifying the genomes in the treated population to obtain multiple different species.

[0071] Specifically, classifying the genomes in the processed population to obtain multiple different species may include: performing similarity clustering on the genomes in the processed population to obtain multiple different species. Of course, there are other ways to classify the genomes, such as classifying the genomes based on the model size corresponding to the genome; this application does not limit this approach.

[0072] After classifying the genomes of the treated population, different species are obtained. Subsequent species-centered performance evaluations can be designed to accelerate convergence. This is an evolutionary approach to speciation, where genome performance evaluation and species selection work together to maintain population diversity while speeding up the search process.

[0073] In the above embodiments, the gene encoding process of multiple initial quantization models generated by random quantization based on the first neural network model may include: performing fusion encoding on the target network layer structure in the initial quantization model, wherein the target network layer includes at least two network layers, and the fusion encoding is to fuse at least two network layers in the first neural network into a single row of encoding.

[0074] In one implementation, to improve the search speed of the neural network, this method performs fusion encoding on specific networks (corresponding to the target network layer structure), as follows: Figure 3 As shown.

[0075] Figure 3In this paper, the Conv and ReLU layers in the neural network are fused and encoded. Specifically, to accelerate the search speed, the Conv+ReLU layers are fused into a single row of encoding, while Maxpool, without weights or biases, is encoded as a separate row. In this application, all neural network layers can be encoded in the above manner, forming a two-dimensional matrix. Figure 3 In the matrix shown, w and b are the number of weight bits and bias bits in the fixed-point representation (which could be any of 8, 16, or 32); (y1, bit1) represents the number of fixed-point bits in the Conv layer output vector and the configuration decimal width. (y2, bit2) is the output representation of the ReLU layer; since Maxpool only has one layer, it is 0.

[0076] Since fusion coding effectively compresses the amount of genomic data, the amount of data to search for subsequent related data will also be reduced, thus speeding up the search process.

[0077] In the above embodiments, performing diversity processing on the genomes of the initial population to obtain a processed population may include: performing random crossover and mutation processing on the genomes of the initial population to obtain a processed population.

[0078] Furthermore, other implementations may include: catastrophic mutation of the latest treated population if, after the first number of population evolutions, no genome meets the termination criteria; or, if, after the first number of population evolutions, no genome meets the termination criteria and no better (lower fitness) genome emerges, catastrophic mutation of the latest treated population is performed. For example, if, after three population evolutions, no genome meets the termination criteria, catastrophic mutation can be controlled, i.e., adding a new genome to the population to ensure genome richness, so that a genome meeting the termination criteria can be found subsequently.

[0079] In this implementation, the introduction of various statistical measures and catastrophes prevents evolutionary search from prematurely maturing, thereby increasing the diversity of species selection while simultaneously promoting innovative biological evolution.

[0080] The termination condition is related to the fitness value of the genome. Determining the fitness value of the genome may include: determining the difference data between the final output of the genome in the smart chip inference and the floating-point output of the first neural network; determining the fitness value of the genome based on the difference data and the first parameter of the chip inference, wherein the first parameter represents the number of frames of images processed per second.

[0081] Of course, the above is only an illustrative implementation of determining the fitness value. In different application scenarios, the fitness value can be determined in different ways. For example, in some scenarios, it can be determined based solely on the difference between the final output of the genome in the smart chip inference and the floating-point output of the first neural network.

[0082] In the above embodiments, the step of selecting multiple genomes from the treated population based on the fitness values ​​of each genome in the treated population to construct a new initial population may include: determining the average fitness value of a second number of genomes selected within each species as the standard fitness value of the corresponding species; and selecting a portion of genomes from a third number of species with the highest standard fitness values ​​to construct a new initial population.

[0083] This implementation constructs a joint fitness function to coordinate the differences in inference time and inference accuracy of intelligent chips, taking into account the characteristics of both metrics. They influence each other during evolution, achieving the best balance between accuracy and inference speed in the mixed-precision quantization model. This is beneficial for improving the generalization and performance of the mixed-precision quantization network and facilitates rapid porting of embedded models.

[0084] The specific implementation of constructing a new initial population will be described in detail in the following section on the specific implementation, and will not be elaborated on here.

[0085] To better understand the content of this application, a specific implementation will be given below. Figure 4 This is a schematic diagram illustrating a specific implementation process of a neural network quantization method disclosed in an embodiment of this application. It can be combined with... Figure 4 Understand the content described below.

[0086] This implementation automatically generates a mixed-precision quantization model with optimal performance in intelligent chip inference based on a neuroevolutionary algorithm. Taking the YOLO (You Only Look Once, a convolutional neural network algorithm for object detection) model as an example, it includes the following:

[0087] 1. Prepare a calibration dataset n and a floating-point YOLO model. Generate p using random quantization. q An initial quantization model.

[0088] This random quantization represents the floating-point weights, biases, and output vectors of each layer of YOLO in a random fixed-point format (8 bits, 16 bits, 32 bits); the fractional part of the output vector of each layer is randomly configured in bits (according to the candidate set mean, median, mode, quantile).

[0089] Decimal bit width calculation:

[0090] The floating-point model is parsed, and floating-point inference is performed on n data points to obtain n output vector sets {Valuej} for each layer. Valuej represents the feature vector of the output of the j-th layer. Assuming that the output of a certain layer is represented by an 8-bit binary fixed-point floating-point number, the formula for calculating the fractional bit width required for this layer is as follows (1):

[0091] bit = 8 - log2(max(|Value) j |))-1 (1)

[0092] The decimal width of n data points in the same network layer is calculated to obtain a list listn = {list1, list2, ..., listn}. Four statistics of listn are obtained: mean, median, mode, and quantile, which can be used as candidate sets for the decimal width of the output of this layer.

[0093] 2. Gene encoding of the initial population: Direct encoding is used on the genomes in the population to obtain an initial population including multiple genomes.

[0094] 3. Perform random crossover and mutation on the genome of the population. Evolution e g If, after a generation, the population fails to improve its optimal fitness, a catastrophe occurs, and the number of genomes in the population (corresponding to the treated population) changes accordingly. For details on random crossover and mutation processes, please refer to [link to relevant documentation]. Figure 5 As shown.

[0095] 4. This implementation uses species-centric performance evaluation and selection collaboration. It maintains population diversity by clustering individuals into different species based on genomic similarity within the population. Considering that the goal of clustering is to minimize intra-cluster compactness and maximize inter-cluster dispersion, this method fuses intra-cluster and inter-cluster distances by maximizing the distance between cluster centers and genomic objects in other clusters. The formula is as follows:

[0096]

[0097] Where, x i It is the i-th genome, m k It is the cluster center of the k-th class, u ik =1 indicates that the i-th genome is assigned to the k-th class (Σu ik =1,u ik ∈{0,1}), η is a parameter that balances intra-class distance and inter-class distance, which changes with the clustering process, and ε is a preset parameter.

[0098] 5. After clustering into different species, calculate the fitness value of each genome in each species. The difference between the final output of the genome in the smart chip inference and the original YOLO floating-point output, combined with the FPS of the chip inference, is used as the fitness value f for that gene. qAssume there are n pieces of data, and the network finally outputs t values. The formula is as shown in (3):

[0099]

[0100] where f e is the floating-point and fixed-point inference output error; f s is the inference speed of the intelligent chip; and respectively represent the k-th output value of the i-th sample after fixed-point and floating-point inferences; FPS is the number of frames of pictures processed per second; the weight parameters λe and λs play an important role in determining the joint adaptation index, and the relationship between the two is f e λ s = f s λ e .

[0101] 6. According to the fitness value, determine whether there is a genome that meets the termination condition in each species. The "termination condition is: f q < Pα, where Pα is the threshold". If there is, terminate the search; otherwise, go to step 7.

[0102] 7. In the species, the average fitness value of several selected genomes is used as the fitness value of the species through the roulette selection method The species are sorted in descending order based on the fitness value, and p u species are selected. Among the p u species, the share of each group is calculated according to formula (4) Randomly select the corresponding genomes as the new population. In addition, use the pre-set λ u to reduce the grouping quantity threshold p k generated every μ u evolution to accelerate the convergence speed.

[0103]

[0104] Repeat steps 3 to 7 until the evolutionary search ends, and the optimal mixed-precision quantization model is obtained through quantization configuration.

[0105] The quantization method based on neural evolution described in this embodiment is designed in a distributed manner, enabling users to conveniently and simply obtain an efficient mixed quantization configuration model. It has the following advantages:

[0106] This method focuses on applying the advantages of the neural evolution algorithm, decomposes the traditional quantization method, automatically searches for the best combination, builds a quantization model suitable for embedded deployment, and completes the fixed-point quantization of the floating-point type.

[0107] Diversity is a catalyst for evolutionary innovation. This method ingeniously uses multiple statistics to map the fractional part of floating-point numbers, increasing the selectivity of mutations, incorporating the characteristics of the output data of each layer, and preserving specific information. It introduces catastrophe operations to avoid premature convergence of the evolutionary search, i.e., getting stuck in a saddle point.

[0108] Some potentially advantageous new genomes may have insufficient initial fitness, negatively impacting the initial iterations and potentially leading to their elimination by evolution. To mitigate this risk, our method introduces a grouping scheme (species) to protect population diversity. Species and population selection work collaboratively to simplify the search process.

[0109] The construction of the joint fitness function dynamically plans the hyperparameters during the evolutionary process, reduces the differences in indicators of different orders of magnitude, and provides more effective supervision signals for the model, thus balancing the inference accuracy and time of the mixed-precision model.

[0110] For the foregoing method embodiments, in order to simplify the description, they are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, because according to this application, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0111] The methods described in the above-disclosed embodiments of this application are detailed in terms of the methods. The methods of this application can be implemented by various forms of apparatus. Therefore, this application also discloses an apparatus. Specific embodiments are given below for detailed description.

[0112] Figure 6 This is a schematic diagram of the structure of a neural network quantization device disclosed in an embodiment of this application. See also... Figure 6 As shown, the quantization device 60 of the neural network may include:

[0113] The encoding processing module 601 is used to perform gene encoding processing on multiple initial quantization models generated by random quantization based on the first neural network model to obtain an initial population including multiple genomes, wherein the number of genomes is the same as the number of initial quantization models.

[0114] The diversity processing module 602 is used to perform diversity processing on the genomes in the initial population to obtain a processed population, wherein the number of genomes in the processed population is greater than the number of genomes in the initial population.

[0115] Termination determination module 603 is used to determine whether there are genomes in the processed population whose fitness values ​​meet the termination conditions.

[0116] The population update module 604 is used to select multiple genomes from the processed population to construct a new initial population when the determination result of the termination determination module is that no genome exists.

[0117] The quantization determination module 605 is used to determine the quantization data of the first neural network model based on the genome that meets the termination condition when the determination result of the termination determination module is that it exists.

[0118] The quantization device of the neural network described in this embodiment uses a neuroevolutionary strategy to automatically search for the best combination of multiple quantization values ​​to construct an efficient and accurate mixed-precision quantization model. Compared with the quantization singularity of traditional embedded neural networks, it expands the quantization mapping method and has good versatility.

[0119] The specific implementation of the quantization device of the aforementioned neural network and its various modules, as well as other possible implementations, can be found in the relevant sections of the method embodiments, and will not be repeated here.

[0120] The quantization device of any neural network described in the above embodiments includes a processor and a memory. The encoding processing module, diversity processing module, termination determination module, population update module, quantization determination module, etc. in the above embodiments are all stored as program modules in the memory, and the processor executes the above program modules stored in the memory to realize the corresponding functions.

[0121] The processor contains a kernel, which retrieves the corresponding program modules from memory. One or more kernels can be configured, and the processing of backtracking data can be achieved by adjusting kernel parameters.

[0122] The memory may include non-permanent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip.

[0123] In an exemplary embodiment, a computer-readable storage medium is also provided, which can be directly loaded into the internal memory of a computer, and contains software code that, after being loaded and executed by the computer, can implement the steps shown in any embodiment of the quantization method for the neural network described above.

[0124] In an exemplary embodiment, a computer program product is also provided, which can be directly loaded into the internal memory of a computer and contains software code. After being loaded and executed by the computer, the computer program can implement the steps shown in any embodiment of the neural network quantization method described above.

[0125] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to the method section.

[0126] It should also be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0127] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0128] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for quantizing a neural network, the method comprising: include: Gene encoding processing is performed on multiple initial quantization models randomly generated based on a first neural network model to obtain an initial population comprising multiple genomes, wherein the number of genomes is the same as the number of initial quantization models; wherein, the gene encoding processing on multiple initial quantization models randomly generated based on a first neural network model includes: performing fusion encoding on the target network layer structure in the initial quantization model, wherein the target network layer comprises at least two network layers, and the fusion encoding is that at least two network layers in the first neural network are fused and represented as a row of encoding; Diversity processing is performed on the genomes in the initial population to obtain a processed population, wherein the number of genomes in the processed population is greater than the number of genomes in the initial population; The genomes of the treated population were classified to obtain multiple different species; Determine whether there exists a genome in the processed population whose fitness value satisfies the termination condition; wherein, determining the fitness value of the genome includes: determining the difference data between the final output of the genome in the smart chip inference and the floating-point output of the first neural network; determining the fitness value of the genome based on the difference data and the first parameter of the chip inference, wherein the first parameter represents the number of frames of images processed per second; If none exist, select multiple genomes from the treated population based on the fitness values ​​of each genome in the treated population to construct a new initial population, and return to the above-described diversity treatment steps; the step of selecting multiple genomes from the treated population based on the fitness values ​​of each genome in the treated population to construct a new initial population includes: determining the average fitness value of a second number of genomes selected within each species as the standard fitness value of the corresponding species; and selecting a portion of genomes from a third number of species with the highest standard fitness values ​​to construct a new initial population; If present, the quantification data for the first neural network model is determined based on the genome that meets the termination condition.

2. The method of quantizing a neural network of claim 1, wherein, The genomes of the treated population are classified to obtain multiple different species, including: The genomes of the treated population were clustered based on similarity to obtain multiple different species.

3. The method of quantizing a neural network of claim 1, wherein, Multiple initial quantization models generated based on the random quantization of the first neural network model include: The floating-point objects in each layer of the first neural network are represented by random fixed-point representations, and the bit width of the fractional part of the output vector of each layer is randomly configured to obtain multiple initial quantization models.

4. The method of quantizing a neural network of claim 1, wherein, The process of performing diversity processing on the genomes of the initial population to obtain a processed population includes: Random crossover and mutation processes are performed on the genomes of the initial population to obtain the processed population.

5. The method of quantizing a neural network according to claim 4, wherein, Each instance of diversity treatment evolves the population once. The methods also include: If, after the first number of population evolutions, no genome still meets the termination criteria, a catastrophic process is performed on the latest treated population.

6. A quantization apparatus of a neural network, characterized by, include: The encoding processing module is used to perform gene encoding processing on multiple initial quantization models generated by random quantization based on the first neural network model to obtain an initial population including multiple genomes, wherein the number of genomes is the same as the number of initial quantization models; the encoding processing module is specifically used to: perform fusion encoding on the target network layer structure in the initial quantization model, wherein the target network layer includes at least two network layers, and the fusion encoding is to fuse at least two network layers in the first neural network into a single row of encoding. A diversity processing module is used to perform diversity processing on the genomes in the initial population to obtain a processed population, wherein the number of genomes in the processed population is greater than the number of genomes in the initial population; The termination determination module is used to determine whether there is a genome in the processed population whose fitness value meets the termination condition; wherein, the determination of the genome's fitness value includes: determining the difference data between the genome's last output in the smart chip inference and the floating-point output of the first neural network; and determining the genome's fitness value based on the difference data and the first parameter of the chip inference, wherein the first parameter represents the number of frames of images processed per second; The population update module is used to select multiple genomes from the processed population to construct a new initial population based on the fitness values ​​of each genome in the processed population when the determination result of the termination determination module is that no such population exists. Specifically, the population update module is used to: determine the average fitness value of a second number of genomes selected within each species as the standard fitness value of the corresponding species; and select a portion of genomes from a third number of species with the highest standard fitness values ​​to construct a new initial population. A quantization determination module is used to determine the quantization data of the first neural network model based on the genome that satisfies the termination condition when the determination result of the termination determination module is that it exists. The quantization device of the neural network is also used to classify the genomes in the processed population to obtain multiple different species.

Images