Neural network compression method and device thereof, electronic device, and storage medium

Abstract

Embodiments of the present application provide a neural network compression method and device, electronic equipment and storage medium, and belong to the technical field of artificial intelligence. The method comprises: performing quantization processing on a target neural network to obtain a plurality of first compressed neural networks; training the plurality of first compressed neural networks to obtain a candidate network sequence; performing iterative compression processing on the candidate network sequence until an iteration end condition is met; and selecting at least one optimized compressed neural network from the candidate network sequence as a target compressed neural network. The embodiments of the present application can obtain a required neural network compression model, which is beneficial to improving the compression accuracy of the neural network.

Description

Technical Field

[0001] This application relates to, but is not limited to, the field of artificial intelligence technology, and particularly to a neural network compression method and apparatus, electronic device, and computer-readable storage medium. Background Technology

[0002] With the continuous development of deep learning technology, the scale of deep learning neural network models is also increasing, reflected in the increasing number of layers and the exponential growth of parameters. Consequently, the memory and energy consumption of neural network models are also increasing. These large-scale models are difficult to apply to memory-constrained devices such as MCUs. Therefore, it is necessary to compress the size of neural network models. Currently, common techniques include pruning, quantization, and knowledge distillation. While these techniques can compress neural network models, they cannot guarantee that the resulting neural network meets the required scale. Therefore, improving the compression accuracy of neural networks has become an urgent technical problem to be solved. Summary of the Invention

[0003] The following is an overview of the subject matter described in detail herein. This overview is not intended to limit the scope of the claims.

[0004] The main objective of this application is to provide a neural network compression method, apparatus, electronic device, and computer-readable storage medium, which aims to improve the compression accuracy of neural networks.

[0005] To achieve the above objectives, a first aspect of this application proposes a neural network compression method, the method comprising:

[0006] The target neural network is quantized to obtain multiple first compressed neural networks;

[0007] The plurality of first compressed neural networks are trained to obtain a candidate network sequence, wherein the candidate network sequence includes a plurality of candidate compressed neural networks respectively corresponding to the plurality of first compressed neural networks;

[0008] The candidate network sequences are compressed and iterated until the iteration termination condition is met;

[0009] From the candidate network sequence, at least one optimized compressed neural network is selected as the target compressed neural network;

[0010] The compression iteration process includes:

[0011] Obtain the neural network architecture search NAS index for each of the candidate compressed neural networks;

[0012] The candidate compressed neural network with the smallest NAS index in the candidate network sequence is quantized to obtain multiple second compressed neural networks;

[0013] Multiple optimized compression neural networks are obtained by training the multiple second compression neural networks;

[0014] The plurality of optimized compressed neural networks are added to the end of the candidate network sequence, and the candidate compressed neural network at the beginning of the candidate network sequence is deleted;

[0015] The iteration termination condition is that the number of compression iterations reaches the preset number of iterations.

[0016] The neural network compression method provided according to the embodiments of this application has at least the following beneficial effects:

[0017] By quantizing and training the target neural network to be compressed, candidate network sequences are obtained to determine the range of possible target neural networks. Then, by performing compression iteration on the candidate network sequences, optimization is achieved, so that the candidate network sequences can gradually meet the compression requirements of the target neural network. In particular, compression iteration is performed based on the magnitude of the NAS index of each candidate compressed neural network, so that the candidate network sequences can be continuously updated and evolved towards a smaller NAS index, thereby meeting the compression requirements of the target neural network. This allows for the selection of the appropriate optimized compressed neural network as the target compressed neural network, ensuring that a neural network compression model that meets the requirements is obtained, which is beneficial to improving the compression accuracy of the neural network.

[0018] In some embodiments, selecting at least one optimized compressed neural network as the target compressed neural network from the candidate network sequence includes:

[0019] From the candidate network sequence, select multiple optimized compressed neural networks whose NAS index is not greater than the preset NAS optimization index as the target compressed neural network.

[0020] By further screening the candidate network sequences, multiple optimized compression neural networks with NAS indices no greater than the preset NAS optimization index can be selected as target compression neural networks. In other words, the NAS indices of the selected multiple optimized compression neural networks meet the requirement of being smaller than the preset NAS optimization index. Therefore, these optimized compression neural networks can be used as target compression neural networks to ensure that a target compression neural network that meets the requirements and has high compression accuracy can be obtained.

[0021] In some embodiments, selecting multiple optimized compressed neural networks from the candidate network sequence whose NAS metric is not greater than a preset NAS optimization metric as the target compressed neural network includes:

[0022] From a plurality of optimized compression neural networks whose NAS index is not greater than the preset NAS optimization index, the optimized compression neural network with the smallest NAS is selected as the target compression neural network.

[0023] Since the NAS index can effectively characterize the compression performance of a neural network, in order to further improve the accuracy of the target compressed neural network, the optimized compressed neural network with the smallest NAS index can be selected from multiple optimized compressed neural networks that meet the requirements and whose NAS index is no greater than the preset NAS optimization index, thus enabling the selection of the target compressed neural network with the highest compression accuracy.

[0024] In some embodiments, obtaining the neural network architecture search NAS metric for each of the candidate compressed neural networks includes:

[0025] For each of the candidate compressed neural networks, the candidate compressed neural network is input into the NAS index model;

[0026] Obtain the peak memory parameters, capacity parameters, latency parameters, and accuracy parameters of the NAS indicator model;

[0027] Based on the NAS metric model, the NAS metric of the candidate compressed neural network is calculated according to the peak memory parameter, the capacity parameter, the latency parameter, and the accuracy parameter.

[0028] Each candidate compressed neural network is further processed using a pre-configured NAS index model. Specifically, the peak memory parameter, capacity parameter, latency parameter, and accuracy parameter of the NAS index model are obtained through the NAS index model. The peak memory parameter can be the maximum value of the sum of the memory usage of the input tensor and the output tensor of each layer, which is a factor affecting SRAM memory. The capacity parameter reflects the size of the NAS index model, the latency parameter reflects the latency impact of the candidate compressed neural network, and the accuracy parameter reflects the calculation accuracy of the NAS index model. This allows for the accurate and reliable calculation of the NAS index of the candidate compressed neural network based on the peak memory parameter, capacity parameter, latency parameter, and accuracy parameter.

[0029] In some embodiments, training the plurality of first compressed neural networks to obtain candidate network sequences includes:

[0030] The plurality of first compressed neural networks are trained to obtain the plurality of candidate compressed neural networks;

[0031] The candidate compressed neural networks are sorted in descending order of the NAS index to obtain a candidate network sequence.

[0032] Sort multiple candidate compressed neural networks in descending order of their NAS (Neural Network Scale) index. This means that the NAS index of the candidate compressed neural networks in the candidate network sequence changes from the front end to the back end. Since the desired candidate compressed neural network is one with a smaller NAS index, when performing compression iteration on such a candidate network sequence, multiple optimized compressed neural networks are added to the end of the candidate network sequence, and the candidate compressed neural network at the beginning of the candidate network sequence is deleted. This allows us to filter out candidate compressed neural networks with larger NAS indices and continuously optimize towards candidate compressed neural networks with smaller NAS indices. This meets the objective requirements of neural network compression and is conducive to obtaining a more accurate and optimized candidate network sequence.

[0033] In some embodiments, the candidate compressed neural network with the smallest NAS index in the candidate network sequence is quantized to obtain multiple second compressed neural networks, including:

[0034] By randomly changing the second quantization parameter of the candidate compressed neural network with the smallest NAS index multiple times, multiple second compressed neural networks corresponding to the second quantization parameter are obtained.

[0035] By randomly changing the second quantization parameter of the candidate compressed neural network with the smallest NAS index multiple times, multiple second compressed neural networks corresponding to the second quantization parameter can be obtained. In other words, each time the second quantization parameter is changed, it is equivalent to compressing the candidate compressed neural network to a corresponding degree. Therefore, by randomly changing the second quantization parameter, a certain number of candidate second compressed neural networks can be obtained, so that further processing can be carried out based on the obtained second compressed neural networks in subsequent steps.

[0036] In some embodiments, the target neural network is quantized to obtain multiple first compressed neural networks, including:

[0037] By randomly changing the first quantization parameter of the target neural network multiple times, multiple first compressed neural networks corresponding to the first quantization parameter are obtained.

[0038] By randomly changing the first quantization parameter of the target neural network multiple times, multiple first compressed neural networks corresponding to the first quantization parameter can be obtained. In other words, each time the first quantization parameter is changed, it is equivalent to compressing the target neural network to a corresponding degree. Therefore, by randomly changing the first quantization parameter, a certain number of candidate first compressed neural networks can be obtained, so that further processing can be carried out based on the obtained first compressed neural networks in subsequent steps.

[0039] To achieve the above objectives, a second aspect of this application provides a neural network compression apparatus, the apparatus comprising:

[0040] The first processing module is used to quantize the target neural network to obtain multiple first compressed neural networks;

[0041] The second processing module is used to train the plurality of first compressed neural networks to obtain a candidate network sequence, wherein the candidate network sequence includes a plurality of candidate compressed neural networks respectively corresponding to the plurality of first compressed neural networks;

[0042] The third processing module is used to perform compression iterative processing on the candidate network sequences until the iteration termination condition is met, wherein the compression iterative processing includes:

[0043] Obtain the neural network architecture search NAS index for each of the candidate compressed neural networks;

[0044] The candidate compressed neural network with the smallest NAS index in the candidate network sequence is quantized to obtain multiple second compressed neural networks;

[0045] Multiple optimized compressed neural networks are obtained by training the multiple second compressed neural networks;

[0046] The plurality of optimized compressed neural networks are added to the end of the candidate network sequence and removed from the candidate compressed neural network at the beginning of the candidate network sequence;

[0047] The iteration termination condition is that the number of compressed iterations reaches a preset number of iterations.

[0048] The fourth processing module is used to select at least one of the optimized compressed neural networks as the target compressed neural network from the candidate network sequence.

[0049] To achieve the above objectives, a fourth aspect of the present application provides a storage medium, which is a computer-readable storage medium for computer-readable storage, wherein the storage medium stores one or more programs that can be executed by one or more processors to implement the method described in the first aspect.

[0050] The neural network compression method, apparatus, electronic device, and storage medium proposed in this application obtain candidate network sequences by quantizing and training the target neural network to be compressed, thereby determining the selectable range of the target neural network. Then, by performing compression iterative processing on the candidate network sequences, optimization is achieved, allowing the candidate network sequences to gradually meet the compression requirements of the target neural network. In particular, compression iterative processing is performed based on the magnitude of the NAS index of each candidate compressed neural network, enabling the candidate network sequences to continuously update and evolve towards a smaller NAS index, thus meeting the compression requirements of the target neural network. This allows for the selection of the appropriate optimized compressed neural network as the target compressed neural network, ensuring a compliant neural network compression model and improving the compression accuracy of the neural network.

[0051] Other features and advantages of this application will be set forth in the description which follows, and will be apparent in part from the description, or may be learned by practicing the application. The objectives and other advantages of this application may be realized and obtained by means of the structures particularly pointed out in the description, claims and drawings. Attached Figure Description

[0052] Figure 1 This is a flowchart of the neural network compression method provided in the embodiments of this application;

[0053] Figure 2 yes Figure 1 A flowchart of step S101 provided in one embodiment;

[0054] Figure 3 yes Figure 1 A flowchart of step S102 provided in one embodiment;

[0055] Figure 4 yes Figure 1 A flowchart of step S103 provided in one embodiment;

[0056] Figure 5 yes Figure 1 A flowchart of step S104 provided in one embodiment;

[0057] Figure 6 yes Figure 5A flowchart of step S501 provided in one embodiment;

[0058] Figure 7 This is a schematic diagram of the execution flow of the neural network compression method provided in the embodiments of this application;

[0059] Figure 8 This is a schematic diagram of the neural network compression device provided in the embodiments of this application;

[0060] Figure 9 This is a schematic diagram of the hardware structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0061] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0062] It should be noted that although functional modules are divided in the device schematic diagram and a logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart. The terms "first," "second," etc., in the specification, claims, and the aforementioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence.

[0063] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0064] First, let's analyze some of the terms used in this application:

[0065] Artificial intelligence (AI) is a new branch of computer science that studies, develops, and applies theories, methods, technologies, and systems to simulate, extend, and expand human intelligence. It aims to understand the essence of intelligence and produce intelligent machines that can react in a way similar to human intelligence. Research in this field includes robotics, speech recognition, image recognition, natural language processing, and expert systems. AI can simulate the information processes of human consciousness and thought. Furthermore, AI utilizes digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceiving the environment, acquiring knowledge, and using that knowledge to achieve optimal results.

[0066] Neural Architecture Search (NAS) is a widely used technique for optimizing neural network architectures. It uses machine learning algorithms to search for hyperparameters that maximize the accuracy of the neural network, aiming to discover architectures that meet specific needs. Essentially, NAS automates the process of manually tuning neural networks to discover more complex architectures. This field encompasses a set of tools and methods that use search strategies to test and evaluate a large number of architectures in the search space, selecting the architecture that best satisfies the given problem objective by maximizing the fitness function.

[0067] Based on this, embodiments of this application provide a neural network compression method and apparatus, electronic device, and computer-readable storage medium, aiming to improve the compression accuracy of neural networks.

[0068] The neural network compression method, apparatus, electronic device, and computer-readable storage medium provided in this application are specifically described through the following embodiments. First, the neural network compression method in this application is described.

[0069] The neural network compression method provided in this application relates to the field of artificial intelligence technology. This method can be applied to a terminal, a server, or software running on either a terminal or a server. In some embodiments, the terminal can be a smartphone, tablet, laptop, desktop computer, etc.; the server can be configured as an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms; the software can be an application implementing the neural network compression method, but is not limited to the above forms.

[0070] This application can be used in a wide variety of general-purpose or special-purpose computer system environments or configurations. Examples include: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices. This application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform specific tasks or implement specific abstract data types. This application can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0071] Figure 1 This is an optional flowchart of the neural network compression method provided in the embodiments of this application. Figure 1 The method may include, but is not limited to, steps S101 to S104:

[0072] Step S101: Quantize the target neural network to obtain multiple first compressed neural networks;

[0073] Step S102: Train multiple first compressed neural networks to obtain candidate network sequences;

[0074] Step S103: Perform compression and iterative processing on the candidate network sequences until the iteration termination condition is met;

[0075] Step S104: Select at least one optimized compressed neural network as the target compressed neural network from the candidate network sequences.

[0076] The candidate network sequence includes multiple candidate compressed neural networks, each corresponding to a multiple first compressed neural network;

[0077] The compression iteration process in step S103 includes:

[0078] Obtain the NAS index for neural network architecture search for each candidate compressed neural network;

[0079] The candidate compressed neural network with the smallest NAS index in the candidate network sequence is quantized to obtain multiple second compressed neural networks.

[0080] Multiple optimized compressed neural networks are obtained by training multiple second compressed neural networks;

[0081] Multiple optimized compressed neural networks are added to the end of the candidate network sequence, and the candidate compressed neural network at the beginning of the candidate network sequence is removed;

[0082] The iteration ends when the number of compressed iterations reaches the preset number of iterations.

[0083] Steps S101 to S104, as illustrated in this embodiment, involve quantizing and training the target neural network to be compressed to obtain candidate network sequences. This determines the selectable range of the target neural network for compression. Then, iterative compression processing is performed on the candidate network sequences to optimize them, gradually ensuring they meet the compression requirements of the target neural network. Specifically, iterative compression processing is based on the NAS (Neural Network Assignment) index of each candidate compressed neural network, allowing the candidate network sequences to continuously evolve towards smaller NAS indices, thus meeting the compression requirements of the target neural network. This facilitates the selection of the appropriate optimized compressed neural network as the target compressed neural network, ensuring a compliant neural network compression model and improving the compression accuracy of the neural network.

[0084] It is important to emphasize that, compared to related technologies where neural network quantization converts floating-point numbers in the model into 8-bit integers, sometimes 8-bit integers cannot maximize the reduction of the neural network size, and the neural network is still relatively large. Reducing it entirely to 4 bits or lower may lead to a serious loss of precision. The embodiments of this application do not adhere to quantizing the network to 8-bit integers, but consider quantizing it to a mixed precision of 2 bits, 4 bits, or 8 bits, and use NAS to accurately determine the specific compressed neural network that meets the requirements. Through actual experiments, it can be determined that the embodiments of this application can further compress the network on the basis of existing network compression schemes, and obtain smaller SRAM and Flash usage, as well as higher precision and lower latency.

[0085] In step S101 of some embodiments, quantization processing is a specialized term in the art for compressing neural networks, which is well known to those skilled in the art. To avoid redundancy, it will not be described in detail here. The following is a specific implementation of step S101.

[0086] Please see Figure 2 In some embodiments, step S101 may include, but is not limited to, step S201:

[0087] Step S201: Randomly change the first quantization parameter of the target neural network multiple times to obtain multiple first compressed neural networks corresponding to the first quantization parameter.

[0088] In this step, by randomly changing the first quantization parameter of the target neural network multiple times, multiple first compressed neural networks corresponding to the first quantization parameter can be obtained. That is, each time the first quantization parameter is changed, it is equivalent to compressing the target neural network to a corresponding degree. Therefore, by randomly changing the first quantization parameter, a certain number of candidate first compressed neural networks can be obtained, so that further processing can be carried out based on the obtained first compressed neural networks in subsequent steps.

[0089] In step S201 of some embodiments, the specific number of times the first quantization parameter of the target neural network is randomly changed is not limited, and can be set according to the application resources, latency requirements, etc. of the scenario.

[0090] In step S102 of some embodiments, there are various ways to train multiple first compressed neural networks to obtain candidate network sequences, which are not limited here. For example, corresponding convolutional neural networks can be selected to train multiple first compressed neural networks respectively, thereby obtaining multiple candidate compressed neural networks to form candidate network sequences.

[0091] Please see Figure 3 In some embodiments, step S102 may include, but is not limited to, steps S301 to S302:

[0092] Step S301: Train multiple first compressed neural networks to obtain multiple candidate compressed neural networks;

[0093] Step S302: Sort the multiple candidate compressed neural networks in descending order of NAS index to obtain a candidate network sequence.

[0094] In this step, multiple candidate compressed neural networks are sorted in descending order of their NAS (Neutral Scale) indices. This means that the NAS indices of the candidate compressed neural networks in the candidate network sequence change from large to small from the front end to the back end. Since the desired candidate compressed neural network is one with a smaller NAS index, when performing compression iteration on such a candidate network sequence, multiple optimized compressed neural networks are added to the end of the candidate network sequence, and the candidate compressed neural network at the beginning of the candidate network sequence is deleted. This allows us to filter out candidate compressed neural networks with larger NAS indices and continuously optimize towards candidate compressed neural networks with smaller NAS indices. This meets the objective requirements of neural network compression and is conducive to obtaining a more accurate and optimized candidate network sequence.

[0095] In step S103 of some embodiments, by performing compression iteration on the candidate network sequence, optimized compressed neural networks with smaller NAS indices are continuously added to the candidate network sequence to optimize the candidate network queue. Furthermore, by continuously deleting candidate compressed neural networks with relatively larger NAS indices from the candidate network sequence, further optimization of the candidate network queue is achieved. It can be understood that by continuously optimizing the network to make its NAS index value smaller, this is similar to simulating the evolutionary process of organisms. In biological evolution, the offspring of the best individuals tend to be excellent, while the worst individuals are eliminated. If each generation allows the best individuals to generate new offspring while eliminating the worst individuals, then the entire population is likely to develop in an increasingly better direction. This is similar to the embodiments of this application where the best neural network generates a new network and is added to the candidate network sequence, while the worst neural network is removed.

[0096] In step S103 of some embodiments, the method of compressing and iterating the candidate network sequence can be set according to the specific application scenario, and is not limited here. The preset number of iterations is preferably at least 10 times, so as to ensure that the network training results can basically meet the general requirements. Of course, the iteration termination condition can also be set according to factors such as iteration cost, latency impact and iteration difficulty, or can be set according to the specific application scenario, and is not limited here.

[0097] Please see Figure 4 In some embodiments, the step S103 of "obtaining the neural network architecture search NAS index of each candidate compressed neural network" may include, but is not limited to, steps S401 to S403:

[0098] Step S401: For each candidate compressed neural network, input the candidate compressed neural network into the NAS index model;

[0099] Step S402: Obtain the peak memory parameters, capacity parameters, latency parameters, and accuracy parameters of the NAS indicator model;

[0100] Step S403: Based on the NAS index model, calculate the NAS index of the candidate compressed neural network according to the peak memory parameter, capacity parameter, latency parameter and accuracy parameter.

[0101] In this step, each candidate compressed neural network is further processed using a pre-configured NAS metric model. Specifically, the peak memory parameter, capacity parameter, latency parameter, and accuracy parameter of the NAS metric model are obtained. The peak memory parameter can be the maximum sum of the memory usage of the input tensor and output tensor of each layer, serving as a factor affecting SRAM memory. The capacity parameter reflects the size of the NAS metric model, the latency parameter reflects the latency impact of the candidate compressed neural network, and the accuracy parameter reflects the calculation accuracy of the NAS metric model. This allows for the accurate and reliable calculation of the NAS metric of the candidate compressed neural network based on the peak memory parameter, capacity parameter, latency parameter, and accuracy parameter.

[0102] In steps S401 to S403 of some embodiments, the NAS indicator model can be set according to the specific scenario. Accordingly, if the set NAS indicator model is different, the peak memory parameter, capacity parameter, latency parameter and accuracy parameter of the NAS indicator model can also be different. This is not limited here. The following is a specific example to illustrate the relevant principles and process.

[0103] Example 1:

[0104] The data for the target neural network model is divided into two types: static data and dynamic data. Static data refers to the weight tensors of each layer after the model has been trained, while dynamic data refers to the input and output values ​​of each layer after the model has run and inferred. Static data is stored in the MCU's Flash memory, while dynamic data is stored in the MCU's SRAM memory; therefore, their capacity bottlenecks are different. In this example, the Flash memory is larger than the SRAM memory and can hold more data.

[0105] For SRAM memory, due to the serial operation characteristic of neural networks, at most one layer's input and output tensors can reside in memory at any given time. Once the computation of this layer is complete, the memory occupied by the input tensors is released, and new memory is allocated for the output tensors of the next layer. Therefore, the bottleneck of SRAM memory is the maximum sum of the memory usage of the input and output tensors of each layer, called the peak memory M.

[0106]

[0107] Where, x i y i Represents the input and output tensors of the i-th layer. These represent the quantization parameters corresponding to the model.

[0108] For Flash memory, since the weight tensors of each layer must exist off-chip so that they can be accessed when needed, the amount of off-chip data is equal to the sum of the sizes of the weight tensors of each layer, which is called the model size S.

[0109]

[0110] Where w i This represents the weight tensor of the i-th layer. This represents the quantization parameters corresponding to the model.

[0111] The M and S values ​​determined above are added to the NAS metric, along with the model latency metric. The aim is to ensure that the network obtained through model search does not have excessively high latency, thus affecting prediction performance. The model latency L can be considered positively correlated with the number of cumulative multiplication operations (MACs), which are the most time-consuming operations in neural network computation. Therefore, the latency L is:

[0112] L = kMACs;

[0113] In addition, considering the accuracy (Accc) of the traditional NAS metric model, the NAS metric is obtained as follows:

[0114] G(Q)=max(λ1(1-Acc(Q)),λ2M(Q),λ3S(Q),λ4L(Q));

[0115] in, This is the set of all quantization parameters. Each quantization parameter can be selected from 2, 4, and 8 bits, meaning it can be quantized to a 2-bit, 4-bit, or 8-bit integer.

[0116] It is understood that one objective of the embodiments of this application is to minimize this index G(Q).

[0117] In some embodiments, step S103, "quantizing the candidate compressed neural network with the smallest NAS index in the candidate network sequence to obtain multiple second compressed neural networks," may include, but is not limited to, step S404:

[0118] Step S404: Randomly change the second quantization parameter of the candidate compressed neural network with the smallest NAS index multiple times to obtain multiple second compressed neural networks corresponding to the second quantization parameter.

[0119] In this step, by randomly changing the second quantization parameter of the candidate compressed neural network with the smallest NAS index multiple times, multiple second compressed neural networks corresponding to the second quantization parameter can be obtained. That is, each time the second quantization parameter is changed, it is equivalent to compressing the candidate compressed neural network to a corresponding degree. Therefore, by randomly changing the second quantization parameter, a certain number of candidate second compressed neural networks can be obtained, so that further processing can be carried out based on the obtained second compressed neural networks in subsequent steps.

[0120] In step S404 of some embodiments, the specific number of times the second quantization parameter of the candidate compressed neural network with the smallest NAS index is randomly changed is not limited, and can be selected and set according to the application resources, latency requirements, etc.

[0121] In step S104 of some embodiments, the number of optimized compression neural networks selected as target compression neural networks is not limited, and there can be one or more. Specific implementation methods are described below.

[0122] Please see Figure 5 In some embodiments, step S104 may include, but is not limited to, step S501:

[0123] Step S501: Select multiple optimized compressed neural networks from the candidate network sequences whose NAS index is no greater than the preset NAS optimization index as the target compressed neural network.

[0124] In this step, by further screening the candidate network sequences, multiple optimized compression neural networks with NAS indices no greater than the preset NAS optimization index can be selected as target compression neural networks. In other words, the NAS indices of the selected multiple optimized compression neural networks meet the requirement of being smaller than the preset NAS optimization index. Therefore, these optimized compression neural networks can be used as target compression neural networks to ensure that a target compression neural network that meets the requirements and has high compression accuracy can be obtained.

[0125] In step S501 of some embodiments, the preset NAS optimization index can be set according to the specific application scenario, and is not limited here.

[0126] Please see Figure 6 In some embodiments, step S501 may include, but is not limited to, step S5011:

[0127] Step S5011: Select the optimized compression neural network with the smallest NAS from multiple optimized compression neural networks whose NAS index is not greater than the preset NAS optimization index as the target compression neural network.

[0128] In this step, since the NAS index can well characterize the compression of the neural network, in order to further improve the accuracy of the target compressed neural network, the optimized compressed neural network with the smallest NAS can be selected from multiple optimized compressed neural networks that meet the requirements and whose NAS index is no greater than the preset NAS optimization index, so as to select the target compressed neural network with the highest compression accuracy.

[0129] The following is a specific example to illustrate the working principle and process of the above embodiments.

[0130] Please see Figure 7 For a given neural network that needs to be compressed, perform the following steps:

[0131] Step 1: Randomly change the quantization parameters of the neural network to obtain a series of initial networks, train these initial networks to obtain a candidate network queue, and calculate the NAS index G of all initial networks in the candidate network queue.

[0132] Step 2: Determine if the NAS iteration count has been reached. If so, end the process directly and return to the network with the smallest G in the candidate network queue. Otherwise, continue to step 3.

[0133] Step 3: Select the network with the smallest G in the candidate network queue, randomly change its quantization parameters to obtain a new quantized network and train it. Then add the optimized network obtained from the training to the rightmost end of the candidate network queue.

[0134] Step 4: Remove the leftmost initialized network from the candidate network queue, and then return to step 2.

[0135] As can be seen, by performing the above steps, a neural network compression model that meets the requirements can be obtained, which is beneficial to improving the compression accuracy of neural networks.

[0136] Please see Figure 8 This application also provides a neural network compression apparatus that can implement the above-described neural network compression method. The apparatus includes:

[0137] The first processing module is used to quantize the target neural network to obtain multiple first compressed neural networks;

[0138] The second processing module is used to train multiple first compressed neural networks to obtain a candidate network sequence, wherein the candidate network sequence includes multiple candidate compressed neural networks that correspond to the multiple first compressed neural networks respectively.

[0139] The third processing module is used to perform compression iterative processing on the candidate network sequences until the iteration termination condition is met; wherein, the compression iterative processing includes:

[0140] Obtain the NAS index for neural network architecture search for each candidate compressed neural network;

[0141] The candidate compressed neural network with the smallest NAS index in the candidate network sequence is quantized to obtain multiple second compressed neural networks.

[0142] Multiple optimized compressed neural networks are obtained by training multiple second compressed neural networks;

[0143] Multiple optimized compressed neural networks are added to the end of the candidate network sequence, and the candidate compressed neural network at the beginning of the candidate network sequence is removed;

[0144] The iteration ends when the number of compressed iterations reaches the preset number of iterations.

[0145] The fourth processing module is used to select at least one optimized compressed neural network as the target compressed neural network from the candidate network sequences.

[0146] The specific implementation of this neural network compression device is basically the same as the specific implementation of the neural network compression method described above, and will not be repeated here.

[0147] This application also provides an electronic device, which includes: a memory, a processor, a program stored in the memory and executable on the processor, and a data bus for communication between the processor and the memory. When the program is executed by the processor, it implements the aforementioned neural network compression method. This electronic device can be any smart terminal, including tablet computers, in-vehicle computers, etc.

[0148] Please see Figure 9 , Figure 9 The hardware structure of an electronic device according to another embodiment is illustrated. The electronic device includes:

[0149] The processor 901 can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this application.

[0150] The memory 902 can be implemented as a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 902 can store the operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 902 and is called and executed by the processor 901 using the neural network compression method of the embodiments of this application.

[0151] The input / output interface 903 is used to implement information input and output;

[0152] The communication interface 904 is used to enable communication and interaction between this device and other devices. Communication can be achieved through wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0153] Bus 905 transmits information between various components of the device (e.g., processor 901, memory 902, input / output interface 903, and communication interface 904);

[0154] The processor 901, memory 902, input / output interface 903, and communication interface 904 are connected to each other within the device via bus 905.

[0155] This application embodiment also provides a storage medium, which is a computer-readable storage medium for computer-readable storage. The storage medium stores one or more programs, which can be executed by one or more processors to implement the above-described neural network compression method.

[0156] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory may optionally include memory remotely located relative to the processor, and these remote memories can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0157] The neural network compression method, neural network compression device, electronic device, and storage medium provided in this application obtain candidate network sequences by quantizing and training the target neural network to be compressed, so as to determine the range of possible target neural networks for compression. Then, by performing compression iteration processing on the candidate network sequences, the candidate network sequences are optimized so that they can gradually meet the compression requirements of the target neural network. In particular, the compression iteration processing is based on the size of the NAS index of each candidate compressed neural network, so that the candidate network sequences can be continuously updated and evolved towards a smaller NAS index, thereby meeting the compression requirements of the target neural network. This allows for the selection of the appropriate optimized compressed neural network as the target compressed neural network, ensuring that a neural network compression model that meets the requirements can be obtained, which is beneficial to improving the compression accuracy of the neural network.

[0158] The embodiments described in this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided by the embodiments of this application. As those skilled in the art will know, with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of this application are also applicable to similar technical problems.

[0159] It will be understood by those skilled in the art that Figure 1-6 The technical solutions shown do not constitute a limitation on the embodiments of this application, and may include more or fewer steps than shown, or combine certain steps, or different steps.

[0160] The foregoing has described specific embodiments of this application; other embodiments are within the scope of the appended claims. In some cases, the actions or steps described in the claims may be performed in a different order than those shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily have to follow the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0161] The various embodiments in this application are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the embodiments of apparatus, devices, and computer-readable storage media are basically similar to the method embodiments, so the descriptions are relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0162] The apparatus, device, computer-readable storage medium and method provided in the embodiments of this application are corresponding. Therefore, the apparatus, device and non-volatile computer storage medium also have similar beneficial technical effects as the corresponding method. Since the beneficial technical effects of the method have been described in detail above, the beneficial technical effects of the corresponding apparatus, device and computer storage medium will not be described again here.

[0163] In the 1990s, improvements to a technology could be clearly distinguished as either hardware improvements (e.g., improvements to the circuit structure of diodes, transistors, switches, etc.) or software improvements (improvements to the methodology). However, with technological advancements, many improvements to the methodology today can be considered direct improvements to the hardware circuit structure. Designers almost always obtain the corresponding hardware circuit structure by programming the improved methodology into the hardware circuit. Therefore, it cannot be said that an improvement to the methodology cannot be implemented using hardware physical modules.

[0164] For example, a Programmable Logic Device (PLD) (such as a Field Programmable Gate Array (FPGA)) is an integrated circuit whose logic function is determined by the user programming the device. Designers can program a digital system onto a PLD themselves, eliminating the need for chip manufacturers to design and fabricate dedicated integrated circuit chips. Furthermore, instead of manually fabricating integrated circuit chips, this programming is now mostly implemented using "logic compiler" software, similar to the software compiler used in program development. The source code before compilation must be written in a specific programming language called a Hardware Description Language (HDL). There is not just one type of HDL, but many, such as:

[0165] ABEL (Advanced Boolean Expression Language); AHDL (Altera Hardware Description Language); Confluence; CUPL (Cornell University Programming Language); HDCal; and JHDL (Java Hardware Description Language); Lava, Lola, MyHDL, PALASM, RHDL (Ruby Hardware Description Language), etc. Currently, among the technologies in this field, VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog are more commonly used. Those skilled in the art should also understand that by simply performing some logic programming on the method flow using the aforementioned hardware description languages ​​and programming it into an integrated circuit, the hardware circuit implementing the logical method flow can be easily obtained.

[0166] The controller can be implemented in any suitable manner. For example, the controller can take the form of a microprocessor or processor and a computer-readable medium storing computer-readable program code (e.g., software or firmware) that can be executed by the (micro)processor, logic gates, switches, application-specific integrated circuits (ASICs), programmable logic controllers, and embedded microcontrollers. Examples of controllers include, but are not limited to, the following microcontrollers:

[0167] The memory controller, including the ARC 625D, Atmel AT91SAM, Microchip IP address PIC18F26K20, and Silicon Labs C8051F320, can also be implemented as part of the memory's control logic. Those skilled in the art will also recognize that, in addition to implementing the controller as purely computer-readable program code, the same functionality can be achieved by logically programming the method steps, making the controller function as logic gates, switches, application-specific integrated circuits (ASICs), programmable logic controllers (PLCs), and embedded microcontrollers. Therefore, such a controller can be considered a hardware component, and the devices included within it for implementing various functions can also be considered structures within that hardware component. Alternatively, the devices for implementing various functions can be considered as both software modules implementing the method and structures within a hardware component.

[0168] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, a computer can be, for example, a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email device, game console, tablet computer, wearable device, or any combination of these devices.

[0169] For ease of description, the above apparatus is described by dividing it into various functional units. Of course, in implementing the embodiments of this application, the functions of each unit can be implemented in one or more software and / or hardware.

[0170] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, embodiments of this application can take the form of entirely hardware embodiments, entirely software embodiments, or embodiments combining software and hardware aspects. Furthermore, embodiments of this application can take the form of computer program products implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0171] This specification is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0172] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0173] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0174] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0175] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0176] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0177] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0178] In this application embodiment, "at least one" refers to one or more, and "more than one" refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent the existence of A alone, A and B simultaneously, or B alone. A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one of the following" and similar expressions refer to any combination of these items, including any combination of singular or plural items. For example, at least one of a, b, and c can represent: a, b, c, a and b, a and c, b and c, or a and b and c, where a, b, and c can be single or multiple.

[0179] The embodiments of this application can be described in the general context of computer-executable instructions, such as program modules, that are executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform a specific task or implement a specific abstract data type. The embodiments of this application can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In a distributed computing environment, program modules can reside in local and remote computer storage media, including storage devices.

[0180] The various embodiments in this application are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0181] The above description is merely an embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principle of this application should be included within the scope of the claims of this application.

Claims

1. A neural network compression method, characterized in that, include: The target neural network is quantized to obtain multiple first compressed neural networks, including: randomly changing the first quantization parameter of the target neural network multiple times to obtain multiple first compressed neural networks corresponding to the first quantization parameter; wherein the first quantization parameter is a quantization parameter selected from 2-bit, 4-bit and 8-bit integers; Training the plurality of first compressed neural networks to obtain a candidate network sequence includes: training the plurality of first compressed neural networks to obtain a plurality of candidate compressed neural networks; sorting the plurality of candidate compressed neural networks in descending order of NAS index to obtain a candidate network sequence, wherein the candidate network sequence includes a plurality of candidate compressed neural networks respectively corresponding to the plurality of first compressed neural networks; The candidate network sequences are compressed and iterated until the iteration termination condition is met; From the candidate network sequence, select at least one optimized compressed neural network as the target compressed neural network; The compression iteration process includes: Obtaining the Neural Network Architecture Search (NAS) index for each of the candidate compressed neural networks includes: for each candidate compressed neural network, inputting the candidate compressed neural network into a NAS index model; obtaining the peak memory parameter, capacity parameter, latency parameter, and accuracy parameter of the NAS index model; and calculating the NAS index of the candidate compressed neural network based on the NAS index model, according to the peak memory parameter, capacity parameter, latency parameter, and accuracy parameter; wherein the peak memory parameter is the maximum value of the sum of the memory usage of the input tensor and the output tensor of each layer; The candidate compressed neural network with the smallest NAS index in the candidate network sequence is quantized to obtain multiple second compressed neural networks; Multiple optimized compression neural networks are obtained by training the multiple second compression neural networks; Add multiple of the optimized compressed neural networks to the very end of the candidate network sequence, and delete the candidate compressed neural network at the very beginning of the candidate network sequence; The iteration termination condition is when the number of compression iterations reaches a preset number of iterations.

2. The neural network compression method according to claim 1, characterized in that, The step of selecting at least one optimized compressed neural network as the target compressed neural network from the candidate network sequence includes: From the candidate network sequence, select multiple optimized compressed neural networks whose NAS index is not greater than the preset NAS optimization index as the target compressed neural network.

3. The neural network compression method according to claim 2, characterized in that, The step of selecting multiple optimized compression neural networks from the candidate network sequence whose NAS index is not greater than a preset NAS optimization index as the target compression neural network includes: From a plurality of optimized compression neural networks whose NAS index is not greater than the preset NAS optimization index, the optimized compression neural network with the smallest NAS is selected as the target compression neural network.

4. The neural network compression method according to claim 1, characterized in that, The candidate compressed neural network with the smallest NAS index in the candidate network sequence is quantized to obtain multiple second compressed neural networks, including: By randomly changing the second quantization parameter of the candidate compressed neural network with the smallest NAS index multiple times, multiple second compressed neural networks corresponding to the second quantization parameter are obtained.

5. A neural network compression device, characterized in that, include: The first processing module is used to quantize the target neural network to obtain multiple first compressed neural networks, including: randomly changing the first quantization parameter of the target neural network multiple times to obtain multiple first compressed neural networks corresponding to the first quantization parameter; wherein the first quantization parameter is a quantization parameter selected from 2-bit, 4-bit and 8-bit integers; The second processing module is used to train the plurality of first compressed neural networks to obtain a candidate network sequence, including: training the plurality of first compressed neural networks to obtain a plurality of candidate compressed neural networks; sorting the plurality of candidate compressed neural networks in descending order of NAS index to obtain a candidate network sequence, wherein the candidate network sequence includes a plurality of candidate compressed neural networks respectively corresponding to the plurality of first compressed neural networks; The third processing module is used to perform compression iterative processing on the candidate network sequences until the iteration termination condition is met, wherein the compression iterative processing includes: Obtaining the Neural Network Architecture Search (NAS) index for each of the candidate compressed neural networks includes: for each candidate compressed neural network, inputting the candidate compressed neural network into a NAS index model; obtaining the peak memory parameter, capacity parameter, latency parameter, and accuracy parameter of the NAS index model; and calculating the NAS index of the candidate compressed neural network based on the NAS index model, according to the peak memory parameter, capacity parameter, latency parameter, and accuracy parameter; wherein the peak memory parameter is the maximum value of the sum of the memory usage of the input tensor and the output tensor of each layer; The candidate compressed neural network with the smallest NAS index in the candidate network sequence is quantized to obtain multiple second compressed neural networks; Multiple optimized compressed neural networks are obtained by training multiple second compressed neural networks; The plurality of optimized compressed neural networks are added to the end of the candidate network sequence and removed from the candidate compressed neural network at the beginning of the candidate network sequence; The iteration termination condition is that the number of compressed iterations reaches a preset number of iterations. The fourth processing module is used to select at least one of the optimized compressed neural networks as the target compressed neural network from the candidate network sequence.

6. An electronic device, comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that the processor, when executing the computer program, implements the neural network compression method as described in any one of claims 1 to 4.

7. A computer-readable storage medium, characterized in that, The device stores computer-executable instructions for performing the neural network compression method according to any one of claims 1 to 4.

Images