Microprocessor-oriented neural network optimization method

Abstract

The invention discloses a neural network optimization method for a microprocessor, and belongs to the technical field of neural network optimization, and the method comprises the steps: generating a plurality of parallel single-core precision branches for each network layer structure of a neural network, and constructing an initial network; performing pruning processing on the initial network by detecting the memory occupation condition of the single-core precision branch in a microprocessor based on a preset memory constraint to obtain a super network; and according to a preset target data set, taking minimization of the loss function as a target, screening out optimal precision branches of the network layer structures in the super network, and integrating the optimal precision branches to obtain an optimal neural network. Therefore, by implementing the method and the device, the problem that a neural network deployed on an edge device is difficult to realize balance in improvement of model precision and processing efficiency in the prior art can be solved.

Description

Technical Field

[0001] This application belongs to the field of neural network optimization technology, specifically relating to a neural network optimization method for microprocessors. Background Technology

[0002] Deploying deep neural networks on edge devices (such as microprocessors) has become a research hotspot. However, microprocessors are characterized by small memory, low computing power, and lack of an operating system, which severely restricts the deployment efficiency of neural network models. To address these issues, quantization techniques are commonly used to convert the precision parameters of neural network weights and activation values ​​from floating-point to low-precision integers. This reduces the model's storage requirements and computational complexity, allowing neural networks to run on resource-constrained edge devices while reducing power consumption and accelerating inference. However, this optimization comes at the cost of decreased precision, which can increase the model error of the neural network.

[0003] To reduce model error, neural architecture search techniques can adaptively search for optimal accuracy parameters based on the sensitivity of different layers and operations within a neural network model. This approach maintains high accuracy while further reducing storage requirements and computational complexity. However, while improving model accuracy, neural architecture search can construct a super-network far exceeding the size of the target model, causing a rapid increase in microprocessor usage that can exceed hardware limitations or result in extremely low data processing efficiency. Therefore, current technologies still struggle to achieve a balance between improving model accuracy and processing efficiency. Summary of the Invention

[0004] This application proposes a neural network optimization method for microprocessors, which can solve the problem in the prior art that it is difficult to achieve a balance between improving model accuracy and processing efficiency for neural networks deployed on edge devices.

[0005] The first aspect of this application provides a neural network optimization method for microprocessors, the method comprising:

[0006] An initial network is constructed by generating several parallel single-kernel precision branches for each network layer structure of the neural network.

[0007] Based on preset memory constraints, the initial network is pruned by detecting the memory usage of the single-core precision branch in the microprocessor to obtain a super network;

[0008] Based on a preset target dataset, with the goal of minimizing the loss function, the optimal precision branches of each network layer structure are selected in the super network, and the optimal precision branches are integrated to obtain the optimal neural network; wherein, the optimal precision branch corresponds to a precision parameter; the precision parameter is the probability of the neural network configuring a precision branch.

[0009] The above scheme finds the most suitable precision configuration for each network parameter in the neural network. First, several candidate precision types are generated for the parameters of each network layer and defined as single-core precision branches, each branch representing a precision type, to construct an initial network. Considering that some precision types consume a large amount of microprocessor memory during runtime, which can severely impact the data processing efficiency of the neural network, branches that may bring high training costs and deployment risks are removed from the initial network by setting memory constraints, retaining branches that will not exceed memory limits. This results in a super network that can be safely deployed on a microprocessor, minimizing model training time. Then, the performance of super networks with various precision configurations is evaluated using a target dataset. By minimizing the loss function, the branch that optimizes both model precision and data processing efficiency is found. Based on the obtained optimal precision branch, the precision configuration of each layer parameter in the neural network is determined, fully utilizing the multi-core processing power of the microprocessor to improve model precision while balancing the model's data processing capacity.

[0010] In one possible implementation of the first aspect, an initial network is constructed by generating several parallel single-kernel precision branches for each network layer of the neural network, specifically as follows:

[0011] By performing post-training quantization on the weight coefficients of each network layer structure, a first threshold single-core precision branch is generated for each network layer structure.

[0012] Wherein, the single-core precision branch represents the data type of the weight coefficient; each single-core precision branch corresponds to a precision parameter;

[0013] The initial network is constructed by adding the single-core precision branch to the neural network metastructure.

[0014] The above scheme generates multiple single-core branches representing different precision types in one go through post-training quantization, providing a variety of different precision type options for the network parameters of each layer of the neural network, and providing data support for subsequent precision parameter selection.

[0015] In one possible implementation of the first aspect, based on a preset memory constraint, the initial network is pruned by detecting the memory usage of the single-core precision branch on the microprocessor to obtain a super network, specifically:

[0016] Calculate the peak memory occupied in the microprocessor when the single-core precision branch is configured into the neural network;

[0017] The peak memory is detected according to the memory constraint. If the peak memory exceeds the second threshold, the corresponding single-core precision branch is deleted from the initial network.

[0018] Based on the pruned initial network, a super network is obtained.

[0019] To improve the efficiency of subsequent optimal branch search, the above scheme first eliminates branches that will inevitably violate microprocessor memory before the search, thus narrowing the search space. By using preset memory constraints, branches that are expected to consume a large amount of GPU memory are deleted, while branches with lower memory requirements are retained. This results in a super network that can be safely deployed on a microprocessor, ensuring that problems such as exploding training costs or excessively long training times will not occur during subsequent training of the neural network.

[0020] In one possible implementation of the first aspect, based on a preset target dataset, and with the objective of minimizing the loss function, the optimal precision branch of each network layer structure in the supernetwork is selected, specifically as follows:

[0021] Based on the target dataset, the single-core precision branches in the super network are screened with the goal of minimizing the single-core loss function, and the optimal single-core precision branch of each network layer structure is determined.

[0022] Based on the accuracy parameters of the optimal single-core precision branch, several network layer structures are extracted from the super network as candidate layers.

[0023] By combining the single-core precision branches in pairs, several dual-core precision branches are added to each candidate layer to construct a dual-branch network;

[0024] With the goal of minimizing the dual-kernel loss function, branch selection is performed on the dual-branch network to obtain the optimal candidate layer precision branch;

[0025] The optimal precision branch is obtained based on the optimal single-core precision branch and the optimal candidate layer precision branch.

[0026] The above scheme performs a two-stage search on the supernetwork to find the optimal precision branch, thereby obtaining a neural network that achieves a balance between model accuracy and data processing efficiency. By searching for the optimal single-core precision branch, the optimal precision configuration corresponding to the network parameters of each layer is first determined. Then, candidate layers that are more sensitive to parameter accuracy are extracted and a dual-core precision search is performed, which significantly reduces the overhead of the secondary search and the risk of model fragmentation. Moreover, the original exponential search space is decomposed into two "polynomial-level" searches, activating only the subspace that matches the hardware resources in each search, thereby compressing the microprocessor processing time from days to hours and improving the model inference efficiency.

[0027] In one possible implementation of the first aspect, based on the target dataset, the single-core precision branches in the supernetwork are screened with the objective of minimizing the single-core loss function to determine the optimal single-core precision branch for each network layer structure, specifically as follows:

[0028] The supernetwork is initialized by initializing the accuracy parameters of the optimal single-core accuracy branch;

[0029] The single-core loss function is constructed based on the task loss and the single-core latency loss; wherein, the task loss is the performance of the super network on the target dataset; and the single-core latency loss is the inference latency of the neural network under different configurations of the single-core precision branches.

[0030] With the goal of minimizing the single-core loss function, the super network is iteratively optimized using the target dataset to obtain the final accuracy parameters of each single-core accuracy branch;

[0031] The optimal single-core precision branch for each network layer structure is determined based on the maximum final precision parameter.

[0032] The above scheme takes into account the data processing latency of the model under different single-core precision configurations to construct a single-core loss function, thereby finding the single-core precision branch corresponding to each layer of network parameters and determining the optimal precision selection for each layer of network parameters under mixed precision configuration.

[0033] In one possible implementation of the first aspect, with the goal of minimizing the single-core loss function, the supernetwork is iteratively optimized using the target dataset to obtain the final accuracy parameters of each single-core accuracy branch, specifically:

[0034] In each iteration of optimization, the accuracy parameters of the single-core accuracy branch are continuously updated using the soft sampling method, and the temperature parameter of the soft sampling is reduced during the continuous update process until the accuracy parameters converge to a preset range, thus obtaining the final accuracy parameters.

[0035] Specifically, the accuracy parameter is updated using the prior probability, sampling noise, and temperature parameter; the temperature parameter is used to control the dispersion of the probability distribution of the accuracy parameter.

[0036] In one possible implementation of the first aspect, several dual-core precision branches are added to each candidate layer by combining the single-core precision branches in pairs to construct a dual-branch network, specifically as follows:

[0037] Arbitrarily combine any two of the single-core precision branches in the candidate layer to obtain several candidate dual-core branches;

[0038] Eliminate the candidate dual-core branches that do not meet the memory constraints to obtain the dual-core precision branch;

[0039] In the candidate layer, all branches except the optimal single-core precision branch are removed, and then the dual-core precision branch is added to construct a dual-branch network.

[0040] The above scheme takes into account that the network parameters of certain layers are more sensitive to accuracy. Therefore, these candidate layers are extracted and subjected to dual-core precision search to provide better accuracy configuration for these network parameters, thereby further improving model performance. Moreover, the secondary search is only performed on candidate layers, and other non-candidate layers will not participate in the search to reduce the search space, thus significantly reducing the overhead of secondary search and the risk of model fragmentation, and improving search efficiency.

[0041] In one possible implementation of the first aspect, the dual-branch network is subjected to branch selection with the objective of minimizing the dual-kernel loss function to obtain the optimal candidate layer precision branch, specifically as follows:

[0042] The dual-core overlapping execution model is introduced into the single-core loss function to obtain the dual-core loss function;

[0043] With the goal of minimizing the dual-core loss function, the accuracy parameters are optimized using gradient descent. Based on the optimization results, the optimal single-core accuracy branch and the dual-core accuracy branch in the dual-branch network are selected, and the optimal candidate layer accuracy branch is output.

[0044] In one possible implementation of the first aspect, a dual-core overlapping execution model is used to retrieve the branch corresponding to the maximum value of the precision parameter in the dual-branch network.

[0045] In one possible implementation of the first aspect, the optimal precision branch is integrated to obtain the optimal neural network, specifically as follows:

[0046] By combining the optimal single-core precision branch of the non-candidate layer with the optimal candidate layer precision branch of the candidate layer, the optimal branch configuration of the neural network is obtained.

[0047] The optimal branch configuration is mapped onto the neural network to obtain the optimal neural network.

[0048] The above solution optimizes the neural network model effectively in a multi-core environment by configuring the parameters of each layer of the neural network with precision. This improves the model's accuracy while balancing its inference efficiency, thereby enhancing the overall performance of the model. Attached Figure Description

[0049] To more clearly illustrate the technical solution of this application, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0050] Figure 1 This is a schematic flowchart illustrating a neural network optimization method for microprocessors provided in an embodiment of this application.

[0051] Figure 2 This is an initial network structure diagram of a neural network optimization method for microprocessors provided in an embodiment of this application;

[0052] Figure 3 This is a structural diagram of a dual-core overlapping execution model of a neural network optimization method for microprocessors provided in an embodiment of this application. Detailed Implementation

[0053] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0054] It should be understood that the step numbers used in the text are for ease of description only and are not intended to limit the order in which the steps are performed.

[0055] First Embodiment

[0056] Currently, reducing the parameter precision of neural network models is often used to decrease the model's storage requirements and computational complexity on microprocessors. However, this method leads to a decrease in model precision, which in turn affects model performance. Although quantization techniques can find the optimal precision configuration for neural networks, quantization constructs a super-network far exceeding the size of the neural network model during the optimization process, causing microprocessor memory consumption to explode, resulting in excessively long optimization times or even optimization process crashes. Therefore, how to search for the optimal precision configuration for neural networks while reducing the search time, and achieving a balance between model precision and model inference efficiency, is the technical problem that this application aims to solve.

[0057] like Figure 1As shown, to address the problem in existing technologies where it is difficult to achieve a balance between improving model accuracy and processing efficiency for neural networks deployed on edge devices, the first embodiment of this application provides a detailed flowchart of a neural network optimization method for microprocessors. This embodiment's neural network optimization method for microprocessors includes steps S1 to S3, detailed below:

[0058] Step S1: An initial network is constructed by generating several parallel single-core precision branches for each network layer structure of the neural network.

[0059] This application embodiment abstracts and models the arithmetic logic unit (ALU), floating-point computing unit (FPU), and multi-core topology of the microprocessor to form a queryable hardware performance table, which is used to guide subsequent acceleration of floating-point calculations and parallel execution of subtasks.

[0060] The hardware performance table allows for the measurement of typical operators such as convolution, depthwise separable convolution, fully connected, and pooling in neural networks under different precision configurations. This helps determine the model performance for various precision configurations and provides support for selecting the optimal precision branch.

[0061] First, while keeping the original neural network metastructure (number of layers, channels, convolution kernels) unchanged, several parallel, optional single-kernel precision branches are generated for each network layer structure to form an initial super network, which is the initial network.

[0062] Optionally, this application embodiment sets five precision types, corresponding to five single-core precision branches, including FP32, FP16, INT8, INT4, and INT2. FP represents floating-point numbers, INT represents integers, and the following data indicates how many bits of binary storage are used.

[0063] Therefore, it can be seen that the precision type directly determines the amount of memory occupied by the neural network parameters in the microprocessor; the higher the precision, the more memory is occupied.

[0064] Specifically, post-training quantization is performed once for the weight coefficients of each layer of the neural network, generating five different single-core precision branches at once. Each single-core precision branch represents a configurable precision type for the weight coefficients. The purpose of post-training quantization is to allow the search to start from a better data distribution. Then, a Q-node is placed at the entry point of each single-core precision branch, and a DQ-node is placed at the exit point, ensuring that any single-core precision branch can be seamlessly connected to the backbone of the neural network.

[0065] For example, embodiments of this application provide Figure 2 This will demonstrate the generated initial network. Figure 2In this network structure, each layer contains five parallel single-core precision branches, each with a trainable precision parameter. This precision parameter represents the probability that the neural network selects and configures that precision type. As subsequent branches search, these precision parameters gradually change from a discrete distribution to a one-hot distribution. Here, OP is an operator, representing the basic unit for performing mixed precision search (such as convolution-batch normalization-ReLU activation commonly found in edge-side convolutional neural networks).

[0066] Step S2: Based on preset memory constraints, the initial network is pruned by detecting the memory usage of the single-core precision branch in the microprocessor to obtain a super network.

[0067] Considering that there are some branches in the initial network that will inevitably consume too much memory, these branches that do not meet the memory constraints are pruned in the entire layer before the search, so as to compress the search space and reduce the memory requirements of the microprocessor, thereby shortening the training time of the model.

[0068] The motivation for pruning the initial network stems from the training costs and deployment risks resulting from the explosion of the "precision x multi-core" combination. If all FP32 / 16 and INT8 / 4 / 2 branches were allowed into the super network, the candidate size for a 52-layer initial network could reach as high as 10^10. 36 A complete search process requires a large amount of GPU memory and more than a day of training. Furthermore, many branches are destined to be undeployable on real microprocessors due to memory limitations, resulting in a huge waste of computing power and electricity.

[0069] Therefore, this embodiment performs a lightweight static analysis on the peak memory of each single-core precision branch before the search. Using a preset model of "maximum layer activation + shared input / output buffer + library overhead," the peak memory occupied by a single-core precision branch in each layer is calculated. This peak memory is compared with a set threshold, and single-core precision branches exceeding the threshold are removed, leaving only branches not exceeding the threshold. This results in a super network that fully satisfies the memory constraints, avoiding model fragmentation and resource waste later on, and laying a feasible and efficient foundation for the subsequent two-stage fine-grained search.

[0070] Step S3: Based on the preset target dataset, with the goal of minimizing the loss function, select the optimal precision branch of each network layer structure in the super network, and integrate the optimal precision branches to obtain the optimal neural network.

[0071] In order to further improve retrieval efficiency and solve the combinatorial explosion problem caused by "precision × multi-core" after compressing the search space, this application embodiment splits the original exponential search space into two "polynomial" searches, namely single-core precision search and multi-core combinatorial search, and activates only the subspace that matches the hardware resources in each stage, thereby compressing the GPU time from days to hours.

[0072] For the first stage of single-core precision search, the training weights and precision parameters of each single-core precision branch are initialized to obtain the initialized super network. Then, based on the predefined single-core loss function, the super network is optimized using the target dataset until the precision parameters of each layer converge to the preset precision range. At this point, the optimal single-core precision branch of each layer and its corresponding maximum precision parameter are output.

[0073] The expression for the single-core loss function is as follows:

[0074]

[0075] In the formula, The task loss is used to measure the model's performance on the target dataset. λ is the single-core latency loss, used to measure the model inference loss under different precision configurations (i.e., different single-core precision branches are selected); λ is a hyperparameter used to balance task loss and single-core latency loss.

[0076] With the goal of minimizing the single-core loss function, the super network is iteratively optimized using the target dataset. During the iterative optimization process, the precision parameters of the branches are soft-sampled using the Gumbel-Softmax distribution, so that the precision parameters are continuously updated during training until they converge to a set precision configuration, indicating that the super network has completed optimization.

[0077] Specifically, the Gumbel-Softmax distribution is a data tool used to process discrete probability distributions, transforming a discrete sampling process into a continuous random process. In this embodiment, soft sampling is used to gradually decrease the temperature parameter, transforming the originally discrete precision parameter into a unique thermal distribution. The update process of the precision parameter is as follows:

[0078]

[0079] In the formula, α i Let π be the precision parameter of the i-th single-core precision branch. i g is the prior probability of the precision parameter. i The noise sampled in the Gumbel-Softmax distribution is τ, which is a temperature parameter used to control the degree of dispersion of the distribution.

[0080] The core idea of ​​one-hot distribution is to convert a variable into a unique binary vector to achieve a mapping from discrete values ​​to continuous vectors, thereby obtaining a numerical format suitable for machine learning models and reducing the complexity of using neural networks.

[0081] After optimization, the optimal single-core precision branch and its corresponding maximum precision parameter are obtained for each layer of the super network structure. The optimal single-core precision branch of each layer represents the best precision selection of the neural network under the mixed precision configuration, and the maximum precision parameter is the maximum probability of the branch being configured.

[0082] Considering that some network layer structures are sensitive to changes in accuracy, this embodiment dynamically determines whether multi-core parallelism is needed based on the accuracy sensitivity of each layer to improve model performance. Based on the maximum accuracy parameter of each optimal single-core accuracy branch, several network layer structures whose maximum accuracy parameter is relatively close to that of other branches within the layer are selected as candidate layers, also known as Top-k layers, where k is the number of candidate layers. The small difference between the candidate layers and other layers in the selection probability of each accuracy type indicates that they are sensitive to changes in accuracy type; therefore, multi-core combination search is needed to further determine a more suitable accuracy configuration. Other non-candidate layer structures will not be searched, and the original optimal single-core accuracy branch will be directly retained.

[0083] In the multi-core combination search, for each candidate layer, the original five single-core precision branches are arbitrarily paired to obtain ten different candidate dual-core branches. Branches that do not meet the internal constraints are removed from these eleven candidate dual-core branches, resulting in dual-core precision branches after memory detection. These dual-core precision branches are combined with the optimal single-core precision branch to construct several optional branches for each candidate layer. The candidate dual-core branches allow the neural network model to use two different precision types simultaneously in the same layer.

[0084] Based on all possible branches of all candidate layers, a new search space is constructed. This search space eliminates all branches except the optimal single-core precision branch, and then the dual-core precision branch is added to form a dual-branch network.

[0085] By incorporating the dual-core overlapping execution model into the single-core loss function, we obtain the dual-core loss function, the specific expression of which is:

[0086]

[0087] In the formula, max(P1,P2) is the dual-core overlapping execution model, P1 is the first dual-core precision branch, and P2 is the second dual-core precision branch.

[0088] For example, Figure 3This is one case of the dual-core overlapping execution model. In the diagram, the first dual-core precision branch of a candidate layer is OP(FP16), and the second is OP(INT4). That is, this dual-core precision branch is a combination of the original single-core precision branches FP16 and INT4. Meanwhile, OP(INT8) in the diagram is the original optimal single-core precision branch. Therefore, it can be seen that the dual-core overlapping execution model is used to retrieve the branch corresponding to the maximum value of the precision parameter in the dual-branch network.

[0089] Within the new search space, with the goal of minimizing the dual-core loss function, the accuracy parameters are optimized using gradient descent. The optimal single-core accuracy branch and the dual-core accuracy branch in the dual-branch network are selected to output the optimal candidate layer accuracy branch and its corresponding maximum accuracy parameter.

[0090] Finally, the optimal single-core precision branch of the non-candidate layer is combined with the optimal candidate layer precision branch of the candidate layer to obtain the optimal branch configuration of the neural network. This optimal branch configuration is then applied as a quantization strategy to the neural network to construct a high-performance optimal neural network model.

[0091] As an improvement to the above solution, the super network is further optimized to flexibly adapt to the hardware characteristics of different MCUs. Specifically, the number and type of precision branches in each layer of the super network are dynamically adjusted according to the MCU's arithmetic logic unit (ALU), floating-point computing unit (FPU), and multi-core topology. For example, for MCUs that support more advanced floating-point operations, the precision of the weights and quantization parameters of the FP32 and FP16 branches is maintained; while for MCUs that prioritize energy efficiency (without FPU and multi-core), only the structure of INT8, INT4, and INT2 branches is maintained to achieve higher computational efficiency and lower power consumption.

[0092] Furthermore, a hardware-aware design was introduced, enabling the supernetwork to perceive the hardware characteristics of the target MCU and adjust its own structure and parameters accordingly. By modeling different MCU hardware, the supernetwork building blocks can query the target MCU's performance metrics in real time, such as computing power, memory bandwidth, and multi-core communication latency, and optimize the precision pruning, quantization parameters, and precision-efficiency tradeoffs for each layer based on this information. This adaptive design not only improves the supernetwork's compatibility with different MCUs but also ensures optimal performance in various hardware environments.

[0093] Furthermore, the precision search process can be implemented using more powerful GPUs, thereby reducing GPU hours and improving search efficiency. Supernetworks are essentially larger neural networks, and their search process requires significant computational resources. Using more powerful GPUs can significantly accelerate this process. Performing precision searches on resource-constrained GPUs limits the number of batches, resulting in more GPU hours spent on large datasets.

[0094] Implementing the embodiments of this application has the following beneficial effects:

[0095] This application embodiment finds the most suitable precision configuration for each network parameter in the neural network. First, several candidate precision types are generated for the parameters of each network layer and defined as single-core precision branches, each branch representing a precision type, to construct an initial network. Considering that some precision types consume a large amount of microprocessor memory during runtime, which can severely impact the data processing efficiency of the neural network, branches that may bring high training costs and deployment risks are removed from the initial network by setting memory constraints, retaining branches that will not exceed memory limits. This results in a super network that can be safely deployed on a microprocessor, minimizing model training time. Then, the performance of super networks with various precision configurations is evaluated using a target dataset. By minimizing the loss function, the branch that optimizes both model precision and data processing efficiency is found. Based on the obtained optimal precision branch, the precision configuration of each layer parameter in the neural network is determined, fully utilizing the multi-core processing power of the microprocessor to improve model precision while balancing the model's data processing capacity.

[0096] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of this application. It should be understood that the above descriptions are merely specific embodiments of this application and are not intended to limit the scope of protection of this application. In particular, it should be noted that any modifications, equivalent substitutions, or improvements made by those skilled in the art within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A neural network optimization method for microprocessors, characterized in that, include: An initial network is constructed by generating several parallel single-kernel precision branches for each network layer structure of the neural network. Based on preset memory constraints, the initial network is pruned by detecting the memory usage of the single-core precision branch in the microprocessor to obtain a super network; Based on a preset target dataset, with the goal of minimizing the loss function, the optimal precision branches of each network layer structure are selected in the super network, and the optimal precision branches are integrated to obtain the optimal neural network; wherein, the optimal precision branch corresponds to a precision parameter; the precision parameter is the probability of the neural network configuring a precision branch.

2. The neural network optimization method for microprocessors according to claim 1, characterized in that, The process involves generating several parallel single-core precision branches for each network layer of the neural network to construct an initial network, specifically as follows: By performing post-training quantization on the weight coefficients of each network layer structure, a first threshold single-core precision branch is generated for each network layer structure. Wherein, the single-core precision branch represents the data type of the weight coefficient; each single-core precision branch corresponds to a precision parameter; The initial network is constructed by adding the single-core precision branch to the neural network metastructure.

3. The neural network optimization method for microprocessors according to claim 1, characterized in that, The process, based on preset memory constraints, involves pruning the initial network by detecting the memory usage of the single-core precision branch on the microprocessor to obtain a super network. Specifically: Calculate the peak memory occupied in the microprocessor when the single-core precision branch is configured into the neural network; The peak memory is detected according to the memory constraint. If the peak memory exceeds the second threshold, the corresponding single-core precision branch is deleted from the initial network. Based on the pruned initial network, a super network is obtained.

4. The neural network optimization method for microprocessors according to claim 1, characterized in that, The step of selecting the optimal precision branch of each network layer structure in the supernetwork based on a preset target dataset and minimizing the loss function is as follows: Based on the target dataset, the single-core precision branches in the super network are screened with the goal of minimizing the single-core loss function, and the optimal single-core precision branch of each network layer structure is determined. Based on the accuracy parameters of the optimal single-core precision branch, several network layer structures are extracted from the super network as candidate layers. By combining the single-core precision branches in pairs, several dual-core precision branches are added to each candidate layer to construct a dual-branch network; With the goal of minimizing the dual-kernel loss function, branch selection is performed on the dual-branch network to obtain the optimal candidate layer precision branch; The optimal precision branch is obtained based on the optimal single-core precision branch and the optimal candidate layer precision branch.

5. The neural network optimization method for microprocessors according to claim 4, characterized in that, The step of filtering the single-core precision branches in the supernetwork based on the target dataset, with the objective of minimizing the single-core loss function, and determining the optimal single-core precision branch for each network layer structure, specifically involves: The supernetwork is initialized by initializing the accuracy parameters of the optimal single-core accuracy branch; The single-core loss function is constructed based on the task loss and the single-core latency loss; wherein, the task loss is the performance of the super network on the target dataset; and the single-core latency loss is the inference latency of the neural network under different configurations of the single-core precision branches. With the goal of minimizing the single-core loss function, the super network is iteratively optimized using the target dataset to obtain the final accuracy parameters of each single-core accuracy branch; The optimal single-core precision branch for each network layer structure is determined based on the maximum final precision parameter.

6. The neural network optimization method for microprocessors according to claim 5, characterized in that, The goal is to minimize the single-core loss function. The supernetwork is iteratively optimized using the target dataset to obtain the final accuracy parameters of each single-core accuracy branch. Specifically: In each iteration of optimization, the accuracy parameters of the single-core accuracy branch are continuously updated using the soft sampling method, and the temperature parameter of the soft sampling is reduced during the continuous update process until the accuracy parameters converge to a preset range, thus obtaining the final accuracy parameters. Specifically, the accuracy parameter is updated using the prior probability, sampling noise, and temperature parameter; the temperature parameter is used to control the dispersion of the probability distribution of the accuracy parameter.

7. The neural network optimization method for microprocessors according to claim 4, characterized in that, The method involves combining the single-core precision branches in pairs to add several dual-core precision branches to each candidate layer, thereby constructing a dual-branch network. Specifically: Arbitrarily combine any two of the single-core precision branches in the candidate layer to obtain several candidate dual-core branches; Eliminate the candidate dual-core branches that do not meet the memory constraints to obtain the dual-core precision branch; In the candidate layer, all branches except the optimal single-core precision branch are removed, and then the dual-core precision branch is added to construct a dual-branch network.

8. The neural network optimization method for microprocessors according to claim 4, characterized in that, The process of minimizing the dual-kernel loss function and then performing branch selection on the dual-branch network to obtain the optimal candidate layer precision branch is as follows: The dual-core overlapping execution model is introduced into the single-core loss function to obtain the dual-core loss function; With the goal of minimizing the dual-core loss function, the accuracy parameters are optimized using gradient descent. Based on the optimization results, the optimal single-core accuracy branch and the dual-core accuracy branch in the dual-branch network are selected, and the optimal candidate layer accuracy branch is output.

9. The neural network optimization method for microprocessors according to claim 8, characterized in that, The dual-core overlapping execution model is used to retrieve the branch corresponding to the maximum value of the precision parameter in the dual-branch network.

10. The neural network optimization method for microprocessors according to any one of claims 1 to 9, characterized in that, The process of integrating the optimal precision branches to obtain the optimal neural network is as follows: By combining the optimal single-core precision branch of the non-candidate layer with the optimal candidate layer precision branch of the candidate layer, the optimal branch configuration of the neural network is obtained. The optimal branch configuration is mapped onto the neural network to obtain the optimal neural network.

Images