Target network acquisition method and device, electronic equipment and readable storage medium
Quantitative training is carried out by partially sharing between the quantitative learning step parameters of the hybrid precision supernet, and the problem of low accuracy after quantitative training of the hybrid precision supernet is solved, and high-precision target network acquisition is achieved.
Patent Information
- Application Number
- CN202311302564.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-10-09
- Publication Date
- 2025-04-11
AI Technical Summary
In the quantitative training of hybrid precision supernet, the quantitative training method of ordinary network structure supernet is directly applied, resulting in the mixed precision supernet after quantitative training, which affects the final accuracy of the target network.
Quantitative training is carried out by partially sharing between the quantitative learning step parameters of the hybrid precision supernet, and the relationship between the quantitative learning step parameters of different bits is defined, so as to improve the accuracy of the hybrid precision supernet after quantization training.
The accuracy of the hybrid accuracy supernet after quantitative training is improved, and the target network with high accuracy is obtained, solving the problem of low accuracy after quantitative training.
Smart Images
Figure CN119808847A_ABST
Abstract
Description
Technical Field
[0001] The embodiments of the present disclosure relate to a method, device, electronic device and readable storage medium for acquiring a target network. Background Art
[0002] In the process of obtaining the target network that executes the target task according to the target task to be executed, the mixed precision model established in the process needs to be quantized. In order to better balance the speed, size and accuracy of the model, it is necessary to quantize the model with lower bits while improving the network accuracy as much as possible.
[0003] Inspired by the network structure search NAS technology, existing technologies first build a mixed precision supernet to include all model accuracies, and then optimize the entire supernet through the idea of weight sharing to solve. However, the supernet based on mixed precision is different from the ordinary network structure supernet. The number of weights of the maximum and minimum sub-models of the ordinary network structure supernet is different, while the weights of the mixed precision supernet are completely shared under different quantization configurations. Therefore, the quantization training method used in related technologies is directly applied to the quantization training of the mixed precision supernet, which will result in low accuracy of the mixed precision supernet after quantization training, which will in turn affect the accuracy of the target network finally obtained. Summary of the invention
[0004] In view of this, the present disclosure provides a target network acquisition method, device, electronic device and readable storage medium, which can improve the accuracy of a mixed precision supernet after quantization training, and thus obtain a high-precision target network.
[0005] The technical solution of the present disclosure is achieved as follows:
[0006] In one embodiment, a method for acquiring a target network is provided, the method comprising:
[0007] Acquire network construction information corresponding to the target task; wherein the network construction information includes: search space information, sample data and search conditions;
[0008] Constructing a first mixed precision supernet according to the search space information;
[0009] Based on the sample data, the first mixed precision supernet is trained with quantization perception in a preset manner to obtain a second mixed precision supernet; wherein the preset manner is a manner of partial sharing between quantization learning step parameters, and the partial sharing between quantization learning step parameters is achieved by defining the relationship between quantization learning step parameters of different bits when performing bit-specific quantization on floating-point weights and activation values;
[0010] Search the second mixed precision supernet for a subnet that meets the search condition to obtain a target network; wherein the target network is a network that performs the target task.
[0011] The relationship between the quantization learning step parameters of different bits is:
[0012]
[0013] And meet the following conditions:
[0014] Among them, s k+m represents the k+m bit quantization learning step parameter, s k represents the k-bit quantization learning step parameter, α k+m Represents a learnable bias parameter, M is the percentage threshold of the increase in the floating-point value range after k+m-bit quantization compared to the floating-point value range after k-bit quantization, and m is an integer greater than 0.
[0015] Wherein, when searching for a subnet satisfying the search condition in the second mixed precision supernet, a preset search strategy is used for searching; the preset search strategy limits the correlation between the weight distribution range and the sampling probability.
[0016] The preset search strategy includes:
[0017] If the floating point weight distribution range of the j+1th layer of the subnet is wider than the floating point weight distribution range of the jth layer, a sampling probability greater than that of the low bits is set for the high bits of the j+1th layer;
[0018] or,
[0019] If the floating point weight distribution range of the j+1th layer is narrower than that of the jth layer, a sampling probability smaller than that of the low bit is set for the high bit of the j+1th layer;
[0020] or,
[0021] If the floating point weight distribution range of the j+1th layer of the subnet is equal to the floating point weight distribution range of the jth layer, the same sampling probability as the low bit is set for the high bit of the j+1th layer.
[0022] Wherein, the method further comprises:
[0023] Obtaining target operating environment information corresponding to the target network;
[0024] The searching the second mixed-precision supernet for a subnet that meets the search condition includes:
[0025] Based on the target operating environment information, the second mixed precision supernet is searched for a subnet that meets the search condition.
[0026] Wherein, the method further comprises:
[0027] Deploy the obtained target network in an environment corresponding to the target operating environment information;
[0028] The target task is performed using the deployed target network.
[0029] In one embodiment, a target network acquisition device is provided, the device comprising:
[0030] An acquisition unit is configured to acquire network construction information corresponding to a target task; wherein the network construction information includes: search space information, sample data and search conditions;
[0031] A construction unit, configured to construct a first mixed precision supernet according to the search space information;
[0032] A quantization training unit is configured to perform quantization-aware training on the first mixed precision supernet in a preset manner based on the sample data to obtain a second mixed precision supernet; wherein the preset manner is a partial sharing manner between quantization learning step parameters, and the partial sharing manner between quantization learning step parameters is achieved by defining a relationship between quantization learning step parameters of different bits when performing bit-specific quantization on floating-point weights and activation values;
[0033] The search unit is configured to search the second mixed precision supernet for a subnet that meets the search condition to obtain a target network; wherein the target network is a network that performs the target task.
[0034] in,
[0035] The search unit is configured to use a preset search strategy to perform a search when searching for a subnet that meets the search condition in the second mixed precision supernet; the preset search strategy defines a correlation between a weight distribution range and a sampling probability.
[0036] In another embodiment, an electronic device is provided, comprising:
[0037] processor;
[0038] a memory for storing executable instructions for the processor;
[0039] The processor is configured to execute the executable instructions to implement the target network acquisition method as described in any one of the above items.
[0040] In another embodiment, a computer-readable storage medium, when at least one instruction in the computer-readable storage medium is executed by a processor of an electronic device, enables the electronic device to implement any of the target network acquisition methods described above.
[0041] It can be seen from the above technical solution that when the first mixed precision supernet constructed based on the search space information corresponding to the target task is subjected to quantization training in the embodiment of the present disclosure, taking into account that the weights of the mixed precision supernet are completely shared under different quantization configurations, quantization training is performed by partially sharing the quantization learning step parameters to improve the accuracy of the mixed precision supernet after quantization training, and then a high-precision target network can be searched based on the high-precision mixed precision supernet. BRIEF DESCRIPTION OF THE DRAWINGS
[0042] In order to more clearly illustrate the technical solutions in the embodiments of the present disclosure, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.
[0043] Figure 1 is a flow chart of a method for acquiring a target network according to an exemplary embodiment;
[0044] Figure 2 is a flow chart of another method for acquiring a target network according to an exemplary embodiment;
[0045] Figure 3 is a flow chart of a network deployment method according to an exemplary embodiment;
[0046] Figure 4 A schematic diagram of partial sharing between quantization learning step parameters of different bits in an example disclosed in the present invention;
[0047] Figure 5 is a schematic diagram of a target network acquisition device according to an exemplary embodiment;
[0048] Figure 6 It is a structural schematic diagram of an electronic device provided by an embodiment of the present disclosure. DETAILED DESCRIPTION
[0049] In order to make the objectives, technical solutions and advantages of the present disclosure more clearly understood, the present disclosure is further described in detail below with reference to the accompanying drawings and examples.
[0050] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.
[0051] The disclosed embodiments take into account the specificity of the mixed-precision supernet, that is, the weights of the mixed-precision supernet are fully shared under different quantization configurations. In the process of implementing the provided target network acquisition method, the mixed-precision supernet is quantized and trained in a manner that partially shares the quantized learning step parameters. This can improve the accuracy of the mixed-precision supernet after quantization training, thereby enabling a high-precision target network to be obtained.
[0052] Figure 1 FIG. 1 is a flow chart of a method for acquiring a target network according to an exemplary embodiment. Figure 1 As shown, the target network acquisition method of the embodiment of the present disclosure mainly includes the following steps 101 to 104.
[0053] Step 101, obtaining network construction information corresponding to the target task; wherein the network construction information includes: search space information, sample data and search conditions.
[0054] In the disclosed embodiment, the task that the user needs to perform may be referred to as a target task, such as image recognition, image classification, image segmentation, etc., but not limited thereto. The network that the user needs to construct to perform the target task is referred to as a target network.
[0055] In one example, the target network may be a neural network.
[0056] The network construction information includes: search space information, sample data and search conditions.
[0057] The search space information may include a variety of information used to construct a mixed precision supernet, such as the definition of input data of the mixed precision supernet, the definition of output data, the definition of the loss function, the number of modules constituting the mixed precision supernet, the branch construction form of the branches in each module of the mixed precision supernet, and the bit parameter of each layer, etc.; the embodiment of the present invention is not limited to this;
[0058] The sample data is used to train the mixed precision supernet;
[0059] The search conditions include performance indicators of the target network, etc., which are used to search the target network from the mixed precision super network.
[0060] Step 102: construct a first mixed precision supernet according to the search space information.
[0061] In the relevant implementation, the first mixed precision supernet constructed is a floating point network, and the first mixed precision supernet can be trained as a floating point network. The trained first mixed precision supernet is then quantized, that is, step 103 is executed.
[0062] Step 103, based on the sample data, a preset method is used to perform quantization-aware training on the first mixed precision supernet to obtain a second mixed precision supernet; wherein the preset method is a method of partially sharing quantization learning step parameters, and the method of partially sharing quantization learning step parameters is achieved by defining the relationship between quantization learning step parameters of different bits when bit-specific quantization is performed on floating-point weights and activation values.
[0063] In the embodiments of the present disclosure, quantization-aware training is used during quantization. When using quantization-aware training (QAT), all weights and activations are "pseudo-quantized" in the forward and backward processes of training, that is, floating-point values are rounded to simulated fixed-point values, but all calculations are still done using floating-point numbers, and all weight adjustments during training are done with the "awareness" of the fact that the model will eventually be quantized; therefore, higher precision can be produced, which is more suitable for the training of high-precision hybrid supernets.
[0064] In related technologies, the quantization parameters of mixed precision supernets cannot be partially shared. Usually, there are two ways to deal with it:
[0065] The first type is that the weights between the quantized learning step parameters are not shared at all;
[0066] This method can effectively alleviate gradient conflicts under different quantization configurations, but the entire search space is huge, and underfitting is easy to occur with limited training times.
[0067] The second type is that the weights between the quantized learning step parameters are fully shared;
[0068] In this method, since the parameters of different quantization bits are completely consistent, the gradient conflict can only be alleviated by adjusting the weight distribution of the network but not by additional parameters, which greatly reduces the ability to alleviate the gradient conflict.
[0069] The disclosed example proposes a method for partially sharing quantization learning step parameters during quantization-aware training. The method for partially sharing quantization learning step parameters is implemented by defining the relationship between quantization learning step parameters of different bits when performing fixed-point quantization on floating-point weights and activation values. Compared with the first method, the implementation of this method is that each quantization bit configuration shares the same parameters and is not prone to underfitting. Compared with the second method, each quantization bit configuration has its own independent parameters, which can better alleviate gradient conflicts.
[0070] Step 104, searching the second mixed precision supernet for a subnet that meets the search condition to obtain a target network; wherein the target network is a network that performs a target task.
[0071] In the disclosed embodiment, before the target network is acquired through search, when the first mixed precision supernet is subjected to sensitive quantization training, the quantization training is performed in a manner of partially sharing the quantization learning step size parameters, which can improve the accuracy of the mixed precision supernet after quantization training, and thus obtain a high-precision target network.
[0072] In the disclosed embodiment, not only an improved quantization-aware training method is used during quantization, but also an improved search strategy is adopted when searching for a target network, that is, a preset search strategy, which can sample a subnet that meets the search conditions with higher accuracy at a faster speed as the target network to be acquired.
[0073] Figure 2 FIG. 1 is a flow chart of another method for acquiring a target network according to an exemplary embodiment. Figure 2 As shown, the target network acquisition method of the embodiment of the present disclosure mainly includes the following steps 201 to 204.
[0074] Step 201, obtaining network construction information corresponding to the target task; wherein the network construction information includes: search space information, sample data and search conditions.
[0075] In the disclosed embodiment, the task that the user needs to perform may be referred to as a target task, such as image recognition, image classification, image segmentation, etc., but not limited thereto. The network that the user needs to construct to perform the target task is referred to as a target network.
[0076] In one example, the target network may be a neural network.
[0077] The network construction information includes: search space information, sample data and search conditions.
[0078] The search space information may include a variety of information used to construct a mixed precision supernet, such as the definition of input data of the mixed precision supernet, the definition of output data, the definition of the loss function, the number of modules constituting the mixed precision supernet, the branch construction form of the branches in each module of the mixed precision supernet, and the bit parameter of each layer, etc.; the embodiment of the present invention is not limited to this;
[0079] The sample data is used to train the mixed precision supernet;
[0080] The search conditions include performance indicators of the target network, etc., which are used to search the target network from the mixed precision super network.
[0081] Step 202: construct a first mixed precision supernet according to the search space information.
[0082] In the relevant implementation, the constructed first mixed precision supernet is a floating point network, and the floating point network training can be performed on the first mixed precision supernet. The trained first mixed precision supernet is then quantized, that is, step 203 is executed.
[0083] Step 203, based on the sample data, quantization-aware training is performed on the first mixed precision supernet in a preset manner to obtain a second mixed precision supernet; wherein the preset manner is a manner of partially sharing quantization learning step parameters, and the manner of partially sharing quantization learning step parameters is achieved by defining the relationship between quantization learning step parameters of different bits when bit-specific quantization is performed on floating-point weights and activation values.
[0084] The first mixed precision supernet includes multiple mixed precision subnets; each mixed precision subnet has multiple layers, each layer has a quantized bit, and the quantized bit of each layer can be the same or different;
[0085] Partial sharing between quantized learning step parameters refers to partial sharing between quantized learning step parameters of different bits between different mixed precision subnetworks.
[0086] The disclosed example proposes a method for partially sharing quantization learning step parameters during quantization-aware training. The method for partially sharing quantization learning step parameters is implemented by defining the relationship between quantization learning step parameters of different bits when performing fixed-point quantization on floating-point weights and activation values. Compared with the method in which quantization learning step parameters are not shared at all, in which each quantization bit configuration has its own independent parameters, this method is less likely to produce underfitting. Compared with the method in which quantization learning step parameters are fully shared, in which each quantization bit configuration shares the same parameters, this method can better alleviate gradient conflicts.
[0087] Step 204, using a preset search strategy to search for a subnet that meets the search conditions in the second mixed precision supernet to obtain a target network; wherein the target network is a network that performs a target task; and the preset search strategy limits the correlation between the weight distribution range and the sampling probability.
[0088] The preset search strategies include:
[0089] If the floating point weight distribution range of the j+1th layer of the subnet is wider than that of the jth layer, a larger sampling probability is set for the high bits of the j+1th layer than for the low bits;
[0090] or,
[0091] If the floating point weight distribution range of the j+1th layer of the subnet is narrower than that of the jth layer, a smaller sampling probability is set for the high bits of the j+1th layer than for the low bits;
[0092] or,
[0093] If the floating point weight distribution range of the j+1th layer of the subnet is equal to the floating point weight distribution range of the jth layer, the same sampling probability as the low bit is set for the high bit of the j+1th layer.
[0094] In the disclosed embodiment, before the target network is acquired through search, when the first mixed precision supernet is subjected to sensitive quantization training, the quantization training is performed in a manner of partial sharing between the quantization learning step parameters, which can improve the accuracy of the mixed precision supernet after quantization training, and thus obtain a high-precision target network; and when searching for the target network, the relationship between the weight distribution range and the sampling probability is considered, and a quantization-sensitive sampling strategy is used, which can improve the speed of searching the target network.
[0095] The embodiment of the present disclosure further provides a network deployment method based on the method for acquiring a target network. Figure 3 is a flow chart of a network deployment method according to an exemplary embodiment. Figure 3 As shown, the network deployment method of the embodiment of the present disclosure mainly includes the following steps 301 to 306.
[0096] Step 301, obtaining network construction information and target operating environment information corresponding to the target task; wherein the network construction information includes: search space information, sample data and search conditions.
[0097] In the disclosed embodiment, the task that the user needs to perform may be referred to as a target task, such as image recognition, image classification, image segmentation, etc., but not limited thereto. The network that the user needs to construct to perform the target task is referred to as a target network.
[0098] In one example, the target network may be a neural network.
[0099] The network construction information includes: search space information, sample data and search conditions.
[0100] The search space information may include a variety of information used to construct a mixed precision supernet, such as the definition of input data of the mixed precision supernet, the definition of output data, the definition of the loss function, the number of modules constituting the mixed precision supernet, the branch construction form of the branches in each module of the mixed precision supernet, and the bit parameter of each layer, etc.; the embodiment of the present invention is not limited to this;
[0101] The sample data is used to train the mixed precision supernet;
[0102] The search conditions include performance indicators of the target network, etc., which are used to search the target network from the mixed precision super network.
[0103] The target operating environment information is the environment information of the target network corresponding to the required deployment environment, and may include: hardware operating environment information and / or software operating environment information.
[0104] Step 302: construct a first mixed precision supernet according to the search space information.
[0105] In the relevant implementation, the constructed first mixed precision supernet is a floating point network, and the first mixed precision supernet can be trained as a floating point network, and the trained first mixed precision supernet is then quantized, that is, step 303 is executed. The embodiment of the present disclosure does not limit the training implementation of the first mixed precision supernet.
[0106] Step 303, based on the sample data, quantization-aware training is performed on the first mixed precision supernet in a preset manner to obtain a second mixed precision supernet; wherein the preset manner is a manner of partially sharing quantization learning step parameters, and the manner of partially sharing quantization learning step parameters is achieved by defining the relationship between quantization learning step parameters of different bits when bit-specific quantization is performed on floating-point weights and activation values.
[0107] The disclosed example proposes a method for partially sharing quantization learning step parameters during quantization-aware training. The method for partially sharing quantization learning step parameters is implemented by defining the relationship between quantization learning step parameters of different bits when performing fixed-point quantization on floating-point weights and activation values. Compared with the method in which weights are not completely shared between quantization learning step parameters, in which each quantization bit configuration has its own independent parameters, this method is less likely to produce underfitting, and compared with the method in which each quantization bit configuration shares the same parameters in which quantization learning step parameters are completely shared, this method can better alleviate gradient conflicts.
[0108] Among them, the relationship between the quantization learning step parameters of different bits is:
[0109]
[0110] And meet the following conditions:
[0111] Among them, s k+m represents the k+m bit quantization learning step parameter, s k represents the k-bit quantization learning step parameter, α k+m Represents a learnable bias parameter, M is the percentage threshold of the increase in the floating-point value range after k+m-bit quantization compared to the floating-point value range after k-bit quantization, and m is an integer greater than 0.
[0112] The relationship between the quantization learning step parameters of different bits set in the embodiment of the present disclosure is reasonable and meets the requirements of sensitive quantization training. The specific process is as follows:
[0113] Assume that the entire search space is A, A=[B 1 , B 2 , ..., B j ], the floating point weights of the mixed precision supernet are W, and the mixed precision supernet is N.
[0114] In each iteration, n different quantization configuration sequences are sampled from the mixed precision supernet. For example, n can be 4, but is not limited to this. A′ 1 , A′ 2 , ..., A′ n , apply each quantization configuration to the mixed precision supernet, adjust the quantization bits of the pseudo-quantization nodes, perform the forward and reverse processes on the supernets under different quantization configurations to obtain the accumulated gradients, and then update the parameters based on the accumulated gradients. The loss function L of the supernet in each iteration is determined as follows:
[0115]
[0116] Where: A′ is any quantized configuration in the quantized configuration sequence; A is the quantized configuration search space in the super network; D trn is the training data, i.e. the sample data used for training; L trn (A′; W, N, D trn ) is the loss function of the sub-network; γ is the weight decay factor; ACT is the activation value of the super-network; R(W) is the second normal form of W.
[0117] The settings for the quantization of floating-point weights and activation values are the same, quantizing the floating-point weights in W and the activation values in ACT. The following takes the quantization of the floating-point weight x as an example:
[0118] The floating point weight x is quantized by the following function Qk (x) Perform k-bit specific point quantization:
[0119]
[0120] Among them, s k , z k is the quantitative learning step parameter to be learned, P k is the upper boundary of the quantized fixed-point value; N k is the lower boundary of the quantized fixed-point value, then the quantized floating-point value range can be expressed as:
[0121] R k =s k ×|P k -N k |=s k ×2 k (2)
[0122] Since the floating-point weights of different bits before quantization are the same, the floating-point value range before quantization is also the same; therefore, assuming that the floating-point value range after quantization is also the same, it can be specifically expressed as: R k =R k+1 , after expanding and simplifying, we get:
[0123]
[0124] The above sharing method can achieve the cooperation between different bits (sharing quantization parameters), but there are also some problems. k =R k+1 This limits the expressive capacity of higher precision bits; for example, after taking clip error and round error into account, higher precision bits may require higher R k Therefore, we hope that different bits can work together and maintain a certain degree of freedom. In order to obtain better expression capacity and alleviate the gradient conflict caused by different expression capacities, the following sharing method is proposed:
[0125]
[0126] In the above formula, α k+1 The additional learnable bias parameter is introduced, which enables different bits to work together without causing gradient conflicts. Furthermore, in the actual training process, since the required bits are not necessarily completely adjacent, the relationship between the two parameters with an interval of m bits is defined as:
[0127]
[0128] Assume that the free amount M of the floating-point value range after quantization between different bits, that is, the percentage threshold of the floating-point value range after k+m bits quantization is increased compared to the floating-point value range after k bits quantization, which can be set to 0.3 in a specific implementation, and is not limited in the embodiments of the present application;
[0129] Then R k+m ≤(1+M)R k ; Combine this formula and substitute formula (5) into formula (2), and after simplification, we get:
[0130]
[0131] At this point, the relationship formula (5) between the quantization learning step parameters of different bits and the constraint condition formula (6) are obtained.
[0132] Figure 4 Schematic diagram of partial sharing between quantization learning step parameters of different bits in the disclosed example. Figure 4 In the example, the quantization learning step size parameters of three different bits are partially shared, and the quantization learning step size parameters corresponding to the blank parts of k1 bit, k2 bit and k3 bit are shared; assuming that s k represents the quantization learning step size parameter corresponding to k1 bits, then the quantization learning step size parameter corresponding to k2 bits can be expressed as: The quantization learning step size corresponding to k3 bits can be expressed as: It can be seen that in s k When s changes k+1 and k+2 will also change accordingly; this makes it less likely to produce underfitting. At the same time, in order to better alleviate the gradient conflict caused by the expression capacity between the quantization learning step size parameters corresponding to different bits, the corresponding bias parameter α is introduced k+2 and α k+1 .
[0133] In the same implementation, the quantization learning step size parameters represented by the vertical line parts corresponding to the k2 bits and the k3 bits are shared;
[0134] The quantization learning step size parameter represented by the horizontal line portion corresponding to the k3 bit is not shared with the quantization learning step size parameters corresponding to other bits;
[0135] This sharing method can well adapt to the specificity of high-precision hybrid supernets, and can obtain a quantized mixed-precision supernet with higher accuracy after quantization training. For the convenience of identification, the arrow part only identifies the blank part corresponding to k1 bits, k2 bits and k3 bits, which represents the sharing of quantization learning steps.
[0136] Step 304, based on the target operating environment information, a preset search strategy is used to search for a subnet that meets the search conditions in the second mixed precision supernet to obtain a target network; wherein the target network is a network that performs the target task; and the preset search strategy limits the correlation between the weight distribution range and the sampling probability.
[0137] In step 303, a second mixed precision supernet after quantization training is obtained, and a subnet A that meets the condition can be searched based on the second mixed precision supernet and the search condition. * The subnet that meets the conditions is the target network to be obtained in the embodiment of the present disclosure, subnet A * It can be expressed by the following formula:
[0138]
[0139] Where: MQ is the quantitative strategy; ∈ is the constraint (such as the size of the model running, the time of the model running, etc.); Accuracy is the accuracy function; D val is the verification data; A is the quantization configuration search space in the hypernetwork; A′ is any quantization configuration in the quantization configuration search space; W is the weight of the mixed precision hypernetwork; N is the mixed precision hypernetwork; Cons(N(A′,W)) is the constraint term on A′ and W.
[0140] In the above-mentioned related technologies, genetic algorithms are usually used for iterative optimization. Assume that P j is the sampling distribution of the j-th layer mixed bit, the traditional genetic algorithm has P 1 =P 2 =P j , That is, all layers share the same sampling distribution, and each bit has an equal probability of being sampled.
[0141] However, the traditional genetic algorithm does not take into account the inherent characteristics of quantization: weights with a wider numerical distribution range require more bits to obtain higher precision when quantized than weights with a narrower numerical distribution. In the embodiment of the present disclosure, the relationship between the floating-point weight distribution range and the sampling probability is specifically considered as follows:
[0142] If the floating point weight distribution range of the j+1th layer is wider than that of the jth layer, a larger sampling probability is set for the high bits of the j+1th layer than for the low bits; for example, the following formula can be used to express it:
[0143]
[0144] Among them, |W j+1 | represents the floating point weight distribution range of the j+1th layer of the corresponding subnet, |W j | represents the distribution range of floating-point weights of the jth layer of the corresponding subnet; represents the sampling probability of k bits of the j+1th layer of the corresponding subnet; represents the sampling probability of the k+1 bit of the j+1th layer of the corresponding subnet;
[0145] or,
[0146] If the floating point weight distribution range of the j+1th layer is narrower than that of the jth layer, then a smaller sampling probability is set for the high bits of the j+1th layer than for the low bits; for example, the following formula can be used to express it:
[0147]
[0148] or,
[0149] If the floating point weight distribution range of the j+1th layer is equal to the floating point weight distribution range of the jth layer, the same sampling probability is set for the high bit of the j+1th layer as for the low bit; for example, it can be expressed by the following formula:
[0150]
[0151] In a specific implementation, a genetic algorithm may be used to perform an optimization search based on the search strategy, or a simulated annealing method may be used to perform an optimization search based on the strategy, etc. This is not limited in the embodiments of the present disclosure.
[0152] The above search strategy can sample the target network with higher accuracy at a faster speed.
[0153] The subnet searched out in step 304 is a directly quantized subnet, which does not need to be retrained and can be directly used as a target network for deployment.
[0154] Step 305: deploy the obtained target network based on the target operating environment information.
[0155] The target network is deployed to a platform having a target operating environment corresponding to the target operating environment information, such as a mobile terminal, a CPU (Central Processing Unit) or a GPU (Graphics Processing Unit).
[0156] Step 306: Use the deployed target network to execute the target task.
[0157] Execute the target task on the platform where the target network is deployed. For example, input the image into the target network of the platform, and the target network performs image recognition, image classification, image segmentation, etc.
[0158] In the disclosed embodiment, a multi-bit collaborative parameter sharing method is adopted when quantizing a mixed precision network. Compared with the sharing method in the related technical solution, it is less likely to produce underfitting and can better alleviate the gradient conflict caused by the expression capacity between different bits; when searching for the target network, the correlation between the floating point weight distribution range and the sampling probability is considered, and a quantization-sensitive sampling strategy is proposed, which can sample the target network with higher accuracy that meets the search conditions at a faster speed; and then the obtained network is deployed to the corresponding environment platform to perform the target task. This solution can adapt to the operating environment and tasks while improving the accuracy of the mixed precision super network, and quickly build a network.
[0159] The following takes image recognition as an example to illustrate the process of obtaining the target network and using it for image recognition:
[0160] The first step is to obtain network construction information and target operating environment information corresponding to the image recognition task to be executed; wherein the network construction information includes: search space information, sample data and search conditions.
[0161] The sample data is used to train the mixed precision supernet; it can be currently collected image data or pre-collected and stored image data.
[0162] The second step is to construct the first mixed precision supernet based on the search space information.
[0163] When building, you can use resnet or mobilenet.
[0164] Here, sample data is usually used to train the floating-point network of the first mixed precision supernet.
[0165] Step 3: Based on the sample data, the first mixed precision supernet is trained with quantization perception by partially sharing the quantization learning step size parameters to obtain a second mixed precision supernet.
[0166] The partial sharing of quantization learning step parameters is achieved by defining the relationship between quantization learning step parameters of different bits when performing bit-by-bit quantization on floating-point weights and activation values;
[0167] The relationship between the quantization learning step parameters of different bits is specifically: there is an exponential correlation change, which makes it less likely to produce underfitting. At the same time, in order to better alleviate the gradient conflict caused by the expression capacity between the quantization learning step parameters corresponding to different bits, the corresponding bias parameters are introduced.
[0168] The improved quantization-aware training method provided in the present application can obtain a second mixed-precision supernet with higher accuracy, thereby laying a foundation for obtaining a high-precision target network. The high accuracy of the target network can enable more accurate recognition when performing image recognition.
[0169] In the fourth step, based on the target operating environment information, a preset search strategy that limits the correlation between the weight distribution range and the sampling probability is used to search for a subnet that meets the search conditions in the second mixed precision supernet to obtain the target network.
[0170] The relationship between the specific limited weight distribution range and sampling probability is:
[0171] This limited search strategy enables us to sample the optimal subnet that meets the conditions at a faster speed as the target network we need.
[0172] Step 5: Deploy the target network based on the target operating environment information.
[0173] In the sixth step, image recognition is performed using the deployed target network.
[0174] Based on the given target task, as well as the limited conditions and environmental information, the disclosed embodiment can automatically acquire and deploy a high-precision target network to execute the target task, and through an improved search method, the search speed is accelerated, the speed of acquiring the target network is increased, and the efficiency of the entire process from proposing the target task to executing the target task is improved.
[0175] Figure 5 is a schematic diagram of a target network acquisition device according to an exemplary embodiment. Figure 5 As shown, the target network acquisition device mainly includes:
[0176] The acquisition unit 501 is configured to acquire network construction information corresponding to the target task; wherein the network construction information includes: search space information, sample data and search conditions;
[0177] A construction unit 502 is configured to construct a first mixed precision supernet according to the search space information;
[0178] The quantization training unit 503 is configured to perform quantization-aware training on the first mixed precision supernet in a preset manner based on the sample data to obtain a second mixed precision supernet; wherein the preset manner is a partial sharing manner between quantization learning step parameters, and the partial sharing manner between quantization learning step parameters is achieved by defining a relationship between quantization learning step parameters of different bits when performing bit-specific quantization on floating-point weights and activation values;
[0179] The search unit 504 is configured to search the second mixed precision supernet for a subnet that meets the search condition to obtain a target network; wherein the target network is a network that performs the target task.
[0180] In some embodiments,
[0181] The search unit 504 is configured to perform a search using a preset search strategy when searching for a subnet that meets the search condition in the second mixed precision supernet; the preset search strategy defines a correlation between a weight distribution range and a sampling probability.
[0182] In some embodiments,
[0183] The preset search strategy includes:
[0184] If the floating point weight distribution range of the j+1th layer of the subnet is wider than the floating point weight distribution range of the jth layer, a sampling probability greater than that of the low bits is set for the high bits of the j+1th layer;
[0185] or,
[0186] If the floating point weight distribution range of the j+1th layer is narrower than that of the jth layer, a sampling probability smaller than that of the low bit is set for the high bit of the j+1th layer;
[0187] or,
[0188] If the floating point weight distribution range of the j+1th layer of the subnet is equal to the floating point weight distribution range of the jth layer, the same sampling probability as the low bit is set for the high bit of the j+1th layer.
[0189] In some embodiments,
[0190] An acquisition unit 501 is configured to acquire target operating environment information corresponding to the target network;
[0191] The search unit 504 is configured to search for a subnet that meets the search condition in the second mixed precision supernet based on the target operating environment information.
[0192] In some embodiments, the apparatus further comprises:
[0193] A deployment unit 505 is configured to execute deployment of the obtained target network in an environment corresponding to the target operating environment information;
[0194] The execution unit 506 is configured to execute the target task using the deployed target network.
[0195] Regarding the target network acquisition device in each of the above embodiments, the specific manner in which each unit performs the operation has been described in detail in the embodiment of the target network acquisition method, and will not be elaborated here.
[0196] It should be noted that: the above embodiment only illustrates the division of the above functional modules. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.
[0197] Figure 6 It is a structural diagram of an electronic device provided by an embodiment of the present disclosure. In some embodiments, the electronic device is a server. The electronic device 600 may have relatively large differences due to different configurations or performances, and may include one or more processors (Central Processing Units, CPU) 601 and one or more memories 602, wherein at least one program code is stored in the memory 602, and the at least one program code is loaded and executed by the processor 601 to implement the multi-person motion recognition method provided by the above-mentioned various embodiments. Of course, the electronic device 600 may also have components such as image acquisition components, wired or wireless network interfaces, keyboards, and input and output interfaces for input and output. The electronic device 600 may also include other components for implementing device functions, which will not be repeated here.
[0198] In an exemplary embodiment, a computer-readable storage medium including at least one instruction is also provided, such as a memory including at least one instruction. The at least one instruction can be executed by a processor in a computer device to complete the target network acquisition method in the above embodiment.
[0199] Optionally, the above-mentioned computer-readable storage medium may be a non-temporary computer-readable storage medium, for example, the non-temporary computer-readable storage medium may include ROM (Read-Only Memory), RAM (Random-Access Memory), CD-ROM (Compact Disc Read-Only Memory), magnetic tape, floppy disk and optical data storage device, etc.
[0200] In an exemplary embodiment, a computer program product is also provided, including one or more instructions, which can be executed by a processor of a computer device to complete the target network acquisition method provided by each of the above embodiments.
[0201] The above description is only a preferred embodiment of the present disclosure and is not intended to limit the present disclosure. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present disclosure shall be included in the scope of protection of the present disclosure.
Claims
1. A method for acquiring a target network, characterized in that: The method comprises: Acquire network construction information corresponding to the target task; wherein the network construction information includes: search space information, sample data and search conditions; Constructing a first mixed precision supernet according to the search space information; Based on the sample data, the first mixed precision supernet is trained with quantization perception in a preset manner to obtain a second mixed precision supernet; wherein the preset manner is a manner of partial sharing between quantization learning step parameters, and the partial sharing between quantization learning step parameters is achieved by defining the relationship between quantization learning step parameters of different bits when performing bit-specific quantization on floating-point weights and activation values; Search the second mixed precision supernet for a subnet that meets the search condition to obtain a target network; wherein the target network is a network that performs the target task.
2. The method according to claim 1, characterized in that The relationship between the quantization learning step parameters of different bits is: And meet the following conditions: Among them, s k+m represents the k+m bit quantization learning step parameter, s k represents the k-bit quantization learning step parameter, α k+m Represents a learnable bias parameter, M is the percentage threshold of the increase in the floating-point value range after k+m-bit quantization compared to the floating-point value range after k-bit quantization, and m is an integer greater than 0.
3. The method according to claim 1, characterized in that When searching for a subnet that meets the search condition in the second mixed-precision supernet, a preset search strategy is used to search; the preset search strategy defines the correlation between the weight distribution range and the sampling probability.
4. The method according to claim 3, characterized in that The preset search strategy includes: If the floating point weight distribution range of the j+1th layer of the subnet is wider than the floating point weight distribution range of the jth layer, a sampling probability greater than that of the low bits is set for the high bits of the j+1th layer; or, If the floating point weight distribution range of the j+1th layer is narrower than that of the jth layer, a sampling probability smaller than that of the low bit is set for the high bit of the j+1th layer; or, If the floating point weight distribution range of the j+1th layer of the subnet is equal to the floating point weight distribution range of the jth layer, the same sampling probability as the low bit is set for the high bit of the j+1th layer.
5. The method according to any one of claims 1 to 4, characterized in that: The method further comprises: Obtaining target operating environment information corresponding to the target network; The searching the second mixed-precision supernet for a subnet that meets the search condition includes: Based on the target operating environment information, the second mixed precision supernet is searched for a subnet that meets the search condition.
6. The method according to claim 5, characterized in that The method further comprises: Deploy the obtained target network in an environment corresponding to the target operating environment information; The target task is performed using the deployed target network.
7. A target network acquisition device, characterized in that: The device comprises: An acquisition unit is configured to acquire network construction information corresponding to a target task; wherein the network construction information includes: search space information, sample data and search conditions; A construction unit, configured to construct a first mixed precision supernet according to the search space information; A quantization training unit is configured to perform quantization-aware training on the first mixed precision supernet in a preset manner based on the sample data to obtain a second mixed precision supernet; wherein the preset manner is a partial sharing manner between quantization learning step parameters, and the partial sharing manner between quantization learning step parameters is achieved by defining a relationship between quantization learning step parameters of different bits when performing bit-specific quantization on floating-point weights and activation values; The search unit is configured to search the second mixed precision supernet for a subnet that meets the search condition to obtain a target network; wherein the target network is a network that performs the target task.
8. The device according to claim 7, characterized in that The search unit is configured to use a preset search strategy to perform a search when searching for a subnet that meets the search condition in the second mixed precision supernet; the preset search strategy defines a correlation between a weight distribution range and a sampling probability.
9. An electronic device, characterized in that: include: processor; a memory for storing executable instructions for the processor; Wherein, the processor is configured to execute the executable instructions to implement the multi-person action recognition method as described in any one of claims 1 to 6.
10. A computer-readable storage medium, characterized in that: When at least one instruction in the computer-readable storage medium is executed by a processor of an electronic device, the electronic device is enabled to implement the multi-person motion recognition method as described in any one of claims 1 to 6.