Image processing method and device, electronic equipment, medium and program product

Abstract

Embodiments of the present disclosure disclose an image processing method and device, electronic equipment, medium and program product. The image processing method comprises: obtaining the quantization precision sensitivity of a plurality of layers in a multi-layer neural network used for image processing; generating an initial population of an evolutionary algorithm based on the quantization precision sensitivity of the layers and random numbers generated for the layers, each member in the initial population corresponding to a corresponding quantization precision configuration of the multi-layer neural network; using a set of evaluation images to evaluate the performance of the members in the initial population, updating the population based on the evaluation results until a member in the population whose performance meets a preset condition appears; and determining the quantization precision of the plurality of layers in the multi-layer neural network according to the member whose performance meets the preset condition.

Description

Technical Field

[0001] The present disclosure relates to the field of image processing technology, and in particular to an image processing method, device, electronic device, medium, and program product. Background Art

[0002] In cloud computing environments such as public clouds, when performing image processing, the proportion of inference requirements for neural network models is increasing, and there is a need to optimize the performance of inference tasks. When the neural network model is quantized into a 16-bit floating point (FP16) model, there is basically no loss of computational accuracy. The neural network model can also be quantized into 8-bit integer (INT8) type operations. The nominal computing power of INT8 is twice that of FP16, but directly quantizing the entire model to INT8 will result in unacceptable precision loss. Therefore, when the FP16 quantization accuracy does not meet the speed requirements, hybrid quantization is required to further improve the performance of neural network model inference while ensuring accuracy, and maximize the use of the computing power of the target hardware. When solving the hybrid quantization optimization solution, the computing resources required to traverse and select the optimization solution are huge, time-consuming, and inefficient, which is difficult to meet actual needs. A method for quickly solving the hybrid quantization optimization solution is needed. Summary of the Invention

[0003] In order to solve the problems in the related art, the embodiments of the present disclosure provide an image processing method, apparatus, electronic device, medium, and program product.

[0004] In a first aspect, an image processing method is provided in an embodiment of the present disclosure.

[0005] Specifically, the image processing method includes:

[0006] Obtaining the quantization accuracy sensitivity of multiple layers in a multi-layer neural network for image processing;

[0007] generating an initial population of an evolutionary algorithm based on the quantization accuracy sensitivity of the layer and a random number generated for the layer, wherein each member of the initial population corresponds to a quantization accuracy configuration corresponding to the multi-layer neural network;

[0008] Using the evaluation image set, evaluating the performance of members in the initial population, and updating the population based on the evaluation results until a member whose performance meets the preset conditions appears in the population;

[0009] The quantization accuracy of multiple layers in the multi-layer neural network is determined according to the members whose performance meets the preset conditions.

[0010] In combination with the first aspect, in a first implementation manner of the first aspect of the present disclosure, the quantization accuracy sensitivity of the layer represents the degree of sensitivity of the performance of the multi-layer neural network to changes in the quantization accuracy of the layer.

[0011] In combination with the first aspect, in a second implementation of the first aspect of the present disclosure, generating an initial population of the evolutionary algorithm based on the quantization accuracy sensitivity of the layer and the random number generated for the layer includes determining the quantization accuracy configuration corresponding to the members in the initial population through the following steps:

[0012] generating a set of random numbers for the member, each random number in the set of random numbers corresponding to a respective one of the plurality of layers;

[0013] generating, for each layer of the plurality of layers, a probability factor for the layer based on a quantization precision sensitivity of the layer and a corresponding random number;

[0014] The quantization precision configuration corresponding to the member is determined based on the probability factor of each layer in the multiple layers.

[0015] In combination with the second implementation of the first aspect, in a third implementation of the first aspect of the present disclosure, using the evaluation image set to evaluate the performance of members of the initial population, and updating the population based on the evaluation results, includes, for the current population:

[0016] determining the performance of the member according to the performance of the multi-layer neural network having the quantization accuracy configuration corresponding to the member in the population when processing the evaluation image set;

[0017] The quantization accuracy configuration corresponding to the members with good performance is used to update the quantization accuracy configuration corresponding to the members with poor performance to obtain an updated population.

[0018] In combination with the third implementation manner of the first aspect, in a fourth implementation manner of the first aspect of the present disclosure, using the quantization precision configuration corresponding to the well-performing member to update the quantization precision configuration of the poorly-performing member includes:

[0019] For a high-precision layer included in the quantization precision configuration corresponding to the good-performing member but not included in the quantization precision configuration corresponding to the poor-performing member, determining, based on the quantization precision sensitivity of the high-precision layer and a random number generated for the high-precision layer, a designated low-precision layer in the quantization precision configuration corresponding to the poor-performing member to be modified to a high-precision layer, and modifying the designated low-precision layer in the quantization precision configuration corresponding to the poor-performing member to a high-precision layer;

[0020] For the high-precision layer in the modified quantization precision configuration of the poorly performing member, based on the quantization sensitivity of the high-precision layer and the random number generated for the high-precision layer, determine the designated high-precision layer to be modified to the low-precision layer in the modified quantization precision configuration of the poorly performing member, and modify the designated high-precision layer in the modified quantization precision configuration of the poorly performing member to a low-precision layer to update the quantization precision configuration corresponding to the poorly performing member.

[0021] In combination with the first aspect, the present disclosure further includes, in a fifth implementation of the first aspect:

[0022] configuring the multi-layer neural network based on quantization accuracy of multiple layers in the multi-layer neural network;

[0023] Use a configured multi-layer neural network for image processing.

[0024] In combination with the fifth implementation manner of the first aspect, in a sixth implementation manner of the first aspect of the present disclosure, the image processing includes any one or more of the following: image classification, target detection, environment segmentation, and face recognition.

[0025] In a second aspect, an image processing device is provided in an embodiment of the present disclosure.

[0026] Specifically, the image processing device includes:

[0027] A quantization accuracy sensitivity acquisition module, used to acquire the quantization accuracy sensitivity of multiple layers in a multi-layer neural network used for image processing;

[0028] An initial population generation module is used to generate an initial population of the evolutionary algorithm based on the quantization accuracy sensitivity of the layer and the random number generated for the layer, wherein each member of the initial population corresponds to a quantization accuracy configuration corresponding to the multi-layer neural network;

[0029] a population updating module, configured to evaluate the performance of members in the initial population using a set of evaluation images, and to update the population based on the evaluation results until a member whose performance meets a preset condition appears in the population;

[0030] The quantization accuracy configuration determination module is used to determine the quantization accuracy of multiple layers in the multi-layer neural network based on the members whose performance meets the preset conditions.

[0031] In combination with the second aspect, in a first implementation manner of the second aspect of the present disclosure, the quantization accuracy sensitivity of the layer represents the degree of sensitivity of the performance of the multi-layer neural network to changes in the quantization accuracy of the layer.

[0032] In conjunction with the second aspect, in a second implementation of the second aspect of the present disclosure, generating an initial population of the evolutionary algorithm based on the quantization accuracy sensitivity of the layer and the random number generated for the layer includes determining the quantization accuracy configuration corresponding to the members in the initial population through the following steps:

[0033] generating a set of random numbers for the member, each random number in the set of random numbers corresponding to a respective one of the plurality of layers;

[0034] generating, for each layer of the plurality of layers, a probability factor for the layer based on a quantization precision sensitivity of the layer and a corresponding random number;

[0035] The quantization precision configuration corresponding to the member is determined based on the probability factor of each layer in the multiple layers.

[0036] In conjunction with the second implementation of the second aspect, in a third implementation of the second aspect of the present disclosure, using the evaluation image set to evaluate the performance of members of the initial population, and updating the population based on the evaluation results, includes, for the current population:

[0037] determining the performance of the member according to the performance of the multi-layer neural network having the quantization accuracy configuration corresponding to the member in the population when processing the evaluation image set;

[0038] The quantization accuracy configuration corresponding to the well-performing members is used to update the quantization accuracy configuration of the poorly performing members to obtain an updated population.

[0039] In combination with the third implementation manner of the second aspect, in a fourth implementation manner of the second aspect of the present disclosure, using the quantization precision configuration corresponding to the good-performing member to update the quantization precision configuration of the poor-performing member includes:

[0040] For a high-precision layer included in the quantization precision configuration corresponding to the good-performing member but not included in the quantization precision configuration corresponding to the poor-performing member, determining, based on the quantization precision sensitivity of the high-precision layer and a random number generated for the high-precision layer, a designated low-precision layer in the quantization precision configuration corresponding to the poor-performing member to be modified to a high-precision layer, and modifying the designated low-precision layer in the quantization precision configuration corresponding to the poor-performing member to a high-precision layer;

[0041] For the high-precision layer in the modified quantization precision configuration of the poorly performing member, based on the quantization sensitivity of the high-precision layer and the random number generated for the high-precision layer, determine the designated high-precision layer to be modified to the low-precision layer in the modified quantization precision configuration of the poorly performing member, and modify the designated high-precision layer in the modified quantization precision configuration of the poorly performing member to a low-precision layer to update the quantization precision configuration of the poorly performing member.

[0042] In conjunction with the second aspect, the present disclosure further includes, in a fifth implementation of the second aspect:

[0043] A multi-layer neural network configuration module, configured to configure the multi-layer neural network based on the quantization accuracy of multiple layers in the multi-layer neural network;

[0044] The image processing module is used to perform image processing using a configured multi-layer neural network.

[0045] In combination with the fifth implementation manner of the second aspect, in a sixth implementation manner of the second aspect of the present disclosure, the image processing includes any one or more of the following: image classification, target detection, environment segmentation, and face recognition.

[0046] In a third aspect, an embodiment of the present disclosure provides an electronic device comprising a memory and a processor, wherein the memory is used to store one or more computer instructions, and wherein the one or more computer instructions are executed by the processor to implement a method as described in any one of the first aspect to the sixth implementation manner of the first aspect.

[0047] In a fourth aspect, an embodiment of the present disclosure provides a computer-readable storage medium on which computer instructions are stored. When the computer instructions are executed by a processor, the method described in the first aspect to the sixth implementation method of the first aspect is implemented.

[0048] In a fifth aspect, an embodiment of the present disclosure provides a computer program product, comprising computer instructions, which, when executed by a processor, implement the method steps described in the first aspect to the sixth implementation of the first aspect.

[0049] According to the technical solution provided by the embodiments of the present disclosure, the quantization accuracy sensitivity of multiple layers in a multi-layer neural network used for image processing is obtained; based on the quantization accuracy sensitivity of the layer and the random number generated for the layer, an initial population of the evolutionary algorithm is generated, and each member of the initial population corresponds to a corresponding quantization accuracy configuration of the multi-layer neural network; using a set of evaluation images, the performance of the members in the initial population is evaluated, and the population is updated based on the evaluation results until a member whose performance meets the preset conditions appears in the population; based on the members whose performance meets the preset conditions, the quantization accuracy of multiple layers in the multi-layer neural network is determined, so that the quantization method of each layer in the multi-layer neural network is obtained at a faster speed, and the calculation speed of the multi-layer neural network is improved under the condition of acceptable accuracy loss.

[0050] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Other features, objectives and advantages of the present disclosure will become more apparent through the following detailed description of non-limiting embodiments in conjunction with the accompanying drawings. In the accompanying drawings:

[0052] Figure 1a An exemplary schematic diagram showing a multi-layer neural network according to an embodiment of the present disclosure;

[0053] Figure 1b An exemplary schematic diagram showing differently behaved members of a population according to an embodiment of the present disclosure;

[0054] Figure 1c An exemplary schematic diagram illustrating a configuration for modifying the quantization accuracy of a poorly performing member according to an embodiment of the present disclosure;

[0055] Figure 1d An exemplary schematic diagram illustrating further modifying the quantization precision configuration of a member after the quantization precision configuration has been modified according to an embodiment of the present disclosure is shown;

[0056] Figure 2 A flowchart showing an image processing method according to an embodiment of the present disclosure is shown;

[0057] Figure 3 A flowchart showing an image processing method according to an embodiment of the present disclosure is shown;

[0058] Figure 4 A flowchart showing an image processing method according to an embodiment of the present disclosure is shown;

[0059] Figure 5 A flowchart showing an image processing method according to an embodiment of the present disclosure is shown;

[0060] Figure 6A flowchart showing an image processing method according to an embodiment of the present disclosure is shown;

[0061] Figure 7 A structural block diagram of an image processing apparatus according to an embodiment of the present disclosure is shown;

[0062] Figure 8 A structural block diagram of an electronic device according to an embodiment of the present disclosure is shown;

[0063] Figure 9 A schematic diagram showing the structure of a computer system suitable for implementing the method according to an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0064] Hereinafter, exemplary embodiments of the present disclosure will be described in detail with reference to the accompanying drawings so that those skilled in the art can easily implement them. In addition, for the sake of clarity, parts not related to the description of the exemplary embodiments are omitted in the accompanying drawings.

[0065] In the present disclosure, it should be understood that terms such as "include" or "have" are intended to indicate the presence of features, numbers, steps, actions, components, parts, or combinations thereof disclosed in the present specification, and are not intended to exclude the possibility that one or more other features, numbers, steps, actions, components, parts, or combinations thereof exist or are added.

[0066] It should also be noted that, in the absence of conflict, the embodiments and features of the embodiments of the present disclosure may be combined with each other. The present disclosure will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0067] In this disclosure, the acquisition of user information or user data is an operation authorized and confirmed by the user, or actively selected by the user.

[0068] As mentioned above, in cloud computing environments such as public clouds, when performing image processing, the proportion of inference requirements of neural network models is increasing, and there is a need to optimize the performance of inference tasks. When the neural network model is quantized into a 16-bit floating point (FP16) model, there is basically no loss of computational accuracy. The neural network model can also be quantized into 8-bit integer (INT8) type operations. The nominal computing power of INT8 is twice that of FP16, but directly quantizing the entire model to INT8 will result in unacceptable precision loss. Therefore, when the FP16 quantization accuracy does not meet the speed requirements, hybrid quantization is required to further improve the performance of neural network model inference while ensuring accuracy, and maximize the use of the computing power of the target hardware. When solving the hybrid quantization optimization solution, the computing resources required to traverse and select the optimization solution are huge, time-consuming, and inefficient, which is difficult to meet actual needs. It is necessary to obtain a method for quickly solving the hybrid quantization optimization solution.

[0069] In order to solve the above problems, the present disclosure proposes an image processing method, apparatus, electronic device, medium and program product.

[0070] According to an embodiment of the present disclosure, an image processing method is used, including: obtaining the quantization accuracy sensitivity of multiple layers in a multi-layer neural network used for image processing; generating an initial population of an evolutionary algorithm based on the quantization accuracy sensitivity of the layer and a random number generated for the layer, each member of the initial population corresponds to a quantization accuracy configuration corresponding to the multi-layer neural network; using an evaluation image set, evaluating the performance of the members in the initial population, and updating the population based on the evaluation results until a member whose performance meets the preset conditions appears in the population; determining the quantization accuracy of multiple layers in the multi-layer neural network based on the members whose performance meets the preset conditions, thereby obtaining the quantization method of each layer in the multi-layer neural network at a faster speed, and improving the calculation speed of the multi-layer neural network under the condition of acceptable accuracy loss.

[0071] Figure 1a An exemplary schematic diagram of a multi-layer neural network according to an embodiment of the present disclosure is shown.

[0072] It can be understood by those skilled in the art that Figure 1a A multi-layer neural network is shown as an example, but does not constitute a limitation to the present disclosure.

[0073] like Figure 1a As shown, the multi-layer neural network 100 includes layers 101, 103, etc., a total of N layers, layer 101 includes nodes 102, layer 103 includes nodes 104, and the connection relationship between the layers is shown in FIG. Figure 1a Not shown.

[0074] In the embodiment of the present disclosure, for N layers such as layers 101, 103, the quantization accuracy sensitivity S1, S2, ..., S N The sensitivity of the quantization accuracy of a layer indicates the sensitivity of the performance of a multi-layer neural network (e.g., inference speed, precision loss, etc.) to the change of the quantization accuracy of the layer. For N layers such as layer 101, 103, etc., the random numbers R11, R12, ..., R1 of the layer are obtained by the random number generation algorithm. N , and calculate the probability factors of each layer separately

[0075] P1 i =S i *R1 i , i=1,2......N

[0076] In the embodiment of the present disclosure, the quantization accuracy configuration of each layer can be determined based on the probability factor of each layer. For example, when P1i When P1 is greater than or equal to a first specific threshold value, for example, 0.5, the i-th layer is quantized using FP16. i When the first specific threshold is less than 0.5, the i-th layer is quantized with INT8. Alternatively, in the N-th layer, P1 is selected. i The largest M1 layers are quantized using FP16, and the remaining layers are quantized using INT8. It is understood that although this disclosure uses FP16 as an example of high quantization precision and INT8 as an example of low quantization precision, other high quantization precision and / or low quantization precision can also be used according to actual needs.

[0077] Those skilled in the art will appreciate that the first specific threshold may also be other values ​​determined based on experience or actual needs, and this disclosure does not limit this.

[0078] When quantizing a layer, you can quantize the operations of the nodes in the layer, such as addition, multiplication, and ReLU operations, and you can also quantize the storage values ​​of the nodes.

[0079] In the embodiment of the present disclosure, the N quantization precision configurations of the above-mentioned N-layer neural network constitute an initial member. K groups of different random numbers R11, R12, ..., R1 are used. N , we can obtain K different quantization accuracy configurations of the N-layer neural network, that is, different initial members, and then obtain an initial population including K initial members.

[0080] Figure 1b An exemplary schematic diagram illustrating differently behaved members of a population according to an embodiment of the present disclosure.

[0081] It can be understood by those skilled in the art that Figure 1b The different representations of members of a population are shown as examples and are not intended to limit the present disclosure.

[0082] like Figure 1b As shown, the initial population 110 includes initial members 111-118.

[0083] It can be understood by those skilled in the art that Figure 1b Schematically, the initial population 110 includes eight initial members 111 to 118. The number of initial members may also be other numbers, which is not limited in the present disclosure.

[0084] Based on the evaluation image set, a multi-layer neural network 100 configured according to the quantized precision corresponding to members 111-118 is used to perform image processing such as object detection, environment segmentation, and face recognition. The performance of the multi-layer neural network 100, such as computational speed and precision loss, is then evaluated. From the initial population 110, T members are selected whose performance outperforms the other members. For example, the three members with the fastest inference speed among those with a precision loss of less than 1% are selected, such as members 111, 112, and 113. These members are considered to be the high-performing member set 119. The remaining members, excluding members 111, 112, and 113, constitute the low-performing member set.

[0085] In the set of members with good performance, a member with good performance is arbitrarily selected, such as 112; in the set of members with poor performance, a member with poor performance is arbitrarily selected, such as 115, and the quantization accuracy configuration of the poor performance member 115 is modified by the member with good performance 112.

[0086] Figure 1c An exemplary schematic diagram illustrating a configuration for modifying the quantization accuracy of a poorly performing member according to an embodiment of the present disclosure.

[0087] Figure 1c The layers of the neural network configured according to the member 112 with good performance and the member 115 with poor performance are shown, wherein the positions of the layers 121 to 127 ... configured according to the member 112 in the neural network correspond to the layers 131 to 137 ... configured according to the member 115. It can be understood that the actual neural network used can be different from the Figure 1c As shown, the number of layers may be the same or different. Figure 1c As shown, among the layers 121, 122, ..., 127, ... configured according to member 112, the shaded layers 121, 122, 123, 124, and 125 use FP16 quantization, while the other layers use INT8 quantization. Among the layers 131, 132, ..., 137, ... configured according to member 115, the shaded layers 131, 134, and 136 use FP16 quantization, while the other layers use INT8 quantization. The layers 122, 123, and 125 configured according to member 112 use high-precision FP16 quantization, while the corresponding layers 132, 133, and 135 configured according to member 115 use low-precision INT8 quantization.

[0088] For layers 122, 123, and 125, obtain the quantization accuracy sensitivity S2, S3, and S5 of the aforementioned layers, and use the random number generation algorithm to generate random numbers R22, R23, and R25, and calculate

[0089] P2 i =S i*R2 i , i=2,3,5

[0090] P22, P23, and P25 are obtained. Among P22, P23, and P25, a second specific threshold greater than, for example, 0.5 or a maximum of, for example, M1 = 2 layers is selected, and the corresponding two layers configured according to member 115 are FP16 quantized. For example, based on layers 122 and 125 configured according to member 112, the corresponding layers 132 and 135 configured according to member 115 are modified from INT8 quantization to FP16 quantization.

[0091] Figure 1d An exemplary schematic diagram is shown of modifying the quantization precision configuration of a member after the quantization precision configuration is modified according to an embodiment of the present disclosure.

[0092] like Figure 1d As shown, Figure 1c After the INT8 quantized layers 132 and 135 are modified to FP16 quantized layers 141 and 142, we get Figure 1d A multi-layer neural network configured according to member 140.

[0093] In the multi-layer neural network configured according to member 140, the layers 131, 141, 134, 142, and 136 use FP16 quantization, and the remaining layers use INT8 quantization. For layers 131, 141, 134, 142, and 136, obtain the quantization accuracy sensitivity S1, S2, S4, S5, and S6 of the aforementioned layers, and generate random numbers R31, R32, R34, R35, and R36 through the random number generation algorithm, and calculate

[0094] P3 i =S i *R3 i , i=1,2,4,5,6

[0095] P31, P32, P34, P35, and P36 are obtained. Among P31, P32, P34, P35, and P36, a third specific threshold value less than, for example, 0.5 or a minimum number of layers, for example, M2 = 2, is selected for INT8 quantization. For example, layers 131 and 142 configured according to member 140 are modified from FP16 quantization to INT8 quantization, resulting in a corresponding multi-layer neural network configured according to member 150.

[0096] In the embodiment of the present disclosure, through the above process, Figure 1bThe initial population 110 in the population is modified by member 115 to obtain member 140, and then modified to obtain member 150. After updating member 115 in the initial population 110 to member 150, a new population is obtained. For the new population, the above process of selecting members with good performance and members with poor performance, modifying the quantization accuracy configuration corresponding to the members with poor performance with the quantization accuracy configuration corresponding to the members with good performance, and modifying the quantization accuracy configuration of the members after the modification of the quantization accuracy configuration is repeated to perform population iterative evolution. Finally, a member that meets the preset conditions is obtained (for example, a member that meets the preset accuracy and speed requirements is obtained, and the population iteration is terminated). For example, when a member with an accuracy loss of less than 1% and an inference speed improvement of 15% is obtained, the population iteration is terminated, and the quantization accuracy configuration corresponding to the member is used as the quantization accuracy configuration of multiple layers of the optimized multi-layer neural network.

[0097] In an embodiment of the present disclosure, a multi-layer neural network can be configured based on the quantization accuracy of multiple layers of the above-mentioned optimized multi-layer neural network, and the configured multi-layer neural network can be used for image processing, such as image classification, target detection, environment segmentation, face recognition and other image processing.

[0098] Those skilled in the art will understand that the optimization of the quantization accuracy of each layer of the multi-layer neural network can be completed in the cloud. The use of multi-layer neural networks for image processing applications such as image classification, target detection, environment segmentation, and face recognition can be completed in the cloud or on the terminal side. If image processing is completed on the terminal side, the optimization of the quantization accuracy configuration of the multi-layer neural network can be completed in the cloud, and the optimized quantization accuracy configuration can be transmitted to the terminal. The multi-layer neural network can be configured using the optimized quantization accuracy configuration at the terminal, and the multi-layer neural network can be used for image processing applications such as image classification, target detection, environment segmentation, and face recognition to achieve energy saving on the terminal side. The terminal only needs to transmit the processed image features to the cloud, saving transmission bandwidth resources.

[0099] Those skilled in the art will appreciate that the above-mentioned multi-layer neural network can also be used for other image processing applications. The multi-layer neural network quantization accuracy configuration scheme disclosed in the embodiments of the present disclosure can also be used to optimize multi-layer neural networks in other application scenarios. For example, the performance of the multi-layer neural network can be evaluated using a test speech set, and the quantization accuracy configuration of the multi-layer neural network can be optimized using the above-mentioned population iteration method. The multi-layer neural network with optimized quantization accuracy configuration can also be used for natural language processing (NLP), and this disclosure does not limit this.

[0100] Figure 2 A flowchart of an image processing method according to an embodiment of the present disclosure is shown.

[0101] like Figure 2As shown, the image processing method includes: steps S201, S202, S203, and S204.

[0102] In step S201 , the quantization accuracy sensitivity of multiple layers in a multi-layer neural network for image processing is obtained.

[0103] In step S202, an initial population of the evolutionary algorithm is generated based on the quantization accuracy sensitivity of the layer and the random number generated for the layer, where each member of the initial population corresponds to a quantization accuracy configuration corresponding to the multi-layer neural network.

[0104] In step S203, the performance of the members in the initial population is evaluated using the evaluation image set, and the population is updated based on the evaluation results until a member whose performance meets the preset conditions appears in the population.

[0105] In step S204, the quantization accuracy of multiple layers in the multi-layer neural network is determined based on the members whose performance meets the preset conditions.

[0106] In the embodiment of the present disclosure, as described above, the quantization accuracy sensitivities S1, S2, S3, S4, S5, S6, S7, S8, S9, S10, S11, S12, S13, S14, S15, S16, S17, S18, S19, S20, S21, S22, S23, S24, S25, S26, S27, S28, S2 N Quantization accuracy sensitivity based on layer S1, S2, S N and random numbers R11, R12, ..., R1 generated by a random number generation algorithm N , obtaining an initial population 110 for the evolutionary algorithm. Initial population 110 includes members 111, ..., 118, each of which corresponds to a quantization accuracy configuration of the multi-layer neural network. Based on the evaluation image set, the performance of the members in the initial population is evaluated. Based on the evaluation results, the poorly performing member 115 is updated to member 140, thereby updating the population until a member appears in the population that meets preset conditions, such as an accuracy loss of less than 1% and an inference speed improvement of 15%. Based on the members that meet the preset conditions, the quantization accuracy of multiple layers of the multi-layer neural network 100 is determined.

[0107] According to an embodiment of the present disclosure, the multiple layers in the multi-layer neural network may be all layers in the multi-layer neural network, or may be a part of the layers in the multi-layer neural network.

[0108] Those skilled in the art will appreciate that the above-mentioned multi-layer neural network can also be used for other image processing applications. The multi-layer neural network quantization accuracy configuration scheme disclosed in the embodiments of the present disclosure can also be used to optimize multi-layer neural networks in other application scenarios. For example, the performance of the multi-layer neural network can be evaluated using a test speech set, and the quantization accuracy configuration of the multi-layer neural network can be optimized using the above-mentioned population iteration method. The multi-layer neural network with optimized quantization accuracy configuration can also be used for natural language processing (NLP), and this disclosure does not limit this.

[0109] According to an embodiment of the present disclosure, the quantization accuracy sensitivity of multiple layers in a multi-layer neural network for image processing is obtained; based on the quantization accuracy sensitivity of the layer and the random number generated for the layer, an initial population of the evolutionary algorithm is generated, and each member of the initial population corresponds to a corresponding quantization accuracy configuration of the multi-layer neural network; using a set of evaluation images, the performance of the members in the initial population is evaluated, and the population is updated based on the evaluation results until members whose performance meets the preset conditions appear in the population; based on the members whose performance meets the preset conditions, the quantization accuracy of multiple layers in the multi-layer neural network is determined, thereby optimizing the quantization accuracy configuration of the multi-layer neural network at a faster speed, and improving the inference speed of the multi-layer neural network under the condition of acceptable accuracy loss.

[0110] In an embodiment of the present disclosure, the performance of a multi-layer neural network may include accuracy loss and running speed. The quantization accuracy sensitivity of each layer represents the sensitivity of the performance of the multi-layer neural network to changes in the quantization accuracy of the layer.

[0111] According to an embodiment of the present disclosure, the sensitivity of the performance of a multi-layer neural network to changes in the quantization precision of the layer is represented by the quantization precision sensitivity of the layer, thereby providing an accurate basis for selecting the quantization precision of each layer.

[0112] Figure 3 A flowchart of an image processing method according to an embodiment of the present disclosure is shown.

[0113] like Figure 3 As shown, in an embodiment of the present disclosure, based on the quantization precision sensitivity of the layer and the random number generated for the layer, an initial population of the evolutionary algorithm is generated, including determining the quantization precision configuration corresponding to the members in the initial population through the following steps. Specifically, steps S301 and S302 are included.

[0114] In step S301 , a random number set is generated for a member, where each random number in the random number set corresponds to a corresponding layer in a plurality of layers.

[0115] In step S302 , for each of the multiple layers, a probability factor of the layer is generated based on the quantization accuracy sensitivity of the layer and a corresponding random number.

[0116] As mentioned above, when determining the quantization precision configuration corresponding to the initial member in the initialization population 110, a random number set {R1 i}, random number R1 in the set i Corresponding to the i-th layer of the multi-layer neural network 100. For the i-th layer, based on P1 i =S i *R1 i Probability factor P1 for generating this layer i .

[0117] In the embodiment of the present disclosure, the quantization accuracy configuration of each layer can be determined based on the probability factor of each layer. For example, when P1 i When P1 is greater than or equal to a first specific threshold value, for example, 0.5, the i-th layer is quantized using FP16. i When the first specific threshold is less than 0.5, the i-th layer is quantized with INT8. Alternatively, in the N-th layer, P1 is selected. i The largest M1 layers are quantized using FP16, and the remaining layers are quantized using INT8. It is understood that although this disclosure uses FP16 as an example of high quantization precision and INT8 as an example of low quantization precision, other high quantization precision and / or low quantization precision can also be used according to actual needs.

[0118] According to an embodiment of the present disclosure, a set of random numbers is generated for members, and each random number in the set of random numbers corresponds to a corresponding layer in a plurality of layers; for each layer in a plurality of layers, a probability factor of the layer is generated based on the quantization accuracy sensitivity of the layer and the corresponding random number, so as to flexibly generate a probability factor for each layer, thereby generating suitable initialization members and obtaining a suitable initialization population.

[0119] Figure 4 A flowchart of an image processing method according to an embodiment of the present disclosure is shown.

[0120] like Figure 4 As shown, the evaluation image set is used to evaluate the performance of the members in the initial population, and the population is updated based on the evaluation results, including adopting steps S401 and S402 for the current population.

[0121] In step S401, the performance of a member is determined based on the performance of a multi-layer neural network having a quantization accuracy configuration corresponding to the member in the population when processing a set of evaluation images.

[0122] In step S402, the quantization precision configuration corresponding to the member with good performance is used to update the quantization precision configuration corresponding to the member with poor performance, thereby obtaining an updated population.

[0123] In the embodiments of the present disclosure, as previously described, performance evaluation is performed based on the evaluation image set, and the performance of the multi-layer neural network, such as the precision loss and inference speed of the multi-layer neural network operation, can be obtained for the multi-layer neural network configured according to the quantization precision corresponding to members 111 to 118 in the population 110. Based on three members with better performance than other members, for example, based on the three members 111, 112, and 113 with the fastest inference speed among the members with precision loss of less than 1%, the well-performing member 112 can be selected. Among the other members other than 111, 112, and 113, the poorly performing member 115 can be selected. The quantization precision configuration corresponding to the poorly performing member 115 is updated using the quantization precision configuration corresponding to the well-performing member 112 to obtain an updated population.

[0124] According to the implementation manner of the present disclosure, the performance of the members is determined by determining the performance of a multi-layer neural network having quantization precision configurations corresponding to the members in the population when processing a set of evaluation images; the quantization precision configurations corresponding to the members with good performance are used to update the quantization precision configurations corresponding to the members with poor performance to obtain an updated population, thereby realizing the evolution of the population and obtaining an optimized quantization precision configuration as soon as possible.

[0125] Figure 5 A flowchart of an image processing method according to an embodiment of the present disclosure is shown.

[0126] like Figure 5 As shown, using the quantization precision configuration corresponding to the member with good performance to update the quantization precision configuration of the member with poor performance includes steps S501 and S502.

[0127] In step S501, for the high-precision layer included in the quantization precision configuration corresponding to the member with good performance but not included in the quantization precision configuration corresponding to the member with poor performance, determine the designated low-precision layer in the quantization precision configuration corresponding to the member with poor performance to be modified to the high-precision layer based on the quantization precision sensitivity of the high-precision layer and the random number generated for the high-precision layer, and modify the designated low-precision layer in the quantization precision configuration corresponding to the member with poor performance to the high-precision layer.

[0128] In step S502, for the high-precision layer in the modified quantization precision configuration of the poorly performing member, based on the quantization sensitivity of the high-precision layer and the random number generated for the high-precision layer, the designated high-precision layer to be changed to the low-precision layer in the modified quantization precision configuration of the poorly performing member is determined, and the designated high-precision layer in the modified quantization precision configuration of the poorly performing member is changed to the low-precision layer to update the quantization precision configuration corresponding to the poorly performing member.

[0129] In the embodiments of the present disclosure, Figure 1cAs shown, the layers 122, 123, and 125 configured according to the member 112 with good performance use high-precision FP16 quantization, while the layers 132, 133, and 135 configured according to the member 115 with poor performance use low-precision INT8 quantization. For the layers 122, 123, and 125, the aforementioned quantization accuracy sensitivities S2, S3, and S5 are used, and the random number generation algorithm is used to generate random numbers R22, R23, and R25, and the calculation is performed.

[0130] P2 i =S i *R2 i , i=2,3,5

[0131] P22, P23, and P25 are obtained. Among P22, P23, and P25, a second specific threshold greater than, for example, 0.5 or a maximum of, for example, M1 = 2 layers is selected. The layers configured according to member 115 corresponding to the selected two layers are quantized to FP16. For example, corresponding to layers 122 and 125 configured according to member 112, layers 132 and 135 configured according to member 115 are modified from INT8 quantization to FP16 quantization.

[0132] like Figure 1d As shown, Figure 1c After the INT8 quantized layers 132 and 135 are modified to FP16 quantized layers 141 and 142, the corresponding Figure 1c 115 members of Figure 1d There are 140 members in the group.

[0133] According to the configuration of member 140, the layers using FP16 quantization are 131, 141, 134, 142, and 136. For layers 131, 141, 134, 142, and 136, the aforementioned quantization accuracy sensitivities S1, S2, S4, S5, and S6 are used, and the random number generation algorithm is used to generate random numbers R31, R32, R34, R35, and R36, and the calculation is performed.

[0134] P3 i =S i *R3 i , i=1,2,4,5,6

[0135] P31, P32, P34, P35, and P36 are obtained. Among P31, P32, P34, P35, and P36, a third specific threshold value less than, for example, 0.5 or a minimum number of layers, for example, M2 = 2, is selected for INT8 quantization. For example, layers 131 and 142 configured according to member 140 are modified from FP16 quantization to INT8 quantization, thereby correspondingly obtaining member 150 from member 140.

[0136] According to an embodiment of the present disclosure, the quantization accuracy configuration of the poorly performing member is updated by using the quantization accuracy configuration corresponding to the good-performing member, including: for the high-precision layer included in the quantization accuracy configuration corresponding to the good-performing member but not included in the quantization accuracy configuration corresponding to the poor-performing member, determining the designated low-precision layer to be modified to the high-precision layer in the quantization accuracy configuration corresponding to the poor-performing member based on the quantization accuracy sensitivity of the high-precision layer and the random number generated for the high-precision layer, and modifying the designated low-precision layer in the quantization accuracy configuration corresponding to the poor-performing member to the high-precision layer; for the high-precision layer in the modified quantization accuracy configuration of the poor-performing member, determining the designated high-precision layer to be modified to the low-precision layer in the modified quantization accuracy configuration of the poor-performing member based on the quantization sensitivity of the high-precision layer and the random number generated for the high-precision layer, and modifying the designated high-precision layer in the modified quantization accuracy configuration of the poor-performing member to the low-precision layer, so as to update the quantization accuracy configuration corresponding to the poor-performing member, thereby realizing the evolution of the population and obtaining the optimized quantization accuracy configuration as soon as possible.

[0137] Figure 6 A flowchart of an image processing method according to an embodiment of the present disclosure is shown.

[0138] like Figure 6 As shown, the image processing method includes and Figure 2 The same S201, S202, S203, S204 also include steps S601, S602.

[0139] In step S601, the multi-layer neural network is configured based on the quantization accuracy of multiple layers in the multi-layer neural network.

[0140] In step S602, the configured multi-layer neural network is used to perform image processing.

[0141] According to an embodiment of the present disclosure, the image processing method also includes: configuring the multi-layer neural network based on the quantization accuracy of multiple layers in the multi-layer neural network; using the configured multi-layer neural network to perform image processing, thereby improving image processing efficiency under the condition of limited loss of quantization accuracy.

[0142] According to an embodiment of the present disclosure, image processing includes any one or more of the following: image classification, target detection, environment segmentation, and face recognition, thereby improving the adaptability of the above-mentioned optimized quantization accuracy configuration.

[0143] Those skilled in the art will understand that the optimization of the quantization accuracy of each layer of the multi-layer neural network can be completed in the cloud. The use of multi-layer neural networks for image processing applications such as image classification, target detection, environment segmentation, and face recognition can be completed in the cloud or on the terminal side. If image processing is completed on the terminal side, the optimization of the quantization accuracy configuration of the multi-layer neural network can be completed in the cloud, and the optimized quantization accuracy configuration can be transmitted to the terminal. The multi-layer neural network is used on the terminal to perform image processing applications such as image classification, target detection, environment segmentation, and face recognition, thereby achieving energy saving on the terminal side. The terminal only needs to transmit the processed image features to the cloud, saving transmission bandwidth resources.

[0144] Figure 7 The following is a block diagram of the structure of an image processing device according to an embodiment of the present disclosure, wherein the device can be implemented as part or all of an electronic device through software, hardware, or a combination of both.

[0145] like Figure 7 As shown, the image processing device 700 includes: a quantization accuracy sensitivity acquisition module 701, an initial population generation module 702, a population update module 703, and a quantization accuracy configuration determination module 704.

[0146] The quantization accuracy sensitivity acquisition module 701 is used to acquire the quantization accuracy sensitivity of multiple layers in a multi-layer neural network used for image processing.

[0147] The initial population generation module 702 is used to generate an initial population of the evolutionary algorithm based on the quantization accuracy sensitivity of the layer and the random number generated for the layer. Each member of the initial population corresponds to a quantization accuracy configuration corresponding to the multi-layer neural network.

[0148] The population updating module 703 is configured to evaluate the performance of members in the initial population using the evaluation image set, and update the population based on the evaluation results until a member whose performance meets the preset conditions appears in the population.

[0149] The quantization accuracy configuration determination module 704 is used to determine the quantization accuracy of multiple layers in the multi-layer neural network based on members whose performance meets preset conditions.

[0150] According to the embodiment of the present disclosure, a quantization accuracy sensitivity acquisition module is used to obtain the quantization accuracy sensitivity of multiple layers in a multi-layer neural network used for image processing; an initial population generation module is used to generate an initial population of the evolutionary algorithm based on the quantization accuracy sensitivity of the layer and the random number generated for the layer, and each member in the initial population corresponds to a corresponding quantization accuracy configuration of the multi-layer neural network; a population update module is used to evaluate the performance of members in the initial population using a set of evaluation images, and update the population based on the evaluation results until members whose performance meets the preset conditions appear in the population; a quantization accuracy configuration determination module is used to determine the quantization accuracy of multiple layers in the multi-layer neural network based on members whose performance meets the preset conditions, thereby optimizing the quantization accuracy configuration of the multi-layer neural network at a faster speed and improving the inference speed of the multi-layer neural network under the condition of acceptable accuracy loss.

[0151] According to an embodiment of the present disclosure, the sensitivity of the performance of a multi-layer neural network to changes in the quantization precision of the layer is represented by the quantization precision sensitivity of the layer, thereby providing an accurate basis for selecting the quantization precision of each layer.

[0152] According to an embodiment of the present disclosure, a set of random numbers is generated for members, and each random number in the set of random numbers corresponds to a corresponding layer in a plurality of layers; for each layer in a plurality of layers, a probability factor of the layer is generated based on the quantization accuracy sensitivity of the layer and the corresponding random number, so as to flexibly generate a probability factor for each layer, thereby generating suitable initialization members and obtaining a suitable initialization population.

[0153] According to the implementation manner of the present disclosure, the performance of the members is determined by determining the performance of a multi-layer neural network having quantization precision configurations corresponding to the members in the population when processing a set of evaluation images; the quantization precision configurations corresponding to the members with good performance are used to update the quantization precision configurations corresponding to the members with poor performance to obtain an updated population, thereby realizing the evolution of the population and obtaining an optimized quantization precision configuration as soon as possible.

[0154] According to an embodiment of the present disclosure, the quantization accuracy configuration of the poorly performing member is updated by using the quantization accuracy configuration corresponding to the good-performing member, including: for the high-precision layer included in the quantization accuracy configuration corresponding to the good-performing member but not included in the quantization accuracy configuration corresponding to the poor-performing member, determining the designated low-precision layer to be modified to the high-precision layer in the quantization accuracy configuration corresponding to the poor-performing member based on the quantization accuracy sensitivity of the high-precision layer and the random number generated for the high-precision layer, and modifying the designated low-precision layer in the quantization accuracy configuration corresponding to the poor-performing member to the high-precision layer; for the high-precision layer in the modified quantization accuracy configuration of the poor-performing member, determining the designated high-precision layer to be modified to the low-precision layer in the modified quantization accuracy configuration of the poor-performing member based on the quantization sensitivity of the high-precision layer and the random number generated for the high-precision layer, and modifying the designated high-precision layer in the modified quantization accuracy configuration of the poor-performing member to the low-precision layer, so as to update the quantization accuracy configuration corresponding to the poor-performing member, thereby realizing the evolution of the population and obtaining the optimized quantization accuracy configuration as soon as possible.

[0155] According to an embodiment of the present disclosure, the system further includes: a multi-layer neural network configuration module for configuring the multi-layer neural network based on the quantization accuracy of multiple layers in the multi-layer neural network; and an image processing module for performing image processing using the configured multi-layer neural network, thereby improving image processing efficiency under the condition of limited loss of quantization accuracy.

[0156] According to an embodiment of the present disclosure, image processing includes any one or more of the following: image classification, target detection, environment segmentation, and face recognition, thereby improving the adaptability of the above-mentioned optimized quantization accuracy configuration.

[0157] The present disclosure also discloses an electronic device, Figure 8 A structural block diagram of an electronic device according to an embodiment of the present disclosure is shown.

[0158] like Figure 8 As shown, the electronic device 800 includes a memory 801 and a processor 802, wherein the memory 801 is used to store one or more computer instructions, wherein the one or more computer instructions are executed by the processor 802 to implement the following steps:

[0159] Obtaining the quantization accuracy sensitivity of multiple layers in a multi-layer neural network for image processing;

[0160] generating an initial population of an evolutionary algorithm based on the quantization accuracy sensitivity of the layer and a random number generated for the layer, wherein each member of the initial population corresponds to a quantization accuracy configuration corresponding to the multi-layer neural network;

[0161] Using the evaluation image set, evaluating the performance of members in the initial population, and updating the population based on the evaluation results until a member whose performance meets the preset conditions appears in the population;

[0162] The quantization accuracy of multiple layers in the multi-layer neural network is determined according to the members whose performance meets the preset conditions.

[0163] In an embodiment of the present disclosure, the quantization accuracy sensitivity of the layer represents the degree to which the performance of the multi-layer neural network is sensitive to changes in the quantization accuracy of the layer.

[0164] In an embodiment of the present disclosure, generating an initial population of the evolutionary algorithm based on the quantization precision sensitivity of the layer and a random number generated for the layer includes determining the quantization precision configuration corresponding to the members in the initial population through the following steps:

[0165] generating a set of random numbers for the member, each random number in the set of random numbers corresponding to a respective one of the plurality of layers;

[0166] generating, for each layer of the plurality of layers, a probability factor for the layer based on a quantization precision sensitivity of the layer and a corresponding random number;

[0167] The quantization precision configuration corresponding to the member is determined based on the probability factor of each layer in the multiple layers.

[0168] In an embodiment of the present disclosure, the use of the evaluation image set to evaluate the performance of members in the initial population and updating the population based on the evaluation results includes, for the current population:

[0169] determining the performance of the member according to the performance of the multi-layer neural network having the quantization accuracy configuration corresponding to the member in the population when processing the evaluation image set;

[0170] The quantization accuracy configuration corresponding to the members with good performance is used to update the quantization accuracy configuration corresponding to the members with poor performance to obtain an updated population.

[0171] In an embodiment of the present disclosure, the updating of the quantization precision configuration of the poorly performing member using the quantization precision configuration corresponding to the well-performing member includes:

[0172] For a high-precision layer included in the quantization precision configuration corresponding to the good-performing member but not included in the quantization precision configuration corresponding to the poor-performing member, determining, based on the quantization precision sensitivity of the high-precision layer and a random number generated for the high-precision layer, a designated low-precision layer in the quantization precision configuration corresponding to the poor-performing member to be modified to a high-precision layer, and modifying the designated low-precision layer in the quantization precision configuration corresponding to the poor-performing member to a high-precision layer;

[0173] For the high-precision layer in the modified quantization precision configuration of the poorly performing member, based on the quantization sensitivity of the high-precision layer and the random number generated for the high-precision layer, determine the designated high-precision layer to be modified to the low-precision layer in the modified quantization precision configuration of the poorly performing member, and modify the designated high-precision layer in the modified quantization precision configuration of the poorly performing member to a low-precision layer to update the quantization precision configuration corresponding to the poorly performing member.

[0174] In an embodiment of the present disclosure, it further includes:

[0175] configuring the multi-layer neural network based on quantization accuracy of multiple layers in the multi-layer neural network;

[0176] Use a configured multi-layer neural network for image processing.

[0177] In an embodiment of the present disclosure, the image processing includes any one or more of the following: image classification, target detection, environment segmentation, and face recognition.

[0178] Figure 9 A schematic diagram showing the structure of a computer system suitable for implementing the method according to an embodiment of the present disclosure is shown.

[0179] like Figure 9 As shown, the computer system 900 includes a processing unit 901, which can execute various processes in the above-mentioned embodiments according to a program stored in a read-only memory (ROM) 902 or a program loaded from a storage unit 908 into a random access memory (RAM) 903. Various programs and data required for the operation of the system 900 are also stored in the RAM 903. The processing unit 901, the ROM 902, and the RAM 903 are connected to each other via a bus 904. An input / output (I / O) interface 905 is also connected to the bus 904.

[0180] The following components are connected to the I / O interface 905: an input section 906 including a keyboard, a mouse, etc.; an output section 907 including a cathode ray tube (CRT), a liquid crystal display (LCD), a speaker, etc.; a storage section 908 including a hard disk, etc.; and a communication section 909 including a network interface card such as a LAN card, a modem, etc. The communication section 909 performs communication processing via a network such as the Internet. A drive 910 is also connected to the I / O interface 905 as needed. A removable medium 911, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 910 as needed so that a computer program read therefrom can be installed into the storage section 908 as needed. Among them, the processing unit 901 can be implemented as a processing unit such as a CPU, a GPU, a TPU, an FPGA, an NPU, etc.

[0181] In particular, according to embodiments of the present disclosure, the methods described above can be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising computer instructions that, when executed by a processor, implement the method steps described above. In such embodiments, the computer program product can be downloaded and installed from a network via the communication portion 909 and / or installed from removable media 911.

[0182] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, program segment or part of code, and the module, program segment or part of code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or can be implemented using a combination of dedicated hardware and computer instructions.

[0183] The units or modules involved in the embodiments described in this disclosure may be implemented by software or programmable hardware. The units or modules described may also be provided in a processor, and the names of these units or modules do not, in certain circumstances, constitute limitations on the units or modules themselves.

[0184] As another aspect, the present disclosure further provides a computer-readable storage medium. This computer-readable storage medium may be included in the electronic device or computer system described in the above embodiments, or may be a standalone computer-readable storage medium not incorporated into the device. The computer-readable storage medium stores one or more programs, which are used by one or more processors to execute the methods described in the present disclosure.

[0185] The above description is merely a preferred embodiment of the present disclosure and an illustration of the technical principles employed. Those skilled in the art should understand that the scope of the invention herein is not limited to the technical solutions formed by the specific combination of the above-mentioned technical features, but also encompasses other technical solutions formed by any combination of the above-mentioned technical features or their equivalents without departing from the inventive concept. For example, a technical solution formed by replacing the above-mentioned features with (but not limited to) technical features with similar functions disclosed in this disclosure.

Claims

1. An image processing method, comprising: Obtaining the quantization accuracy sensitivity of multiple layers in a multi-layer neural network for image processing; generating an initial population of an evolutionary algorithm based on the quantization precision sensitivity of the layer and a random number generated for the layer, wherein each member of the initial population corresponds to a corresponding quantization precision configuration of the multi-layer neural network, the quantization precision configuration being determined based on a probability factor of each layer in the plurality of layers; the probability factor being generated based on the quantization precision sensitivity of the layer and the random number generated for the layer; Using the evaluation image set, evaluating the performance of members in the initial population, and updating the population based on the evaluation results until a member whose performance meets the preset conditions appears in the population; The quantization accuracy of multiple layers in the multi-layer neural network is determined according to the members whose performance meets the preset conditions.

2. The method according to claim 1, wherein Generating an initial population of the evolutionary algorithm based on the quantization precision sensitivity of the layer and a random number generated for the layer includes determining the quantization precision configuration corresponding to the members in the initial population through the following steps: generating a set of random numbers for the member, each random number in the set of random numbers corresponding to a respective one of the plurality of layers; generating, for each layer of the plurality of layers, a probability factor for the layer based on a quantization precision sensitivity of the layer and a corresponding random number; The quantization precision configuration corresponding to the member is determined based on the probability factor of each layer in the multiple layers.

3. The method according to claim 2, wherein: The using the evaluation image set to evaluate the performance of the members in the initial population and updating the population based on the evaluation results includes, for the current population: determining the performance of the member according to the performance of the multi-layer neural network having the quantization accuracy configuration corresponding to the member in the population when processing the evaluation image set; The quantization accuracy configuration corresponding to the members with good performance is used to update the quantization accuracy configuration corresponding to the members with poor performance to obtain an updated population.

4. The method according to claim 3, wherein: The updating of the quantization precision configuration of the poorly performing member by using the quantization precision configuration corresponding to the good-performing member includes: For a high-precision layer included in the quantization precision configuration corresponding to the good-performing member but not included in the quantization precision configuration corresponding to the poor-performing member, determining, based on the quantization precision sensitivity of the high-precision layer and a random number generated for the high-precision layer, a designated low-precision layer in the quantization precision configuration corresponding to the poor-performing member to be modified to a high-precision layer, and modifying the designated low-precision layer in the quantization precision configuration corresponding to the poor-performing member to a high-precision layer; For the high-precision layer in the modified quantization precision configuration of the poorly performing member, based on the quantization sensitivity of the high-precision layer and the random number generated for the high-precision layer, determine the designated high-precision layer to be modified to the low-precision layer in the modified quantization precision configuration of the poorly performing member, and modify the designated high-precision layer in the modified quantization precision configuration of the poorly performing member to a low-precision layer to update the quantization precision configuration corresponding to the poorly performing member.

5. The method according to claim 1, wherein Also includes: configuring the multi-layer neural network based on quantization accuracy of multiple layers in the multi-layer neural network; Use a configured multi-layer neural network for image processing.

6. The method according to claim 5, wherein: The image processing includes any one or more of the following: image classification, target detection, environment segmentation, and face recognition.

7. An image processing device comprising: A quantization accuracy sensitivity acquisition module, used to acquire the quantization accuracy sensitivity of multiple layers in a multi-layer neural network used for image processing; an initial population generation module, configured to generate an initial population of the evolutionary algorithm based on the quantization accuracy sensitivity of the layer and a random number generated for the layer, wherein each member of the initial population corresponds to a quantization accuracy configuration corresponding to the multi-layer neural network, and the quantization accuracy configuration is determined based on a probability factor of each layer in the plurality of layers; The probability factor is generated based on the quantization accuracy sensitivity of the layer and a random number generated for the layer; a population updating module, configured to evaluate the performance of members in the initial population using a set of evaluation images, and to update the population based on the evaluation results until a member whose performance meets a preset condition appears in the population; The quantization accuracy configuration determination module is used to determine the quantization accuracy of multiple layers in the multi-layer neural network based on the members whose performance meets the preset conditions.

8. An electronic device comprising a memory and a processor; wherein: The memory is configured to store one or more computer instructions, wherein the one or more computer instructions are executed by the processor to implement the method steps according to any one of claims 1 to 6.

9. A readable storage medium having computer instructions stored thereon, wherein the computer instructions, when executed by a processor, implement the method steps described in any one of claims 1 to 6.

10. A computer program product comprising computer instructions, which, when executed by a processor, implement the method steps according to any one of claims 1 to 6.

Images