A data processing method, network training method, electronic equipment and storage medium

Abstract

Embodiments of the present application provide a data processing method, a network training method, an electronic device and a storage medium. The method comprises: obtaining input data of a network layer of a deep neural network, original weights of the network layer being decomposed into N sub-weights for quantization processing; performing convolution processing on the input data and the N sub-weights respectively to obtain N intermediate data corresponding to the N sub-weights; and performing summation processing on the N intermediate data to obtain output data of the network layer. In this way, the original weights of the network layer are decomposed into the N sub-weights for quantization processing, and the efficiency and performance of the network can be flexibly balanced by adjusting the value of N.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer, in particular to a data processing method, a network training method, an electronic device and a storage medium. BACKGROUND

[0002] With the development of computer technology, deep neural networks are widely used in image denoising, face recognition, and living body verification and other application scenarios. However, the calculation amount of deep neural networks is generally very large, and the huge calculation amount limits the application scenarios of deep neural networks. In the terminal scene with limited computing power, using quantization means to reduce the calculation amount of deep neural networks is an effective means.

[0003] However, when the current deep neural network is quantized, the weights of each network layer in the deep neural network are quantized with a unified specification, such as 2 bits, 4 bits, or 8 bits. This makes the flexibility of deep neural network design poor, and cannot flexibly balance the efficiency and performance of the network. SUMMARY

[0004] The purpose of the embodiments of the present application is to provide a data processing method, a network training method, an electronic device and a storage medium to flexibly balance the efficiency and performance of the network. The specific technical solutions are as follows:

[0005] In a first aspect, the embodiments of the present application provide a data processing method, which comprises:

[0006] obtaining input data of a network layer of a deep neural network, wherein original weights of the network layer are decomposed into N sub-weights for quantization processing;

[0007] performing convolution processing on the input data and the N sub-weights respectively to obtain N intermediate data corresponding to the N sub-weights;

[0008] performing summation processing on the N intermediate data to obtain output data of the network layer.

[0009] Optionally, the sub-weight is a value obtained by multiplying the quantized weight corresponding to the sub-weight and a first scale coefficient;

[0010] The step of performing convolution processing on the input data and the N sub-weights respectively to obtain N intermediate data corresponding to the N sub-weights comprises:

[0011] performing quantization processing on the input data to obtain quantized data corresponding to the input data and a second scale coefficient;

[0012] Convolve the quantized data with quantized weights corresponding to the N sub-weights respectively to obtain N convolution data corresponding to the N sub-weights;

[0013] Multiply each convolution data with the first scale coefficient and the second scale coefficient respectively to obtain N intermediate data corresponding to the N sub-weights.

[0014] Optionally, the quantized weights consist of -1, 0 and 1.

[0015] The sum of the N sub-weights is less than a preset threshold value from the original weight, or the ratio of the sum of the N sub-weights to the original weight is less than a preset ratio.

[0016] Optionally, the input data is a first image, and the output data is a second image, and the noise of the first image is higher than that of the second image.

[0017] In a second aspect, an embodiment of the present application provides a network training method, and the method comprises:

[0018] Obtaining a training set, wherein the training set comprises sample data and expected data corresponding to the sample data;

[0019] Inputting the sample data into a first deep neural network to obtain first prediction data, wherein the first deep neural network comprises at least one network layer.

[0020] Determining a first loss value according to the expected data and the first prediction data.

[0021] If it is determined that the first deep neural network converges according to the first loss value, decomposing original weights of the network layer into N sub-weights subjected to quantization to obtain a second deep neural network.

[0022] Optionally, the method further comprises:

[0023] If it is determined that the first deep neural network does not converge according to the first loss value, adjusting a first parameter of the first deep neural network, and re-executing the step of inputting the sample data into the first deep neural network to obtain the first prediction data; wherein the first parameter comprises the original weights.

[0024] Optionally, after obtaining the second deep neural network, the method further comprises:

[0025] Inputting the sample data into the second deep neural network to obtain second prediction data.

[0026] Determining a second loss value according to the expected data and the second prediction data.

[0027] if it is determined that the second deep neural network converges according to the second loss value, ending the training of the second deep neural network;

[0028] if it is determined that the second deep neural network does not converge according to the second loss value, adjusting a second parameter of the second deep neural network, and re-executing the step of inputting the sample data into the second deep neural network to obtain second prediction data, the second parameter including a scale coefficient of the N sub-weights.

[0029] Optionally, the sub-weight is a value obtained by multiplying a quantized weight corresponding to the sub-weight and a first scale coefficient, the quantized weight consisting of -1, 0 and 1.

[0030] The sum of the N sub-weights and the difference between the original weight are less than a preset threshold, or the ratio of the sum of the N sub-weights to the original weight is less than a preset ratio.

[0031] In a third aspect, an embodiment of the present application provides a data processing apparatus, the apparatus comprising:

[0032] an acquisition unit configured to acquire input data of a network layer of a deep neural network, an original weight of the network layer being decomposed into N sub-weights subjected to quantization processing;

[0033] a convolution unit configured to perform convolution processing on the input data and the N sub-weights respectively to obtain N intermediate data corresponding to the N sub-weights;

[0034] a summation unit configured to perform summation processing on the N intermediate data to obtain output data of the network layer.

[0035] Optionally, the sub-weight is a value obtained by multiplying a quantized weight corresponding to the sub-weight and a first scale coefficient, the quantized weight consisting of -1, 0 and 1.

[0036] The convolution unit is specifically configured to:

[0037] perform quantization processing on the input data to obtain quantized data corresponding to the input data and a second scale coefficient;

[0038] perform convolution processing on the quantized data and quantized weights corresponding to the N sub-weights respectively to obtain N convolution data corresponding to the N sub-weights;

[0039] perform multiplication processing on each quantized convolution and the first scale coefficient and the second scale coefficient respectively to obtain N intermediate data corresponding to the N sub-weights.

[0040] Optionally, the quantized weight consists of -1, 0 and 1.

[0041] a difference between the sum of the N sub-weights and the original weight is less than a preset threshold, or a ratio between the sum of the N sub-weights and the original weight is less than a preset ratio.

[0042] Optionally, the input data is a first image, and the output data is a second image, and noise of the first image is higher than noise of the second image.

[0043] In a fourth aspect, an embodiment of the present application provides a network training apparatus, and the apparatus comprises:

[0044] an obtaining unit configured to obtain a training set, the training set comprising sample data and expected data corresponding to the sample data;

[0045] a first input unit configured to input the sample data into a first deep neural network to obtain first predicted data, wherein the first deep neural network comprises at least one network layer;

[0046] a first determining unit configured to determine a first loss value according to the expected data and the first predicted data;

[0047] a decomposing unit configured to, if it is determined that the first deep neural network converges according to the first loss value, decompose an original weight of the network layer into N quantized sub-weights to obtain a second deep neural network.

[0048] Optionally, the apparatus further comprises:

[0049] a first adjusting unit configured to, if it is determined that the first deep neural network does not converge according to the first loss value, adjust a first parameter of the first deep neural network, and re-perform the step of inputting the sample data into the first deep neural network to obtain the first predicted data; wherein the first parameter comprises the original weight.

[0050] Optionally, the apparatus further comprises:

[0051] a second input unit configured to, after obtaining the second deep neural network, input the sample data into the second deep neural network to obtain second predicted data;

[0052] a second determining unit configured to determine a second loss value according to the expected data and the second predicted data;

[0053] an ending unit configured to, if it is determined that the second deep neural network converges according to the second loss value, end the training of the second deep neural network.

[0054] The second adjusting unit is configured to adjust a second parameter of the second deep neural network if it is determined that the second deep neural network is not converged according to the second loss value, and re-perform the step of inputting the sample data into the second deep neural network to obtain second predicted data, wherein the second parameter comprises a scale coefficient of the N sub-weights.

[0055] Optionally, the sub-weight is a value obtained by multiplying a quantized weight and a first scale coefficient, wherein the quantized weight is composed of -1, 0 and 1.

[0056] The sum of the N sub-weights is less than a preset threshold value from the original weight, or a ratio of the sum of the N sub-weights to the original weight is less than a preset ratio value.

[0057] In a fifth aspect, an embodiment of the present application provides an electronic device, comprising a processor and a machine readable storage medium, wherein the machine readable storage medium stores machine executable instructions which can be executed by the processor, and the processor is prompted by the machine executable instructions to implement any of the data processing method steps or the network training method.

[0058] In a sixth aspect, an embodiment of the present application provides a computer readable storage medium, characterized in that the computer readable storage medium stores a computer program product, and the computer program product is executed by a processor to implement any of the data processing method steps or the network training method.

[0059] The embodiment of the present application has the following beneficial effects:

[0060] In the technical scheme provided by the embodiment of the present application, the original weight of the network layer is decomposed into N sub-weights, and the N sub-weights are quantized. Compared with the convolution processing of the original weight which is not quantized, the convolution processing of the input data of the network layer and the quantized N sub-weights reduces the consumption of computing resources and expands the application scenarios of the deep neural network. In addition, the deep neural network comprises a plurality of network layers, and the number of sub-weights decomposed from the original weight of each network layer can be different, that is, the number N of sub-weights decomposed from the original weight of each network layer can be set according to actual needs, which greatly improves the flexibility of the design of the deep neural network and balances the efficiency and performance of the network.

[0061] Of course, implementing any product or method of the present application does not necessarily require all the advantages described above. BRIEF DESCRIPTION OF DRAWINGS

[0062] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the accompanying drawings needed to be used in the embodiments or prior art description will be briefly introduced as follows. Obviously, the accompanying drawings in the following description only constitute some embodiments of the present application, and for those skilled in the art, other embodiments can also be obtained based on these accompanying drawings.

[0063] Figure 1 The first flowchart of the data processing method provided by the embodiments of the present application;

[0064] Figure 2 One schematic diagram for decomposing the original weight into sub-weights composed of three values of -1, 0 and 1;

[0065] Figure 3 The second flowchart of the data processing method provided by the embodiments of the present application;

[0066] Figure 4 The third flowchart of the data processing method provided by the embodiments of the present application;

[0067] Figure 5 The first flowchart of the network training method provided by the embodiments of the present application;

[0068] Figure 6 The second flowchart of the network training method provided by the embodiments of the present application;

[0069] Figure 7 The third flowchart of the network training method provided by the embodiments of the present application;

[0070] Figure 8 One structural schematic diagram of the data processing apparatus provided by the embodiments of the present application;

[0071] Figure 9 One structural schematic diagram of the network training apparatus provided by the embodiments of the present application;

[0072] Figure 10 The first structural schematic diagram of the electronic device for implementing the data processing method provided by the embodiments of the present application;

[0073] Figure 11 The second structural schematic diagram of the electronic device for implementing the network training method provided by the embodiments of the present application. DETAILED DESCRIPTION

[0074] With reference to the drawings of the embodiments of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described. Obviously, the described embodiments are only a part of the embodiments of the present application, but not all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art based on the present application are within the scope of protection of the present application.

[0075] In the related art, when the deep neural network is quantized, the weights of each network layer in the deep neural network are quantized by using a uniform specification, which makes the flexibility of the deep neural network design poor and unable to flexibly balance the efficiency and performance of the network.

[0076] In addition, if the weights of each network layer in the deep neural network are quantized by using a uniform specification with a low bit specification, the network layer with high sensitivity is used to process data, which causes loss of effective data; if the weights of each network layer in the deep neural network are quantized by using a uniform specification with a high bit specification, the network layer with low sensitivity is used to process data, which causes waste of computing resources.

[0077] To solve the above problems, the present application provides a data processing method, which can be applied to a server, a personal computer, a mobile phone or a tablet computer, and the like electronic device with a deep neural network. For ease of understanding, the following describes the electronic device as the execution subject, which does not limit the present application.

[0078] In addition, in the embodiments of the present application, the deep neural network can include a plurality of network layers, and each network layer can execute the method. The following only takes one network layer as an example for description, which does not limit the present application.

[0079] The data processing method provided by the embodiments of the present application will be described in detail below through specific embodiments.

[0080] Referring to Figure 1 , Figure 1 The first flowchart of the data processing method provided by the embodiments of the present application is shown in FIG. 1. The data processing method includes the following steps.

[0081] In step S11, the input data of the network layer of the deep neural network is obtained, and the original weight of the network layer is decomposed into N sub-weights for quantization processing.

[0082] In the embodiments of the present application, the deep neural network can be a convolutional neural network, a deep convolutional neural network or other neural networks, which are not limited.

[0083] For each network layer of the deep neural network, the electronic device obtains input data of the network layer. The input data of the network layer can be input data of the deep neural network or output data of a previous network layer in the deep neural network, without limitation, due to different positions of the network layer in the deep neural network.

[0084] In the embodiments of the present application, the original weight of a network layer is decomposed into N sub-weights for quantization processing. Correspondingly, a single convolution of the network layer is also decomposed into N sub-convolutions.

[0085] The electronic device can set the value of N according to the sensitivity of the network layer. The higher the sensitivity of the network layer, the larger the value of N; otherwise, the smaller the sensitivity of the network layer, the smaller the value of N.

[0086] In step S12, the input data is respectively convolved with the N sub-weights to obtain N intermediate data corresponding to the N sub-weights.

[0087] In the embodiments of the present application, for each sub-weight, the electronic device convolves the input data with the sub-weight, that is, the electronic device multiplies each element in the sub-weight with the input data and adds the products to obtain an intermediate data corresponding to the sub-weight.

[0088] In the embodiments of the present application, the original weight of the network layer is decomposed into N sub-weights, so the electronic device obtains N intermediate data.

[0089] In step S13, the N intermediate data are summed to obtain output data of the network layer.

[0090] The electronic device sums the N intermediate data to obtain the output data of the network layer. If the network layer is the last layer of the deep neural network, the output data of the network layer can be the final output data of the deep neural network; if the network layer is the first layer or an intermediate layer of the deep neural network, the output data of the network layer can be input data of a next network layer of the deep neural network, and the processing procedure of the input data by the next network layer can refer to the description in steps S11-S13.

[0091] In the technical scheme provided in the embodiments of the present application, the original weight of the network layer is decomposed into N sub-weights, and the N sub-weights are quantized. Compared with convolution processing of the original weight without quantization, the convolution processing of the input data of the network layer and the quantized N sub-weights reduces the consumption of computing resources and expands the application scenarios of the deep neural network. In addition, the deep neural network includes multiple network layers, and the number of sub-weights obtained by decomposing the original weight of each network layer can be different, that is, the number N of sub-weights obtained by decomposing the original weight of each network layer can be set according to actual needs, which greatly improves the flexibility of the design of the deep neural network and can flexibly balance the efficiency and performance of the network.

[0092] The technical scheme provided in the embodiments of the present application can be applied to application scenarios such as image denoising and face recognition. In different application scenarios, the technical scheme provided in the embodiments of the present application is applied, and the difference lies in the input data and the output data of the network layer. For example, in different image denoising scenarios, the technical scheme provided in the embodiments of the present application is applied, the input data of the network layer is a first image, the output data of the network layer is a second image, and the noise of the first image is higher than that of the second image.

[0093] In an embodiment of the present application, the sub-weight obtained by decomposing the original weight is a value obtained by multiplying the quantized weight corresponding to the sub-weight and a first scale coefficient. For example, the quantized weight of the sub-weight i is represented by T i , and the first scale coefficient is represented by α i , then the sub-weight i can be represented by (T i *α i ), and the value range of i is 1-N, at this time, the original weight W f can be represented as: T1*α1+T2*α2+…+T N *α N .

[0094] In the embodiments of the present application, the difference between the sum of the N sub-weights quantized and the original weight is less than a preset threshold, or the ratio of the sum of the N sub-weights quantized and the original weight is less than a preset ratio. The preset threshold and the preset ratio can also be set according to actual needs, as long as the negative impact of the decomposition on the final output result is reduced as much as possible. The method of decomposing the original weight is not limited in the embodiments of the present application.

[0095] In an optional embodiment, the method of decomposing the original weight can be that the sum of the N sub-weights after decomposition is approximately equal to the original weight W f , that is, W f ≈T1*α1+T2*α2+…+T N *α N .

[0096] In the embodiments of the present application, the quantization weights of different sub-weights can be different or the same, and the first scale coefficients of different sub-weights can be different or the same.

[0097] In the embodiments of the present application, the quantization weight is an integer, for example, the quantization weight can include but is not limited to -1, 1, 2, 3 or 4, etc., which is not limited. The first scale coefficient is a floating point number or an integer, which can be determined according to actual conditions.

[0098] In an optional embodiment, the quantization weight is composed of -1, 0 and 1. For example, N is 3, that is, the original weight of the network layer is decomposed into 3, and the size of the convolution kernel is 3*3, then an original weight can be decomposed into 3 sub-weights, and the quantization weights corresponding to the 3 sub-weights are as shown in Figure 2 The quantization weight is composed of three values -1, 0 and 1, which can also be called a ternary weight.

[0099] In the embodiments of the present application, the quantization weight can be composed of -1, 0 and 1. In this case, the electronic device uses the quantization weight for convolution processing, which can effectively avoid multiplication operation, thereby improving the calculation efficiency, the hardware running efficiency is high, and the loss of computing resources is reduced.

[0100] Based on the sub-weights of the above quantization processing, the embodiments of the present application also provide a data processing method, as shown in Figure 3 The steps S31 and S35 are the same as steps S11 and S13. Steps S32-S34 are an implementation of step S12.

[0101] Step S32, quantizing the input data to obtain quantization data corresponding to the input data and a second scale coefficient.

[0102] In the embodiments of the present application, after obtaining the input data, the electronic device can quantize the input data to obtain quantization data corresponding to the input data and a second scale coefficient. The quantization specification of the input data can be set according to actual conditions, for example, 2bit, 4bit or 8bit, etc., which is not limited.

[0103] The quantization data corresponding to the input data is represented by I int , and the second scale coefficient is represented by S in , then the input data can be represented as I int *S in .

[0104] Step S33, convolving the quantization data with the quantization weights corresponding to the N sub-weights respectively to obtain N convolution data corresponding to the N sub-weights.

[0105] In the embodiments of the present application, for each sub-weight, the electronic device performs convolution processing on the quantized data I int corresponding to the sub-weight, to obtain the convolution data corresponding to the sub-weight. For example, the convolution processing is denoted by the symbol , the quantized data corresponding to the input data is denoted by I int , and the quantized weight of the sub-weight i is denoted by T i , then the convolution data corresponding to the sub-weight i can be denoted by the expression .

[0106] In step S34, the electronic device respectively performs product processing on each convolution data and the first scale coefficient and the second scale coefficient, to obtain N intermediate data corresponding to the N sub-weights.

[0107] In the embodiments of the present application, the intermediate data corresponding to the sub-weight i can be denoted by Y i . For the sub-weight i, the electronic device performs product processing on the convolution data corresponding to the sub-weight i and the first scale coefficient S and the second scale coefficient a in , to obtain an expression of Y i . i

[0108]

[0109] wherein S in and a i are constants, and denotes convolution processing of multiplying first and then adding. Placing the constants S in and a i inside or outside the parentheses does not affect the result.

[0110] Therefore, the formula (1) can be transformed to obtain another expression of Y i .

[0111]

[0112] In the above formula, (I int *S in ) denotes input data, and (T i * a i ) denotes a sub-weight. Therefore, the above formula can also be denoted as performing convolution processing on the input data and the N sub-weights respectively, to obtain N intermediate data corresponding to the N sub-weights.

[0113] ​In the technical solution of this application embodiment, the electronic device quantizes both the input data and the sub-weights, and uses the quantized data to perform convolution operations, which improves the computational efficiency of data processing, has high hardware operating efficiency, and reduces the consumption of computing resources.

[0114] The following is combined with Figure 4 The flowchart shown illustrates the data processing method, providing a detailed explanation of the data processing method provided in this application embodiment. The original weights are decomposed into N sub-weights, and the quantization weights of these sub-weights consist of three values: {-1, 0, 1}, i.e., the quantization weights are three-valued weights. Correspondingly, the convolutions of the network layers are decomposed into three-valued convolutions. Figure 4 In Chinese, convolution processing uses symbols express, This indicates that corresponding elements are added together.

[0115] The electronic device quantizes the input data to obtain the quantized data I corresponding to the input data. int Second scale coefficient S in The electronic device will quantize the input data (I... int *S in ) respectively with the N three-valued weights (T) derived from the original weight decomposition i *α i Perform convolution processing (also known as ternary convolution processing), as shown in formula (2) above, to obtain N intermediate data Y corresponding to N ternary weights. i The electronic device performs summation on n intermediate data points Yi, i.e., it performs... The process yields the output data of the network layer, which is then quantized to (O) int *S out ), where O int S is the quantized data corresponding to the output data. out This is the third scale coefficient.

[0116] In one embodiment of this application, a network training method is also provided, such as... Figure 5 As shown, the method may include the following steps:

[0117] Step S51: Obtain the training set, which includes sample data and the expected data corresponding to the sample data.

[0118] In this embodiment, the number of sample data in the training set can be set according to actual needs. For example, if the accuracy requirement of the network is high, the number of sample data can be set to a larger value; if the performance of the electronic device is poor, the number of sample data can be set to a smaller value.

[0119] In the embodiments of the present application, the training set can be a data set pre-stored in the electronic device, or a data set obtained by the electronic device from a preset database, and no limitation is made thereto.

[0120] In step S52, the sample data is input into the first deep neural network to obtain first prediction data, where the first deep neural network comprises at least one network layer.

[0121] In step S52, the original weights of the network layers of the first deep neural network are not decomposed. The electronic device inputs the sample data into the first deep neural network, and the first deep neural network outputs first prediction data after processing the input data.

[0122] In step S53, a first loss value is determined according to the expected data and the first prediction data.

[0123] In the embodiments of the present application, the electronic device can calculate the difference between the expected data and the first prediction data corresponding to the sample data, and take the difference as the first loss value.

[0124] The electronic device can also use a Mean Squared Error (MSE) function as a loss function, and determine the first loss value by using the expected data and the first prediction data corresponding to the sample data.

[0125] In the embodiments of the present application, the electronic device can also use other algorithms to determine the first loss value, and no limitation is made thereto.

[0126] In step S54, if it is determined that the first deep neural network converges according to the first loss value, the original weights of the network layers are decomposed into N quantized sub-weights to obtain a second deep neural network.

[0127] In the embodiments of the present application, the electronic device can pre-set a loss threshold, i.e., a preset loss threshold. If the determined first loss value is less than the preset loss threshold, the electronic device can determine that the first deep neural network converges, end the training of the first deep neural network, and then decompose the original weights of the network layers in the first deep neural network into N quantized sub-weights to obtain a second deep neural network.

[0128] The method and principle of decomposing the original weights can be referred to the related description of the above Figures 1-4 part.

[0129] In an optional embodiment, the sub-weight is a value obtained by multiplying the quantized weight corresponding to the sub-weight by the first scale factor, and the quantized weight consists of -1, 0 and 1; the difference between the sum of the N sub-weights and the original weight is less than a preset threshold, and the ratio of the sum of the N quantized sub-weights to the original weight is less than a preset ratio.

[0130] In the technical scheme provided in the embodiment, the electronic device trains the first deep neural network using the training set without decomposing the original weight of the network layer, and then decomposes the original weight of the network layer into N quantized sub-weights after the training of the first deep neural network is completed. In the technical scheme provided in the embodiment, the deep neural network can fully learn the data rule of the sample data, and the accuracy of data processing of the deep neural network is improved. Moreover, since the original weight of the network layer is not decomposed when the first deep neural network is trained, the weight adjusted when the first deep neural network is trained is only the original weight, instead of the N sub-weights, so that the number of parameters to be adjusted is reduced, and the training efficiency is improved.

[0131] Based on the network training method, the embodiment of the present application further provides a network training method, as shown in Figure 6 The steps S61-S64 are the same as the steps S51-S54, and details are not repeated here.

[0132] In step S65, if it is determined that the first deep neural network does not converge according to the first loss value, the first parameter of the first deep neural network is adjusted, and step S62 is re-executed; wherein the first parameter includes the original weight.

[0133] In the embodiment, the electronic device can use the back propagation algorithm or the gradient descent algorithm to adjust the first parameter of the first deep neural network, and re-execute step S62 to continue to iteratively train the first deep neural network until it is determined that the first deep neural network converges.

[0134] Because the first parameter of the first deep neural network is adjusted, the first deep neural network will change after the adjustment. At this time, the first prediction data output by the re-sample data will change. This is a repeated training process, and the end condition is to determine that the first deep neural network converges.

[0135] In the technical scheme provided in the embodiment, the electronic device trains the first deep neural network using the training set without decomposing the original weight of the network layer, and then decomposes the original weight of the network layer into N quantized sub-weights after the training of the first deep neural network is completed. Since the original weight of the network layer is not decomposed when the first deep neural network is trained, the weight adjusted when the first deep neural network is trained is only the original weight, instead of the N sub-weights, so that the number of parameters to be adjusted is reduced, and the training efficiency is improved.

[0136] In order to further improve the accuracy of data processing of the deep neural network, the embodiment of the present application further provides a network training scheme, as shown in Figure 7As shown, in the method, steps S71-S74 are the same as steps S51-S54.

[0137] Step S75, inputting the sample data into the second deep neural network to obtain second prediction data. Step S75 can refer to the related description in the part of step S52.

[0138] Step S76, determining a second loss value according to the expected data and the second prediction data. Step S76 can refer to the related description in the part of step S53.

[0139] Step S77, if it is determined that the second deep neural network converges according to the second loss value, ending the training of the second deep neural network. Step S77 can refer to the related description in the part of step S54.

[0140] Step S78, if it is determined that the second deep neural network does not converge according to the second loss value, adjusting a second parameter of the second deep neural network, and re-executing the step of inputting the sample data into the second deep neural network to obtain the second prediction data, the second parameter including a scale coefficient of the N sub-weights. Step S75 can refer to the related description in the part of step S65.

[0141] Figure 7 The description of steps S75-S78 is relatively simple, and specific reference can be made to the related description in the part of the above Figures 5-6 description, which will not be repeated here.

[0142] In the technical scheme provided by the embodiments of the present application, after the original weight is decomposed into N sub-weights, the second deep neural network is obtained, and then the training set is used to re-train the second deep neural network, which reduces the error caused by the decomposition of the original weight and further improves the accuracy of data processing of the deep neural network.

[0143] Corresponding to the above data processing method, the embodiments of the present application also provide a data processing device, as Figure 8 shown, the device comprises:

[0144] The acquisition unit 81 is configured to acquire input data of a network layer of a deep neural network, and original weights of the network layer are decomposed into N sub-weights subjected to quantization processing.

[0145] The convolution unit 82 is configured to perform convolution processing on the input data and the N sub-weights respectively to obtain N intermediate data corresponding to the N sub-weights.

[0146] The summation unit 83 is configured to perform summation processing on the N intermediate data to obtain output data of the network layer.

[0147] In an optional embodiment, the sub-weight is a value obtained by multiplying the quantized weight corresponding to the sub-weight and the first scale coefficient.

[0148] The convolution unit 82 can be specifically used for:

[0149] quantizing the input data to obtain quantized data corresponding to the input data and a second scale coefficient;

[0150] performing convolution processing on the quantized data and quantized weights corresponding to the N sub-weights respectively to obtain N convolution data corresponding to the N sub-weights;

[0151] performing product processing on each convolution data and the first scale coefficient and the second scale coefficient respectively to obtain N intermediate data corresponding to the N sub-weights.

[0152] In an optional embodiment, the quantized weights consist of -1, 0 and 1;

[0153] The sum of the N sub-weights and the original weight has a difference less than a preset threshold, or the sum of the N sub-weights and the original weight has a ratio less than a preset ratio.

[0154] In an optional embodiment, the input data is a first image, and the output data is a second image, and the first image has higher noise than the second image.

[0155] Corresponding to the network training method described above, the embodiments of the present application also provide a network training device, as shown in Figure 9 The device comprises:

[0156] The acquisition unit 91 is configured to acquire a training set, the training set comprising sample data and expected data corresponding to the sample data;

[0157] The first input unit 92 is configured to input the sample data into a first deep neural network to obtain first predicted data, wherein the first deep neural network comprises at least one network layer;

[0158] The first determination unit 93 is configured to determine a first loss value according to the expected data and the first predicted data;

[0159] The decomposition unit 94 is configured to, if it is determined that the first deep neural network converges according to the first loss value, decompose original weights of the network layer into N quantized sub-weights to obtain a second deep neural network.

[0160] In an optional embodiment, the network training device described above can further comprise:

[0161] The first adjustment unit is configured to, if it is determined that the first deep neural network does not converge according to the first loss value, adjust a first parameter of the first deep neural network, and re-perform the step of inputting the sample data into the first deep neural network to obtain the first predicted data; wherein the first parameter comprises the original weights.

[0162] In an optional embodiment, the network training apparatus may further include:

[0163] The second input unit is used to input sample data into the second deep neural network after obtaining the second deep neural network to obtain the second prediction data;

[0164] The second determining unit is used to determine the second loss value based on the expected data and the second predicted data;

[0165] The termination unit is used to terminate the training of the second deep neural network if the second deep neural network is determined to have converged based on the second loss value.

[0166] The second adjustment unit is used to adjust the second parameters of the second deep neural network if it is determined from the second loss value that the second deep neural network has not converged, and to re-execute the step of inputting sample data into the second deep neural network to obtain the second prediction data. The second parameters include the scaling coefficients of N sub-weights.

[0167] In an optional embodiment, the sub-weight is the value obtained by multiplying the quantization weight corresponding to the sub-weight with the first scale coefficient, and the quantization weight consists of -1, 0 and 1;

[0168] The difference between the sum of the N sub-weights and the original weight is less than a preset threshold, or the ratio of the sum of the N sub-weights to the original weight is less than a preset ratio.

[0169] In the technical solution provided in this application embodiment, the original weights of the network layer are decomposed into N sub-weights, and these N sub-weights are quantized. Compared to convolution with the unquantized original weights, convolving the input data of the network layer with each of the N quantized sub-weights reduces the consumption of computational resources and expands the application scenarios of deep neural networks. Furthermore, since deep neural networks include multiple network layers, the number of sub-weights obtained from the decomposition of the original weights of each network layer can be different. That is, the number N of sub-weights obtained from the decomposition of the original weights of each network layer can be set according to actual needs. This greatly improves the flexibility of deep neural network design and allows for a flexible balance between network efficiency and performance.

[0170] Corresponding to the above data processing method, embodiments of this application also provide an electronic device, such as... Figure 10 As shown, the device includes a processor 101 and a machine-readable storage medium 102. The machine-readable storage medium 102 stores machine-executable instructions that can be executed by the processor 101. The processor 101 is prompted by the machine-executable instructions to implement any of the above-described data processing method steps.

[0171] Corresponding to the above-described network training method, this application also provides an electronic device, such as...Figure 11 As shown, the apparatus includes a processor 111 and a machine readable storage medium 112, the machine readable storage medium 112 stores machine executable instructions capable of being executed by the processor 111, and the processor 111 is prompted by the machine executable instructions to implement any of the above network training method steps.

[0172] The machine readable storage medium can include a random access memory (RAM), and can also include a non-volatile memory (NVM), for example at least one disk memory. Optionally, the machine readable storage medium can also be at least one storage device located away from the aforementioned processor.

[0173] The processor can be a general purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component.

[0174] In yet another embodiment provided in the present application, a computer readable storage medium is also provided, the computer readable storage medium stores a computer program, and the computer program is executed by a processor to implement any of the above data processing method steps.

[0175] In yet another embodiment provided in the present application, a computer readable storage medium is also provided, the computer readable storage medium stores a computer program, and the computer program is executed by a processor to implement any of the above network training method steps.

[0176] In yet another embodiment provided in the present application, a computer program product containing instructions is also provided, when the computer program product runs on a computer, the computer program product causes the computer to execute any of the above data processing method steps.

[0177] In yet another embodiment provided in the present application, a computer program product containing instructions is also provided, when the computer program product runs on a computer, the computer program product causes the computer to execute any of the above network training method steps.

[0178] In the embodiments described above, all or some of the steps can be implemented by hardware, software, firmware or any combination thereof. When implemented by software, all or some of the steps can be implemented in the form of one or more computer programs or program elements. The computer programs reside (at least temporarily) in a memory of a computer during execution. The memory can be a RAM memory, a flash memory, a ROM memory, an EPROM memory, or any other suitable memory. The memory can be integral to or separate from the computer. The computer programs can be written in any suitable programming language, such as C, C++, Java, Visual Basic, etc. The computer programs can be written in assembly or machine language, if desired. The computer programs can be distributed over network coupled file servers, or can be distributed by any other suitable means.

[0179] It is to be understood that the terminology “first”, “second”, etc. is used in this document only to distinguish one entity or operation from another entity or operation, and does not necessarily imply or suggest any such actual relationship or order between the entities or operations. Also, the terms “comprises”, “comprising”, or any other variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by an occurrence of the phrase “comprises a” does not exclude the existence of additional identical elements in the process, method, article, or apparatus including the element.

[0180] Each of the embodiments in the present specification is described in a related manner, and the same or similar parts between the embodiments can be referred to each other. Each of the embodiments focuses on the difference from other embodiments. In particular, for the system embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the part of the description of the method embodiments.

[0181] The above merely provides the preferred embodiment of the present application, and not intended to limit the protection scope of the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A data processing method, characterized in that, The method includes: The input data of the network layer of a deep neural network is obtained. The original weights of the network layer are decomposed into N sub-weights after quantization. The input data is a first image. The deep neural network is an image processing network. The N sub-weights are obtained by decomposing the original weights of the network layer in the trained deep neural network. The N is set according to the sensitivity of the network layer. The higher the sensitivity of the network layer, the larger the N is. The lower the sensitivity of the network layer, the smaller the N is. The input data is convolved with each of the N sub-weights to obtain N intermediate data corresponding to the N sub-weights; The N intermediate data are summed to obtain the output data of the network layer. The output data is a second image, and the noise of the first image is higher than that of the second image.

2. The method according to claim 1, characterized in that, The sub-weight is the value obtained by multiplying the quantization weight corresponding to the sub-weight with the first scale coefficient; The step of convolving the input data with the N sub-weights respectively to obtain N intermediate data corresponding to the N sub-weights includes: The input data is quantized to obtain the quantized data and the second scale coefficient corresponding to the input data; The quantized data is convolved with the quantized weights corresponding to the N sub-weights to obtain N convolutional data corresponding to the N sub-weights; Each convolutional data point is multiplied by the first scale coefficient and the second scale coefficient to obtain N intermediate data points corresponding to the N sub-weights.

3. The method according to claim 2, characterized in that, The quantization weights consist of -1, 0, and 1; The difference between the sum of the N sub-weights and the original weight is less than a preset threshold, or the ratio of the sum of the N sub-weights to the original weight is less than a preset ratio.

4. A network training method, characterized in that, The method includes Obtain a training set, which includes sample data and expected data corresponding to the sample data. The sample data and the expected data are images, and the noise of the sample data is higher than the noise of the expected data. The sample data is input into a first deep neural network to obtain first prediction data, wherein the first deep neural network includes at least one network layer and is an image processing network. Based on the expected data and the first predicted data, a first loss value is determined; If the convergence of the first deep neural network is determined based on the first loss value, the original weights of the network layer are decomposed into N quantized sub-weights to obtain a second deep neural network. The N is set according to the sensitivity of the network layer. The higher the sensitivity of the network layer, the larger the N; the lower the sensitivity of the network layer, the smaller the N.

5. The method according to claim 4, characterized in that, The method further includes: If it is determined that the first deep neural network has not converged based on the first loss value, then the first parameter of the first deep neural network is adjusted, and the step of inputting the sample data into the first deep neural network to obtain the first predicted data is re-executed; wherein, the first parameter includes the original weight.

6. The method according to claim 4, characterized in that, After obtaining the second deep neural network, the process also includes: The sample data is input into the second deep neural network to obtain the second prediction data; Based on the expected data and the second predicted data, a second loss value is determined; If the second deep neural network is determined to have converged based on the second loss value, then the training of the second deep neural network ends. If it is determined that the second deep neural network has not converged based on the second loss value, then the second parameters of the second deep neural network are adjusted, and the step of inputting the sample data into the second deep neural network to obtain the second prediction data is executed again. The second parameters include the scaling coefficients of the N sub-weights.

7. The method according to any one of claims 4-6, characterized in that, The sub-weight is the value obtained by multiplying the quantization weight corresponding to the sub-weight with the first scale coefficient, and the quantization weight consists of -1, 0 and 1; The difference between the sum of the N sub-weights and the original weight is less than a preset threshold, or the ratio of the sum of the N sub-weights to the original weight is less than a preset ratio.

8. An electronic device, characterized in that, The method includes a processor and a machine-readable storage medium storing machine-executable instructions that can be executed by the processor, the processor being prompted by the machine-executable instructions to perform the steps of the method according to any one of claims 1-3 or 4-7.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program product, which, when executed by a processor, implements the steps of the method described in any one of claims 1-3 or 4-7.

Images