Network training method and device, network optimization method and data processing method

Abstract

The present disclosure provides a network training method and device, a network optimization method and a data processing method. The network training method comprises: quantizing a sample neural network to obtain a sample quantization matrix in a first state, the matrix being used to indicate the type of an operator in the network and the connection relationship between the operators; performing optimization function prediction on the sample quantization matrix in the mth state through a reinforcement network in the mth state to determine a sample optimization function in the mth state; optimizing the sample neural network in the mth state through the sample optimization function in the mth state to obtain a sample neural network in the m+1th state; training the reinforcement network in the mth state according to the reward values of the sample neural networks in the mth state and the m+1th state to obtain a reinforcement network in the m+1th state; and determining the trained reinforcement network under the condition that a training condition is met. The embodiments of the present disclosure can improve the training effect of the reinforcement network.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of computer, and in particular, to a network training method and device, a network optimization method and device, a data processing method and device, an electronic device, and a computer readable medium. BACKGROUND

[0002] With the continuous development of machine learning technology, machine learning has been widely applied in various fields such as computer vision, speech recognition, and natural language processing. In machine learning, various neural network models (such as convolutional neural network CNN, spiking neural network SNN, etc.) are often used to perform various tasks (such as image processing tasks, speech recognition tasks, etc.). The neural network usually includes various operators (OP, operators), such as two-dimensional convolution operator Conv2d, addition operator Add, activation operator Relu and Sigmoid, etc., which are used to implement various computing processes of the neural network.

[0003] Before deploying the neural network to an electronic device, the neural network can usually be optimized, such as operator fusion, invalid node removal, etc., so as to improve the processing efficiency of the neural network. For example, the multiplication operator Mul can be merged into the two-dimensional convolution operator Conv2d, so as to reduce the amount of calculation and reduce the data carrying process.

[0004] In related technologies, a first optimization function is usually manually selected, and the first optimization function is used to update the neural network to be optimized to obtain a first optimized neural network. Then a second optimization function is manually selected, and the first optimized neural network is continuously updated to obtain a second optimized neural network. In this way, the neural network is optimized multiple times until the preset condition (such as the amount of calculation no longer decreases) is met, and the optimization is stopped, and the final optimization result is obtained. Alternatively, multiple optimization functions with fixed or random execution order are used to sequentially optimize the neural network, and the final optimization result is obtained.

[0005] However, the type selection of the optimization function and / or the order of the optimization function will affect the optimization result of the neural network. Therefore, the above processing mode of manually selecting the optimization function or using a fixed or random optimization function order to optimize the neural network has low accuracy of the optimization result and low optimization efficiency. SUMMARY

[0006] The present disclosure provides a network training method and device, a network optimization method and device, a data processing method and device, an electronic device, and a computer readable medium.

[0007] In a first aspect, the present disclosure provides a network training method, comprising:

[0008] The sample neural network is quantized to obtain a sample quantization matrix of the first state, the sample quantization matrix being used to indicate a type of operator and a connection relationship between operators in the sample neural network; an optimization function of the sample quantization matrix of the mth state is predicted by the reinforcement network of the mth state, m being an integer greater than or equal to 1, the optimization function being used to fuse operators of a neural network; the sample neural network of the mth state corresponding to the sample quantization matrix of the mth state is optimized by the sample optimization function of the mth state to obtain a sample neural network of an (m+1)th state; the reinforcement network of the mth state is trained according to a reward value of the sample neural network of the mth state and a reward value of the sample neural network of the (m+1)th state to obtain a reinforcement network of the (m+1)th state; in a case where the reinforcement network of the (m+1)th state meets a training condition, the reinforcement network of the (m+1)th state is determined as a trained reinforcement network, wherein the reinforcement network is used to predict an optimization function matched with a target network to be optimized, and the target network is used to perform a data processing task.

[0009] In some possible implementation manners, the method further includes: determining a reward value of the sample neural network of the (m+1)th state according to a performance parameter of the sample neural network of the (m+1)th state, the performance parameter including at least one of a calculation amount of the neural network, a processing time length, a processing accuracy, and a storage space occupation.

[0010] In some possible implementation manners, the method further includes: processing preset sample data by the sample neural network of the (m+1)th state to determine a processing result of the sample data and a processing time length of the sample neural network of the (m+1)th state; and determining a processing accuracy of the sample neural network of the (m+1)th state according to labeled information of the sample data and the processing result.

[0011] In some possible implementation manners, the training of the reinforcement network of the mth state according to the reward value of the sample neural network of the mth state and the reward value of the sample neural network of the (m+1)th state to obtain the reinforcement network of the (m+1)th state includes: determining a reward difference of the mth optimization according to a difference between the reward value of the sample neural network of the (m+1)th state and the reward value of the sample neural network of the mth state; determining a network loss of the mth optimization according to the reward difference of the mth optimization; and training the reinforcement network of the mth state according to the network loss of the mth optimization to obtain the reinforcement network of the (m+1)th state.

[0012] In some possible implementation manners, the reinforcement network comprises any one of an artificial neural network (ANN), a spiking neural network (SNN), or a neural network that combines the artificial neural network (ANN) and the spiking neural network (SNN).

[0013] In some possible implementation manners, the data processing task comprises any one of an image processing task, a speech processing task, a text processing task, or a video processing task.

[0014] In a second aspect, the present disclosure provides a network optimization method, which comprises:

[0015] quantizing a target network to be optimized to obtain a quantized matrix of the target network, performing optimization function prediction on the quantized matrix of the target network by a reinforcement network to determine a target optimization function that matches the target network from a plurality of preset optimization functions, and optimizing the target network by the target optimization function to obtain an optimized target network, wherein the reinforcement network is trained according to the network training method described above.

[0016] In a third aspect, the present disclosure provides a data processing method, which comprises processing data to be processed by a target network to obtain a processing result of the data to be processed, wherein the target network is optimized according to the network optimization method described above.

[0017] In a fourth aspect, the present disclosure provides a network training apparatus, which comprises:

[0018] A first quantization module configured to quantize a sample neural network to obtain a sample quantized matrix in a first state, the sample quantized matrix being used to indicate the types of operators in the sample neural network and the connection relationships between the operators.

[0019] A first prediction module configured to perform optimization function prediction on the sample quantized matrix in the mth state by a reinforcement network in the mth state to determine a sample optimization function in the mth state, m being an integer greater than or equal to 1, the optimization function being used to fuse the operators of a neural network, and the reinforcement network in the first state being an untrained reinforcement network.

[0020] A first optimization module configured to optimize a sample neural network in the mth state corresponding to the sample quantized matrix in the mth state by the sample optimization function in the mth state to obtain a sample neural network in an (m+1)th state.

[0021] A training module configured to train the reinforcement network in the mth state according to a reward value of the sample neural network in the mth state and a reward value of the sample neural network in the (m+1)th state to obtain a reinforcement network in an (m+1)th state.

[0022] a network determination module configured to determine the reinforcement network in the m+1th state as a trained reinforcement network if the reinforcement network in the m+1th state meets a training condition,

[0023] The reinforcement network is configured to predict an optimization function matched with a target network to be optimized, and the target network is configured to perform a data processing task.

[0024] In some possible implementation manners, the apparatus further includes a benefit value determination module configured to determine a benefit value of the sample neural network in the m+1th state according to a performance parameter of the sample neural network in the m+1th state, the performance parameter including at least one of a calculation amount, a processing time length, a processing accuracy, and a storage space occupation of the neural network.

[0025] In some possible implementation manners, the apparatus further includes a result determination module configured to determine a processing result of preset sample data by the sample neural network in the m+1th state and a processing time length of the sample neural network in the m+1th state, and an accuracy determination module configured to determine a processing accuracy of the sample neural network in the m+1th state according to labeled information of the sample data and the processing result.

[0026] In some possible implementation manners, the training module is configured to: determine a benefit difference of the mth optimization according to a difference between the benefit value of the sample neural network in the m+1th state and the benefit value of the sample neural network in the mth state; determine a network loss of the mth optimization according to the benefit difference of the mth optimization; and train the reinforcement network in the mth state according to the network loss of the mth optimization to obtain the reinforcement network in the m+1th state.

[0027] In some possible implementation manners, the reinforcement network includes any one of an artificial neural network (ANN), a spiking neural network (SNN), or a neural network that combines the ANN and the SNN.

[0028] In some possible implementation manners, the data processing task includes any one of an image processing task, a speech processing task, a text processing task, or a video processing task.

[0029] In a fifth aspect, the present disclosure provides a network optimization apparatus, which includes:

[0030] The second quantization module is configured to quantize the target network to be optimized to obtain a quantization matrix of the target network; the second prediction module is configured to perform optimization function prediction on the quantization matrix of the target network by using the reinforcement network, and determine a target optimization function matched with the target network from a plurality of preset optimization functions; and the third optimization module is configured to optimize the target network by using the target optimization function to obtain an optimized target network, wherein the reinforcement network is trained according to the network training method.

[0031] In a sixth aspect, the present disclosure provides a data processing apparatus, comprising: a data processing module configured to process data to be processed by using a target network to obtain a processing result of the data to be processed, wherein the target network is optimized according to the network optimization method.

[0032] In a seventh aspect, the present disclosure provides an electronic device, comprising: at least one processor; and a memory connected with the at least one processor in communication; wherein the memory stores one or more computer programs executable by the at least one processor, and the one or more computer programs are executed by the at least one processor to enable the at least one processor to execute the network training method, or the network optimization method, or the data processing method.

[0033] In an eighth aspect, the present disclosure provides a computer readable medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the network training method, or the network optimization method, or the data processing method.

[0034] The embodiments provided by the present disclosure can quantize a sample neural network into a matrix, predict an optimization function matched with the sample neural network by using a reinforcement network, optimize the sample neural network by using the optimization function, and train the reinforcement network by using the reward values of the sample neural network before and after optimization, thereby improving the training effect of the reinforcement network, making the prediction of the optimization function by the trained reinforcement network more accurate, and improving the accuracy and efficiency of network optimization.

[0035] It should be understood that the content described in this section is not intended to identify key or important features of the embodiments of the present disclosure, nor is it used to limit the scope of the present disclosure. Other features of the present disclosure will become apparent from the following description. BRIEF DESCRIPTION OF DRAWINGS

[0036] The accompanying drawings are included to provide a further understanding of the present disclosure and are incorporated in and constitute a part of the specification, illustrate embodiments of the present disclosure and are used to explain the present disclosure and are not intended to limit the present disclosure. The above and other features and advantages of the present disclosure will become more apparent from the detailed description in conjunction with the accompanying drawings, in which:

[0037] Figure 1 A flowchart of a network training method provided for an embodiment of the present disclosure;

[0038] Figure 2 A schematic diagram of a neural network provided for an embodiment of the present disclosure;

[0039] Figure 3 A schematic diagram of a processing procedure of a network training method provided for an embodiment of the present disclosure;

[0040] Figure 4 A flowchart of a network optimization method provided for an embodiment of the present disclosure;

[0041] Figure 5 A block diagram of a network training apparatus provided for an embodiment of the present disclosure;

[0042] Figure 6 A block diagram of an electronic device provided for an embodiment of the present disclosure. DETAILED DESCRIPTION

[0043] In order for those skilled in the art to better understand the technical solutions of the present disclosure, the exemplary embodiments of the present disclosure are described below in conjunction with the accompanying drawings, which include various details of the embodiments of the present disclosure to help understanding, and should be considered only as exemplary. Therefore, those skilled in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Also, for the sake of clarity and conciseness, the description below omits the description of well-known functions and structures.

[0044] The embodiments of the present disclosure and the features in the embodiments can be combined with each other without conflict.

[0045] As used herein, the term "and / or" includes any and all combinations of one or more of the associated listed items.

[0046] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the disclosure. As used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms "comprises" and / or "comprising," when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof. "Coupled" or "connected" or similar terms are not restricted to physical or mechanical connections or associations, but can also include electrical connections, whether direct or indirect.

[0047] Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art. It will be further understood that terms, such as those defined in commonly used dictionaries, should be interpreted as having a meaning that is consistent with their meaning in the context of the relevant art and the present disclosure, and will not be interpreted in an overly literal or overly formal sense unless expressly so defined herein.

[0048] In the related art, reinforcement learning is a learning method in machine learning, which emphasizes how to act based on an environment to achieve the maximum expected reward. Reinforcement learning has at least the following basic concepts: environment, agent, state, action, and reward.

[0049] The environment is an external system in which the agent is located, can perceive the system and can make certain actions based on the perceived state. For example, in the game MR (Montezuma's Revenge), the environment is an 80x80 pixel game interface. The state of the environment is represented as S, where St is a state of the environment at time t.

[0050] The agent is a system embedded in the environment, which can change the state of the environment by taking actions. For example, in MR, the agent is the small man controlled by the player, who can make a move (up, down, left, right or jump) according to the current state of the environment, thereby changing the state of the environment.

[0051] The action refers to the behavior of the agent. In MR, it refers to the up, down, left, right or jump operation of the small man. The action of the agent is represented as A, where At is an action of the agent at time t.

[0052] The state refers to a time slice of the current environment. For example, in MR, it is an 80x80 pixel picture at a specific time.

[0053] The reward is a scalar, which refers to an environment reward for a current action or state. For example, in the MR, it refers to a reward defined by the system, which can be a global reward such as Game Over or Win at the end of a game round, or a local reward such as obtaining a "key" or going to another "room". The reward of the environment is denoted as R, where At is an action taken by the agent in state St at time t, and Rt+1 is the corresponding reward. The system will obtain the value of the evaluated action at time t+1, which can be referred to as Q value, representing the expectation of the sum of rewards from the state of the agent selecting this action to the final state. The system will also obtain the value of the evaluated state at time t+1, which can be referred to as V value, representing the expectation of the sum of rewards from the state of the agent to the final state.

[0054] According to an embodiment of the present disclosure, a reinforcement network is provided, and a training sample set for the reinforcement network is provided, the reinforcement network being configured to predict an optimization function matched with a sample neural network by reinforcement learning; and the sample neural network in the training sample set is configured to train the reinforcement network.

[0055] According to the network training method of the embodiment of the present disclosure, the reinforcement network can be used to determine the optimization function matched with the sample neural network, and the sample neural network is optimized by using the optimization function; and the reinforcement network is trained by using the reward values of the sample neural network before and after optimization, so that the prediction of the trained reinforcement network on the optimization function is more accurate, thereby improving the accuracy and optimization efficiency of the neural network optimization.

[0056] The network training method according to the embodiment of the present disclosure can be executed by an electronic device such as a terminal device or a server. The terminal device can be a vehicle-mounted device, a user equipment (UE), a mobile device, a user terminal, a terminal, a cellular phone, a cordless phone, a personal digital assistant (PDA), a handheld device, a computing device, a vehicle-mounted device, a wearable device, etc. The method can be implemented by a processor invoking computer-readable instructions stored in a memory. Alternatively, the method can be executed by a server.

[0057] Figure 1 A flowchart of a network training method according to an embodiment of the present disclosure is provided. Referring to Figure 1 , the method comprises:

[0058] In step S11, the sample neural network is quantized to obtain a sample quantization matrix of the first state, and the sample quantization matrix is used to indicate the types of operators in the sample neural network and the connection relationship between the operators.

[0059] In step S12, an optimization function of the mth state is determined by the reinforcement network of the mth state to predict the sample quantization matrix of the mth state, where m is an integer greater than or equal to 1, and the optimization function is used to fuse the operators of the neural network.

[0060] In step S13, the sample neural network of the mth state corresponding to the sample quantization matrix of the mth state is optimized by the sample optimization function of the mth state to obtain a sample neural network of an (m+1)th state.

[0061] In step S14, the reinforcement network of the mth state is trained according to the reward value of the sample neural network of the mth state and the reward value of the sample neural network of the (m+1)th state to obtain a reinforcement network of the (m+1)th state.

[0062] In step S15, if the reinforcement network of the (m+1)th state meets the training condition, the reinforcement network of the (m+1)th state is determined as the trained reinforcement network.

[0063] The reinforcement network is used to predict an optimization function matched with a target network to be optimized, and the target network is used to perform a data processing task.

[0064] For example, the target network to be optimized can be a trained neural network that can directly perform a corresponding data processing task. However, the execution efficiency or accuracy of the target network can be low, and the operators in the target network can be optimized. The target network can be any type of neural network, such as an artificial neural network (ANN) or a spiking neural network (SNN), and the present disclosure does not limit the type of neural network.

[0065] In some possible implementations, the data processing task performed by the target network can be any type of task, such as any of an image processing task, a speech processing task, a text processing task, or a video processing task, and the present disclosure does not limit the type of task performed by the target network.

[0066] In some possible implementations, a reinforcement network can be preset to predict an optimization function matched with the target network. The reinforcement network can be any type of network, such as an artificial neural network (ANN), a spiking neural network (SNN), or a neural network that combines an artificial neural network (ANN) and a spiking neural network (SNN).

[0067] In one example, the reinforcement network is a convolutional neural network, which includes multiple fully connected layers, such as three fully connected layers, and can also include convolutional layers, activation layers, and the like. The present disclosure does not limit the specific type and network structure of the reinforcement network.

[0068] Before applying the reinforcement network, the reinforcement network can be trained through steps S11-S15.

[0069] In some possible implementation manners, a training sample set for the reinforcement network can be provided, the training sample set including at least one trained sample neural network and sample data corresponding to each sample neural network, the sample data having annotation information. The sample neural network can be any type of neural network such as an artificial neural network (ANN) or a spiking neural network (SNN), and the present disclosure does not limit this.

[0070] In some possible implementation manners, the training sample set can include sample neural networks of various types for processing various tasks. In the training process, the reinforcement network can be trained using multiple sample neural networks, each sample neural network performing multiple rounds of training in a training batch, so that the reinforcement network can be adapted to various neural networks.

[0071] For any training batch, when the first round of training starts, the initial sample neural network corresponding to the training batch can be quantized in step S11 to obtain a sample quantization matrix in a first state, so as to be input into the reinforcement network for processing.

[0072] In some possible implementation manners, the sample quantization matrix is used to indicate the types of operators in the sample neural network and the connection relationship between the operators. For example, the rows and columns of the sample quantization matrix can correspond to the operators of the sample neural network respectively, and the values of the row and column positions corresponding to the same operator correspond to the type of the operator; the values of the row and column positions corresponding to the operators having a connection relationship are 1, and the values of the row and column positions corresponding to the operators not having a connection relationship are 0.

[0073] Figure 2 A schematic diagram of a neural network provided by an embodiment of the present disclosure.

[0074] Referring to Figure 2 The neural network is a commonly used residual network (Resnet), which includes left and right branches. The left branch includes, in sequence, a two-dimensional convolution operator Conv2d, a bias addition operator BiasAdd, and a batch normalization operator BN. The right branch includes, in sequence, a two-dimensional convolution operator Conv2d, a bias addition operator BiasAdd, a batch normalization operator BN, an activation operator Relu, a two-dimensional convolution operator Conv2d, a bias addition operator BiasAdd, a batch normalization operator BN, and an activation operator Relu. Both the left and right branches are connected to an addition operator Add.

[0075] The input data is input into the two-dimensional convolution operator Conv2d on the left and right branch respectively for calculation. After each operator is processed in turn, the calculation result of the last addition operator Add is output as the processing result of the residual network. The calculation result of the former operator is used as the input of the latter operator, that is, there is a connection relationship between the former operator and the latter operator.

[0076] In some possible implementation manners, the two-dimensional convolution operator Conv2d, the bias addition operator BiasAdd, the batch normalization operator BN, the activation operator Relu, and the addition operator Add can be respectively assigned values of 2, 3, 4, 5, and 6. The assignment of the row and column positions of the operator pair with the connection relationship is 1, and the assignment of the row and column positions of the operator pair without the connection relationship is 0. Thus, the following quantization matrix (also referred to as a state table) can be obtained:

[0077] Table 1: Value table of the quantization matrix

[0078]

[0079] As can be seen, the residual network shown in FIG. 2 corresponds to the 12*12 quantization matrix shown in Table 1. Figure 2

[0080] It should be understood that the above is only an exemplary quantization manner of the sample neural network. A person skilled in the art can set any quantization manner according to actual conditions, as long as the type of the operator in the sample neural network and the connection relationship between the operators can be indicated. The present disclosure does not limit this.

[0081] In some possible implementation manners, after the sample quantization matrix of the sample neural network is obtained, the optimization function prediction can be performed by the reinforcement network in step S12.

[0082] In some possible implementation manners, the sample quantization matrix in the first state can be used as an initial state in the reinforcement learning, input into the initial reinforcement network (referred to as the reinforcement network in the first state) for processing, and output the prediction values corresponding to the plurality of optimization functions to be selected. That is, the value of each optimization function for the sample quantization matrix in the first state (initial state). The greater the prediction value, the higher the value and the greater the benefit, and the better the optimization effect of the corresponding optimization function on the sample neural network; otherwise, the smaller the prediction value, the lower the value and the smaller the benefit, and the worse the optimization effect of the corresponding optimization function on the sample neural network.

[0083] In some possible implementation manners, the output result of the reinforcement network can be a vector, the length of the vector corresponds to the number of optimization functions to be selected, and the value of each element in the vector corresponds to the prediction value of the optimization function. The prediction value can also be represented as a confidence degree, a score, etc., and the present disclosure does not limit this.​

[0084] In some possible implementation manners, the optimization function corresponding to the maximum prediction value in the vector output by the reinforcement network can be taken as the sample optimization function of the first state.

[0085] In some possible implementation manners, the plurality of optimization functions to be selected can be optimization functions in a preset function library, and are respectively used for fusing the operators of the sample neural network to be optimized. The fusion manner is associated with the types of the operators of the sample neural network and the connection relationship between the operators, and can also be associated with the structure of the hardware device used for executing the sample neural network. It should be understood that the number and types of the optimization functions can be set by the person skilled in the art according to actual conditions, and the present disclosure does not limit this.

[0086] In some possible implementation manners, the initial sample neural network (which can be referred to as the sample neural network in the first state) can be optimized by using the sample optimization function of the first state to obtain a sample neural network in a second state, so as to complete the current optimization process. The present disclosure does not limit the specific manner of network optimization.

[0087] In some possible implementation manners, the performance parameters of the sample neural network before and after optimization can be used to determine the benefit values of the sample neural network before and after optimization, and the network loss of the first optimization can be determined according to the difference between the benefit values. The reinforcement network in the first state is trained according to the network loss, the network parameters of the reinforcement network are updated, and the first round of training process is completed. The reinforcement network can be trained by using the BP (back propagation) manner, and the present disclosure does not limit the specific training manner of the reinforcement network.

[0088] The performance parameters include, for example, at least one of the calculation amount, the processing time length, the processing accuracy, and the occupied storage space of the neural network, and the present disclosure does not limit this.

[0089] For any mth round (m is an integer greater than or equal to 1) of training in the training batch, when m = 1, the sample quantization matrix in the first state is directly used; when m > 1, the sample neural network in the mth state is quantized to obtain a sample quantization matrix in the mth state. The specific quantization manner is similar to the foregoing, which will not be described herein again.

[0090] In some possible implementation manners, in step S12, the sample quantization matrix in the mth state can be taken as the mth state in the reinforcement learning, input into the reinforcement network in the mth state for processing, and the sample optimization function in the mth state is determined from the vector output by the reinforcement network. The specific determination manner is similar to the foregoing, which will not be described herein again.

[0091] In some possible implementation manners, in step S13, the sample neural network in the mth state can be optimized by using the sample optimization function in the mth state to obtain a sample neural network in an (m+1)th state. The disclosure does not limit the specific manner of network optimization.

[0092] In some possible implementation manners, in step S14, according to the reward difference between the reward values of the sample neural network in the mth state and the sample neural network in the (m+1)th state, a network loss of the mth optimization can be determined, and then the reinforcement network is trained according to the network loss to update the network parameters of the reinforcement network, and the mth training process is completed. The disclosure does not limit the specific training manner.

[0093] In some possible implementation manners, in step S15, if the reinforcement network in the (m+1)th state meets the training condition, the reinforcement network in the (m+1)th state can be determined as the reinforcement network after the training of the training batch, and the training process of the training batch is completed. Otherwise, if the neural network in the (m+1)th state does not meet the preset condition, the training process (steps S12-S14) can be continued. The training condition can be, for example, that the reinforcement network converges, or reaches a preset number of training rounds, and the disclosure does not limit this.

[0094] In some possible implementation manners, a plurality of sample neural networks in the training sample set can be used to train the reinforcement network respectively. In this way, through the training of multiple training batches, the reinforcement network suitable for predicting the optimization function of various neural networks can be obtained, and the adaptability and accuracy of the reinforcement network can be improved.

[0095] According to the embodiments of the disclosure, the sample neural network can be quantized into a matrix, the optimization function matched with the sample neural network can be predicted by using the reinforcement network, the sample neural network can be optimized by using the optimization function, and the reinforcement network can be trained by using the reward values of the sample neural network before and after the optimization, so that the training effect of the reinforcement network can be improved, the prediction of the optimization function by the trained reinforcement network is more accurate, and thus the accuracy and optimization efficiency of network optimization can be improved.

[0096] The network training method according to the embodiments of the disclosure will be described below.

[0097] As described above, for any training batch, when the first round of training starts, the initial sample neural network corresponding to the training batch can be quantized in step S11 to obtain a sample quantization matrix in a first state, so as to be input into the reinforcement network for processing.

[0098] In some possible implementation manners, the sample quantization matrix of the first state can be taken as an initial state in reinforcement learning, input into an initial reinforcement network (a reinforcement network of the first state) for processing, and a predicted value corresponding to the to-be-selected optimization function is output; and the sample optimization function of the first state is determined according to the predicted value. That is, the optimization function corresponding to the maximum predicted value in the vector output by the reinforcement network is taken as the sample optimization function of the first state.

[0099] In some possible implementation manners, the sample neural network of the first state can be optimized by using the sample optimization function of the first state to obtain a sample neural network of a second state, so as to complete the optimization process. The specific optimization manner of the network is not limited in the present disclosure.

[0100] In some possible implementation manners, the performance parameters of the sample neural network before and after optimization can be used to determine the revenue values of the sample neural network before and after optimization, and the network loss of the first optimization is determined according to the difference between the revenue values. The reinforcement network is trained according to the network loss, the network parameters of the reinforcement network are updated, and the first round of training process is completed. The specific training manner is not limited in the present disclosure.

[0101] For any mth round (m is an integer greater than or equal to 1) of training in the training batch, when m = 1, the sample quantization matrix of the first state is directly used; when m > 1, the sample neural network of the mth state is quantized to obtain a sample quantization matrix of the mth state. The specific quantization manner is similar to the foregoing, which will not be described herein again.

[0102] In some possible implementation manners, in step S12, the sample quantization matrix of the mth state can be taken as the mth state in reinforcement learning, input into the reinforcement network for processing, and the sample optimization function of the mth state is determined from the vector output by the reinforcement network. The specific determination manner is similar to the foregoing, which will not be described herein again.

[0103] In some possible implementation manners, in step S13, the sample neural network of the mth state is optimized by using the sample optimization function of the mth state, and a sample neural network of an (m+1)th state can be obtained. The specific optimization manner of the network is not limited in the present disclosure.

[0104] In some possible implementation manners, the revenue value of the sample neural network of the (m+1)th state is obtained, so as to calculate the network loss. The revenue value can be determined by the performance parameters of the sample neural network, and the performance parameters can include at least one of the calculation amount, the processing time length, the processing accuracy, and the occupied storage space of the neural network. The content included in the performance parameters is not limited in the present disclosure.

[0105] In some possible implementation manners, the network optimization method according to the embodiment of the present disclosure can further include the following for the sample neural network in the m+1th state:

[0106] processing the preset sample data by using the sample neural network in the m+1th state, determining a processing result of the sample data and a processing duration of the sample neural network in the m+1th state;

[0107] determining a processing accuracy of the sample neural network in the m+1th state according to the annotation information of the sample data and the processing result.

[0108] For example, the training sample set can include sample data corresponding to the sample neural network, and the type of the sample data depends on the task type of the sample neural network, for example, when the sample neural network is used to perform an image processing task, the sample data can be an image, which is not limited in the present disclosure.

[0109] In some possible implementation manners, the sample data can be input to the sample neural network in the m+1th state, and the processing result of the sample data can be output, and the processing duration and other parameters in the processing process of the sample neural network in the m+1th state can also be determined.

[0110] In some possible implementation manners, the processing accuracy of the sample neural network in the m+1th state can be determined according to the annotation information of the sample data and the processing result, for example, the position difference between the annotated face frame and the detected face frame, the detection rate of the face frame in multiple samples, and the like, and the specific calculation manner of the processing accuracy is not limited in the present disclosure.

[0111] In this way, the performance parameter of the sample neural network in the m+1th state can be determined.

[0112] In some possible implementation manners, the calculation amount of the sample neural network in the m+1th state can be directly calculated according to the operators included in the sample neural network in the m+1th state, for example, the convolution operator includes multiplication and addition, which are two basic calculation amounts, and the sample neural network in the m+1th state can be directly stored to determine the occupied storage space of the sample neural network in the m+1th state.

[0113] In some possible implementation manners, the network optimization method according to the embodiment of the present disclosure can further include the following for the sample neural network in the m+1th state:

[0114] determining a benefit value of the sample neural network in the m+1th state according to the performance parameter of the sample neural network in the m+1th state, the performance parameter including at least one of the calculation amount, the processing duration, the processing accuracy and the occupied storage space of the neural network.

[0115] That is, according to the performance parameter of the sample neural network in the m+1 state, the benefit value of the sample neural network in the m+1 state can be determined. The benefit value can be negatively correlated with parameters such as calculation amount, processing time, occupied storage space, and positively correlated with parameters such as processing accuracy. For example, the smaller the calculation amount, the greater the benefit value; the higher the processing accuracy, the greater the benefit value.

[0116] It should be understood that a person skilled in the art can set the type of performance parameter used to determine the benefit value and the corresponding relationship between the performance parameter and the benefit value according to actual conditions, and the present disclosure does not limit this.

[0117] In this way, the benefit value of the sample neural network can be determined, thereby improving the accuracy of network loss calculation and improving the training effect of the reinforcement network.

[0118] After obtaining the network loss, the network loss of the reinforcement network in the m state can be determined in step S14.

[0119] In some possible implementation manners, step S14 can include:

[0120] According to the difference between the benefit value of the sample neural network in the m+1 state and the benefit value of the sample neural network in the m state, a benefit difference of the mth optimization is determined.

[0121] According to the benefit difference of the mth optimization, a network loss of the mth optimization is determined.

[0122] According to the network loss of the mth optimization, the reinforcement network in the m state is trained to obtain a reinforcement network in the m+1 state.

[0123] For example, the benefit value of the sample neural network in the m state can be represented as mac m , and the benefit value of the sample neural network in the m+1 state can be represented as mac m+1 . The benefit value of the sample neural network in the m state can be obtained in m-1 rounds of training; the benefit value when m=1 is the benefit value of the initial sample neural network.

[0124] In some possible implementation manners, according to the difference between the benefit value of the sample neural network in the m+1 state and the benefit value of the sample neural network in the m state, a benefit difference r m of the mth optimization can be determined, which can be represented as: r m = mac m+1 -mac m .

[0125] In some possible implementation manners, the network loss of the mth optimization loss m may be determined according to the reward difference of the mth optimization m , for example, expressed as: loss m+1 = (mac m -mac 2 ). It should be understood that a person skilled in the art can set the expression of the network loss according to actual conditions, and the present disclosure does not limit this.

[0126] In some possible implementation manners, the network loss of the mth optimization loss m may be back propagated to update the network parameters of the mth state reinforcement network, to obtain the reinforcement network of the m+1th state, thereby completing the mth round of training process.

[0127] In this way, the network loss of the reinforcement network can be obtained, and the reinforcement network can be trained according to the loss, thereby improving the training effect of the reinforcement network.

[0128] In some possible implementation manners, in step S15, if the reinforcement network of the m+1th state meets the training condition, the reinforcement network of the m+1th state can be determined as the reinforcement network trained in the training batch, and the training process of the training batch is completed. Otherwise, if the neural network of the m+1th state does not meet the preset condition, the above training process (steps S12-S14) can be continued. The training condition may, for example, be that the reinforcement network converges, or reaches a preset number of training rounds, and the present disclosure does not limit this.

[0129] Figure 3 A schematic diagram of the processing process of the network training method provided by the embodiment of the present disclosure is shown. Referring to Figure 3 , the network training process includes a state set 31, a reinforcement network 32, and an action set 33. The state set 31 includes M states S0, S1, …, SM. M-1 ; the action set 33 includes M selected value functions Q max (S0, a0), Q max (S1, a1), …, Q max (S M-1 , a M-1 ). Wherein a0, a1, …, a M-1 respectively represent the function serial numbers of the selected value functions in the plurality of value functions to be selected, and M is an integer greater than 1.

[0130] During the training process, the reinforcement network optimizes the sample neural network through reinforcement learning. For the first round of training of a training batch, the quantization matrix of the sample neural network in the first state is taken as the initial state S0, which is input into the initial reinforcement network 32, and the reinforcement network 32 outputs the action, i.e., the optimization function (i.e., the value function) Q max (S0, a0) of the first state. The sample neural network is optimized through Q max (S0, a0) to obtain the sample neural network in the second state.

[0131] In an example, the reward values of the sample neural networks in the first and second states (S0 and S1) can be calculated respectively, the reward difference of the first optimization is determined according to the reward values; the network loss is determined according to the reward difference; and the reinforcement network is trained according to the network loss to obtain the reinforcement network in the second state.

[0132] In an example, for the second round of training of the training batch, the sample neural network in the second state is quantized to obtain the quantization matrix in the second state, which is taken as the second state S1 and input into the reinforcement network 32 in the second state; the reinforcement network 32 outputs the action, i.e., the optimization function (value function) Q max (S1, a1) of the second state; the sample neural network in the second state is optimized through Q max (S1, a1) to obtain the sample neural network in the third state. Further, the reinforcement network is trained according to the reward values of the sample neural networks in the second and third states. In this way, the reinforcement network can be trained for multiple rounds.

[0133] In an example, the reinforcement network in the Mth state that meets the training condition can be determined as the trained reinforcement network of the training batch, thereby completing the optimization process of the training batch.

[0134] After training through multiple training batches, the reinforcement network suitable for optimization function prediction of various neural networks can be obtained, which improves the adaptability and accuracy of the reinforcement network.

[0135] After the training of the reinforcement network is completed, the reinforcement network can be used to predict the optimization function of the neural network to be optimized.

[0136] According to an embodiment of the present disclosure, a network optimization method is also provided. The network optimization method can be executed by an electronic device such as a terminal device or a server, and the terminal device can be a vehicle-mounted device, a user equipment (UE), a mobile device, a user terminal, a terminal, a cellular phone, a cordless phone, a personal digital assistant (PDA), a handheld device, a computing device, a vehicle-mounted device, a wearable device, etc. The method can be implemented by a processor invoking computer-readable instructions stored in a memory. Alternatively, the method can be executed by a server.

[0137] Figure 4 A flowchart of a network optimization method according to an embodiment of the present disclosure is shown in FIG. 4. As shown in FIG. 4, the method includes the following steps. Figure 4

[0138] In step S41, the target network to be optimized is quantized to obtain a quantization matrix of the target network.

[0139] In step S42, an optimization function prediction is performed on the quantization matrix of the target network by a reinforcement network, and a target optimization function matching the target network is determined from a plurality of preset optimization functions.

[0140] In step S43, the target network is optimized by the target optimization function to obtain an optimized target network, wherein the reinforcement network is trained according to the network training method described above.

[0141] For example, the target network to be optimized can be a trained neural network that can directly perform a corresponding data processing task. However, the execution efficiency or accuracy of the target network can be low, and the operators in the target network can be optimized. The target network can be any type of neural network such as an artificial neural network (ANN) or a spiking neural network (SNN), and the present disclosure does not limit the type of the target network.

[0142] In some possible implementation manners, in step S41, the target network can be quantized to obtain a quantization matrix of the target network. The specific quantization manner is not repeated here.

[0143] In some possible implementation manners, in step S42, the quantization matrix can be input into the reinforcement network as an initial state in reinforcement learning; the values of the elements in the vector output by the reinforcement network are prediction values corresponding to the plurality of optimization functions; and then, the optimization function with the largest prediction value can be determined as the target optimization function matching the target network.

[0144] ​In some possible implementation manners, in step S43, the target network can be optimized by the target optimization function to obtain an optimized target network. The present disclosure does not limit the specific manner of network optimization.

[0145] In some possible implementation manners, after one prediction and optimization is completed, it can be determined whether the optimized target network meets a preset condition, for example, a calculation amount is less than or equal to a preset calculation amount threshold, the calculation amount is increasingly smaller and a change amount is relatively small or remains unchanged, and the present disclosure does not limit the specific setting manner of the preset condition.

[0146] In some possible implementation manners, if the optimized target network meets the preset condition, the optimized target network in this time can be used as a final optimized network, and the optimization process is ended; otherwise, if the preset condition is not met, the steps S41-S43 can be continued to process, and the multiple prediction and optimization processes are performed.

[0147] In this way, the target network to be optimized can be quantified into a matrix, the optimal optimization function can be predicted by strengthening the network, and the target network is optimized by using the optimization function, so that the accuracy and optimization efficiency of network optimization are improved.

[0148] In some possible implementation manners, whether the preset condition is met can be determined according to a performance parameter of the optimized target network. The performance parameter includes at least one of a calculation amount, a processing accuracy, a processing time length, and an occupied storage space of the target network.

[0149] In some possible implementation manners, a sample set corresponding to the target network can be set, and the sample set can include samples and their annotation information. The type of the sample depends on the task type of the target network, for example, when the target network is used to perform an image processing task, the sample can be an image, and the present disclosure does not limit this.

[0150] In some possible implementation manners, the sample can be input into the optimized target network, and a processing result of the sample is output; and a processing time length and other parameters in the processing process of the optimized target network can also be determined.

[0151] In some possible implementation manners, the processing accuracy of the optimized target network can be determined according to the annotation information and the processing result of the sample, for example, a position difference between an annotated face frame and a detected face frame, a face frame detection rate in multiple samples, and the present disclosure does not limit the specific calculation manner of the processing accuracy.

[0152] In some possible implementation manners, the calculation amount of the optimized target network can be directly calculated according to the operators included in the optimized target network, for example, the convolution operator includes multiplication and addition, which are two basic calculation amounts, and the optimized target network can be directly stored to determine the storage space occupied by the network.

[0153] In some possible implementation manners, a threshold corresponding to each performance parameter can be set, and after the threshold is reached, the optimized target network is considered to meet the preset condition; or a difference threshold corresponding to each performance parameter can be set, and after the difference between the performance parameters of the target network before and after optimization reaches the difference threshold, the optimized target network is considered to meet the preset condition. It should be understood that a person skilled in the art can set the preset condition as a combination of any conditions according to actual conditions, and the present disclosure does not limit this.

[0154] In this way, the optimization effect of the target network can be evaluated through the preset condition, so as to obtain an optimal target network.

[0155] According to the embodiments of the present disclosure, a data processing method is also provided, which includes: processing the to-be-processed data through a target network to obtain a processing result of the to-be-processed data, wherein the target network is obtained by optimizing the network according to the network optimization method.

[0156] That is, the target network optimized according to the network optimization method can be used to perform a corresponding processing task, process to-be-processed data, and obtain a processing result of the to-be-processed data. The type of the to-be-processed data is adapted to the processing task, for example, image, voice, text, video, and the like, and the present disclosure does not limit this.

[0157] According to the network training method of the embodiments of the present disclosure, a reinforcement network based on reinforcement learning is introduced, the reinforcement network is trained through a sample neural network, a loss function is calculated according to the benefits of the sample neural network updated twice in succession, and the reinforcement network is updated by using the loss function, so as to improve the accuracy of the reinforcement network in predicting the optimization function; and the reinforcement network is trained through multiple types of sample neural networks, so as to improve the adaptability of the reinforcement network to the type of the neural network to be optimized.

[0158] In the network optimization process, the reinforcement network can select the best optimization function (or called value function) for the target network to be optimized, and optimize the network by using the optimization function. Compared with the way of manually specifying the optimization function in the related art, the accuracy and optimization efficiency of the neural network optimization can be significantly improved. The target network optimized in this way has faster processing speed, smaller calculation amount, higher processing accuracy, and smaller storage space occupied by the network when running in a CPU, GPU, AI chip, or the like.

[0159] The network training, network optimization, and data processing methods according to embodiments of this disclosure can be applied to various application scenarios such as target recognition and target detection, to optimize the corresponding neural network (e.g., inference model) so that the neural network can quickly recognize targets in images and perform rapid detection.

[0160] Figure 5 This is a block diagram of a network training device provided in an embodiment of the present disclosure.

[0161] Reference Figure 5 This disclosure provides a network training apparatus, which includes:

[0162] The first quantization module 51 is used to quantize the sample neural network to obtain the sample quantization matrix of the first state. The sample quantization matrix is ​​used to indicate the type of operators in the sample neural network and the connection relationship between operators.

[0163] The first prediction module 52 is used to predict the optimization function of the sample quantization matrix of the m-th state through the reinforcement network of the m-th state, and determine the sample optimization function of the m-th state, where m is an integer greater than or equal to 1. The optimization function is used to fuse the operators of the neural network, and the reinforcement network of the first state is an untrained reinforcement network.

[0164] The first optimization module 53 is used to optimize the sample neural network of the m-th state corresponding to the sample quantization matrix of the m-th state through the sample optimization function of the m-th state to obtain the sample neural network of the (m+1)-th state.

[0165] Training module 54 is used to train the reinforcement network in the m-th state based on the profit value of the sample neural network in the m-th state and the profit value of the sample neural network in the (m+1)-th state, so as to obtain the reinforcement network in the (m+1)-th state.

[0166] The network determination module 55 is used to determine the enhanced network in the (m+1)th state as the trained enhanced network if the enhanced network in the (m+1)th state satisfies the training conditions.

[0167] The reinforcement network is used to predict the optimization function that matches the target network to be optimized, and the target network is used to perform data processing tasks.

[0168] In some possible implementations, the apparatus further includes: a revenue value determination module, configured to determine the revenue value of the sample neural network in the (m+1)th state based on the performance parameters of the sample neural network in the (m+1)th state, wherein the performance parameters include at least one of the following: computational load, processing time, processing accuracy, and storage space occupied by the neural network.

[0169] In some possible implementation manners, the apparatus further includes: a result determination module, configured to process preset sample data by the sample neural network in the m+1th state, to determine a processing result of the sample data and a processing time length of the sample neural network in the m+1th state; and an accuracy determination module, configured to determine a processing accuracy of the sample neural network in the m+1th state according to labeled information of the sample data and the processing result.

[0170] In some possible implementation manners, the training module is configured to: determine a revenue difference of the mth optimization according to a difference between the revenue value of the sample neural network in the m+1th state and the revenue value of the sample neural network in the mth state; determine a network loss of the mth optimization according to the revenue difference of the mth optimization; and train the reinforcement network in the mth state according to the network loss of the mth optimization, to obtain the reinforcement network in the m+1th state.

[0171] In some possible implementation manners, the reinforcement network includes any one of an artificial neural network (ANN), a spiking neural network (SNN), and a neural network in which an artificial neural network (ANN) and a spiking neural network (SNN) are fused.

[0172] In some possible implementation manners, the data processing task includes any one of an image processing task, a speech processing task, a text processing task, and a video processing task.

[0173] According to an embodiment of the present disclosure, a network optimization apparatus is also provided, which includes: a second quantization module configured to quantize a target network to be optimized, to obtain a quantization matrix of the target network; a second prediction module configured to perform optimization function prediction on the quantization matrix of the target network by a reinforcement network, to determine a target optimization function matching the target network from a plurality of preset optimization functions; and a third optimization module configured to optimize the target network by the target optimization function, to obtain an optimized target network, wherein the reinforcement network is trained according to the network training method described above.

[0174] According to an embodiment of the present disclosure, a data processing apparatus is also provided, which includes: a data processing module configured to process data to be processed by a target network, to obtain a processing result of the data to be processed, wherein the target network is optimized according to the network optimization method described above.

[0175] Figure 6 A block diagram of an electronic device according to an embodiment of the present disclosure is provided.

[0176] Reference is made to Figure 6The electronic device provided by the embodiments of the present disclosure includes at least one processor 501 and a memory 502 connected with the at least one processor 501; the memory 502 stores one or more computer programs executable by the at least one processor 501; and the one or more computer programs are executed by the at least one processor 501 to enable the at least one processor 501 to perform the network training method and / or the network optimization method described above.

[0177] In addition, the embodiments of the present disclosure further provide a computer readable medium having a computer program stored thereon, wherein the computer program, when executed by a processing core, implements the network training method and / or the network optimization method described above.

[0178] Those skilled in the art can understand that all or some of the steps in the method disclosed above, the functions of the modules / units in the system and the device can be implemented as software, firmware, hardware and their appropriate combinations. In the hardware implementation, the division between the functional modules / units mentioned in the above description does not necessarily correspond to the division of physical components; for example, one physical component can have multiple functions, or one function or step can be performed by several physical components in cooperation. Some or all of the physical components can be implemented as software executed by a processor, such as a central processing unit, a digital signal processor or a microprocessor, or as hardware, or as an integrated circuit, such as an application specific integrated circuit. Such software can be distributed on a computer readable medium, which can include computer storage media (or non-transitory media) and communication media (or transitory media). As known to those skilled in the art, the term computer storage media includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tapes, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and can be accessed by a computer. In addition, as known to those skilled in the art, communication media generally includes computer readable instructions, data structures, program modules or other data in modulated data signals such as carrier waves or other transport mechanisms, and can include any information delivery medium.

[0179] Example embodiments have been disclosed herein and, although specific terms are employed, they are used in a generic and descriptive sense only and not for purposes of limitation. In some instances, it will be apparent to those skilled in the art that features, characteristics or / and elements described in connection with a particular embodiment can be used in conjunction with other embodiments unless otherwise explicitly stated. Accordingly, it will be understood that various changes in form and details can be made without departing from the scope of the disclosure as set forth in the appended claims.

Claims

1. A network training method, characterized in that, The method further includes: The sample neural network is quantized to obtain the sample quantization matrix of the first state. The sample quantization matrix is ​​used to indicate the type of operator in the sample neural network and the connection relationship between the operators. The optimization function is predicted by the reinforcement network of the m-th state to optimize the sample quantization matrix of the m-th state, and the sample optimization function of the m-th state is determined. m is an integer greater than or equal to 1. The optimization function is used to fuse the operators of the neural network. The reinforcement network of the first state is an untrained reinforcement network. By using the sample optimization function of the m-th state, the sample neural network of the m-th state corresponding to the sample quantization matrix of the m-th state is optimized to obtain the sample neural network of the (m+1)-th state. Based on the reward value of the sample neural network in the m-th state and the reward value of the sample neural network in the (m+1)-th state, the reinforcement network in the m-th state is trained to obtain the reinforcement network in the (m+1)-th state. If the reinforcement network in the (m+1)th state satisfies the training conditions, the reinforcement network in the (m+1)th state is determined as the trained reinforcement network. The reinforcement network is used to predict the optimization function that matches the target network to be optimized, and the target network is used to perform data processing tasks; the reinforcement network includes any one of artificial neural networks (ANN), spiking neural networks (SNN), and neural networks that combine artificial neural networks (ANN) and spiking neural networks (SNN).

2. The method according to claim 1, characterized in that, The method further includes: Based on the performance parameters of the sample neural network in the (m+1)th state, the profit value of the sample neural network in the (m+1)th state is determined. The performance parameters include at least one of the following: computational cost, processing time, processing accuracy, and storage space occupied by the neural network.

3. The method according to claim 2, characterized in that, The method further includes: The sample neural network in the (m+1)th state processes the preset sample data to determine the processing result of the sample data and the processing time of the sample neural network in the (m+1)th state. Based on the annotation information of the sample data and the processing results, the processing accuracy of the sample neural network in the (m+1)th state is determined.

4. The method according to claim 1, characterized in that, The step of training the reinforcement network in the m-th state based on the reward value of the sample neural network in the m-th state and the reward value of the sample neural network in the (m+1)-th state to obtain the reinforcement network in the (m+1)-th state includes: The profit difference for the m-th optimization is determined based on the difference between the profit value of the sample neural network in the (m+1)-th state and the profit value of the sample neural network in the m-th state. Based on the profit difference of the m-th optimization, determine the network loss of the m-th optimization; Based on the network loss optimized in the mth iteration, the reinforcement network in the mth state is trained to obtain the reinforcement network in the (m+1)th state.

5. The method according to any one of claims 1-4, characterized in that, The data processing task includes any one of the following: image processing task, voice processing task, text processing task, and video processing task.

6. A network optimization method, characterized in that, The method includes: The target network to be optimized is quantized to obtain the quantization matrix of the target network; The optimization function prediction of the quantization matrix of the target network is performed by strengthening the network, and the target optimization function that matches the target network is determined from a plurality of preset optimization functions; The target network is optimized using the objective optimization function to obtain the optimized target network. The reinforcement network is trained using the network training method according to any one of claims 1-5.

7. A data processing method, characterized in that, include: The target network is used to process the data to be processed to obtain the processing result of the data to be processed, wherein the target network is optimized by the network optimization method according to claim 6.

8. A network training device, comprising: The first quantization module is used to quantize the sample neural network to obtain the sample quantization matrix of the first state. The sample quantization matrix is ​​used to indicate the type of operators in the sample neural network and the connection relationship between operators. The first prediction module is used to predict the optimization function of the sample quantization matrix of the m-th state through the reinforcement network of the m-th state, and to determine the sample optimization function of the m-th state, where m is an integer greater than or equal to 1. The optimization function is used to fuse the operators of the neural network, and the reinforcement network of the first state is an untrained reinforcement network. The first optimization module is used to optimize the sample neural network of the m-th state corresponding to the sample quantization matrix of the m-th state through the sample optimization function of the m-th state, so as to obtain the sample neural network of the (m+1)-th state. The training module is used to train the reinforcement network in the m-th state based on the reward value of the sample neural network in the m-th state and the reward value of the sample neural network in the (m+1)-th state, so as to obtain the reinforcement network in the (m+1)-th state. The network determination module is used to determine the enhanced network in the (m+1)th state as the trained enhanced network if the enhanced network in the (m+1)th state satisfies the training conditions. The reinforcement network is used to predict the optimization function that matches the target network to be optimized, and the target network is used to perform data processing tasks; the reinforcement network includes any one of artificial neural networks (ANN), spiking neural networks (SNN), and neural networks that combine artificial neural networks (ANN) and spiking neural networks (SNN).

9. An electronic device, comprising: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores one or more computer programs that can be executed by the at least one processor, the one or more computer programs being executed by the at least one processor to enable the at least one processor to perform the network training method as described in any one of claims 1-5, or the network optimization method as described in claim 6, or the data processing method as described in claim 7.

10. A computer-readable medium having a computer program stored thereon, wherein, When the computer program is executed by the processor, it implements the network training method as described in any one of claims 1-5, the network optimization method as described in claim 6, or the data processing method as described in claim 7.

Images