Data processing method and apparatus, electronic device and storage medium

Abstract

The present disclosure relates to the technical field of computers, and relates to a data processing method and apparatus, an electronic device and a storage medium. The method comprises: acquiring network parameter information of a neural network in a current iteration state; on the basis of a parameter adjustment mode of the neural network and the network parameter information of the current iteration state, determining network adjustment information of the current iteration state, wherein when the parameter adjustment mode is parameter perturbation, the network adjustment information comprises a perturbation processing result, and when the parameter adjustment mode is gradient sparsification, the network adjustment information comprises target gradient information; and adjusting the network parameter information of the current iteration state on the basis of adjustment information of the current iteration state to obtain network parameter information of the next iteration state so as to train the neural network.

Description

Data processing method and device, electronic device, and storage medium TECHNICAL FIELD

[0001] Embodiments of the present disclosure relate to the technical field of computer, and particularly relate to a data processing method and device, electronic device, and computer readable storage medium. BACKGROUND

[0002] The learning or training of a neural network is essentially solving an optimization problem, and a gradient descent-based optimization method is currently widely used. The Back Propagation (BP) algorithm is an efficient algorithm for explicitly solving the gradient, but it requires saving the intermediate state variables of the neural network (such as the activation information of each layer) and updating in order from back to front. SUMMARY

[0003] The present disclosure provides a data processing method and device, electronic device, and computer readable storage medium.

[0004] In a first aspect, the present disclosure provides a data processing method, comprising:

[0005] obtaining network parameter information of a neural network in a current iteration state;

[0006] determining network adjustment information of the current iteration state according to a parameter adjustment mode of the neural network and the network parameter information of the current iteration state; wherein in the case of parameter perturbation as the parameter adjustment mode, the network adjustment information comprises a perturbation processing result; in the case of gradient sparsification as the parameter adjustment mode, the network adjustment information comprises target gradient information;

[0007] adjusting the network parameter information of the current iteration state according to the adjustment information of the current iteration state to obtain network parameter information of a next iteration state, so as to train the neural network.

[0008] In some possible implementation manners, the network parameter information comprises weight information, and in the case of parameter perturbation as the parameter adjustment mode,

[0009] determining the adjustment information of the current iteration state according to the parameter adjustment mode of the neural network and the network parameter information of the current iteration state comprises:

[0010] performing perturbation processing on first weight information of the network layer in the current iteration state to obtain second weight information after perturbation;

[0011] determining a perturbed neural network according to the second weight information of the plurality of network layers;

[0012] The sample data in the training set is processed by the disturbed neural network to obtain a disturbance processing result.

[0013] According to the adjustment information of the current iteration state, the network parameter information of the current iteration state is adjusted to obtain the network parameter information of the next iteration state, so as to train the neural network, comprising:

[0014] According to the disturbance processing result, the label result of the sample data, and the input random variable and the output random variable of each network layer in the current iteration state, the first weight information of each network layer is adjusted respectively, so as to train the neural network, wherein the input random variable and the output random variable are respectively associated with the dimension of the input data and the dimension of the output data of the corresponding network layer.

[0015] In some possible implementation manners, in the case that the parameter adjustment manner is parameter disturbance,

[0016] According to the parameter adjustment manner of the neural network and the network parameter information of the current iteration state, the adjustment information of the current iteration state is determined, comprising:

[0017] The network parameter information of the neural network in the current iteration state is disturbed to obtain a disturbed neural network;

[0018] The sample data in the training set is processed by the disturbed neural network to obtain a disturbance processing result.

[0019] According to the target evaluation value of the current iteration state and the target reference value of the neural network in the current iteration state, the disturbance processing result is determined, and the target reference value of the current iteration state is obtained by adjusting the target reference value of the previous iteration state;

[0020] According to the adjustment information of the current iteration state, the network parameter information of the current iteration state is adjusted to obtain the network parameter information of the next iteration state, so as to train the neural network, comprising:

[0021] According to the disturbance processing result and the random disturbance variable of the neural network in the current iteration state, the network parameter information of the current iteration state is adjusted to obtain the network parameter information of the next iteration state, so as to train the neural network.

[0022] In some possible implementation manners, in the case that the parameter adjustment manner is gradient sparsity, the method further comprises:

[0023] Obtaining initial gradient information of the neural network in the current iteration state;

[0024] According to the parameter adjustment manner of the neural network and the network parameter information of the current iteration state, the network adjustment information of the current iteration state is determined, comprising:

[0025] determine the gradient sparsity of the current iteration state according to the iteration information of the neural network and the preset gradient sparse distribution information;

[0026] perform sparse processing on the initial gradient information according to the gradient sparsity of the current iteration state, to obtain target gradient information of the current iteration state;

[0027] adjust the network parameter information of the current iteration state according to the adjustment information of the current iteration state, to obtain network parameter information of a next iteration state, so as to train the neural network, comprising:

[0028] adjust the network parameter information of the current iteration state based on the target gradient information of the current iteration state, to obtain network parameter information of a next iteration state, so as to train the neural network.

[0029] In a second aspect, the present disclosure provides a data processing apparatus, comprising:

[0030] a parameter information acquisition module, configured to acquire network parameter information of a neural network in a current iteration state;

[0031] an adjustment information determination module, configured to determine network adjustment information of the current iteration state according to a parameter adjustment manner of the neural network and the network parameter information of the current iteration state; wherein, in the case that the parameter adjustment manner is parameter perturbation, the network adjustment information comprises a perturbation processing result; and in the case that the parameter adjustment manner is gradient sparsity, the network adjustment information comprises target gradient information;

[0032] a network adjustment module, configured to adjust the network parameter information of the current iteration state according to the adjustment information of the current iteration state, to obtain network parameter information of a next iteration state, so as to train the neural network.

[0033] In a third aspect, the present disclosure provides an electronic device, comprising: at least one processor; and a memory communicatively connected with the at least one processor; wherein the memory stores one or more computer programs executable by the at least one processor, and the one or more computer programs are executed by the at least one processor to enable the at least one processor to execute the above-mentioned data processing method.

[0034] In a fourth aspect, the present disclosure provides a computer-readable storage medium, which stores a computer program, wherein the computer program, when executed by a processor / processing core, implements the above-mentioned data processing method.

[0035] In a fifth aspect, the present disclosure provides a computer program product comprising computer readable code, or a non-transitory computer readable storage medium carrying computer readable code, which when run in a processor of an electronic device, the processor in the electronic device performs the data processing method described above.

[0036] According to the parameter adjustment manner of the neural network, the embodiments provided by the present disclosure can determine the network adjustment information of the current iteration state; when the parameter adjustment manner is parameter perturbation, the perturbation processing result after the network parameter information is perturbed is determined; when the parameter adjustment manner is gradient sparsity, the sparsified target gradient information is determined; and then the network parameter information of the current iteration state is adjusted according to the perturbation processing result or the target gradient information, to obtain the network parameter information of the next iteration state, so as to realize the training of the neural network, thereby improving the training efficiency, reducing the memory overhead and the calculation overhead in network training, and improving the universality and learning performance of network training.

[0037] It should be understood that the content described in this part is not intended to identify key or important features of the embodiments of the present disclosure, nor is it used to limit the scope of the present disclosure. Other features of the present disclosure will become apparent from the following description. BRIEF DESCRIPTION OF DRAWINGS

[0038] The accompanying drawings are included to provide a further understanding of the present disclosure and constitute a part of the specification, which together with the embodiments of the present disclosure serve to explain the present disclosure, and do not constitute a limitation of the present disclosure. The above and other features and advantages will become more apparent from the detailed description of the specific example embodiments, with reference to the accompanying drawings, in which:

[0039] FIG. 1 is a flowchart of a data processing method according to an embodiment of the present disclosure;

[0040] FIG. 2 is a flowchart of a data processing method according to an embodiment of the present disclosure;

[0041] FIG. 3 is a schematic diagram of the training effect of a data processing method according to an embodiment of the present disclosure;

[0042] FIG. 4 is a flowchart of a data processing method according to an embodiment of the present disclosure;

[0043] FIG. 5 is a schematic diagram of a data processing method according to an embodiment of the present disclosure;

[0044] FIG. 6 is a flowchart of a data processing method according to an embodiment of the present disclosure;

[0045] FIG. 7 is a schematic diagram of gradient sparsity distribution information according to an embodiment of the present disclosure;

[0046] FIG. 8 is a schematic diagram of gradient sparsity distribution information according to an embodiment of the present disclosure;

[0047] FIG. 9 is a schematic diagram of gradient sparsity distribution information according to an embodiment of the present disclosure;

[0048] FIG. 10 is a schematic diagram of gradient sparsity distribution information according to an embodiment of the present disclosure;

[0049] FIG. 11 is a schematic diagram of gradient sparsity distribution information according to an embodiment of the present disclosure;

[0050] FIG. 12 is a block diagram of a data processing apparatus according to an embodiment of the present disclosure;

[0051] FIG. 13 is a block diagram of a data processing apparatus according to an embodiment of the present disclosure;

[0052] FIG. 14 is a block diagram of a data processing apparatus according to an embodiment of the present disclosure

[0053] FIG. 15 is a block diagram of a data processing apparatus according to an embodiment of the present disclosure;

[0054] FIG. 16 is a block diagram of a many-core system according to an embodiment of the present disclosure;

[0055] FIG. 17 is a block diagram of an electronic device according to an embodiment of the present disclosure;

[0056] FIG. 18 is a block diagram of an electronic device according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0057] In order for those skilled in the art to better understand the technical solutions of the present disclosure, the following describes exemplary embodiments of the present disclosure with reference to the accompanying drawings, including various details of the embodiments of the present disclosure to help understanding, which should be considered only as exemplary. Therefore, those of ordinary skill in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Also, in order to be clear and concise, the description below omits the description of well-known functions and structures.

[0058] In the case of no conflict, each embodiment of the present disclosure and each feature in the embodiments can be combined with each other.

[0059] As used herein, the term "and / or" includes any and all combinations of one or more of the associated listed items.

[0060] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the disclosure. As used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms "comprises" and / or "comprising," when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof. "Coupled" or "connected" or similar terms are not necessarily limited to a direct or physical connection, but can include an electrical connection, whether direct or indirect.

[0061] Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art. It will be further understood that terms, such as those defined in commonly used dictionaries, should be interpreted as having a meaning that is consistent with their meaning in the context of the relevant art and the present disclosure and will not be interpreted in an overly literal or overly formal sense unless expressly so defined herein.

[0062] Generally, the training of a neural network can be modeled as the following optimization formula: min θ E (x,y)～D L[f(x; θ), y] (1)

[0063] In formula (1), L represents a target function of neural network training; f represents a neural network to be optimized; θ represents parameters of the neural network, mainly composed of weights w; x represents input data of the neural network, y represents output labeled data of the neural network, and (x, y) represents sample data sets composed of input data and output labeled data. Formula (1) represents that the training target of the neural network is to minimize the statistical average value E (x,y)～D of the target function L on the sample data set (x, y).

[0064] For a brain-like spiking neural network (SNN), the training method in the related art generally includes a local learning method based on bionic synaptic plasticity and a gradient descent method based on BP. The local learning method based on bionic synaptic plasticity can include, for example, Hebb rule, synaptic timing dependent plasticity (STDP) and its variants. For example, the weight update formula of Hebb rule is as follows: Δw = ηxy T (2)

[0065] In formula (2), Δw represents the weight update amount; η represents the learning rate; x and y represent the pulse activation states of the neurons in the front and rear two layers, respectively, and y T represents the transpose of y.

[0066] In the BP-based gradient descent method, the gradient update formula is as follows:

[0067] In formula (3), Δw represents the weight update amount; η represents the learning rate; represents the gradient of the objective function L with respect to the weight parameter, which can be obtained by the BP algorithm or the chain rule.

[0068] However, the local learning method based on the bionic synaptic plasticity is generally applicable to an unsupervised setting, and the training effect is significantly weaker than the BP-based gradient descent method. Moreover, the universality of such a method is weak, and it is difficult to be applied to a neural network with diverse structures, such as a Transformer architecture with a self-attention mechanism.

[0069] Although the BP-based gradient descent method has good training effect and strong universality, the memory and computing overheads in the training are large. Especially when training a recurrent neural network (RNN) or a spiking neural network (SNN), the corresponding back propagation through time (BPTT) algorithm will make the memory requirement for saving the intermediate state more significant, so that it cannot be implemented when processing a long sequence. For the training of an RNN or SNN network with a sequence length of n, the activation value to be saved is n times that in one forward pass (length 1), which greatly limits the length of the processable sequence. For example, if the SNN model is time-discretized at 1 ms, 100,000 iterations are required to process a time signal of 100 s, which is unacceptable in terms of memory overhead.

[0070] Moreover, for a many-core system architecture including multiple processing cores, such as a brain-like computing chip or a neuromorphic chip, when training is performed by using the BP algorithm, the gradient update of each layer of the deployed neural network on each processing core requires additional synchronization logic, and generally needs to rely on an additional controller, such as a CPU, for coordinated control of each computing core, which brings additional overhead and limits the on-chip learning application of the neural network.

[0071] In addition, the BP algorithm generally needs to maintain high precision (for example, FP16, FP32) for data during back propagation, which not only increases the data transmission amount, but also correspondingly increases the calculation amount. Generally, the numerical precision supported by a brain-like chip is low (for example, int8 or fp16), which also affects the numerical precision and training stability in the BP algorithm. Therefore, the method based on BP is difficult to be efficiently deployed on a decentralized many-core brain-like chip, and additional control overhead is required.

[0072] According to an embodiment of the present disclosure, a data processing method is provided, which can determine network adjustment information of a current iteration state according to a parameter adjustment mode of a neural network in one iteration training; when the parameter adjustment mode is parameter perturbation, an adjustment value of network parameter information can be obtained by combining local perturbation of each network layer weight of the neural network with global signal of the entire neural network, or by perturbing and determining an evaluation value of an inference result after perturbation; when the parameter adjustment mode is gradient sparsity, target gradient information after sparsity is determined; then, network parameter information of the current iteration state is adjusted according to the perturbation processing result or the target gradient information, to obtain network parameter information of a next iteration state, so as to realize training of the network.

[0073] Through this training mode, the intermediate state of the network at each time step does not need to be saved, additional control overhead is not required, and high-precision gradient signals do not need to be returned, so that the memory overhead and the calculation overhead in training can be significantly reduced, and the deployment efficiency on a device with low numerical precision (for example, a many-core system architecture) can be improved.

[0074] The data processing method according to the embodiment of the present disclosure can be executed by an electronic device such as a terminal device or a server. The terminal device can be a vehicle-mounted device, a user equipment (UE), a mobile device, a user terminal, a terminal, a cellular phone, a cordless phone, a personal digital assistant (PDA), a handheld device, a computing device, a vehicle-mounted device, a wearable device, etc. The method can be realized by a processor calling computer readable program instructions stored in a memory. Alternatively, the method can be executed by a server.

[0075] FIG. 1 is a flowchart of a data processing method according to an embodiment of the present disclosure. Referring to FIG. 1, the method includes steps S11-S13:

[0076] In step S11, network parameter information of a neural network in a current iteration state is obtained.

[0077] In step S12, network adjustment information of the current iteration state is determined according to the parameter adjustment mode of the neural network and the network parameter information of the current iteration state; wherein, in the case of parameter perturbation as the parameter adjustment mode, the network adjustment information includes a perturbation processing result; in the case of gradient sparsity as the parameter adjustment mode, the network adjustment information includes target gradient information.

[0078] In step S13, the network parameter information of the current iteration state is adjusted according to the adjustment information of the current iteration state, to obtain network parameter information of a next iteration state, so as to train the neural network.

[0079] For example, the neural network to be trained can be any type of neural network, such as a convolutional neural network (CNN), a spiking neural network (SNN), a recurrent neural network (RNN), etc. The neural network includes multiple network layers, such as convolutional layers, activation layers, fully connected layers, etc. The type of the neural network and the type of the network layers in the neural network are not limited in the present disclosure.

[0080] In some possible implementation manners, the neural network to be trained can be used to perform a target task, which includes any one of an image processing task, a speech processing task, a text processing task, and a video processing task. Correspondingly, the sample data of the neural network can include any one of an image, a speech, a text, and a video. The present disclosure does not limit this.

[0081] It should be noted that the neural network to be trained can be a relatively independent network with task processing capability, or can be a component of another neural network or cooperate with another neural network to perform a task. The present disclosure does not limit this.

[0082] In some possible implementation manners, the neural network training can include multiple iteration processes. At the beginning of each iteration, the network parameter information of the neural network in the current iteration state can be determined in step S11. The network parameter information can include weight information and bias information of each network layer of the neural network, and the present disclosure does not limit the specific content of the network parameter information.

[0083] In some possible implementation manners, in step S12, the network adjustment information of the current iteration state is determined according to the parameter adjustment mode of the neural network and the network parameter information of the current iteration state. The parameter adjustment mode can include parameter perturbation or gradient sparsity.

[0084] In the case of parameter perturbation as the parameter adjustment mode, the adjustment value of the network parameter information can be obtained by combining the local perturbation of each network layer weight of the neural network with the global signal of the entire neural network, or by perturbing and determining the evaluation value of the inference result after the perturbation.

[0085] In the case where the parameter adjustment manner is gradient sparsification, the initial gradient information of the neural network in the current iteration state can be sparsified according to the gradient sparsity of the current iteration state, and the target gradient information after sparsification is determined.

[0086] In some possible implementation manners, in step S13, the network parameter information in the current iteration state is adjusted according to the perturbation processing result or the target gradient information, and the network parameter information in the next iteration state is obtained, so as to realize one training iteration of the neural network.

[0087] According to the embodiments of the present disclosure, the network adjustment information corresponding to the parameter adjustment manner of the neural network can be determined in one iteration training, and then the network parameter information in the current iteration state is adjusted according to the network adjustment information, so as to realize the training of the network. Through this training manner, the intermediate states of the network at each time step do not need to be saved, no additional control overhead is needed, and no high-precision gradient signal needs to be returned, so that the memory overhead and the calculation overhead in the training can be significantly reduced, and the deployment efficiency of the device with low numerical precision (for example, a many-core system architecture) can be improved.

[0088] The data processing method according to the embodiments of the present disclosure will be described below.

[0089] FIG. 2 is a flowchart of a data processing method according to an embodiment of the present disclosure.

[0090] In some possible implementation manners, the network parameter information includes weight information, in the case where the parameter adjustment manner is parameter perturbation, referring to FIG. 2, step S12 includes steps S121-S123:

[0091] In step S121, for any network layer in the plurality of network layers of the neural network, the first weight information of the network layer in the current iteration state is perturbed to obtain the second weight information after perturbation.

[0092] In step S122, the perturbed neural network is determined according to the second weight information of the plurality of network layers.

[0093] In step S123, the sample data in the training set is processed by the perturbed neural network to obtain the perturbation processing result.

[0094] In step S13, step S131 is included:

[0095] In step S131, the first weight information of the plurality of network layers is respectively adjusted according to the perturbation processing result, the label result of the sample data, and the input random variable and the output random variable of each network layer in the current iteration state, so as to train the neural network, wherein the input random variable and the output random variable are respectively associated with the dimension of the input data and the dimension of the output data of the corresponding network layer.

[0096] For example, the neural network training can include a plurality of iteration processes, and the weight of each network layer of the neural network is the first weight information of the current iteration state at the beginning of each iteration.

[0097] In step S121, for any network layer, the first weight information of the network layer in the current iteration state can be subjected to perturbation processing to obtain second weight information after perturbation. The perturbation processing can be random perturbation, that is, a random value (which can be referred to as a perturbation information amount) is added to or subtracted from each first weight information, the random value is within a certain value range (for example, -1 to 1), and satisfies a certain distribution rule, such as Gaussian distribution, Bernoulli distribution, etc. Other perturbation methods can also be used for perturbation processing, which are not limited in the present disclosure.

[0098] In some possible implementation manners, in step S122, after the perturbation processing is respectively performed on each network layer of the neural network, the second weight information of the plurality of network layers can be obtained, so as to determine the neural network after perturbation.

[0099] In some possible implementation manners, in step S123, the sample data in the training set is processed by using the neural network after perturbation to obtain a perturbation processing result. For example, a part of the sample data, such as a batch of sample data, can be processed to obtain a perturbation processing result of the part of sample data. The perturbation processing result is used to represent the influence on the network processing result after the perturbation processing is performed on each network layer of the neural network.

[0100] In some possible implementation manners, in step S131, according to the difference between the perturbation processing result and the label result of the sample data, a global weight update polarity (which can be referred to as a global weight update value) of the neural network can be determined, and the global weight update value can represent the weight update direction that makes the loss of the neural network smaller. The global weight update value can be represented by only 1 bit, such as 1 or -1.

[0101] After the global weight update value is obtained, the first weight information of each network layer in the current iteration state can be respectively adjusted according to the input random variable and the output random variable of each network layer in the current iteration state to obtain the first weight information of the next iteration state, so as to obtain the neural network in the next iteration state, and complete one iteration process.

[0102] In this way, each iteration makes the loss of the neural network smaller. After a plurality of iteration processes, the trained neural network can be obtained when a training condition (for example, network convergence or reaching a preset number of iterations) is met.

[0103] The input random variable and the output random variable are respectively associated with the dimension of the input data and the dimension of the output data of the corresponding network layer. For example, the input random variable is a column vector with the same dimension as the input data, and the output random variable is a column vector with the same dimension as the output data. The specific form of the input random variable and the output random variable is not limited in the present disclosure.

[0104] According to an embodiment of the present disclosure, the weight information of each network layer of the neural network in the current iteration state can be perturbed respectively to realize local perturbation of the weight of each network layer. The sample data is processed by the perturbed neural network to obtain a perturbation processing result, and then the update direction of the global neural network is determined and the weight information of each network layer is adjusted respectively according to the perturbation processing result, the sample label result, and the input random variable and the output random variable of each network layer in the current iteration state, so as to reduce the memory overhead and the calculation overhead in network training, and improve the generality and learning performance of network training.

[0105] In the neural network training using the BP algorithm, the update direction of the weight needs to depend on the error of the back propagation. In the embodiment of the present disclosure, the update value of the weight is determined by the weight update direction and the weight update polarity, the weight update direction is locally independent in each network layer in the form of a random variable, and does not depend on the global external signal; the weight update polarity needs to be calculated based on the result of the forward output of the entire neural network, that is, it depends on the global evaluation of the neural network.

[0106] As described above, in any iteration in the neural network training, for any network layer of the neural network, the first weight information of the network layer in the current iteration state can be perturbed in step S121 to obtain the second weight information after perturbation.

[0107] Before perturbation, the input random variable and the output random variable of each network layer can be set, and the input random variable and the output random variable are column vectors, and the input random variable and the output random variable are respectively associated with the dimension of the input data and the dimension of the output data of the corresponding network layer.

[0108] In some possible implementation manners, the input random variable and the output random variable can satisfy Gaussian distribution, Bernoulli distribution, or the like, the input random variable can be the same as or a multiple of the dimension of the input data of the corresponding network layer, and the output random variable can be the same as or a multiple of the dimension of the output data of the corresponding network layer. The disclosure does not limit the specific length and distribution of the input random variable and the output random variable.

[0109] In some possible implementation manners, for any network layer in the plurality of network layers, the input random variable and the output random variable of the network layer in the current iteration state are Bernoulli random variables, the dimension of the input random variable is the same as the dimension of the input data of the network layer in the current iteration state, and the dimension of the output random variable is the same as the dimension of the output data of the network layer in the current iteration state.

[0110] In some possible implementation manners, the input random variable and the output random variable that satisfy the preset distribution can be randomly generated for each network layer at the beginning of each iteration, and the amount of perturbation information of each network layer can be determined according to the input random variable and the output random variable of each network layer.

[0111] In some possible implementation manners, the amount of perturbation information includes the product value of the input random variable, the transpose of the output random variable, and a perturbation coefficient, and the perturbation coefficient is used to represent the degree of random perturbation on the weight.

[0112] That is, for any network layer, the input random variable of the network layer can be multiplied by the transpose of the output random variable to obtain a matrix matched with the weight information of the network layer, and the matrix can be multiplied by a perturbation coefficient to obtain the amount of perturbation information, facilitating subsequent perturbation on the weight information of the network layer. The perturbation coefficient is used to represent the degree of random perturbation on the weight, and can be 0.1 or 0.01, and the disclosure does not limit the specific value of the perturbation coefficient.

[0113] In some possible implementation manners, step S121 can include:

[0114] The value of the first weight information of the network layer in the current iteration state is increased by the amount of perturbation information to obtain forward perturbation weight information, and the value of the first weight information of the network layer in the current iteration state is decreased by the amount of perturbation information to obtain negative perturbation weight information, the second weight information includes the forward perturbation weight information and the negative perturbation weight information, and the amount of perturbation information is associated with the input random variable and the output random variable of the network layer.

[0115] That is, for any network layer, the weight information of the network layer can be respectively perturbed in a positive direction and a negative direction. The value of the first weight information of the network layer in the current iteration state can be increased by an amount of perturbation information to obtain positive perturbation weight information, and the value of the first weight information of the network layer in the current iteration state can be reduced by an amount of perturbation information to obtain negative perturbation weight information. The second weight information includes the positive perturbation weight information and the negative perturbation weight information.

[0116] In this way, the first weight information of each network layer of the neural network is perturbed to obtain the second weight information of the plurality of network layers of the neural network. In step S122, the perturbed neural network is determined according to the second weight information of the plurality of network layers. Step S122 can include:

[0117] The positive perturbed neural network is determined according to the positive perturbation weight information of the plurality of network layers, and the negative perturbed neural network is determined according to the negative weight information of the plurality of network layers, and the perturbed neural network includes the positive perturbed neural network and the negative perturbed neural network.

[0118] That is, the positive perturbation weight information of the plurality of network layers of the neural network can be respectively stored to obtain the positive perturbed neural network, and the negative perturbation weight information of the plurality of network layers of the neural network can be respectively stored to obtain the negative perturbed neural network, and the positive perturbed neural network and the negative perturbed neural network are taken as the perturbed neural network.

[0119] In this way, the update direction of the weight can be locally generated independently in each network layer of the neural network, which facilitates improving the processing efficiency in a parallel manner, and the intermediate state of the network at each time step does not need to be saved, no additional control overhead is required, and no high-precision gradient signal needs to be returned, thereby significantly reducing the memory overhead and the calculation overhead in the training.

[0120] After obtaining the perturbed neural network, the perturbed neural network can be globally evaluated to determine the weight update polarity of the neural network. In step S123, the sample data in the training set is processed by the perturbed neural network to obtain a perturbation processing result. Step S123 can include:

[0121] The sample data is processed by the positive perturbed neural network to obtain a positive perturbation processing result, and the sample data is processed by the negative perturbed neural network to obtain a negative perturbation processing result, and the perturbation processing result includes the positive perturbation processing result and the negative perturbation processing result.

[0122] That is, the sample data can be processed by the positive perturbation neural network and the negative perturbation neural network respectively to obtain the positive perturbation processing result and the negative perturbation processing result. Wherein, a batch of sample data can be processed to obtain the positive perturbation processing result and the negative perturbation processing result of the part of sample data as the perturbation processing result. The perturbation processing result is used to represent the influence on the network processing result after the perturbation processing of each network layer of the neural network.

[0123] In some possible implementation ways, in step S131, the first weight information of the plurality of network layers is adjusted respectively according to the perturbation processing result, the label result of the sample data, and the input random variable and the output random variable of each network layer in the current iteration state, so as to train the neural network. Wherein, step S131 can include:

[0124] determining the global weight update value of the neural network according to the perturbation processing result, the label result of the sample data, and the preset loss function;

[0125] generating the weight adjustment information of the network layer according to the input random variable and the output random variable of the network layer in the current iteration state and the global weight update value;

[0126] adjusting the first weight information of the network layer in the current iteration state according to the weight adjustment information of the network layer to obtain the first weight information of the network layer in the next iteration state;

[0127] determining the neural network in the next iteration state according to the first weight information of the plurality of network layers in the next iteration state.

[0128] For example, after obtaining the positive perturbation processing result and the negative perturbation processing result of a batch of sample data, the global weight update polarity (which can be referred to as the global weight update value) of the neural network can be determined according to the difference between the perturbation processing result and the label result of the sample data. Wherein, a loss function (such as cross entropy loss, L2 loss, etc.) can be preset, and the global weight update value of the neural network can be determined according to the perturbation processing result, the label result of the sample data, and the preset loss function. The specific type of the loss function is not limited in the present disclosure.

[0129] In some possible implementation ways, the step of determining the global weight update value of the neural network according to the perturbation processing result, the label result of the sample data, and the preset loss function can include:

[0130] generating the global weight update value according to the first value of the loss function under the label result of the sample data and the positive perturbation processing result, and the second value of the loss function under the label result of the sample data and the negative perturbation processing result.

[0131] The first value represents the impact of a positive perturbation on the network loss, and the second value represents the impact of a negative perturbation on the network loss. Based on the difference between the impacts of positive and negative perturbations on the network processing results, the global weight update value b can be determined. z Global weight update value b z It can be represented using only 1 bit, for example, as 1 or -1. In this way, during weight updates, the entire neural network only needs to synchronize this 1 bit of signal, which greatly reduces the amount of synchronization data and bandwidth requirements.

[0132] In some possible implementations, the global weight update value is 1 if the difference between the first and second values ​​is greater than or equal to 0, and -1 if the difference is less than 0. That is, if the difference between the first and second values ​​is greater than or equal to 0, it indicates that the loss function value is larger after a positive perturbation, the weight update polarity is positive, and the global weight update value is set to 1; if the difference between the first and second values ​​is less than 0, it indicates that the loss function value is larger after a negative perturbation, the weight update polarity is negative, and the global weight update value is set to -1.

[0133] In some possible implementations, the global weight update value b can be determined using the following formula. z b z =sgn[L[f(x;w+εb x b y T ),y]-L[f(x;w-εb x b y T ),y] (4)

[0134] In formula (4), sgn[] represents the sign function, which outputs 1 when its input is greater than or equal to 0, and -1 otherwise; f represents the neural network; x represents the sample data; y represents the labeling result of the sample data; w represents the first weight information of the current iteration state; b x Indicates the input random variable; b y T This indicates that the output random variable b y The transpose of ; ε represents the perturbation coefficient, used to indicate the degree of random perturbation of the weights, and can take values ​​of 0.1 or 0.01, etc.

[0135] In formula (4), w+εb x b y T This represents a positive random perturbation applied to each layer of the neural network, resulting in the positive perturbation processing result f(x; w+εb).x b y T );w-εb x b y T represents a negative random perturbation performed by each network layer of the neural network, and a negative perturbation processing result f(x; w-εb x b y T );in the case of Bernoulli random perturbation, b z indicates the polarity of the Bernoulli random search.

[0136] It should be understood that formula (4) is the update formula of the global weight update value b z in the supervised learning scenario. The scheme of the present disclosure is also applicable to unsupervised, semi-supervised, reinforcement learning, etc. scenarios. Those skilled in the art can set the update formula of the global weight update value b z according to the actual situation, and the present disclosure does not limit this.

[0137] In some possible implementation manners, the above update formula (4) is also applicable when training a convolutional or self-attention structure. At this time, the corresponding perturbation generated needs to be consistent with the parameter size to be optimized.

[0138] In this way, the perturbed neural network can be globally evaluated to determine the weight update direction of the neural network, thereby improving the efficiency of network weight updating.

[0139] In some possible implementation manners, after obtaining the global weight update value, the weight adjustment information of each network layer of the neural network can be determined respectively to adjust the weights. For any network layer, the weight adjustment information of the network layer is generated according to the input random variable and the output random variable of the network layer in the current iteration state and the global weight update value. Wherein, the step includes:

[0140] The weight adjustment information is generated according to the product of the input random variable, the transpose of the output random variable, and the global weight update value.

[0141] In some possible implementation manners, the weight adjustment information of the network layer can be generated according to the following formula: Δw=-ηb x b y T b z (5)

[0142] Wherein, Δw represents the weight adjustment information, η represents the learning rate, b x represents an input random variable consistent with the dimension of the input data (i.e. the input neuron), and b yT denotes the transpose of the output data (i.e., output neurons) output random variable b y z denotes the global weight update value, and b x and b y are column vectors. The learning rate η can be 0.1 or 0.3, and the specific value of the learning rate η is not limited in the present disclosure.

[0143] In the case of Bernoulli random disturbance, the input random variable and the output random variable are both Bernoulli random vectors, and the sampling results of each element of the Bernoulli random vector are -1 or 1. In the training process, the probabilities of the two values can be made equal and fixed, i.e., both are 0.5. It should be understood that the probabilities of the sampling results of each element of the Bernoulli random vector being -1 or 1 can also change with the training process, which is not limited in the present disclosure.

[0144] where b z is represented by 1 bit, so the update method of such a weight matrix is mainly composed of XOR operation, and the calculation amount is extremely low. b x b y T determines the weight update direction of the present solution, and generates each network layer locally and independently.

[0145] In this way, each network layer can generate its own weight adjustment information locally and independently, which reduces the calculation amount and facilitates improving the processing efficiency through parallel mode.

[0146] In some possible implementation manners, the first weight information of the network layer in the current iteration state is adjusted according to the weight adjustment information of the network layer, to obtain the first weight information of the network layer in the next iteration state, so as to realize the local independent adjustment of each network layer and facilitate improving the processing efficiency through parallel mode.

[0147] In some possible implementation manners, the first weight information of the plurality of network layers is adjusted respectively to obtain the first weight information of the plurality of network layers in the next iteration state, so as to determine the neural network in the next iteration state and complete one iteration training process.

[0148] In this way, each iteration makes the loss of the neural network smaller. After a plurality of iteration processes, the trained neural network can be obtained when the training condition (for example, network convergence or reaching the preset iteration number) is met.

[0149] The convergence analysis of the network training method according to the embodiments of the present disclosure is as follows: b z is the loss function at the point x along b x b y ​T The binary directional derivative of the direction is thus updated according to the weight update formula (5), and the loss function value is generally reduced after the update, and thus the training process can be completed.

[0150] FIG. 3 is a schematic diagram of a training effect of a data processing method according to an embodiment of the present disclosure.

[0151] In an example, the neural network to be trained is used to perform an MNIST (a kind of image dataset) image classification task, and the neural network includes 6 convolutional layers and 1 fully connected layer. FIG. 3 shows the prediction accuracy of the neural network on a test dataset during the training according to the technical solution of the embodiment of the present disclosure, in which the horizontal axis represents the number of iterations, and the vertical axis represents the prediction accuracy of the neural network. It can be seen that according to the data processing method of the embodiment of the present disclosure, the neural network can effectively converge, the convergence speed is fast, and the training effect is improved.

[0152] In some possible implementation manners, in the data processing method according to the embodiment of the present disclosure, the trained neural network includes a recurrent neural network RNN and / or a spiking neural network SNN.

[0153] For the recurrent neural network RNN and the spiking neural network SNN, the corresponding backpropagation through time BPTT algorithm will make the memory requirement for saving the intermediate state more significant, so that it cannot be implemented when processing a long sequence.

[0154] According to the data processing method of the embodiment of the present disclosure, since the intermediate state of the network at each time step does not need to be saved, and a high-precision gradient signal does not need to be back-propagated, the reduction effect of the memory overhead and the calculation overhead is more obvious when training the recurrent neural network RNN and the spiking neural network SNN, and the training of a long sequence that cannot be trained by the original BPTT algorithm can be implemented.

[0155] The network training method according to the embodiment of the present disclosure is more friendly to hardware deployment, especially on a many-core system. In learning, only a local perturbation needs to be generated, and then a global weight update value of 1 bit is calculated according to the prediction of the network and is synchronized to each computing core, so that the learning can be completed. It is worth noting that the data amount of the perturbation signal for the weight is O(m+n), that is, the perturbation signal of the weight matrix is generated by the neuron and the outer product, and the perturbation parameter amount of O(mn) is needed when the perturbation is generated according to the weight. It can be seen that the processing manner according to the embodiment of the present disclosure is significantly better than the manner of generating the perturbation according to the weight.

[0156] In some possible implementation manners, the network training method according to the embodiment of the present disclosure can be applied to a many-core system, and the many-core system includes a plurality of processing cores.

[0157] The weight adjustment information of the network layer is generated according to the input random variable and the output random variable of the network layer in the current iteration state and the global weight update value, and includes:

[0158] The target processing core receives the global weight update value, and processes the input random variable and the output random variable and the received global weight update value to obtain the weight adjustment information of the network layer. The target processing core is a processing core corresponding to the network layer in the many-core system;

[0159] The first weight information of the network layer in the next iteration state is obtained by adjusting the first weight information of the network layer in the current iteration state according to the weight adjustment information of the network layer, and includes:

[0160] The target processing core processes the weight adjustment information and the first weight information currently stored in the target processing core to obtain the processed first weight information, and updates the first weight information currently stored in the target processing core.

[0161] For example, in the case that the network training method of the embodiment of the present disclosure is applied to the many-core system, the network layers of the neural network are deployed to each processing core of the many-core system respectively; each network layer can correspond to one or more processing cores. For any network layer, the processing core corresponding to the network layer in the many-core system can be referred to as a target processing core, and the weight information of the network layer is stored in the target processing core or the storage space corresponding to the target processing core. At the beginning of each iteration of neural network training, for any network layer of the neural network, the first weight information of the network layer in the current iteration state is stored in the target processing core of the network layer.

[0162] In some possible implementation manners, local perturbation can be performed in step S121. Before perturbation, the target processing core can randomly generate input random variables and output random variables satisfying a preset distribution rule; a perturbation information quantity is generated according to the input random variables, the output random variables and a preset perturbation coefficient; the first weight information is positively and negatively perturbed according to the perturbation information quantity to generate and cache the second weight information. This process can be implemented in parallel by the target processing cores of each network layer to improve processing efficiency.

[0163] In some possible implementation manners, after perturbation of each network layer of the neural network, the positively perturbed neural network and the negatively perturbed neural network are obtained in step S122; then, in step S123, a batch of sample data is taken, and the positively perturbed neural network and the negatively perturbed neural network are used to process the batch of sample data to obtain the positively perturbed processing result and the negatively perturbed processing result of the batch of sample data. This process can be implemented by the target processing cores of each network layer.

[0164] In some possible implementation manners, according to the positive perturbation processing result, the negative perturbation processing result, and the label result of the sample data, the global weight update value b can be determined according to formula (4) z The global weight update value is synchronized to the target processing core corresponding to each network layer of the neural network. This step can be implemented by a specific processing core or one or more target processing cores.

[0165] In some possible implementation manners, for any network layer, the target processing core corresponding to the network layer receives the global weight update value; according to the input random variable and the output random variable of the network layer stored in the target processing core and the global weight update value, the weight adjustment information Aw of the network layer can be generated according to formula (5); then, the target processing core adjusts the first weight information currently stored in the target processing core according to the weight adjustment information; and the first weight information currently stored in the target processing core is updated to obtain the first weight information of the network layer in the next iteration state. This process can be implemented in parallel by the target processing cores of each network layer, improving the processing efficiency.

[0166] In this way, the intermediate state of the network at each time step does not need to be saved, no additional control overhead is needed, and no high-precision gradient signal needs to be returned, which can significantly reduce the memory overhead and the calculation overhead in training, realize efficient deployment on a decentralized many-core system, and further improve the processing efficiency in a parallel manner in the many-core system. Moreover, the training can be normally performed in the case where the numerical precision supported by the many-core system is low (int8 or fp16), and the stability of the training is maintained.

[0167] As described above, at the beginning of each iteration, the input random variable and the output random variable that satisfy the preset distribution rule can be randomly generated for each network layer. The input random variable and the output random variable can each be a Bernoulli random variable, and each element of the Bernoulli random variable has a value of 1 or -1.

[0168] In some possible implementation manners, in the process of the network training method in the embodiments of the present disclosure in multiple iterations:

[0169] In each iteration, the probability that each element of the Bernoulli random variable has a value of 1 is 0.5; or

[0170] In each iteration, the probability that each element of the Bernoulli random variable has a value of 1 is determined according to the current iteration number, the value of the probability in the previous iteration, the value of the Bernoulli random variable in the previous iteration, and the value of the Bernoulli random variable in the current iteration.

[0171] That is, in the process of multiple iterations of training, the probabilities of the two values of -1 or 1 of each element of the controllable Bernoulli random variable (including the input random variable and the output random variable) are equal and fixed, that is, both are 0.5, so as to ensure the randomness and stability of the perturbation processing, and make the network convergence process more stable.

[0172] In some possible implementation manners, the probabilities of the two values of -1 or 1 of each element of the controllable Bernoulli random variable (including the input random variable and the output random variable) can be changed with the training process, so as to improve the convergence speed of the neural network, alleviate the problem of falling into a local minimum, and improve the training effect of the network.

[0173] In some possible implementation manners, for any iteration in the multiple iterations, let the number of previous iterations be k times, and the current iteration be the (k+1)th iteration, where k is an integer. In the current iteration, the probability of each element of the Bernoulli random variable taking the value 1 is p k+1 , and correspondingly, the probability of each element taking the value -1 is 1-p k+1 . The value of the probability p k+1 may be determined according to the following formula: p k+1 =g[{p i} i≤k ,{b i} i≤k ,{b z} i≤k ,{x} i≤k ,{y} i≤k , θ,k] (6)

[0174] In formula (6), p k+1 represents the value of the probability of each element taking the value 1 in the (k+1)th iteration, {p i} i≤k represents a set of values of the probability of each element taking the value 1 in the previous iterations (i.e., the first k iterations), {b i} i≤k represents a set of Bernoulli random variables in the first k iterations, {b z} i≤k represents a set of global weight update values in the first k iterations, {x} i≤k represents a set of input data of the neural network in the first k iterations, {y} i≤k represents a set of output data of the neural network in the first k iterations, θ represents parameters of the neural network, including first weight information of multiple network layers of the neural network, g is a function for updating the probability, and i and k are integers.

[0175] It can be seen that in the iteration called current iteration, the probability of each element of the Bernoulli random variable taking value 1 can be determined according to the number of previous iterations, the value of the probability in the previous iteration, the value of the Bernoulli random variable in the previous iteration, and the value of the Bernoulli random variable in the current iteration.

[0176] In some possible implementation manners, in the k+1th iteration of the multiple iterations, the value of the probability p k+1 may be determined according to the following formula: p k+1 = clip[p k + λb k b z , δ] (7)

[0177] In formula (7), p k+1 denotes the value of the probability of each element taking value 1 in the k+1th iteration, p k denotes the value of the probability in the kth iteration, b k denotes the Bernoulli random variable in the kth iteration, b z denotes the global weight update value in the kth iteration, λ denotes an update parameter for reflecting an update speed, for example, can take a value of 0.1, and the clip[p, δ] function denotes a clipping function for clipping each element of the probability p to a range greater than or equal to δ and less than or equal to (1-δ), and δ is a constant taking a value between 0 and 1. The disclosure does not limit the specific values of λ and δ.

[0178] The meaning of formulas (6)-(7) is that the probability of each element of the Bernoulli random variable taking value 1 can be increased or decreased according to the direction of weight update in the iteration, and thresholded by the clip function, to maintain a certain exploration ability.

[0179] By adjusting the probability of each element of the Bernoulli random variable taking value 1 along with the training process, the processing manner can play a role of momentum, so that each iteration is accelerated to update in the direction of network convergence, thereby improving the convergence speed of the neural network, relieving the problem of falling into a local minimum, and further improving the training effect of the network.

[0180] According to the data processing method provided in the embodiments of the present disclosure, the neural network training can be performed based on local disturbance learning and global signal regulation, the local disturbance of each network layer weight of the neural network is combined with the global signal of the entire neural network in one iteration training, the adjustment value of the weight is obtained, and then the weight is adjusted to realize the training of the network. Through this training method, the general optimization capability and high learning performance of the BP algorithm and the local learning and hardware-friendly advantages of the Hebb, STDP and other bionic local learning algorithms are combined, and the advantages are fused. At the same time, the problems of the intermediate state saving and additional scheduling control of the BP method and the problems of poor generality and poor learning performance of the Hebb method are avoided.

[0181] According to the data processing method provided in the embodiments of the present disclosure, the hardware is more friendly in implementation, and is more suitable for deployment to a many-core system (for example, a many-core brain-like computing chip). The intermediate state of the network at each time step does not need to be saved, and a high-precision gradient signal does not need to be fed back, so that the memory overhead and the calculation overhead in the training can be significantly reduced, and the deployment efficiency of a device with low numerical precision (for example, a many-core system architecture) can be improved.

[0182] According to the data processing method provided in the embodiments of the present disclosure, the differentiability of the network is not limited, and the non-differentiable network can also be applied. The method has a wider application range than the BP, and is suitable for the optimization of the brain-like spiking neural network, the optimization of the deep artificial neural network, and other types of optimization problems.

[0183] As described above, although the gradient descent method based on the BP has good training effect and strong universality, the memory overhead and the calculation overhead in the training are large. Moreover, for a many-core system architecture including a plurality of processing cores, for example, a brain-like computing chip or a neuromorphic chip, when the training is performed by using the BP algorithm, the gradient update of each layer of the neural network deployed on each processing core needs additional synchronization logic. Generally, an additional controller such as a CPU needs to be used to coordinate and control the calculation cores, which brings additional overhead and limits the on-chip learning application of the neural network.

[0184] In addition, the BP algorithm generally needs to maintain high precision (for example, FP16 (16-bit floating point number) or FP32 (32-bit floating point number)) when performing back propagation, which not only increases the data transmission amount but also correspondingly increases the calculation amount. Generally, the numerical precision supported by the brain-like chip is low (for example, int8 or FP16), which also affects the numerical precision and the training stability in the BP algorithm.

[0185] Moreover, although the method based on gradient estimation can avoid the problem of intermediate state saving, the gradient needs to be determined through multiple feedforward, which leads to low training efficiency.

[0186] For example, the gradient estimation formula (8) of the simultaneous perturbation stochastic approximation (SPSA) is as follows:

[0187] where L is the objective function, ε is the perturbation step, θ represents the network parameters of the neural network, and B represents a batch of sample data. z is a random noise perturbation satisfying certain conditions, and can be selected as a Gaussian distribution or a Bernoulli distribution. When the Bernoulli distribution is selected, each element of z is binary, and can be selected as {-1, 1}. is the accurate gradient of the network parameters θ, is the estimated gradient. In fact, is the directional derivative of L along the noise z direction, which can indicate whether the descending direction of the objective function is along the z direction or the -z direction, so as to complete the optimization similar to the gradient descent. Moreover, the SPSA method needs to perform forward propagation twice (once along the z direction and once along the -z direction) at each gradient estimation, thereby resulting in a relatively low training efficiency.

[0188] Therefore, the embodiments of the present disclosure provide a data processing method, which can perturb the network parameters of the neural network in the current iteration state, obtain a perturbed neural network, and perform a forward propagation inference on the sample data by using the perturbed neural network, so as to obtain a target evaluation value of the perturbed neural network in the current iteration state. By the target evaluation value in the current iteration state and a target reference value of the neural network in the current iteration state, the perturbation processing result of the neural network can be determined, and then the network parameter information in the current iteration state can be adjusted in combination with a random perturbation variable in the current iteration state to obtain network parameter information in a next iteration state. Through the above process, the training of the neural network can be realized. Moreover, only one forward propagation needs to be performed in each iteration process, and the number of forward propagations is relatively small, so that the training efficiency can be improved, and the memory consumption and the calculation consumption in the network training can be reduced.

[0189] FIG. 4 is a flowchart of a data processing method provided by the embodiments of the present disclosure.

[0190] In some possible implementation manners, in the case where the parameter adjustment manner is parameter perturbation, referring to FIG. 4, step S12 includes steps S124-S126, where:

[0191] Step S124: perturbing the network parameter information of the neural network in the current iteration state to obtain a perturbed neural network.

[0192] Step S125, the sample data in the training set is inferred once through the disturbed neural network, and a target evaluation value of the disturbed neural network in the current iteration state is obtained.

[0193] Step S126, a disturbance processing result is determined according to the target evaluation value in the current iteration state and a target reference value of the neural network in the current iteration state, and the target reference value in the current iteration state is obtained by adjusting the target reference value in the previous iteration state.

[0194] Step S13 includes step S132, wherein:

[0195] Step S132, the network parameter information in the current iteration state is adjusted according to the disturbance processing result and the random disturbance variable of the neural network in the current iteration state, and the network parameter information in the next iteration state is obtained to train the neural network.

[0196] For example, the training of the neural network can include multiple iteration processes, and at the beginning of each iteration, the parameters of each network layer of the neural network are the network parameter information in the current iteration state. The parameters of the network layer of the neural network can include weight parameters and bias parameters, and the embodiments of the present disclosure are not limited thereto.

[0197] In some optional implementation manners, in step S124, the network parameter information of the neural network in the current iteration state can be disturbed to obtain the disturbed neural network. The disturbance processing can be random disturbance, that is, a random value is added to or subtracted from each network parameter information, and the random value can be in a certain value range (for example, -1-1) and satisfy a certain distribution rule (for example, Gaussian distribution, Bernoulli distribution, etc.). In addition, other disturbance methods can be used to disturb the network parameter information of the neural network, and the embodiments of the present disclosure are not limited thereto.

[0198] In some optional implementation manners, in step S125, the sample data in the training set can be inferred once by using the disturbed neural network, so as to obtain the target evaluation value of the disturbed neural network in the current iteration state. In one iteration process, a part of sample data can be fed forward once, for example, a batch of sample data is processed. The target evaluation value can reflect the degree of disturbance of the disturbed neural network, that is, the degree of inconsistency between the prediction value and the true value of the disturbed neural network.

[0199] In some optional implementation, in step S126, the perturbation processing result can be determined according to the target evaluation value of the current iteration state and the target reference value of the neural network in the current iteration state, through which it can be reflected whether the influence of adding the perturbation to the neural network on the training effect is positive or negative, which provides an important basis for subsequent adjustment of the network parameter information. The target reference value of the current iteration state is obtained by adjusting the target reference value of the previous iteration state.

[0200] In some optional implementation, in step S132, the influence of the perturbation added to the neural network on the training effect of the neural network can be determined according to the perturbation processing result, and therefore, the network parameter information of the current iteration state can be adjusted based on the perturbation processing result and the random perturbation variable of the neural network in the current iteration state, to obtain the network parameter information of the next iteration state, and further obtain the neural network in the next iteration state, to complete one iteration process.

[0201] In this way, each iteration makes the loss of the neural network smaller. After multiple iteration processes, the trained neural network can be obtained when the training condition (for example, network convergence or reaching the preset number of iterations) is met.

[0202] It should be noted that, according to the above content, in the training process of the neural network, only one feedforward process (corresponding to step S125) needs to be performed, and the required computing resource overhead and storage resource overhead are relatively small, and the training efficiency can be effectively improved.

[0203] According to the embodiments of the present disclosure, the network parameters of the neural network in the current iteration state can be perturbed to obtain the perturbed neural network, and the sample data can be fed forward once by using the perturbed neural network, to obtain the target evaluation value of the perturbed neural network in the current iteration state. Through the target evaluation value of the current iteration state and the target reference value of the neural network in the current iteration state, the perturbation processing result of the neural network can be determined, and then the network parameter information of the current iteration state can be adjusted in combination with the random perturbation variable of the current iteration state to obtain the network parameter information of the next iteration state. Through the above process, the training of the neural network can be realized. In each iteration process, only one feedforward process needs to be performed, and the number of feedforward is relatively small, so that the training efficiency can be improved, and the memory overhead and computing overhead in network training can be reduced.

[0204] In the neural network training using the BP algorithm, multiple feedforward processes may be required in each gradient estimation (for example, two feedforward processes are required for SPSA), and the resource overhead required increases with the number of feedforward processes, and the processing time also increases accordingly, which may result in a long training time and affect the training efficiency. In the embodiments of the present disclosure, a perturbation is first added to the neural network, and then one feedforward process is performed based on the perturbed neural network to obtain a corresponding target evaluation value, and then the perturbation effect is determined according to the target evaluation value and the target reference value, and then the network parameter information is adjusted according to the perturbation effect, thereby completing one iteration process of the neural network. That is, only one feedforward process is required in one iteration process to accurately adjust the network parameter information, the resource overhead is effectively reduced, and the training time is also reduced, thereby improving the training efficiency.

[0205] As described previously, in any iteration in the neural network training, a corresponding perturbation can be added to the network parameter of the current iteration state of the neural network in step S124 to obtain the perturbed neural network.

[0206] Before the perturbation, a corresponding random perturbation variable can be set, and the random perturbation variable is added to the corresponding network parameter to obtain the perturbed network parameter, and then the perturbed neural network is obtained based on the perturbed network parameter.

[0207] In some optional implementations, the network parameter information of the current iteration state of the neural network is perturbed to obtain the perturbed neural network, including: determining a random perturbation variable corresponding to each network parameter of the current iteration state of the neural network; superimposing the random perturbation variable on the corresponding network parameter to obtain a perturbed network parameter; and obtaining the perturbed neural network based on the perturbed network parameter. The random perturbation variable can be subject to a Gaussian distribution, a Bernoulli distribution, etc.

[0208] For example, one random perturbation variable is generated based on a preset distribution, the random perturbation variable includes a group of perturbation data with the same number as the network parameters of the neural network, and the perturbation data can include positive values greater than zero, zero, and negative values less than zero. For the network parameter information of each network layer of the neural network, the corresponding perturbation data is added to obtain a group of perturbed network parameters, and the perturbed neural network is determined based on the perturbed network parameters.

[0209] For example, in the kth iteration, if the network parameter information of the current iteration state of the neural network includes network parameters θ k , and the generated random perturbation variable is g k , the perturbed network parameter θ k + g kAnd, when performing processing of sample data based on the perturbed neural network, the perturbed network parameter θ k +g k ’ is equivalent to a parameter search direction, that is, a forward feed of the sample is to be made in the direction of θ k +g k ’.

[0210] After obtaining the perturbed neural network, data processing can be performed by using the perturbed neural network to obtain a target evaluation value of the perturbed neural network in the current iteration state. The target evaluation value can be a value obtained by evaluating the inference ability (or prediction ability) of the perturbed neural network according to a loss function.

[0211] In some optional implementations, in step S125, the target evaluation value of the perturbed neural network in the current iteration state is obtained by performing inference on the sample data in the training set by using the perturbed neural network, including: inputting the sample data into the perturbed neural network to obtain a prediction result of the sample data; determining a function value of a preset loss function according to the prediction result of the sample data and a label result of the sample data, and taking the function value as the target evaluation value of the current iteration state.

[0212] For example, for the kth iteration, the neural network is represented as f k , the perturbed neural network after adding the perturbation processing is represented as f k ’, and the training set includes multiple sample data (x, y), where x is input data, and y is a label result (i.e., expected output data) corresponding to the input data. Based on this, one forward feed processing of the input data in the sample data can be performed by using the perturbed neural network f k ’ to obtain a prediction result y’ = f k ’ (x). If the preset loss function is L, based on the prediction result y’ and the label result y, a function value of the loss function can be obtained, and the function value is taken as the target evaluation value L cur_k of the perturbed neural network in the current iteration state.

[0213] In some optional implementations, the perturbation processing result can be determined according to the target evaluation value and a corresponding target reference value. The perturbation processing result can represent whether the perturbation added to the neural network has a positive impact on the training result. Moreover, for the target reference value of the current iteration state, it is obtained by adjusting the target reference value of the previous iteration state.

[0214] In some optional implementation manners, the perturbation processing result is determined according to the target evaluation value of the current iteration state and the target reference value of the neural network at the current iteration state, including: in a case where the target evaluation value of the current iteration state is less than the target reference value of the current iteration state, determining that the perturbation processing result is a positive perturbation processing result; in a case where the target evaluation value of the current iteration state is greater than or equal to the target reference value of the current iteration state, determining that the perturbation processing result is a negative perturbation processing result; wherein the perturbation processing result includes the positive perturbation processing result and the negative perturbation processing result. The target reference value of the current iteration state is a determined value in the last iteration process; for example, the target reference value of the kth iteration state can be obtained by adjusting the target reference value of the (k-1)th iteration state according to the target evaluation value of the (k-1)th iteration state.

[0215] That is, theoretically, the smaller the target evaluation value, the smaller the loss of the neural network, indicating that the added perturbation is a perturbation that will have a positive effect on training; on the contrary, the larger the target evaluation value, the larger the loss of the neural network, indicating that the added perturbation is a perturbation that will have a reverse effect on training. The target reference value of the current iteration state is used as a comparison benchmark to measure the size of the target evaluation value. If the target evaluation value is less than the target reference value, it indicates that the perturbation is a positive perturbation, and the corresponding perturbation processing result is a positive perturbation processing result; if the target evaluation value is greater than the target reference value, it indicates that the perturbation is a reverse perturbation, and the corresponding perturbation processing result is a negative perturbation processing result; if the target evaluation value is equal to the target reference value, it indicates that the perturbation does not have a positive effect or a reverse effect on training and does not have a promoting effect, and therefore the corresponding perturbation processing result is also attributed to the negative perturbation processing result.

[0216] After the perturbation processing result is determined, the adjustment manner of the network parameter of the current iteration state can be determined, and the adjustment is performed based on the adjustment manner to obtain the network parameter information of the next iteration state.

[0217] In some optional implementation manners, the network parameter information of the next iteration state is obtained by adjusting the network parameter information of the current iteration state according to the perturbation processing result and the random perturbation variable of the neural network at the current iteration state, including: performing one inference on the random perturbation variable according to the perturbation processing result to obtain parameter adjustment information; and determining the network parameter information of the next iteration state according to the parameter adjustment information and the network parameter information of the current iteration state. The random perturbation variable is the perturbation information added to the network parameter of the current iteration state.

[0218] That is, the perturbation processing result can represent whether the added perturbation is effective for training, and thus some adjustment can be made to the random perturbation variable according to the perturbation processing result to obtain parameter adjustment information, and the network parameter information of the next iteration state can be obtained by adjusting the network parameter information of the current iteration state based on the parameter adjustment information.

[0219] Further, the generated parameter adjustment information is different for different perturbation processing results, so that accurate and reasonable adjustment of the network parameter information can be achieved.

[0220] In some optional implementations, the perturbation processing result includes a positive perturbation processing result and a negative perturbation processing result; and correspondingly, the processing of the random perturbation variable according to the perturbation processing result to obtain the parameter adjustment information includes: in the case that the perturbation processing result is the positive perturbation processing result, determining first parameter adjustment information according to a product value of the random perturbation variable and the learning rate of the current iteration state; in the case that the perturbation processing result is the negative perturbation processing result, determining an inverse vector of the random perturbation variable, and determining second parameter adjustment information according to a product value of the inverse vector, a preset update coefficient and the learning rate of the current iteration state; and the parameter adjustment information includes the first parameter adjustment information and the second parameter adjustment information.

[0221] In some optional implementations, the second parameter adjustment information can be determined by formula 1: △θ=-λη k g k ’ 1

[0222] wherein, △θ represents the second parameter adjustment information, g k ’ represents the random perturbation variable, -g k ’ represents the inverse vector of the random perturbation variable, η k is the learning rate, and λ is an update coefficient for representing the update degree of the network parameter, and λ is a positive number less than 1. And λ is set to a positive number less than 1 mainly to reduce the degree of single update and alleviate the deviation of the update operation from the expectation due to the inaccurate adjustment direction of the parameter.

[0223] In some optional implementations, the network parameter information includes a network parameter value, and the parameter adjustment information includes a parameter adjustment value; and correspondingly, the network parameter information of the next iteration state is determined according to the parameter adjustment information and the network parameter information of the current iteration state, including: subtracting the corresponding parameter adjustment value from the network parameter value of the current iteration state to obtain the network parameter value of the next iteration state.

[0224] It should be noted that the direction in which the value of the loss function decreases most steeply is exactly opposite to the gradient direction, and therefore, after determining the perturbation processing result based on the target evaluation value in the current iteration state and the target reference value of the neural network in the current iteration state, it can be determined whether the perturbation direction of the added random perturbation variable can make the loss value decrease. If it can make the loss value decrease, it means that the opposite direction of the perturbation direction of the random perturbation variable should be the correct gradient direction, and therefore, subtracting the corresponding parameter adjustment value from the network parameter value in the current iteration state can make the network parameter update towards the correct gradient direction (i.e. the direction in which the loss function value decreases).

[0225] Exemplarily, the random perturbation variable is g k If the target evaluation value L cur_k in the current iteration state is less than the target reference value L k in the current iteration state, it means that the random perturbation variable g k ' can reduce the value of the loss function, and therefore, it is determined that the random perturbation direction is feasible, and the perturbation processing result is a positive perturbation processing result. Based on this, the product value η k g k ' of the random perturbation variable g k ' and the learning rate η k in the current iteration state is taken as the first parameter adjustment information.

[0226] If the target evaluation value L cur_k in the current iteration state is greater than or equal to the target reference value L k in the current iteration state, it means that the random perturbation variable g k ' cannot reduce the value of the loss function, and therefore, it is determined that the random perturbation direction is not feasible, and the perturbation processing result is a negative perturbation processing result. Based on this, it is determined that the opposite direction -g k ' of the random perturbation variable is a direction that may cause the loss function value to decrease, and therefore, the opposite direction -g k ' of the random perturbation variable is determined, and the product value (-λ k g k ' of the opposite vector -g k ', the preset update coefficient λ and the learning rate η k in the current iteration state is taken as the second parameter adjustment information.

[0227] The update coefficient is used to represent the update degree of the network parameter, and the update coefficient can be set to a small positive number (for example, 0.1). This is because when the perturbation processing result is a negative perturbation processing result, only the forward feeding and loss evaluation in the g k ' direction are performed, and the actual forward feeding and loss evaluation in the opposite direction -g k ' are not performed, but are based on gk 'Fedforward and loss assessment in the opposite direction -g' k 'Feedforward and loss evaluation may result in positive perturbation. Therefore, in order to both update the network parameters and mitigate the large-scale erroneous updates of the network parameters caused by inaccurate gradient update direction, the update coefficient is set to a small positive number.'

[0228] Based on this, if the network parameters of the neural network in the current iteration state are θ k Then, based on the first parameter adjustment information, the network parameters θ for the next iteration state can be determined. k+1 For (θ) k -η k g k Based on the first parameter adjustment information, the network parameters θ for the next iteration state can be determined. k+1 For (θ) k -(-λη) k g k '))=(θ k +λη k g k ').

[0229] Figure 5 is a schematic diagram of a data processing method provided in an embodiment of this disclosure, illustrating two different update scenarios through a and b. Figure 5 shows the correlation between the random perturbation variable added to the network parameter information of the neural network and the gradient update direction of the network parameter information.

[0230] Referring to a in Figure 5, in the k-th iteration of the neural network, if the random perturbation variable of the network parameter information in the current iteration state (i.e., the k-th iteration) is g... k The perturbation direction is shown in Figure a. The perturbed neural network is then used to process the sample data in the training set to obtain the target evaluation value L of the perturbed neural network in the current iteration state. cur_k And compare it with the target reference value L of the neural network in the current iteration state. k If L cur_k <L k This indicates that the random disturbance variable g k This can reduce the value of the loss function; therefore, the gradient direction is determined to be -g. k Furthermore, this is compared with the learning rate η of the current iteration state. k Multiplying these yields the gradient direction – η – of the network parameters in the current iteration state. k g k As shown in the direction of the k-th gradient update in a (i.e., the perturbation direction is opposite to the gradient direction).

[0231] Referring to b in FIG. 5, in the kth iteration of the neural network, if it is determined that the random perturbation variable of the network parameter information of the neural network in the current iteration state (i.e., the kth iteration) is g k ', the perturbation direction is as shown in b. The perturbed neural network is used to process the sample data in the training set, and the target evaluation value L cur_k of the perturbed neural network in the current iteration state is obtained. k If L cur_k ≥ L k , it indicates that the random perturbation variable g k ' cannot reduce the value of the loss function, and therefore, the gradient direction is determined to be g k '. Further, the product of g k ' and a small update coefficient λ is multiplied by the learning rate η k of the current iteration state, and the product is further multiplied by g k ', i.e., the gradient direction λη k g k ' of the network parameter information in the current iteration state is obtained, as shown in the kth gradient update direction in b (i.e., the perturbation direction and the gradient direction are opposite).

[0232] It should be noted that if the perturbation direction of the perturbation random variable is the direction in which the value of the loss function decreases, then the gradient direction is exactly opposite to the perturbation direction. Therefore, in a, the gradient direction is opposite to the perturbation direction, and in b, the gradient direction is the same as the perturbation direction.

[0233] Further, comparing a and b, it can be seen that the gradient amplitude of the kth update in a is greater than the gradient amplitude of the kth update in b, because the gradient amplitude in b is additionally multiplied by a small update coefficient. This is mainly because in b, the perturbation random variable in the perturbation direction of -g k ' is not actually fed forward and the loss value is estimated, and the selection of g k ' as the gradient direction is only a speculation under the premise that g k ' cannot reduce the loss value. In order to avoid the speculation error from causing a large update of the network parameter information in the wrong gradient direction, a small update coefficient is used to adjust the gradient amplitude and reduce the intensity of single update.

[0234] Through the above method, after multiple iterations, the network parameter information of the neural network can gradually approach the target solution, and finally a neural network satisfying the training condition is obtained. The target solution can be understood as the optimal solution of the loss function of the neural network.

[0235] In some optional implementation manners, the method can further include: adjusting the target reference value of the current iteration state according to the perturbation processing result and the target evaluation value of the current iteration state, to obtain a target reference value of a next iteration state, so as to adjust the network parameter information of the next iteration state based on the target reference value of the next iteration state.

[0236] In the case that the perturbation processing results are different, the adjustment manner of the target reference value can also be different accordingly, so as to obtain a more accurate and reasonable target reference value of the next iteration state.

[0237] In some optional implementation manners, the perturbation processing result includes a positive perturbation processing result and a negative perturbation processing result. Correspondingly, the adjusting the target reference value of the current iteration state according to the perturbation processing result and the target evaluation value of the current iteration state, to obtain a target reference value of a next iteration state, includes: in the case that the perturbation processing result is the positive perturbation processing result, calculating a weighted value of the target evaluation value of the current iteration state and the target reference value of the current iteration state based on a preset weight adjustment coefficient, taking the weighted value as the target reference value of the next iteration state, or taking the target evaluation value of the current iteration state as the target reference value of the next iteration state; in the case that the perturbation processing result is the negative perturbation processing result, taking the target reference value of the current iteration state as the target reference value of the next iteration state.

[0238] In some optional implementation manners, in the case that the perturbation processing result is the positive perturbation processing result, the target reference value of the next iteration state can be determined by formula 2: L k+1 = γL k + (1-γ)L cur_k 2

[0239] In the case that the perturbation processing result is the negative perturbation processing result, the target reference value of the next iteration state can be determined by formula 3: L k+1 = L cur_k 3

[0240] Wherein, L k+1 is the target reference value of the next iteration state, L k is the target reference value of the current iteration state, L cur_k is the target evaluation value of the current iteration state, and γ is the weight adjustment coefficient.

[0241] It can be known that, by the above adjustment manner, the value of the loss function generally presents a downward trend, so as to ensure that the neural network is trained in the correct direction, and the trained neural network has better task processing capability.

[0242] For example, in the kth iteration, if the target evaluation value Lcur_k The target reference value L is less than the current iteration state. k Then the target reference value L for the next iteration state can be set. k+1 =γL k +(1-γ)L cur_k Where γ is the weight adjustment coefficient. Alternatively, the target reference value L for the next iteration state can be set directly. k+1 =L cur_k The difference between the two implementation methods mentioned above lies in the fact that the L determined in the first implementation method... k+1 Overall, it's smoother, but requires corresponding weighted calculations. L is determined based on the second implementation method. k+1 It is now simpler and faster.

[0243] If the target evaluation value of the current iteration state is L cur_k The target reference value L is greater than or equal to the current iteration state. k Then the target reference value L for the next iteration state can be set. k+1 =L cur_k .

[0244] In addition, the predicted value of the target evaluation can be maintained based on some estimation methods. For example, the value of the loss function can be estimated using methods such as Kalman filtering. This disclosure does not limit the scope of the embodiments.

[0245] In summary, in this embodiment, the historical information of the loss function value is used to indicate the update direction. Compared with methods such as SPSA, only one feedforward is needed in each iteration to determine a more accurate update direction. The number of feedforward operations is relatively small, which can improve training efficiency and reduce memory and computational overhead in network training.

[0246] In some possible implementations, the data processing method according to embodiments of this disclosure can be applied to a many-core system, which includes multiple processing cores and a neural network including multiple network layers. Accordingly, based on the perturbation processing result and the random perturbation variables of the neural network in the current iteration state, the network parameter information of the current iteration state is adjusted to obtain the network parameter information of the next iteration state. This includes: the target processing core perturbs the network parameter information of the neural network layer in the current iteration state to obtain the perturbated network layer. The target processing core is the processing core in the many-core system corresponding to the network layer (i.e., the processing core on which the network layer is deployed).

[0247] Exemplarily, the perturbation processing result can be received by the target processing core, and adjustment is performed according to the perturbation processing result and network parameter information of a current iteration state corresponding to the network layer deployed in the target processing core, to obtain network parameter information of a next iteration state of the network layer. The network parameter information of the next iteration state of the plurality of network layers of the neural network constitutes the neural network in the next iteration state.

[0248] For example, in the case that the data processing method of the embodiments of the present disclosure is applied to a many-core system, the network layers of the neural network can be respectively deployed to each processing core of the many-core system; each network layer can correspond to one or more processing cores, and one or more network layers can also be deployed in one processing core. For any network layer, the processing core corresponding to the network layer in the many-core system can be referred to as a target processing core, and the network parameter information of the network layer is stored in the target processing core or a storage space corresponding to the target processing core. At the beginning of each iteration of the training of the neural network, for any network layer of the neural network, the network parameter information of the network layer in the current iteration state is stored in the target processing core of the network layer.

[0249] In some possible implementation manners, the local perturbation can be performed in step S124. The perturbation processing of the network parameter information of the neural network in the current iteration state to obtain the perturbed neural network includes: performing, by the target processing core, the perturbation processing on the network parameter information of the network layer of the neural network in the current iteration state to obtain a perturbed network layer, and the target processing core is a processing core corresponding to the network layer in the many-core system.

[0250] Exemplarily, before the perturbation, the target processing core can randomly generate a random perturbation variable satisfying a preset distribution rule and corresponding to the network parameter of the network layer deployed in the target processing core, and superimpose the random perturbation variable on the corresponding network parameter to obtain a perturbed network parameter of the network layer; and the perturbed neural network can be obtained according to the perturbed network parameters of the plurality of network layers of the neural network. Since the above process can be implemented in parallel by the target processing cores of the plurality of network layers, the processing efficiency can be improved.

[0251] In some possible implementation manners, after the perturbation of each network layer of the neural network, the target processing core can perform inference on the sample data based on the perturbed network layer in step S125 to obtain a target evaluation value of the perturbed neural network in the current iteration state.

[0252] Exemplarily, a batch of sample data can be taken, and a forward propagation is performed on the sample data by the perturbed neural network to obtain a prediction result of the batch of sample data, which is compared with a corresponding label result to obtain a target evaluation value of the perturbed neural network in the current iteration state in combination with a preset loss function. The sample data can be input into the many-core system in a pipeline manner, and each processing core determines a data flow direction according to a connection relationship of network layers of the neural network, and then performs processing of the sample data in a pipeline manner and in an order.

[0253] Further, in step S126, a perturbation processing result can be determined by a preset processing core in the many-core system according to the target evaluation value in the current iteration state and the target reference value of the neural network in the current iteration state, and the perturbation processing result is synchronized to each target processing core.

[0254] In other words, this step can be implemented by a certain specific processing core or one or more of the target processing cores (i.e., the preset processing core). Accordingly, the preset processing core determines the perturbation processing result according to the target evaluation value in the current iteration state and the target reference value of the neural network in the current iteration state, and synchronizes the perturbation processing result to the processing core in which the network layer is deployed in the many-core system.

[0255] After the target processing core obtains the perturbation processing result, the network parameter information of the network layer in the current iteration state is processed by the target processing core according to the perturbation processing result and the random perturbation variable of the network layer in the current iteration state in step S132 to obtain network parameter information of the network layer in the next iteration state.

[0256] Exemplarily, the random perturbation variable is processed according to the perturbation processing result to obtain parameter adjustment information of the network layer deployed in the target processing core, and the network parameter information of the next iteration state of the network layer is determined according to the parameter adjustment information and the network parameter information of the current iteration state of the network layer. The neural network in the next iteration state is obtained according to the network parameter information of the next iteration state of the plurality of network layers of the neural network.

[0257] In some possible implementation manners, for any network layer, a target processing core corresponding to the network layer receives a perturbation processing result, and if the perturbation processing result is a positive perturbation processing result, the target processing core determines a first parameter adjustment value of the network layer according to a product value of the random perturbation variable and the learning rate of the current iteration state, and then subtracts the first parameter adjustment value from the network parameter value of the current iteration state of the network layer to obtain the network parameter value of the next iteration state of the network layer. Conversely, if the perturbation processing result is a negative perturbation processing result, the target processing core first determines an inverse vector of the random perturbation variable, and then determines a second parameter adjustment value according to a product value of the inverse vector, a preset update coefficient and the learning rate of the current iteration state, and subtracts the second parameter adjustment value from the network parameter value of the current iteration state of the network layer to obtain the network parameter value of the next iteration state of the network layer. Since the above network parameter adjustment process can be implemented in parallel by the target processing cores of each network layer, the processing efficiency can be improved.

[0258] To sum up, by using the processing mode based on the many-core system, on the one hand, the intermediate states of the network at each time step do not need to be saved, no additional control overhead is needed, and high-precision gradient signals do not need to be returned, which can significantly reduce the memory overhead and the calculation overhead in the training, realize efficient deployment on the decentralized many-core system, and coordinate without central control; on the other hand, the processing efficiency can be further improved in a parallel execution manner in the many-core system; in addition, the data processing method based on the embodiments of the present disclosure only needs to perform a forward propagation once in one iteration process, the data processing amount is small, the resource overhead of the many-core system can be saved, and the training efficiency can be improved. Moreover, the neural network can be normally trained in the case that the numerical precision supported by the many-core system is low (int8 or fp16), and the stability of the training is maintained.

[0259] In some optional implementation manners, the data processing method of the embodiments of the present disclosure can also be applied to training of a neural network based on Back Propagation Through Time (BPTT); wherein the neural network comprises at least one of a recurrent neural network and a spiking neural network. The Back Propagation Through Time is an extension method of the back propagation algorithm, which can propagate errors from the output layer of the network to the hidden layer and the input layer of the neural network, so as to update the weights of the neural network. Moreover, since the BPTT method can process sequence data of any length, it can better handle long-term dependencies, propagate errors from the output layer to any time step, and thus is suitable for tasks such as speech recognition and natural language processing.

[0260] That is, the method of the embodiments of the present disclosure can be used to train recurrent neural networks, spiking neural networks, etc. in a backward propagation in time, and these neural networks can be deployed on a decentralized many-core system (e.g., a many-core chip) for on-chip training.

[0261] In some optional implementations, the training of the neural network is modeled as the following optimization formula: min θ E (x,y)～D L[f(x; θ), y]

[0262] For the kth iteration, the gradient-based optimization process can be represented as follows: θ k+1 = θ k - η k g k

[0263] where g k is the gradient of this iteration, and η k represents the learning rate of this iteration.

[0264] Each iteration process can be completed by the following steps: first, determining the search direction of the gradient; second, determining the positive and negative polarity of the search direction; third, maintaining the estimated value of the loss function, indicating the state of optimization. Details are as follows:

[0265] First, in the kth iteration of the neural network, a set of perturbation data with the same number of parameters as the neural network is randomly generated, denoted as g k ', which is added to the network parameters of the current iteration state as the gradient search direction of the network parameters. Then, based on the perturbed neural network, a forward pass is performed on a batch of sample data to evaluate the loss function value L cur_k at this time. Where the perturbation g k ' can be Gaussian distribution or Bernoulli distribution, and when Bernoulli distribution is taken, each perturbation data in g k ' takes the value of -1 or 1.

[0266] Second, judge the size relationship between L cur_k and the target reference value L k maintained. If L cur_k < L k , it means that the perturbation g k ' direction can reduce the loss function value. Therefore, the search direction g k of the gradient can be determined as g k ', and combined with the gradient optimization formula, the network parameter information θ k+1 of the next iteration state can be determined as θ k - η k g k '.

[0267] Conversely, if L cur_k ≥L k This indicates that the disturbance g k The direction does not decrease the loss function value, and its opposite direction -g k ' is a direction that might decrease the loss function value. However, since it is not based on -g k The perturbed neural network performs feedforward and loss value evaluation. Therefore, the update coefficient λ is set to a small value (e.g., 0.1). In summary, the gradient update direction g is determined. k Corresponding to (-λg) k Furthermore, by combining the gradient optimization formula, the network parameter information θ for the next iteration state can be determined. k+1 For θ k +λη k g k '.

[0268] Finally, it is also necessary to maintain the estimated value of the loss function, that is, to determine the target reference value L for the next iteration state. k+1 The update logic is as follows: if the target evaluation value L of the current iteration state... cur_k The target reference value L is less than the current iteration state. k Then the target reference value L for the next iteration state can be set. k+1 =γL k +(1-γ)L cur_k Alternatively, you can directly set the target reference value L for the next iteration state. k+1 =L cur_k Where γ is the weight adjustment coefficient; if the target evaluation value L of the current iteration state... cur_k The target reference value L is greater than or equal to the current iteration state. k Then the target reference value L for the next iteration state can be set. k+1 =L cur_k Therefore, the target reference value is non-increasing during the iteration process, causing the loss function value to decrease overall, thereby gradually improving the accuracy of the neural network. Ultimately, a well-trained neural network with the expected accuracy is obtained, and the corresponding target task can be performed based on the trained neural network.

[0269] As described above, although the BP-based gradient descent method has good training effect and strong universality, the memory and calculation overheads in training are large. Moreover, for a many-core system architecture, such as a brain-like computing chip or a neuromorphic chip, when training is performed by using the BP algorithm, gradient update of each layer of the deployed neural network on each processing core requires additional synchronization logic, and generally, an additional controller, such as a CPU, is needed to coordinate and control the computing cores, which brings additional overhead and limits the on-chip learning application of the neural network.

[0270] In addition, the BP algorithm generally needs to maintain high precision (for example, FP16 (16-bit floating point number) or FP32 (32-bit floating point number)) for data during back propagation, which not only increases the data transmission amount but also correspondingly increases the calculation amount. Generally, the numerical precision supported by a brain-like chip is low (for example, int8 or FP16), which also affects the numerical precision and training stability in the BP algorithm.

[0271] Moreover, the gradient estimation-based method can avoid the problem of intermediate state saving, but generally needs multiple feedforward to determine the gradient, resulting in inefficient training.

[0272] The zero-order gradient method such as SPSA will perturb each weight element during the entire training process, which does not conform to the overall trend of increasing gradient sparsity when training by using the BP algorithm, and thus is not conducive to the convergence of the neural network. In addition, if a part of the weight elements is selected for update at each iteration during the entire process, the convergence speed will be slow. Therefore, the static and constant gradient sparsity cannot meet the requirements of convergence speed at the beginning of training and convergence accuracy at the end of training.

[0273] Therefore, the embodiments of the present disclosure provide a data processing method, which can selectively select part of the gradient information for sparse processing according to the training requirements for different iteration processes when training the neural network, so as to realize dynamic sparse processing of the gradient, and adjust the number of network parameters participating in the update through the dynamic sparse processing of the gradient, so as to balance the convergence efficiency and convergence accuracy of the training and improve the training efficiency.

[0274] FIG. 6 is a flowchart of a data processing method provided by the embodiments of the present disclosure. In some possible implementation manners, in the case where the parameter adjustment manner is gradient sparsity, referring to FIG. 6, the method further includes the following steps.

[0275] In step S14, initial gradient information of the neural network in the current iteration state is obtained.

[0276] In step S12, steps S127 and S128 are included, in which:

[0277] In step S127, the gradient sparsity of the current iteration state is determined according to the iteration information of the neural network and the preset gradient sparsity distribution information.

[0278] In step S128, the initial gradient information is sparsified according to the gradient sparsity of the current iteration state to obtain target gradient information of the current iteration state.

[0279] In step S13, step S133 is included, in which:

[0280] In step S133, the network parameter information of the next iteration state is obtained by adjusting the network parameter information of the current iteration state based on the target gradient information of the current iteration state, so as to train the neural network.

[0281] For example, the training of the neural network can include multiple iteration processes. At the beginning of each iteration, the parameters of each network layer of the neural network are the network parameter information of the current iteration state. The parameters of the network layer of the neural network can include weight parameters and bias parameters, and the embodiments of the present disclosure are not limited thereto.

[0282] In some optional implementations, in steps S11 and S14, the network parameter information of the neural network in the current iteration state and the initial gradient information can be obtained respectively. The initial gradient information is the information preliminarily determined for gradient updating of the network parameter information of the current iteration state.

[0283] In some optional implementations, the initial gradient information can be gradient information obtained based on zero-order gradient estimation. The zero-order gradient estimation can be based on optimization of the target function (for example, the loss function of the neural network) to be optimized, without the need for derivative solving, and thus is suitable for situations where gradient calculation is difficult or impossible (for example, the target function is not differentiable, or the gradient calculation cost is high). That is, the zero-order gradient estimation does not need to calculate the gradient, but estimates a zero-order gradient through a corresponding estimation method.

[0284] In some optional implementations, for the target function of the neural network, in the case where the target function is derivable, the initial gradient information can be obtained by calculating the first-order or multi-order derivative of the target function, and in the case where the target function is not derivable, the initial gradient information can be obtained based on zero-order gradient estimation.

[0285] For example, a sampling and difference-based operator can be used to estimate the zero-order gradient. Assuming that f(x) is the target function of the neural network, the zero-order gradient of f(x) at x can be calculated as

[0286] ​wherein ε is a small positive number given in advance, μ is a random variable obeying p(μ), p(μ) is a distribution with mean 0 and covariance matrix of unit matrix specified in advance, and a standard normal distribution is usually used.

[0287] Therefore, only a few points are sampled from p(μ) to estimate the zero-order gradient. After the zero-order gradient is determined, it can be treated as a normal gradient for subsequent data processing.

[0288] In summary, the zero-order gradient estimation is mainly based on the difference method to obtain a reasonable generalization of the gradient. Since the difference operation does not require the function to be derivable, it can be applied to the optimization of derivable or non-derivable objective functions. Since the dimension of μ is the dimension of all network parameters of the neural network, when the number of parameters of the neural network is large, the variance in the calculation process based on the zero-order gradient estimation may be large, which may lead to difficulty in convergence. Based on this, the zero-order gradient estimation method can be applied to the processing of part of the network parameters of the neural network, or as an auxiliary optimization means in the parameter update process. For example, in the training process of the neural network, the network parameters of the neural network can be adjusted based on the alternating update mode of "derivable objective function + normal gradient + learning rate" and "non-derivable objective function + zero-order gradient estimation + small learning rate". After obtaining the initial gradient information by either method, the initial gradient information can be sparsified based on the gradient sparsity distribution information.

[0289] It should be noted that in the related art, the network parameter information of the current iteration state is usually directly updated based on the initial gradient information to obtain the network parameter information of the next iteration state, thereby realizing one iteration process of the neural network.

[0290] However, in the embodiments of the present disclosure, in order to realize the dynamic sparsification of the gradient, after obtaining the initial gradient information, the initial gradient information is processed based on a series of steps to realize the sparsification of the initial gradient information in the current iteration process, to obtain the sparse gradient information (i.e., the target gradient information), and then the network parameter information of the current iteration state is adjusted based on the sparse gradient information to obtain the network parameter information of the next iteration state, thereby realizing one iteration process of the neural network.

[0291] In some optional implementation, in step S127, the gradient sparsity of the current iteration state is determined according to the iteration information of the neural network and the preset gradient sparsity distribution information. The iteration information of the neural network can reflect the iteration of the neural network, and can include the total number of iterations, the current number of iterations, etc., which are not limited by the embodiments of the present disclosure. The gradient sparsity distribution information is about the distribution of the sparsity of the gradient, and can reflect the value law of the sparsity of the gradient. Moreover, the gradient sparsity distribution information is associated with the iteration process of the neural network, so that the gradient sparsity of the current iteration state can be determined in combination with the gradient sparsity distribution information after the iteration information of the neural network is determined.

[0292] In some optional implementation, in step S128, the initial gradient information is sparsified according to the gradient sparsity of the current iteration state after the gradient sparsity of the current iteration state is determined, so as to obtain the target gradient information of the current iteration state. Compared with the initial gradient information, the sparsity of the target gradient information can be improved to a certain extent, so that the convergence speed can be improved to a certain extent.

[0293] In some optional implementation, in step S133, the network parameter information of the current iteration state is adjusted based on the target gradient information of the current iteration state, so as to obtain the network parameter information of the next iteration state, and further obtain the neural network of the next iteration state, thereby completing the iteration process of the neural network.

[0294] In this way, in each iteration process, the initial gradient information is sparsified by using the gradient sparsity corresponding to the iteration process, and the network parameter information is updated by using the obtained target gradient information. After a plurality of iteration processes, the trained neural network can be obtained under the condition that the training condition (for example, the network converges or reaches the preset number of iterations) is met.

[0295] According to an embodiment of the present disclosure, network parameter information and initial gradient information of the neural network in a current iteration state are acquired; gradient sparsity of the current iteration state is determined according to iteration information of the neural network and preset gradient sparsity distribution information; the initial gradient information is sparsified according to the gradient sparsity of the current iteration state, to obtain target gradient information of the current iteration state; and the network parameter information of the current iteration state is adjusted based on the target gradient information of the current iteration state, to obtain network parameter information of a next iteration state, so as to train the neural network. Therefore, in the embodiment of the present disclosure, when training the neural network, for different iteration processes, part of the gradient information can be selectively selected for sparsification according to training requirements, so that dynamic sparsification of the gradient can be realized, and the number of network parameters participating in updating can be adjusted through dynamic sparsification of the gradient, so that the convergence efficiency and convergence accuracy of training are taken into account, and the training efficiency is improved.

[0296] When the neural network is trained by using the BP algorithm, the same gradient sparsity is usually used to update the network parameter information of the neural network in each iteration process. For example, if a higher convergence speed is required at the beginning of training, a gradient sparsity with a larger value needs to be taken, and as the training is performed, if a higher convergence accuracy is required at the end of training, a gradient sparsity with a smaller value needs to be taken. Therefore, it is difficult to take into account the individualized requirements of convergence speed and convergence accuracy at different training stages by using a static constant gradient sparsity.

[0297] In the embodiment of the present disclosure, gradient sparsity distribution information matched with the individualized requirements of convergence speed and convergence accuracy of the neural network at different training stages is set, the gradient sparsity corresponding to the current iteration state is determined in each iteration process, the initial gradient information of the current iteration state is sparsified based on the gradient sparsity, to obtain target gradient information of the current iteration state, and then the network parameter information of the current iteration state is adjusted based on the target gradient information, to obtain network parameter information of a next iteration state. In this way, the sparsification of the gradient is performed according to the preset trend in the whole training process of the neural network, the convergence speed and convergence accuracy at different training stages are accurately controlled, the convergence efficiency and convergence accuracy of training are taken into account, and the computing resource and storage resource overheads are reduced.

[0298] As described above, in any iteration in the training of the neural network, the network parameter information and the initial gradient information of the neural network in the current iteration state can be acquired in steps S11 and S14, respectively.

[0299] Exemplarily, first, the network parameter information of the neural network in the current iteration state is acquired, and the network parameter information is subjected to perturbation processing to obtain a perturbed neural network. Based on the perturbed neural network, feedforward of sample data in the training set is performed to obtain a prediction result. According to the prediction result and the label result of the sample data, a loss function value is calculated. The polarity of the perturbation is determined through the loss function value, and the initial gradient information is determined according to the polarity of the perturbation.

[0300] After the network parameter information in the current iteration state and the initial gradient information are obtained, the gradient sparsity in the current iteration state can be determined in step S127.

[0301] In some optional implementation manners, the gradient sparsity distribution information is used to represent the correlation between the gradient sparsity and the iteration number. In other words, according to the gradient sparsity distribution information, the value change of the gradient sparsity with the iteration can be determined.

[0302] In some optional implementation manners, the iteration information at least includes the current iteration number, and the gradient sparsity distribution information includes the value of the gradient sparsity corresponding to each iteration number (i.e., the value of the gradient sparsity or the gradient sparsity value). Correspondingly, according to the iteration information of the neural network and the preset gradient sparsity distribution information, the gradient sparsity in the current iteration state is determined, including: determining the target value of the gradient sparsity corresponding to the current iteration number in the gradient sparsity distribution information, and taking the target value as the gradient sparsity in the current iteration state.

[0303] Exemplarily, the gradient sparsity distribution information can be represented as D~(K, GS), where K represents the iteration number, and K = 1, 2, …, N (N > 1), GS represents the gradient sparsity, and GS = gs1, gs2, …, gsN, gsk represents the value of the gradient sparsity corresponding to the kth iteration, k ∈ K. Based on this, for the kth iteration process, the target value of the gradient sparsity corresponding to the kth iteration can be determined through D~(K, GS) as gsk, and thus the gradient sparsity in the current iteration state (i.e., the kth iteration) can be determined as gsk.

[0304] Exemplarily, the gradient sparsity distribution information can be represented as:

[0305] {D1 ~ GS1(K = [1, N1)), D2 ~ GS2(K = [N1, N2)), D3 ~ GS3(K = [N2, N])}, wherein K represents the number of iterations, and K = 1, 2, …, N, N > 1, N1 e K, N2 e K, N1 < N2 < N, wherein GS1, GS2 and GS3 are all functions of the number of iterations, and can be the same or different. And when K = [1, N1), the gradient sparsity value is determined based on the first function GS1, when K = [N1, N2), the gradient sparsity value is determined based on the second function GS2, and when K = [N2, N], the gradient sparsity value is determined based on the third function GS3. Based on this, for the kth iteration process, first determine which value interval k is located in among [1, N1), [N1, N2) and [N2, N]. If it is determined that k is located in [N1, N2), then the target value of the gradient sparsity corresponding to the kth iteration can be determined through the second function GS2, that is, GS2(k), so the gradient sparsity of the current iteration state (i.e. the kth iteration) can be determined as GS2(k). If k is located in other value intervals, the determination method of the corresponding gradient sparsity is similar, which will not be described here.

[0306] In some optional implementations, the gradient sparsity distribution information can include at least one of the following: the gradient sparsity increases with the increase of the number of iterations, the gradient sparsity decreases first and then gradually increases with the increase of the number of iterations, and the gradient sparsity oscillates and increases with the increase of the number of iterations.

[0307] Wherein, the gradient sparsity increases with the increase of the number of iterations, which means that as the iteration proceeds, the value of the gradient sparsity also increases accordingly, so that the training process can converge faster. The gradient sparsity decreases first and then gradually increases with the increase of the number of iterations, which means that in the initial training stage, the gradient sparsity decreases from a larger value to obtain a faster training speed, and when the gradient sparsity decreases to a certain target value or reaches a certain number of iterations, the value of the gradient sparsity gradually increases to obtain higher convergence accuracy. The gradient sparsity oscillates and increases with the increase of the number of iterations, which means that the gradient sparsity has an overall increasing trend to speed up the convergence, but the increase of the gradient sparsity is realized in the form of oscillation.

[0308] In some optional implementations, the gradient sparsity distribution information can be determined based on a cosine annealing strategy and / or a training warm-up strategy.

[0309] In the cosine annealing strategy, the annealing is derived from the heating and cooling process of metals in the metallurgical field, which means reaching a stable state by continuously reducing the system temperature. In the field of machine learning, the cosine annealing can change the change rule (or value rule) of the gradient sparsity, so as to explore a wider super parameter space and find the corresponding target solution. For example, the cosine annealing strategy can be implemented based on a cosine function. In the cosine function, the cosine value first slowly decreases, then accelerates, and then slowly decreases again as x (i.e. angle) increases. Combining this decreasing mode with the gradient sparsity, the corresponding gradient sparsity distribution information can be obtained. In other words, in the gradient sparsity distribution information determined based on the cosine annealing strategy, the gradient sparsity value first slowly decreases, then accelerates, and finally slowly decreases as the iteration number increases.

[0310] The warm up strategy means that the gradient sparsity is set to a very small value at the beginning of the training of the neural network, and then gradually increases to the pre-set gradient sparsity value, and then the gradient sparsity value is slowly reduced. The process of changing the gradient sparsity from a very small value to the pre-set gradient sparsity value is called warm up. It should be noted that the initialization of the network parameters such as weights is random. Therefore, if a larger gradient sparsity is used at the beginning of the training, it may cause instability of the neural network. Based on this, the gradient sparsity value is preheated in the initial training stage, which can alleviate the problem of instability caused by too large gradient sparsity value in the initial stage, thereby improving the stability of the training.

[0311] In some optional implementations, the above-mentioned multiple gradient sparsity distributions can also be combined or combined to obtain new gradient sparsity distribution information. For example, the cosine annealing strategy and the warm up strategy can be combined to obtain a new strategy, and the gradient sparsity distribution information can be determined based on the new strategy. For another example, the cosine annealing strategy can be periodically restarted to realize periodic gradient sparsity change.

[0312] The gradient sparsity distribution information of the embodiments of the present disclosure will be described below with reference to FIGS. 7-11.

[0313] FIG. 7 is a schematic diagram of a gradient sparsity distribution information provided by an embodiment of the present disclosure. Referring to FIG. 7, a gradient sparsity distribution determined based on a cosine annealing strategy is shown. As shown in FIG. 7, in the gradient sparsity distribution, the gradient sparsity value (i.e. gradient sparsity value) first slowly decreases, then accelerates, and finally slowly decreases as the iteration number increases.

[0314] FIG. 8 is a schematic diagram of gradient sparsity distribution information provided by an embodiment of the present disclosure. Referring to FIG. 8, a gradient sparsity distribution determined based on a training warm-up strategy is shown. As shown in FIG. 8, in the gradient sparsity distribution, at an initial stage of iteration, the gradient sparsity value (i.e., gradient sparsity value) is set to a very small value, and as the number of iterations increases, the gradient sparsity value increases to a preset gradient sparsity value (corresponding to the k1th iteration), and then as the number of iterations increases, the gradient sparsity value decreases slowly.

[0315] FIG. 9 is a schematic diagram of gradient sparsity distribution information provided by an embodiment of the present disclosure. Referring to FIG. 9, a gradient sparsity distribution determined based on a training warm-up strategy and a cosine annealing strategy is shown. As shown in FIG. 9, in the gradient sparsity distribution, at an initial stage of iteration, the gradient sparsity value is set to a very small value, and as the number of iterations increases, the gradient sparsity value (i.e., gradient sparsity value) increases to a preset gradient sparsity value (corresponding to the k2th iteration), and after the k2th iteration, the gradient sparsity value changes in accordance with the value change of the cosine function, i.e., as the number of iterations increases, the gradient sparsity value first decreases slowly, then accelerates, and finally decreases slowly.

[0316] FIG. 10 is a schematic diagram of gradient sparsity distribution information provided by an embodiment of the present disclosure. Referring to FIG. 10, a gradient sparsity distribution in which the gradient sparsity value increases in an oscillating manner as the number of iterations increases is shown. As shown in FIG. 10, as the number of iterations increases, the gradient sparsity value as a whole shows an increasing trend, and is increased in an oscillating form. The gradient sparsity distribution shown in FIG. 10 can be obtained by multiplying or adding a cosine function and a preset exponential function, and the present disclosure does not limit this.

[0317] FIG. 11 is a schematic diagram of gradient sparsity distribution information provided by an embodiment of the present disclosure. Referring to FIG. 11, a gradient sparsity distribution determined based on a training warm-up strategy and a periodic restart method is shown, which can also be referred to as a gradient sparsity distribution with restart. As shown in FIG. 11, in the gradient sparsity distribution, one iteration process is divided into multiple periods, and in each period, the gradient sparsity value conforms to the value rule of the cosine function, i.e., first decreases slowly, then accelerates, and finally decreases slowly. In other words, in each period, the gradient sparsity value decreases from a higher value to a lower value, and then returns to the higher value to start the next period. The size of the period can be determined according to the size of the training set, the complexity of the neural network, etc., and the present disclosure does not limit this.

[0318] It should be noted that the optimization problem of the objective function of the neural network can present a multi-peak form, that is, in addition to the global optimal solution, there can be multiple local optimal solutions. In the gradient update process, it can fall into a local minimum value. Through the gradient sparsity distribution with restart and the gradient sparsity distribution in an oscillating increasing manner, the sparsity of the gradient can be suddenly increased, so as to jump out of the local minimum value and find a path leading to the global minimum value.

[0319] It should be noted that the above gradient sparsity distribution information is only an example. When training the neural network, the corresponding gradient sparsity distribution information can be set according to the training requirements to meet the individualized requirements of convergence speed and convergence accuracy in different training stages, so as to obtain a neural network meeting the expected training purpose.

[0320] In some optional implementations, after determining the gradient sparsity of the current iteration state, the initial gradient information of the current iteration state can be subjected to corresponding sparsification processing. If the gradient sparsity is higher, the sparsification degree is higher, and the target gradient information obtained is information with higher sparsity; otherwise, if the gradient sparsity is lower, the sparsification degree is lower, and the target gradient information obtained is information with lower sparsity.

[0321] In some optional implementations, the initial gradient information is subjected to sparsification processing according to the gradient sparsity of the current iteration state to obtain the target gradient information of the current iteration state, including: determining candidate gradient information from the initial gradient information according to the gradient sparsity of the current iteration state; setting the value of the candidate gradient information in the initial gradient information to a preset sparse value to obtain the target gradient information of the current iteration state. The preset sparse value can be zero or other values, and the embodiments of the present disclosure do not limit this. When the preset sparse value is zero, the network parameter corresponding to the gradient value does not participate in gradient update, and does not need to be subjected to corresponding calculation, so that resource overhead can be saved.

[0322] That is, a part of the initial gradient information can be selected as candidate gradient information to be sparsified according to the gradient sparsity, and subsequent sparsification processing of the candidate gradient information can be performed to obtain the corresponding target gradient information.

[0323] In some optional implementations, the candidate gradient information can be selected from the initial gradient information in a random manner, or can be selected according to the specific value of the initial gradient information, or can be selected according to the importance of the gradient, and the embodiments of the present disclosure do not limit this.

[0324] In some optional implementation, the candidate gradient information is determined from the initial gradient information according to the gradient sparsity of the current iteration state, including: determining the second parameter quantity of the candidate gradient information according to the gradient sparsity of the current iteration state and the first parameter quantity of the initial gradient information; and selecting randomly from the initial gradient information according to the second parameter quantity to obtain the candidate gradient information.

[0325] For example, the gradient sparsity of the current iteration state is β, 0 < β < 1, and the first parameter quantity of the initial gradient information is N, N > 1, so the second parameter quantity of the candidate gradient information is β × N, that is, the number of gradient values that are sparsified is β × N. Based on this, β × N gradient values are randomly selected from the N gradient values of the initial gradient information, and the candidate gradient information is obtained. If β × N is not an integer, it can be rounded and the rounding result is taken as the second parameter quantity.

[0326] In some optional implementation, the candidate gradient information is determined from the initial gradient information according to the gradient sparsity of the current iteration state, including: determining the second parameter quantity of the candidate gradient information according to the gradient sparsity of the current iteration state and the first parameter quantity of the initial gradient information; and selecting randomly from the initial gradient information according to the second parameter quantity to obtain the candidate gradient information.

[0327] In this way, the gradient values with larger values in the initial gradient information can be selected as much as possible, so that the problem of accuracy decline caused by gradient sparsification can be alleviated to some extent.

[0328] For example, the gradient sparsity of the current iteration state is β, 0 < β < 1, and the first parameter quantity of the initial gradient information is N, N > 1, so the second parameter quantity of the candidate gradient information is β × N, that is, the number of gradient values that are sparsified is β × N. Based on this, β × N gradient values are randomly selected from the N gradient values of the initial gradient information, and the candidate gradient information is obtained. If β × N is not an integer, it can be rounded and the rounding result is taken as the second parameter quantity.

[0329] In some optional implementations, the candidate gradient information is determined from the initial gradient information according to the gradient sparsity of the current iteration state, including: determining the second parameter quantity of the candidate gradient information according to the gradient sparsity of the current iteration state and the first parameter quantity of the initial gradient information; and selecting the second parameter quantity of gradient values from the initial gradient information as the candidate gradient information according to the importance of each gradient value in the initial gradient information, where the importance of any gradient value in the candidate gradient information is greater than or equal to the importance of the gradient value in the initial gradient information that is not selected. The importance of the gradient value is related to the function of the network layer corresponding to the gradient, and the embodiments of the present disclosure do not limit how to determine the importance of the gradient value.

[0330] For example, the gradient sparsity of the current iteration state is β, 0 < β < 1, the first parameter quantity of the initial gradient information is N, N > 1, and thus the second parameter quantity of the candidate gradient information is β × N, that is, the number of gradient values that are sparsified is β × N. Based on this, the N gradient values of the initial gradient information are arranged in descending order of importance to obtain a second gradient value sequence, and the first β × N gradient values in the second gradient value sequence are selected as the candidate gradient information. If β × N is not an integer, it can be rounded and the rounding result is taken as the second parameter quantity.

[0331] After the sparsification of the initial gradient information is completed to obtain the target gradient information of the current iteration state, the network parameter information of the current iteration state can be adjusted based on the target gradient information of the current iteration state to obtain the network parameter information of the next iteration state.

[0332] In some optional implementations, the network parameter information of the next iteration state is obtained by adjusting the network parameter information of the current iteration state based on the target gradient information of the current iteration state, including: determining the parameter adjustment value of the current iteration state according to the target gradient information of the current iteration state and the learning rate of the current iteration state; and subtracting the corresponding parameter adjustment value from the network parameter value of the current iteration state to obtain the network parameter value of the next iteration state.

[0333] For example, for the kth iteration, if the network parameter of the neural network in the current iteration state is θ k , the learning rate is η k , and the target gradient information is g k , the parameter adjustment value Δθ k of the current iteration state can be determined as η k g k , and the network parameter θ k+1 of the next iteration state is determined as (θ k - η k g k ).

[0334] In some optional implementation, the data processing method according to the embodiments of the present disclosure can be applied to a many-core system including a plurality of processing cores, and the neural network includes a plurality of network layers. Accordingly, in steps S11 and S14, the network parameter information and the initial gradient information of the neural network in the current iteration state are respectively obtained, including: obtaining, by a target processing core, the network parameter information and the corresponding initial gradient information of the network layer of the neural network in the current iteration state, the target processing core being a processing core corresponding to the network layer in the many-core system. In other words, the network parameter information and the corresponding initial gradient information of the network layer deployed in the target processing core in the current iteration state can be obtained by the target processing core.

[0335] For example, in the case that the data processing method according to the embodiments of the present disclosure is applied to a many-core system, the network layers of the neural network can be respectively deployed to each processing core of the many-core system. Each network layer can correspond to one or more processing cores, and one or more network layers can also be deployed in one processing core. For any network layer, the processing core corresponding to the network layer in the many-core system can be referred to as a target processing core, and the network parameter information of the network layer is stored in the target processing core or the storage space corresponding to the target processing core.

[0336] At the beginning of each iteration of the neural network training, for any network layer of the neural network, the network parameter information of the network layer in the current iteration state is stored in the target processing core of the network layer. On this basis, the target processing core can randomly generate a random perturbation variable satisfying a preset distribution rule and corresponding to the network parameter of the network layer deployed in the target processing core, and add the random perturbation variable to the corresponding network parameter to obtain the perturbed network parameter of the network layer. Wherein, according to the perturbed network parameters of the plurality of network layers of the neural network, the perturbed neural network can be obtained. Since the above process can be implemented in parallel by the target processing cores of each network layer, the processing efficiency can be improved.

[0337] Further, after perturbing each network layer of the neural network, a batch of sample data can be taken, and the perturbed neural network can perform feedforward processing based on the sample data, and determine the initial gradient information of the network layer deployed in the target processing core in the current iteration state according to the feedforward processing result and the random perturbation variable. Wherein, the sample data can be input into the many-core system in a pipelined manner, and each processing core can determine the data flow direction according to the connection relationship of the network layers of the neural network, and then perform the processing of the sample data in a pipelined and orderly manner, with high processing efficiency.

[0338] Further, in step S127, the gradient sparsity of the current iteration state is determined according to the iteration information of the neural network and the preset gradient sparsity distribution information, including: the target processing core determines the gradient sparsity of the current iteration state of the network layer according to the iteration information of the neural network and the gradient sparsity distribution information, or the target processing core obtains the gradient sparsity of the current iteration state of the network layer from the preset processing core of the many-core system.

[0339] For example, the target processing core can determine the gradient sparsity of the corresponding network layer in the current iteration state according to the iteration information of the neural network and the gradient sparsity distribution information. In addition, the process of determining the gradient sparsity of the network layer in the current iteration state can also be implemented by a specific processing core (for example, a preset processing core) in the many-core system, and the target processing core only needs to obtain the gradient sparsity of the current iteration state from the specific processing core. The preset processing core is one or more processing cores designated in advance.

[0340] In some optional implementation manners, in step S128, the initial gradient information is sparsified according to the gradient sparsity of the current iteration state to obtain the target gradient information of the current iteration state, including: the target processing core sparsifies the initial gradient information of the network layer according to the gradient sparsity of the network layer in the current iteration state to obtain the target gradient information of the network layer in the current iteration state.

[0341] In some optional implementation manners, in step S133, the target processing core can adjust the network parameter information of the network layer in the current iteration state based on the target gradient information of the corresponding network layer in the current iteration state to obtain the network parameter information of the network layer in the next iteration state. Based on the plurality of processing cores in the many-core system, the network parameter information of the next iteration state of the neural network can be obtained, and the neural network in the next iteration state is determined. The above iteration process is repeated, and when the training condition is met, the trained neural network can be obtained, and the corresponding target task can be executed based on the trained neural network.

[0342] In summary, by using the processing method based on the many-core system, on the one hand, the intermediate state of the network at each time step does not need to be saved, no additional control overhead is needed, and no high-precision gradient signal needs to be returned, which can significantly reduce the memory and computing overheads in the training process, and realize efficient deployment on the decentralized many-core system; on the other hand, the processing efficiency can be further improved in a parallel execution manner in the many-core system; in addition, the data processing method based on the embodiments of the present disclosure can set the corresponding gradient sparsity for different iteration processes, so as to meet the precise requirements of convergence speed and convergence accuracy at different training stages, and the gradient sparsification can reduce the data processing amount to a certain extent, thereby saving the resource overhead of the many-core system and improving the training efficiency. Moreover, the neural network can be normally trained under the condition that the numerical precision supported by the many-core system is low (int8 or FP16), and the stability of the training is maintained.

[0343] In some optional implementations, the data processing method of the embodiments of the present disclosure can also be applied to a training process based on Back Propagation Through Time (BPTT) for a neural network; wherein the neural network comprises at least one of a recurrent neural network and a spiking neural network. The Back Propagation Through Time is an extension method of the back propagation algorithm, which can propagate the error from the output layer of the network to the hidden layer and the input layer of the neural network, so as to update the weights of the neural network. Moreover, since the BPTT method can process sequence data of any length and can better handle long-term dependencies, the error is propagated from the output layer to any time step, and thus the BPTT method is suitable for tasks such as speech recognition and natural language processing.

[0344] That is, the method of the embodiments of the present disclosure can be used to train the recurrent neural network, the spiking neural network and the like by using the Back Propagation Through Time, so as to dynamically sparsify the gradient according to the training requirements, and adjust the number of network parameters participating in the update through the dynamic sparsification of the gradient, so as to balance the convergence efficiency and convergence accuracy of the training and improve the training efficiency. Moreover, these neural networks can be deployed on a decentralized many-core system (for example, a many-core chip) for on-chip training.

[0345] It can be understood that the above-mentioned various method embodiments of the present disclosure can be combined with each other to form combined embodiments without violating the principle logic. Limited by the length of the present disclosure, the present disclosure will not be described again. It can be understood by those skilled in the art that the specific execution order of each step in the above-mentioned method should be determined according to its function and possible internal logic.

[0346] In addition, the present disclosure also provides a data processing apparatus, an electronic device, a computer readable storage medium, and a computer program product, which can be used to implement any of the data processing methods provided by the present disclosure. The corresponding technical solutions and descriptions are described in the method section and are not repeated here.

[0347] FIG. 12 is a block diagram of a data processing apparatus according to an embodiment of the present disclosure.

[0348] Referring to FIG. 12, the data processing apparatus according to an embodiment of the present disclosure includes modules 31-33, wherein:

[0349] The parameter information obtaining module 31 is configured to obtain network parameter information of the neural network in a current iteration state.

[0350] The adjustment information determining module 32 is configured to determine network adjustment information of the current iteration state according to the parameter adjustment mode of the neural network and the network parameter information of the current iteration state. In the case of parameter perturbation, the network adjustment information includes a perturbation processing result. In the case of gradient sparsification, the network adjustment information includes target gradient information.

[0351] The network adjustment module 33 is configured to adjust the network parameter information of the current iteration state according to the adjustment information of the current iteration state to obtain network parameter information of a next iteration state, so as to train the neural network.

[0352] FIG. 13 is a block diagram of a data processing apparatus according to an embodiment of the present disclosure. Referring to FIG. 13, the data processing apparatus according to an embodiment of the present disclosure includes an adjustment information determining module 32 including modules 321-323, wherein:

[0353] The perturbation module 321 is configured to perform perturbation processing on first weight information of a network layer in a current iteration state for any network layer of a plurality of network layers of the neural network to obtain second weight information after perturbation.

[0354] The network determining module 322 is configured to determine a perturbed neural network according to the second weight information of the plurality of network layers.

[0355] The sample processing module 323 is configured to process sample data in a training set through the perturbed neural network to obtain a perturbation processing result.

[0356] The network adjustment module 33 of the device comprises a module 331, wherein: the weight adjustment module 331 is configured to adjust the first weight information of the plurality of network layers according to the perturbation processing result, the label result of the sample data, and the input random variable and the output random variable of each of the plurality of network layers in the current iteration state, so as to train the neural network, wherein the input random variable and the output random variable are respectively associated with the dimension of the input data and the dimension of the output data of the corresponding network layer.

[0357] In some possible implementation manners, the weight adjustment module 331 comprises:

[0358] The update value determination sub-module is configured to determine the global weight update value of the neural network according to the perturbation processing result, the label result of the sample data, and a preset loss function;

[0359] The adjustment information generation sub-module is configured to, for any network layer in the plurality of network layers, generate the weight adjustment information of the network layer according to the input random variable and the output random variable of the network layer in the current iteration state and the global weight update value;

[0360] The weight adjustment sub-module is configured to adjust the first weight information of the network layer in the current iteration state according to the weight adjustment information of the network layer, to obtain the first weight information of the network layer in the next iteration state;

[0361] The network determination sub-module is configured to determine the neural network in the next iteration state according to the first weight information of the plurality of network layers in the next iteration state.

[0362] In some possible implementation manners, the perturbation module 321 is configured to:

[0363] add a perturbation information amount to the value of the first weight information of the network layer in the current iteration state to obtain forward perturbation weight information, and subtract the perturbation information amount from the value of the first weight information of the network layer in the current iteration state to obtain backward perturbation weight information, the second weight information comprises the forward perturbation weight information and the backward perturbation weight information, and the perturbation information amount is associated with the input random variable and the output random variable of the network layer;

[0364] The perturbed neural network is determined according to the second weight information of the plurality of network layers, comprising:

[0365] The forward perturbed neural network is determined according to the forward perturbation weight information of the plurality of network layers, and the backward perturbed neural network is determined according to the backward weight information of the plurality of network layers, and the perturbed neural network comprises the forward perturbed neural network and the backward perturbed neural network;

[0366] The sample data in the training set is processed through the perturbed neural network to obtain the perturbation processing result, comprising:

[0367] The sample data is processed by the neural network after the positive perturbation to obtain a positive perturbation processing result, and the sample data is processed by the neural network after the negative perturbation to obtain a negative perturbation processing result. The perturbation processing result includes the positive perturbation processing result and the negative perturbation processing result.

[0368] In some possible implementation manners, the updating value determination sub-module is configured to:

[0369] The global weight updating value is generated according to a first value of the loss function under the labeled result of the sample data and the positive perturbation processing result, and a second value of the loss function under the labeled result of the sample data and the negative perturbation processing result.

[0370] In some possible implementation manners, the perturbation information quantity includes a product value of the input random variable, the transpose of the output random variable, and a perturbation coefficient. The perturbation coefficient is used to represent a degree of random perturbation on the weight.

[0371] In some possible implementation manners, when a difference between the first value and the second value is greater than or equal to 0, the global weight updating value is 1; and when the difference between the first value and the second value is less than 0, the global weight updating value is -1.

[0372] In some possible implementation manners, the adjustment information generation sub-module is configured to:

[0373] The weight adjustment information is generated according to a product of the input random variable, the transpose of the output random variable, and the global weight updating value.

[0374] In some possible implementation manners, for any network layer in the plurality of network layers, the input random variable and the output random variable of the network layer in the current iteration state are Bernoulli random variables, a dimension of the input random variable is the same as a dimension of the input data of the network layer in the current iteration state, and a dimension of the output random variable is the same as a dimension of the output data of the network layer in the current iteration state.

[0375] In some possible implementation manners, the device includes a plurality of iteration processes, and each element of the Bernoulli random variable is valued as 1 or -1.

[0376] In each iteration, a probability that each element of the Bernoulli random variable is valued as 1 is 0.5; or

[0377] In each iteration, a probability that each element of the Bernoulli random variable is valued as 1 is determined according to a previous iteration number, a value of the probability in the previous iteration, a value of the Bernoulli random variable in the previous iteration, and a value of the Bernoulli random variable in the current iteration.

[0378] In some possible implementation manners, the neural network comprises a recurrent neural network (RNN) and / or a spiking neural network (SNN).

[0379] In some possible implementation manners, the device is applied to a many-core system, and the many-core system comprises a plurality of processing cores.

[0380] The adjustment information generation submodule is configured to: receive, by the target processing core, the global weight update value, and process the input random variable, the output random variable, and the received global weight update value to obtain the weight adjustment information of the network layer, the target processing core being a processing core corresponding to the network layer in the many-core system.

[0381] The weight adjustment submodule is configured to: process, by the target processing core, the weight adjustment information and the first weight information currently stored by the target processing core to obtain processed first weight information, and update the first weight information currently stored by the target processing core.

[0382] FIG. 14 is a block diagram of a data processing device provided in an embodiment of the present disclosure. Referring to FIG. 14, the data processing device provided in the embodiment of the present disclosure comprises an adjustment information determination module 32 of the device, which comprises modules 324-326.

[0383] The first perturbation module 324 is configured to perform perturbation processing on the network parameter information of the neural network in the current iteration state to obtain a perturbed neural network.

[0384] The processing module 325 is configured to process, by the perturbed neural network, the sample data in the training set to obtain a target evaluation value of the perturbed neural network in the current iteration state.

[0385] The first determination module 326 is configured to determine the perturbation processing result according to the target evaluation value in the current iteration state and a target reference value of the neural network in the current iteration state.

[0386] The network adjustment module 33 of the device comprises a module 332, wherein the first adjustment module 332 is configured to adjust the network parameter information in the current iteration state according to the perturbation processing result and a random perturbation variable of the neural network in the current iteration state to obtain network parameter information in a next iteration state, so as to train the neural network.

[0387] In the embodiments provided by the present disclosure, the network parameter information of the neural network in the current iteration state can be disturbed by the disturbance module, and a disturbed neural network is obtained. The feedforward inference of the sample data is performed by the processing module using the disturbed neural network, so that the target evaluation value of the disturbed neural network in the current iteration state is obtained. The disturbance processing result of the neural network can be determined by the determination module through the target evaluation value of the current iteration state and the target reference value of the neural network in the current iteration state. Then, the network parameter information of the current iteration state is adjusted by the adjustment module combined with the random disturbance variable of the current iteration state, and the network parameter information of the next iteration state is obtained. Through the above process, the training of the neural network can be realized. In each iteration process, only one feedforward processing is required, and the number of feedforward times is relatively small, so that the training efficiency can be improved, and the memory overhead and the calculation overhead in the network training can be reduced.

[0388] FIG. 15 is a block diagram of a data processing apparatus provided by an embodiment of the present disclosure. Referring to FIG. 15, the data processing apparatus provided by the embodiment of the present disclosure further includes:

[0389] The gradient obtaining module 34 is configured to obtain initial gradient information of the neural network in the current iteration state.

[0390] The adjustment information determining module 32 of the apparatus includes modules 327-328, wherein:

[0391] The second determining module 327 is configured to determine the gradient sparsity of the current iteration state according to the iteration information of the neural network and the preset gradient sparse distribution information.

[0392] The sparse module 328 is configured to perform sparse processing on the initial gradient information according to the gradient sparsity of the current iteration state, and obtain target gradient information of the current iteration state.

[0393] The network adjustment module 33 of the apparatus includes a module 333, wherein the second adjustment module 333 is configured to adjust the network parameter information of the current iteration state based on the target gradient information of the current iteration state, and obtain the network parameter information of the next iteration state, so as to train the neural network.

[0394] In the embodiments provided by the present disclosure, the network parameter information and the initial gradient information of the neural network in a current iteration state are acquired by an acquisition module; the gradient sparsity of the current iteration state is determined according to the iteration information of the neural network and preset gradient sparse distribution information by a determination module; the initial gradient information is sparsified according to the gradient sparsity of the current iteration state by a sparsification module to obtain target gradient information of the current iteration state; and the network parameter information of the current iteration state is adjusted based on the target gradient information of the current iteration state by an adjustment module to obtain network parameter information of a next iteration state, so as to train the neural network. Therefore, in the embodiments of the present disclosure, when training the neural network, for different iteration processes, part of the gradient information can be selectively selected for sparsification according to the training requirements, so that dynamic sparsification of the gradient can be realized, and the number of network parameters participating in the update is adjusted through the dynamic sparsification of the gradient, so that the convergence efficiency and the convergence accuracy of the training are taken into account, and the training efficiency is improved.

[0395] In another aspect, the present disclosure provides a many-core system.

[0396] FIG. 16 is a block diagram of a many-core system according to an embodiment of the present disclosure. Referring to FIG. 16, the many-core system 800 includes a plurality of processing cores, wherein at least one network layer of a neural network is deployed in the processing cores, and the many-core system can be used to execute the data processing method according to any one of the embodiments of the present disclosure. For details, refer to related content of the embodiments of the present disclosure, which will not be described here.

[0397] In addition, the present disclosure also provides an electronic device, a computer readable storage medium and a computer program product.

[0398] FIG. 17 is a block diagram of an electronic device according to an embodiment of the present disclosure.

[0399] Referring to FIG. 17, the electronic device according to an embodiment of the present disclosure includes at least one processor 901, at least one memory 902, and one or more I / O interfaces 903 connected between the processor 901 and the memory 902. The memory 902 stores one or more computer programs executable by the at least one processor 901. The one or more computer programs are executed by the at least one processor 901 to enable the at least one processor 901 to execute the data processing method according to any one of the embodiments of the present disclosure.

[0400] FIG. 18 is a block diagram of an electronic device according to an embodiment of the present disclosure.

[0401] Referring to FIG. 18, the electronic device according to an embodiment of the disclosure includes a plurality of processing cores 1001 and a network-on-chip 1002, wherein the plurality of processing cores 1001 are connected to the network-on-chip 1002, and the network-on-chip 1002 is configured to interact data between the plurality of processing cores 1001 and external data.

[0402] The one or more processing cores 1001 store one or more instructions, and the one or more instructions are executed by the one or more processing cores 1001 to enable the one or more processing cores 1001 to perform the data processing method according to any one of the embodiments of the disclosure.

[0403] In some embodiments, the electronic device can be a brain-like chip. Since the brain-like chip can use vectorized computing, the brain-like chip needs to load parameters such as weight information of a neural network model from an external memory such as a double data rate (DDR) synchronous dynamic random access memory. Therefore, the batch processing has higher operation efficiency.

[0404] The embodiment of the disclosure further provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor / processing core, implements the data processing method described above. The computer-readable storage medium can be a volatile or non-volatile computer-readable storage medium.

[0405] The embodiment of the disclosure further provides a computer program product including computer-readable code or a non-volatile computer-readable storage medium carrying computer-readable code, and when the computer-readable code is executed in a processor of an electronic device, the processor in the electronic device performs the data processing method described above.

[0406] Those skilled in the art can understand that all or some of the steps in the above disclosed method, the functions of the modules / units in the system and the device can be implemented as software, firmware, hardware and appropriate combinations thereof. In the hardware implementation, the division between the functional modules / units mentioned in the above description does not necessarily correspond to the division of physical components; for example, one physical component can have multiple functions, or one function or step can be performed by several physical components in cooperation. Some or all of the physical components can be implemented as software executed by a processor such as a central processing unit, a digital signal processor or a microprocessor, or as hardware, or as an integrated circuit such as an application specific integrated circuit. Such software can be distributed on a computer-readable storage medium, which can include computer storage media (or non-transitory media) and communication media (or transitory media).

[0407] As those skilled in the art will appreciate, the term computer storage media includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable program instructions, data structures, program modules or other data. Computer storage media include, but are not limited to, random access memory (RAM), read only memory (ROM), erasable programmable read only memory (EPROM), static random access memory (SRAM), flash memory or other memory technology, compact disc read only memory (CD-ROM), digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store the desired information and which can be accessed by a computer. Further, as those skilled in the art will appreciate, communication media typically embodies computer readable program instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transport mechanism and includes any information delivery media. As used herein, the term "exemplary" means serving as an example, instance, or illustration. Any implementation described herein as exemplary is not necessarily to be construed as preferred or advantageous over other implementations.

[0408] Computer readable program instructions described herein can be downloaded to respective computing / processing devices from a computer readable storage medium or to an external computer or external storage device via a network, for example, the Internet, a local area network, a wide area network and / or a wireless network. The network can comprise copper transmission cables, optical transmission fibers, wireless transmission, routers, firewalls, switches, gateway computers and / or edge servers. A network adapter card or network interface in each computing / processing device receives computer readable program instructions from the network and forwards the computer readable program instructions for storage in a computer readable storage medium within the respective computing / processing device.

[0409] Computer readable program instructions for carrying out operations of the present disclosure can be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine dependent instructions, microcode, firmware instructions, state setting data, or any combination of one or more of the above in any combination, written in any combination of one or more programming languages, including object oriented programming languages such as Smalltalk, C++ or the like, and conventional procedural programming languages such as "C" or the like. The computer readable program instructions can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider). In some embodiments, electronic circuitry including, for example, programmable logic circuitry, field-programmable gate arrays (FPGA), or programmable logic arrays (PLA) can execute the computer readable program instructions by utilizing state information of the computer readable program instructions to personalize the electronic circuitry, in order to perform aspects of the present disclosure.

[0410] The computer program product described herein can be embodied in a specific manner by hardware, software, or a combination thereof. In an optional embodiment, the computer program product is embodied as a computer storage medium. In another optional embodiment, the computer program product is embodied as a software product, such as a software development kit (SDK), and the like.

[0411] The various aspects of the present disclosure are described herein with reference to flowcharts and / or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the present disclosure. It should be understood that each block of the flowcharts and / or block diagrams, and combinations of blocks in the flowcharts and / or block diagrams, can be implemented by computer readable program instructions.

[0412] These computer readable program instructions can be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks. These computer readable program instructions can also be stored in a computer readable storage medium that can include a non-transitory computer readable storage medium that can be a computer- readable storage medium having no data storage cycles that change state. The instructions can be executed by one or more processors of a computer, other programmable data processing apparatus, or other devices to produce a computer-implemented process such that the instructions which execute via the one or more processors of the computer or other programmable data processing devices create means for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks. The computer program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other devices to cause a series of operational steps to be performed on the computer, other programmable apparatus or other devices to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0413] The computer readable program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other device to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0414] The flow and block diagrams in the drawings show architectural, functional, and operational representations of possible implementations of systems, methods, and computer program products according to the present disclosure. In this regard, each block in the flow and block diagrams can represent a module, a segment, or a portion of instructions, which comprises one or more executable instructions for implementing the specified logical function(s). In some alternative implementations, the functions noted in the blocks can occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may

[0415] Example embodiments have been disclosed and, although a specific terminology is employed, it is merely for the convenience of the reader and is not intended to be limiting. In some instances, specific details have been included for the purpose of providing a thorough understanding of the embodiments presented, but it will be apparent to those skilled in the art that embodiments of the application can be practiced without these specific details. In some instances, features, characteristics and / or elements described in connection with a particular embodiment can be used in conjunction with, or switched to, features, characteristics and / or elements described in connection with other embodiments, unless otherwise explicitly stated. Accordingly, it will be understood that various changes in form and details can be made without departing from the scope of the disclosure as set forth in the appended claims.

Claims

1. A data processing method, wherein, The method comprises: obtaining network parameter information of a neural network in a current iteration state; determining network adjustment information of the current iteration state according to a parameter adjustment mode of the neural network and the network parameter information of the current iteration state; wherein, in the case that the parameter adjustment mode is parameter perturbation, the network adjustment information comprises a perturbation processing result; in the case that the parameter adjustment mode is gradient sparsification, the network adjustment information comprises target gradient information; adjusting the network parameter information of the current iteration state according to the adjustment information of the current iteration state to obtain network parameter information of a next iteration state, so as to train the neural network.

2. The method of claim 1, wherein, The network parameter information comprises weight information, and in the case that the parameter adjustment mode is parameter perturbation, The method comprises: for any network layer in a plurality of network layers of a neural network, performing perturbation processing on first weight information of the network layer in a current iteration state to obtain second weight information after perturbation; determining a neural network after perturbation according to the second weight information of the plurality of network layers; processing sample data in a training set through the neural network after perturbation to obtain a perturbation processing result; The method comprises: adjusting the first weight information of the plurality of network layers respectively according to the perturbation processing result, a label result of the sample data, and input random variables and output random variables of the plurality of network layers in the current iteration state, so as to train the neural network, wherein the input random variables and the output random variables are respectively associated with the dimensions of input data and the dimensions of output data of the corresponding network layers.

3. The method of claim 2, wherein, The method comprises: determining a global weight update value of the neural network according to the perturbation processing result, the label result of the sample data, and a preset loss function; for any network layer in the plurality of network layers, generating weight adjustment information of the network layer according to the input random variables and the output random variables of the network layer in the current iteration state and the global weight update value; adjusting the first weight information of the network layer in the current iteration state according to the weight adjustment information of the network layer to obtain first weight information of the network layer in a next iteration state; determining a neural network in the next iteration state according to the first weight information of the plurality of network layers in the next iteration state.

4. The method of claim 3, wherein, The method comprises: The value of the first weight information of the network layer in the current iteration state is increased by a disturbance information amount to obtain forward disturbance weight information, and the value of the first weight information of the network layer in the current iteration state is reduced by the disturbance information amount to obtain negative disturbance weight information, the second weight information includes the forward disturbance weight information and the negative disturbance weight information, and the disturbance information amount is associated with input random variables and output random variables of the network layer; The method comprises the following steps: According to the forward disturbance weight information of the plurality of network layers, a forward disturbance neural network is determined, and according to the negative weight information of the plurality of network layers, a negative disturbance neural network is determined, and the disturbance neural network includes the forward disturbance neural network and the negative disturbance neural network; The method comprises the following steps: The sample data in the training set is processed through the forward disturbance neural network to obtain a forward disturbance processing result, and the sample data is processed through the negative disturbance neural network to obtain a negative disturbance processing result, and the disturbance processing result includes the forward disturbance processing result and the negative disturbance processing result.

5. The method of claim 4, wherein, The method comprises the following steps: According to the loss function, a first value of the loss function under the annotation result of the sample data and the forward disturbance processing result, and a second value of the loss function under the annotation result of the sample data and the negative disturbance processing result, the global weight update value of the neural network is generated.

6. The method of claim 4, wherein, The disturbance information amount includes the product value of the input random variable, the transpose of the output random variable and a disturbance coefficient, and the disturbance coefficient is used to represent the degree of random disturbance to the weight.

7. The method of claim 5, wherein, In the case where the difference between the first value and the second value is greater than or equal to 0, the global weight update value is 1; In the case where the difference between the first value and the second value is less than 0, the global weight update value is -1.

8. The method of claim 3, wherein, The method comprises the following steps: According to the product of the input random variable, the transpose of the output random variable and the global weight update value, the weight adjustment information of the network layer is generated.

9. The method of any one of claims 3 to 8, wherein, For any network layer in the plurality of network layers, the input random variable and the output random variable of the network layer in the current iteration state are Bernoulli random variables, the dimension of the input random variable is the same as the dimension of the input data of the network layer in the current iteration state, and the dimension of the output random variable is the same as the dimension of the output data of the network layer in the current iteration state.

10. The method of claim 9, wherein, The method comprises a plurality of iteration processes, and each element of the Bernoulli random variable is 1 or -1. The method comprises a plurality of iteration processes, and each element of the Bernoulli random variable is 1 or -1. In each iteration, each element of the Bernoulli random variable has a probability of 0.5 of taking the value 1; or In each iteration, each element of the Bernoulli random variable has a probability of taking the value 1, which is determined according to the number of previous iterations, the values of the probabilities in the previous iterations, the values of the Bernoulli random variable in the previous iterations, and the value of the Bernoulli random variable in the current iteration.

11. The method of any one of claims 3 to 8, wherein, The method is applied to a many-core system, and the many-core system includes a plurality of processing cores; The method further includes: The target processing core receives the global weight update value and processes the input random variable, the output random variable, and the received global weight update value to obtain the weight adjustment information of the network layer, the target processing core being a processing core corresponding to the network layer in the many-core system; The method further includes: The target processing core processes the weight adjustment information and the currently stored first weight information of the target processing core to obtain processed first weight information, and updates the currently stored first weight information of the target processing core.

12. The method of claim 1, wherein, In a case where the parameter adjustment mode is parameter perturbation, The method further includes: The method further includes: The perturbed neural network is obtained by perturbing the network parameter information of the neural network in the current iteration state; The target evaluation value of the perturbed neural network in the current iteration state is obtained by performing inference on the sample data in the training set by using the perturbed neural network; The perturbation processing result is determined according to the target evaluation value of the current iteration state and a target reference value of the neural network in the current iteration state, the target reference value of the current iteration state being obtained by adjusting a target reference value of a previous iteration state; The method further includes:

13. The method of claim 12, wherein, The network parameter information of the next iteration state is obtained by adjusting the network parameter information of the current iteration state according to the perturbation processing result and a random perturbation variable of the neural network in the current iteration state, so as to train the neural network. The method further includes: The parameter adjustment information is obtained by processing the random perturbation variable according to the perturbation processing result; The network parameter information of the next iteration state is determined according to the parameter adjustment information and the network parameter information of the current iteration state.

14. The method of claim 13, wherein, The perturbation processing result includes a positive perturbation processing result and a negative perturbation processing result. The processing of the random perturbation variable according to the perturbation processing result to obtain parameter adjustment information includes: In a case where the perturbation processing result is the positive perturbation processing result, determining first parameter adjustment information according to a product value of the random perturbation variable and a learning rate of the current iteration state; In a case where the perturbation processing result is the negative perturbation processing result, determining a reverse vector of the random perturbation variable, and determining second parameter adjustment information according to a product value of the reverse vector, a preset update coefficient and the learning rate of the current iteration state; The parameter adjustment information includes the first parameter adjustment information and the second parameter adjustment information.

15. The method of claim 14, wherein, The second parameter adjustment information is determined by formula 1: Δθ = -λη k g k ' 1 wherein, Δθ represents the second parameter adjustment information, g k ’ represents the random disturbance variable, -g k ’ represents the inverse vector of the random disturbance variable, η k is a learning rate, λ is an update coefficient used to represent the update degree of the network parameters, and λ is a positive number less than 1.

16. The method of claim 13, wherein, The network parameter information includes a network parameter value, and the parameter adjustment information includes a parameter adjustment value; The determination of the network parameter information of the next iteration state according to the parameter adjustment information and the network parameter information of the current iteration state includes: Obtaining a network parameter value of the next iteration state according to the network parameter value of the current iteration state and the corresponding parameter adjustment value.

17. The method of any one of claims 12-16, wherein, The method further includes: Adjusting a target reference value of the current iteration state according to the perturbation processing result and a target evaluation value of the current iteration state to obtain a target reference value of a next iteration state, so as to adjust the network parameter information of the next iteration state based on the target reference value of the next iteration state.

18. The method of claim 17, wherein, The perturbation processing result includes a positive perturbation processing result and a negative perturbation processing result. The adjustment of the target reference value of the current iteration state according to the perturbation processing result and a target evaluation value of the current iteration state to obtain a target reference value of a next iteration state includes: In a case where the perturbation processing result is the positive perturbation processing result, calculating a weighted value of the target evaluation value of the current iteration state and the target reference value of the current iteration state based on a preset weight adjustment coefficient, taking the weighted value as the target reference value of the next iteration state, or taking the target evaluation value of the current iteration state as the target reference value of the next iteration state; In a case where the perturbation processing result is the negative perturbation processing result, taking the target reference value of the current iteration state as the target reference value of the next iteration state.

19. The method of claim 18, wherein, In the case that the perturbation processing result is the forward perturbation processing result, a target reference value of the next iteration state is determined by Formula 2: L k+1 = γL k + (1 - γ)L cur_k 2 In a case where the perturbation processing result is the negative perturbation processing result, the target reference value of the next iteration state is determined by formula 3: L k+1 = L cur_k 3 wherein, L k+1 is the target reference value of the next iteration state, L k is the target reference value of the current iteration state, L cur_k is the target evaluation value of the current iteration state, and γ is the weight adjustment coefficient.

20. The method of claim 12, wherein, The inference of the sample data in the training set by the perturbed neural network to obtain a target evaluation value of the perturbed neural network in the current iteration state includes: Inputting the sample data into the perturbed neural network to obtain a prediction result of the sample data; Determining a function value of a preset loss function according to the sample data prediction result and a label result of the sample data, and taking the function value as the target evaluation value of the current iteration state.

21. The method of claim 12, wherein, determining a perturbation processing result according to the target evaluation value of the current iteration state and the target reference value of the neural network in the current iteration state, wherein the target evaluation value of the current iteration state is obtained by performing inference on sample data in a training set by using the perturbed neural network in the current iteration state, and the target reference value of the neural network in the current iteration state is obtained by performing inference on the sample data in the training set by using the neural network in the current iteration state; in a case where the target evaluation value of the current iteration state is less than the target reference value of the current iteration state, determining the perturbation processing result as a positive perturbation processing result; in a case where the target evaluation value of the current iteration state is greater than or equal to the target reference value of the current iteration state, determining the perturbation processing result as a negative perturbation processing result; wherein the perturbation processing result comprises the positive perturbation processing result and the negative perturbation processing result.

22. The method of claim 12, wherein, the perturbation processing on the network parameter information of the neural network in the current iteration state to obtain a perturbed neural network, comprising: determining a random perturbation variable corresponding to each network parameter of the neural network in the current iteration state; superimposing the random perturbation variable on the corresponding network parameter to obtain a perturbed network parameter; obtaining the perturbed neural network based on the perturbed network parameter.

23. The method of claim 12, wherein, The method is applied to a many-core system, and the many-core system comprises a plurality of processing cores. wherein the perturbation processing on the network parameter information of the neural network in the current iteration state to obtain a perturbed neural network, comprising: performing perturbation processing on the network parameter information of the network layer of the neural network in the current iteration state by a target processing core to obtain a perturbed network layer, the target processing core being a processing core corresponding to the network layer in the many-core system; wherein the obtaining of the target evaluation value of the perturbed neural network in the current iteration state by performing inference on sample data in a training set once, comprises: performing inference on the sample data based on the perturbed network layer by the target processing core to obtain the target evaluation value of the perturbed neural network in the current iteration state; wherein the determining of the perturbation processing result according to the target evaluation value of the current iteration state and the target reference value of the neural network in the current iteration state, comprises: determining the perturbation processing result according to the target evaluation value of the current iteration state and the target reference value of the neural network in the current iteration state by a preset processing core in the many-core system, and synchronizing the perturbation processing result to each target processing core; wherein the adjusting of the network parameter information of the current iteration state according to the perturbation processing result and the random perturbation variable of the neural network in the current iteration state to obtain network parameter information of a next iteration state, comprises: processing the network parameter information of the network layer in the current iteration state according to the perturbation processing result and the random perturbation variable of the network layer in the current iteration state by the target processing core to obtain network parameter information of the network layer in the next iteration state.

24. The method of claim 1, wherein, in a case where the parameter adjustment mode is gradient sparsity, the method further comprises: obtaining initial gradient information of the neural network in the current iteration state; the determining of the network adjustment information of the current iteration state according to the parameter adjustment mode of the neural network and the network parameter information of the current iteration state, comprises: determine the gradient sparsity of the current iteration state according to the iteration information of the neural network and preset gradient sparsity distribution information; perform sparsification processing on the initial gradient information according to the gradient sparsity of the current iteration state, to obtain target gradient information of the current iteration state; the adjusting the network parameter information of the current iteration state according to the adjustment information of the current iteration state to obtain network parameter information of a next iteration state to train the neural network, comprises: adjusting the network parameter information of the current iteration state based on the target gradient information of the current iteration state to obtain network parameter information of a next iteration state to train the neural network.

25. The method of claim 24, wherein, The iteration information at least includes the current iteration number, and the gradient sparsity distribution information includes the value of the gradient sparsity corresponding to each iteration number; the determining the gradient sparsity of the current iteration state according to the iteration information of the neural network and preset gradient sparsity distribution information, comprises: determining a target value of the gradient sparsity corresponding to the current iteration number in the gradient sparsity distribution information, and taking the target value as the gradient sparsity of the current iteration state.

26. The method of claim 24 or 25, wherein, The gradient sparsity distribution information is used to represent the correlation between the gradient sparsity and the iteration number. The gradient sparsity distribution information comprises at least one of the following: the gradient sparsity increases with the increase of the iteration number, the gradient sparsity decreases first and then gradually increases with the increase of the iteration number, and the gradient sparsity oscillates and increases with the increase of the iteration number.

27. The method of claim 24, wherein, The sparsification processing on the initial gradient information according to the gradient sparsity of the current iteration state to obtain target gradient information of the current iteration state, comprises: determining the candidate gradient information from the initial gradient information according to the gradient sparsity of the current iteration state; setting the value of the candidate gradient information in the initial gradient information to a preset sparse value to obtain the target gradient information of the current iteration state.

28. The method of claim 27, wherein, The determining the candidate gradient information from the initial gradient information according to the gradient sparsity of the current iteration state, comprises: determining the second parameter quantity of the candidate gradient information according to the gradient sparsity of the current iteration state and the first parameter quantity of the initial gradient information; randomly selecting in the initial gradient information according to the second parameter quantity to obtain the candidate gradient information.

29. The method of claim 24, wherein, The adjusting the network parameter information of the current iteration state based on the target gradient information of the current iteration state to obtain network parameter information of a next iteration state, comprises: determining a parameter adjustment value of the current iteration state according to the target gradient information of the current iteration state and a learning rate of the current iteration state; obtaining the network parameter value of the next iteration state according to the network parameter value of the current iteration state and the corresponding parameter adjustment value.

30. The method of claim 24, wherein, The method is applied to a many-core system, and the many-core system includes a plurality of processing cores. The method comprises the following steps: obtaining network parameter information of a neural network in a current iteration state, and obtaining initial gradient information of the neural network in the current iteration state. obtaining, by a target processing core, network parameter information and corresponding initial gradient information of a network layer of the neural network in a current iteration state, the target processing core being a processing core corresponding to the network layer in the many-core system; wherein the determining the gradient sparsity of the current iteration state according to the iteration information of the neural network and the preset gradient sparse distribution information comprises: determining, by the target processing core, the gradient sparsity of the current iteration state of the network layer according to the iteration information of the neural network and the gradient sparse distribution information, or the target processing core obtaining the gradient sparsity of the current iteration state of the network layer from a preset processing core of the many-core system; wherein the sparsifying the initial gradient information according to the gradient sparsity of the current iteration state to obtain target gradient information of the current iteration state comprises: sparsifying, by the target processing core, the initial gradient information of the network layer according to the gradient sparsity of the network layer in the current iteration state to obtain target gradient information of the network layer in the current iteration state; wherein the adjusting the network parameter information of the current iteration state based on the target gradient information of the current iteration state to obtain network parameter information of a next iteration state comprises: adjusting, by the target processing core, the network parameter information of the network layer in the current iteration state based on the target gradient information of the network layer in the current iteration state to obtain network parameter information of the network layer in the next iteration state.

31. The method of any one of claims 1-8, 12, 23, 24, 30, wherein, The method is applied to a training process based on backward propagation along time for the neural network. The neural network comprises at least one of a recurrent neural network and a spiking neural network.

32. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein, The processor executes the computer program to implement the data processing method of any one of claims 1-31.

33. A computer readable storage medium having stored thereon a computer program, wherein, The computer program is executed by the processor to implement the data processing method of any one of claims 1-31.

34. A computer program product comprising computer readable code, or a non-transitory computer readable storage medium having computer readable code embodied thereon, wherein, When the computer readable code runs in the processor of the electronic device, the processor in the electronic device executes the data processing method of any one of claims 1-31.

Images