Neural network training method and device based on depth time alignment gradient enhancement

Abstract

The invention discloses a neural network training method and device based on depth time alignment gradient enhancement. The training method comprises the following steps: acquiring input sample data; inputting the input sample data into a neural network model based on depth time alignment gradient enhancement to obtain prediction category probability data corresponding to the category of the input sample data; according to the prediction category probability data and the sample label data, adjusting each parameter of the neural network model based on depth time alignment gradient enhancement to obtain a trained neural network model, the neural network model based on depth time alignment gradient enhancement comprising an input layer, K stages and an output layer, the first stage to the (K-1) th stage each comprise a convolutional network based on depth time alignment gradient enhancement and an auxiliary classifier network, the Kth stage comprises a convolutional network based on depth time alignment gradient enhancement, and K is a positive integer greater than 1.

Description

Technical Field

[0001] The present disclosure generally relates to the field of artificial intelligence and brain-like learning technology, and more specifically, to a training method and device for a neural network based on deep time-aligned gradient enhancement. Background Art

[0002] In related fields, neural networks have made significant progress in the field of modern artificial intelligence, but they still have the following shortcomings: (1) High data requirements: Neural networks require a large amount of training data to effectively learn and improve performance, which is not conducive to industries or fields with less data; (2) High demand for computing resources: The training and deployment of neural networks usually require high-performance computing resources, which may lead to high operating costs. In particular, for deep neural networks, the training process is time-consuming and computationally complex, resulting in excessively high costs; (3) Black box nature: Neural networks are often regarded as "black boxes" because their decision-making process is difficult to explain. This opacity is particularly problematic in areas that require a high degree of trust and explainability. (4) Long-term dependency problem: When processing long sequences of data, classical neural networks (such as recurrent neural networks) may encounter the problem of gradient vanishing or gradient exploding, making it difficult for the model to capture long-distance dependencies. (5) Susceptibility to overfitting and underfitting: Neural networks are prone to overfitting, that is, they perform well on training data but have poor generalization ability on new data, and if the amount of data is insufficient, it may lead to underfitting. (6) Design complexity: The design and architecture selection of neural networks are very complex. For example, when faced with different application problems and scenarios, how to choose a suitable network structure is still challenging.

[0003] Therefore, in order to solve the problems existing in the related fields, an improved neural network is needed. Summary of the invention

[0004] The embodiments of the present disclosure provide a training method and device for a neural network based on deep time-aligned gradient enhancement and a data recognition method and device. By adopting a convolutional network based on deep time-aligned gradient enhancement and an auxiliary classifier network, gradient enhancement and spatiotemporal deep supervision can be achieved, the problem that the gradient may disappear or deviate during the training process is solved and the gradient flow is enhanced, thereby improving the training efficiency and model performance of the neural network model.

[0005] In one general aspect, a training method for a neural network based on deep time alignment gradient enhancement is provided, the training method comprising: obtaining input sample data, the input sample data comprising sample image data and / or sample text data and corresponding sample label data; inputting the input sample data into a neural network model based on deep time alignment gradient enhancement to obtain predicted category probability data corresponding to the category of the input sample data; adjusting various parameters of the neural network model based on deep time alignment gradient enhancement according to the predicted category probability data and the sample label data to obtain a trained neural network model, wherein the neural network model based on deep time alignment gradient enhancement comprises an input layer, K stages and an output layer, the 1st stage to the K-1th stage each comprise a convolutional network based on deep time alignment gradient enhancement and an auxiliary classifier network, and the Kth stage comprises a convolutional network based on deep time alignment gradient enhancement, wherein K is a positive integer greater than 1.

[0006] Optionally, the following formula can be used to calculate the gradient parameters in the convolutional network based on deep time-aligned gradient enhancement: , in, ,and ,in, t represents the time step, represents the gradient parameter, Indicates t The membrane potential at each time step is Indicates the preset trigger threshold. Indicates t The L2 norm metric at the time step, σ1 represents the L2 norm metric at the first time step, express The normalization factor of Represents preset hyperparameters.

[0007] Optionally, the step of inputting the input sample data into a neural network model based on deep time alignment gradient enhancement to obtain predicted category probability data corresponding to the category of the input sample data may include: inputting the input sample data into the input layer, the first stage to the K-1th stage in a forward propagation manner to obtain K-1 predicted category probability vectors at each time step; inputting the K-1th predicted category probability vector into the Kth stage and the output layer in a forward propagation manner to obtain predicted category probability data corresponding to the category of the input sample data at each time step.

[0008] Optionally, the step of adjusting various parameters of the neural network model based on deep time alignment gradient enhancement according to the predicted category probability data and the sample label data to obtain a trained neural network model may include: obtaining a total time step for model training; and calculating a loss value for adjusting various parameters based on the following formula according to the predicted category probability data, the sample label data and the total time step: , in, = ( ),in, represents the loss value, K represents the total number of stages and k The value range is [1, K -1], T represents the total time step, t Represents the time step and its value range is [1, T ], represents the cross entropy loss function, Indicated in t The next time step k The predicted category probability output of each stage, Indicated in t The predicted category probability data at time steps and the predicted category probability data before global average pooling, represents the sample label data, represents the network function of the auxiliary classifier network, Indicated in t The time step k The characteristic vector of each stage is obtained; and each parameter is adjusted based on the loss value to obtain a trained neural network model.

[0009] Optionally, each of the 1st to K-1th stages may include a deep time-aligned gradient enhancement-based convolutional network and an auxiliary classifier network connected in sequence, and wherein the auxiliary classifier network may include three convolutional layers, a global average pooling layer and a fully connected layer connected in sequence.

[0010] In another general aspect, a data recognition method is provided, the data recognition method comprising: obtaining input data to be recognized, the input data comprising image data and / or text data; inputting the input data to be recognized into a neural network model based on deep time alignment gradient enhancement, and obtaining corresponding predicted category probability data as a data recognition result, wherein the neural network model based on deep time alignment gradient enhancement is trained using the training method described above.

[0011] In another general aspect, a training device for a neural network based on deep time alignment gradient enhancement is provided, the training device comprising: an acquisition module, configured to: acquire input sample data, the input sample data comprising sample image data and / or sample text data and corresponding sample label data; a prediction module, configured to: input the input sample data into a neural network model based on deep time alignment gradient enhancement, and obtain predicted category probability data corresponding to the category of the input sample data; a training module, configured to: adjust various parameters of the neural network model based on deep time alignment gradient enhancement according to the predicted category probability data and the sample label data, and obtain a trained neural network model, wherein the neural network model based on deep time alignment gradient enhancement comprises an input layer, K stages and an output layer, the 1st stage to the K-1th stage each comprise a convolutional network based on deep time alignment gradient enhancement and an auxiliary classifier network, and the Kth stage comprises a convolutional network based on deep time alignment gradient enhancement, wherein K is a positive integer greater than 1.

[0012] Optionally, the following formula can be used to calculate the gradient parameters in the convolutional network based on deep time-aligned gradient enhancement: , in, ,and ,in, t represents the time step, represents the gradient parameter, Indicates t The membrane potential at each time step is Indicates the preset trigger threshold. Indicates t The L2 norm metric at the time step, σ1 represents the L2 norm metric at the first time step, express The normalization factor of Represents preset hyperparameters.

[0013] Optionally, the prediction module inputs the input sample data into a neural network model based on deep time alignment gradient enhancement, and the operation of obtaining predicted category probability data corresponding to the category of the input sample data may include: inputting the input sample data into the input layer, the 1st stage to the K-1th stage in a forward propagation manner to obtain K-1 predicted category probability vectors at each time step; inputting the K-1th predicted category probability vector into the Kth stage and the output layer in a forward propagation manner to obtain predicted category probability data corresponding to the category of the input sample data at each time step.

[0014] Optionally, the training module adjusts various parameters of the neural network model based on deep time alignment gradient enhancement according to the predicted category probability data and the sample label data, and the operation of obtaining the trained neural network model may include: obtaining a total time step for model training; and calculating a loss value for adjusting various parameters based on the following formula according to the predicted category probability data, the sample label data and the total time step: , in, = ( ),in, represents the loss value, K represents the total number of stages and k The value range is [1, K -1], T represents the total time step, t Represents the time step and its value range is [1, T ], represents the cross entropy loss function, Indicated in t The next time step k The predicted category probability output of each stage, Indicated in t The predicted category probability data at time steps and the predicted category probability data before global average pooling, represents the sample label data, represents the network function of the auxiliary classifier network, Indicated in t The time step k The characteristic vector of each stage is obtained; and each parameter is adjusted based on the loss value to obtain a trained neural network model.

[0015] Optionally, each of the 1st to K-1th stages may include a deep time-aligned gradient enhancement-based convolutional network and an auxiliary classifier network connected in sequence, and wherein the auxiliary classifier network may include three convolutional layers, a global average pooling layer and a fully connected layer connected in sequence.

[0016] In another general aspect, a data recognition device is provided, comprising: an acquisition module, configured to: acquire input data to be recognized, wherein the input data comprises image data and / or text data; a recognition module, configured to: input the input data to be recognized into a neural network model based on deep time alignment gradient enhancement, and obtain corresponding predicted category probability data as a data recognition result, wherein the neural network model based on deep time alignment gradient enhancement is trained using the training method described above.

[0017] In another general aspect, a computer program product is provided, comprising a computer program / instruction, which, when executed by a processor, implements the training method of a neural network based on deep time-aligned gradient enhancement as described above and the data recognition method as described above.

[0018] In another general aspect, a computer-readable storage medium is provided, which, when the instructions in the computer-readable storage medium are executed by a processor of an electronic device / server, enables the electronic device / server to perform the training method of a neural network based on deep time-aligned gradient enhancement as described above and the data recognition method as described above.

[0019] In another general aspect, a computing device is provided, comprising: at least one processor; and at least one memory storing computer executable instructions, wherein the computer executable instructions, when executed by the at least one processor, cause the at least one processor to execute the training method of the neural network based on deep time-aligned gradient enhancement as described above and the data recognition method as described above.

[0020] According to the training method and device of the neural network based on deep time alignment gradient enhancement and the data recognition method and device of the embodiment of the present disclosure, by adopting the convolutional network based on deep time alignment gradient enhancement and the auxiliary classifier network, gradient enhancement and spatiotemporal depth supervision can be realized, the problem that the gradient may disappear or deviate during the training process is solved and the gradient flow is enhanced, thereby improving the training efficiency and model performance of the neural network model. In addition, by dynamically adjusting the proxy gradient function using the deviation metric between the membrane potential and the trigger threshold, the gradient calculation for different time steps is balanced, and the gradient disappearance problem in the deep network of the model is solved. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The above and other objects and features of the embodiments of the present disclosure will become more apparent through the following description in conjunction with the accompanying drawings showing the embodiments, in which: Figure 1is a flowchart illustrating a training method of a neural network based on deep time-aligned gradient enhancement according to an embodiment of the present disclosure; Figure 2 is a schematic diagram showing the structural framework of a neural network model based on deep time-aligned gradient enhancement according to an embodiment of the present disclosure; Figure 3 is a flow chart illustrating a data identification method according to an embodiment of the present disclosure; Figure 4 is a block diagram showing a training apparatus for a neural network based on deep time-aligned gradient enhancement according to an embodiment of the present disclosure; Figure 5 is a block diagram showing a data identification device according to an embodiment of the present disclosure; Figure 6 is a block diagram illustrating a computing device according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0022] The following specific embodiments are provided to help the reader obtain a comprehensive understanding of the methods, devices and / or systems described herein. However, after understanding the disclosure of the present application, various changes, modifications and equivalents of the methods, devices and / or systems described herein will be clear. For example, the order of operations described herein is only an example and is not limited to those orders set forth herein, but can be changed as will be clear after understanding the disclosure of the present application, except for operations that must occur in a specific order. In addition, for greater clarity and simplicity, the description of features known in the art may be omitted.

[0023] Reference will now be made in detail to the embodiments of the present disclosure, examples of which are illustrated in the accompanying drawings, wherein like reference numerals refer to like parts throughout. The embodiments will be described below with reference to the drawings in order to explain the present disclosure.

[0024] As mentioned above, in the related art, the existing neural network algorithms may have a series of problems as mentioned above. In addition, for example, a specific spiking neural network (Spiking Neural Network, SNN), which is a biologically inspired computing model with dynamic spatiotemporal connections and a pulse-based communication mechanism. Unlike traditional artificial neural networks (ANN), SNN operates through discrete pulse signals instead of continuous activation, which has advantages in computing efficiency and energy efficiency. Especially in neuromorphic hardware implementation, SNN has lower power consumption and stronger real-time processing capabilities due to its event-driven computing method (computation is performed only when a pulse is received), which is suitable for energy-constrained application scenarios.

[0025] However, due to its complex neural dynamics and non-differentiable pulse communication mechanism, spiking neural networks still have the following problems in the efficient training process. For example, due to the non-differentiability of the pulse activation function, the propagation of the gradient in the back-propagation process becomes complicated, which greatly increases the difficulty of training the spiking neural network. For example, during the training process, the gradient in the back-propagation process gradually disappears, affecting the effectiveness of the optimization process. Existing methods improve the learning performance of spiking neural networks by adding residual connection methods, regularization methods or attention mechanisms.

[0026] However, these existing methods have failed to effectively solve the problem of the difficulty of gradient propagation in the spatiotemporal dimension during the training of spiking neural networks. Furthermore, in the training process involving multi-layer networks and multiple time steps, as the time step increases, the membrane potential distribution of spiking neurons deviates from the trigger threshold, resulting in a decrease in the gradient amplitude of back propagation. Existing methods are difficult to overcome the above-mentioned problems of gradient vanishing and unbalanced training process, resulting in poor optimization results. In addition, existing optimization methods require adjusting the network architecture or adding additional control modules to a certain extent, which not only increases the difficulty of implementation, but may also lead to additional overhead in the inference stage, limiting its versatility in practical applications.

[0027] In order to solve the above and / or other problems, the present disclosure proposes a training method and device for a neural network based on deep time-aligned gradient enhancement and a data recognition method and device. By adopting a convolutional network based on deep time-aligned gradient enhancement and an auxiliary classifier network, gradient enhancement and spatiotemporal deep supervision can be achieved, which solves the problem that the gradient may disappear or deviate during the training process and enhances the gradient flow, thereby improving the training efficiency and model performance of the neural network model.

[0028] Refer to the following Figures 1 to 6 The present invention describes in detail a training method and device for a neural network based on deep time-aligned gradient enhancement and a data recognition method and device according to an embodiment of the present invention.

[0029] First, refer to Figure 1 to Figure 2 To describe a training method for a neural network based on deep time-aligned gradient enhancement according to an embodiment of the present disclosure.

[0030] Figure 1 is a flowchart illustrating a training method 100 of a neural network based on deep time-aligned gradient enhancement according to an embodiment of the present disclosure. Figure 2 is a schematic diagram showing the structural framework of a neural network model based on deep time-aligned gradient enhancement according to an embodiment of the present disclosure.

[0031] Reference Figure 1According to an embodiment of the present disclosure, in step S101, input sample data is obtained. Here, the input sample data includes sample image data and / or sample text data and corresponding sample label data.

[0032] According to an embodiment of the present disclosure, in step S102, input sample data is input into a neural network model based on deep time alignment gradient enhancement to obtain predicted category probability data corresponding to the category of the input sample data.

[0033] For example, a neural network model based on deep time alignment gradient enhancement includes an input layer, K stages (for example, the "stage" here can be regarded as a "hidden layer") and an output layer, and the first stage to the K-1th stage each include a convolutional network based on deep time alignment gradient enhancement (for example, as Figure 2 Conv and LIFNode in stages 1 to K-1 in the figure) and the auxiliary classifier network , and the Kth stage includes a convolutional network based on deep temporal alignment gradient boosting (e.g., as Figure 2 Here, K is a positive integer greater than 1 (for example, it can also be greater than 2, greater than 3, etc.), and its corresponding k It can represent the stage number of the stage or the stage number of the network stage.

[0034] For example, the output layer of the present disclosure may be a fully connected layer (e.g., Figure 2 FC as shown), and global average pooling can be performed on the output data before it is output by FC (e.g., as Figure 2 GAP) treatment as shown.

[0035] By adding auxiliary classifier networks to supervise the outputs of multiple time steps and multiple network stages, the gradient flow in the backpropagation path is enhanced, thereby improving the training efficiency and network performance of the model.

[0036] As an example, each of the 1st to K-1th stages may include a deep time-aligned gradient enhancement-based convolutional network and an auxiliary classifier network connected in sequence, and wherein the auxiliary classifier network may include three convolutional layers, a global average pooling layer, and a fully connected layer connected in sequence.

[0037] By constructing the auxiliary classifier network to include three convolutional layers, a global average pooling layer and a fully connected layer, a balance between network performance and computational complexity and amount can be achieved, for example, the network performance is relatively optimal and the computational complexity and amount are appropriate (for example, relatively small).

[0038] As an example, step S102 may further include steps S1021 and S1022: In step S1021, the input sample data is input to the input layer, the 1st stage to the K-1th stage in a forward propagation manner to obtain K-1 predicted category probability vectors at each time step (eg, iteration step).

[0039] In step S1022, the K-1th predicted category probability vector is input to the Kth stage and output layer in a forward propagation manner to obtain predicted category probability data corresponding to the category of the input sample data at each time step.

[0040] By performing direct supervision at multiple network stages and multiple time steps (e.g., direct supervision at the spatial scale of the first K-1 stages and direct supervision at the temporal scale of the Kth stage), the gradient flow of the model in the back-propagation path is enhanced, thereby accelerating the training process and improving network performance.

[0041] According to an embodiment of the present disclosure, in step S103, various parameters of the neural network model based on deep time alignment gradient enhancement are adjusted according to the predicted category probability data and the sample label data to obtain a trained neural network model.

[0042] As an example, the following formulas (1) to (3) can be used to calculate the gradient parameters in a convolutional network based on deep time-aligned gradient enhancement: (1) (2) (3) in, t represents the time step, represents the gradient parameter (e.g., Figure 2 In ), Indicates t The membrane potential at each time step is represents the preset trigger threshold, σ1 represents the L2 norm metric at the first time step, (For example, Figure 2 In ) indicates the t The L2 norm metric at time steps, express The normalization factor of Figure 2 In ( )to ( )), Represents preset hyperparameters.

[0043] Here, the above metrics To quantify at each time step t , membrane potential With preset trigger threshold For example, The larger the value, the longer the time step. t , membrane potential The further away from the threshold , which is usually the reason why the gradient is small.

[0044] In addition, the above normalization factor Used to dynamically adjust the proxy gradient function according to changes in the membrane potential distribution. In addition, preset hyperparameters Used to control the time step t The membrane potential distribution deviation ratio between time step 1 and For example, when When That is, this normalization factor ensures that the proxy gradient function is responsive to changes in the input membrane potential distribution between time steps.

[0045] In addition, for the above formula (1), by changing the membrane potential Relative to Scaling is performed to calculate the proxy gradient. The above scaling operation does not change and The size relationship between them, but it can make them stay away from Moreover, input data with a flatter distribution can obtain more effective gradients, thus effectively avoiding the gradient vanishing problem.

[0046] By measuring the deviation between the membrane potential and the preset trigger threshold (i.e., the distribution of the membrane potential), the proxy gradient function is dynamically adjusted to balance the gradient calculations at different time steps, thus solving the gradient vanishing problem of the model in deep networks.

[0047] In addition, for example, compared with the existing method of using a smooth function instead of a pulse activation function to approximate the non-differentiable part in the pulse neural network, the problem of improving the gradient only in the shallow network but not in the deep network in the prior art is solved, that is, the training method of the present invention solves the gradient vanishing problem of the model in the deep network, effectively solves the gradient propagation problem of the pulse neural network in the time and space dimensions, and greatly improves the optimization effect of the model.

[0048] In addition, as an example, step S103 may further include steps S1031 and S1032: In step S1031, the total time steps used for model training are obtained.

[0049] In step S1032, according to the predicted category probability data, the sample label data and the total time step, the loss value for adjusting each parameter is calculated based on the following formula: (4) in, = ( ),in, represents the loss value (e.g. Figure 2 ), K represents the total number of stages and k The value range is [1, K -1], T represents the total time step, t Represents the time step and its value range is [1, T ], represents the cross entropy loss function, Indicated in t The next time step k The predicted category probability output of each stage, Indicated in t The time step K The predicted category probability output of each stage is the data of the predicted category probability data before global average pooling. represents sample label data, A network function representing the auxiliary classifier network (e.g., Figure 2 In ), Indicated in t The time step k The feature vector of each stage.

[0050] As an example, see Figure 2 , the above formula is illustrated by taking a pulse neural network including K network stages as an example. In order to achieve deep supervision, an auxiliary classifier network is introduced after network stages 1 to K-1. , and each time step t The final classification output (i.e., the predicted class probability corresponding to the class of the input data) is expressed as .

[0051] Next, by combining the feature maps at different time steps Input into the auxiliary classifier network to obtain additional classification prediction data, as shown in the following formula (5): (5) Finally, through the above loss function To calculate the cross entropy loss function for all classification predictions.

[0052] By performing direct supervision at multiple network stages and time steps, the disclosed method can further enhance the gradient flow of the model in back-propagation, thereby accelerating the training process and improving the performance of the spiking neural network.

[0053] In step S1033, various parameters are adjusted based on the loss value to obtain a trained neural network model.

[0054] By applying the cross entropy loss function to all classification prediction results (predicted category probabilities) to calculate the model loss as described above, the problem that the gradient may disappear or deviate during the training process is solved, and the gradient flow of the model in the return path is further improved, thereby enhancing the deep learning ability of the neural network algorithm.

[0055] In addition, for pulse neural networks, the above-mentioned training method disclosed in the present invention can solve the problem that the existing methods cannot maintain effective gradient transfer in both time and space dimensions at the same time, fully consider the collaborative optimization of gradients in time and space dimensions, and greatly improve the optimization effect on the model, for example, it improves the deep learning ability of pulse neural networks.

[0056] It should be noted that although the above description uses a pulse neural network as an example to illustrate the training method and corresponding effects of the present disclosure, the present disclosure is not limited to pulse neural networks, and can also be applied to other neural networks to achieve effects similar to those described above.

[0057] Here, it should be noted that the above Figure 1 The description of each step is merely exemplary, and the steps in the method according to the present disclosure are not limited thereto.

[0058] Next, refer to Figure 3 To describe the data identification method according to an embodiment of the present disclosure. Figure 3 is a flow chart illustrating a data identification method 300 according to an embodiment of the present disclosure.

[0059] According to an embodiment of the present disclosure, in step S301, input data to be recognized is obtained. Here, the input data includes image data and / or text data.

[0060] According to an embodiment of the present disclosure, in step S302, the input data to be identified is input into a neural network model based on deep time alignment gradient enhancement to obtain corresponding predicted category probability data as a data recognition result.

[0061] Here, the neural network model based on deep time alignment gradient enhancement is trained using the above training method.

[0062] Next, refer to Figure 4 To describe a training device for a neural network based on deep time-aligned gradient enhancement according to an embodiment of the present disclosure.

[0063] Figure 4 4 is a block diagram showing a training apparatus 400 for a neural network based on deep time-aligned gradient enhancement according to an embodiment of the present disclosure.

[0064] Reference Figure 4 According to an embodiment of the present disclosure, a training device 400 for a neural network based on deep time-aligned gradient enhancement may include an acquisition module 410, a prediction module 420 and a training module 430.

[0065] According to an embodiment of the present disclosure, the acquisition module 410 may execute: acquiring input sample data. Here, the input sample data includes sample image data and / or sample text data and corresponding sample label data.

[0066] According to an embodiment of the present disclosure, the prediction module 420 may execute: inputting the input sample data into a neural network model based on deep time alignment gradient enhancement to obtain predicted category probability data corresponding to the category of the input sample data.

[0067] Here, the neural network model based on deep time alignment gradient enhancement includes an input layer, K stages and an output layer. Specifically, the first stage to the K-1th stage each include a convolutional network based on deep time alignment gradient enhancement and an auxiliary classifier network, and the Kth stage includes a convolutional network based on deep time alignment gradient enhancement. Here, K is a positive integer greater than 1.

[0068] For example, each of the 1st to K-1th stages may include a convolutional network based on deep time-aligned gradient enhancement and an auxiliary classifier network connected in sequence. In addition, for example, the auxiliary classifier network may include three convolutional layers, a global average pooling layer, and a fully connected layer connected in sequence.

[0069] As an example, the prediction module 420 inputs the input sample data into a neural network model based on deep time-aligned gradient enhancement, and the operation of obtaining predicted category probability data corresponding to the category of the input sample data may include: inputting the input sample data into the input layer, the 1st stage to the K-1th stage in a forward propagation manner, and obtaining K-1 predicted category probability vectors at each time step; inputting the K-1th predicted category probability vector into the Kth stage and the output layer in a forward propagation manner, and obtaining predicted category probability data corresponding to the category of the input sample data at each time step.

[0070] According to an embodiment of the present disclosure, the training module 430 may execute: adjusting various parameters of the neural network model based on deep time alignment gradient enhancement according to the predicted category probability data and the sample label data to obtain a trained neural network model.

[0071] As an example, the above formulas (1) to (3) can be used to calculate the gradient parameters in a convolutional network based on deep time-aligned gradient enhancement.

[0072] In addition, as an example, the training module 430 adjusts the various parameters of the neural network model based on deep time alignment gradient enhancement according to the predicted category probability data and the sample label data, and the operation of obtaining the trained neural network model may include: obtaining the total time step for model training; calculating the loss value for adjusting the various parameters based on the above formula (4) according to the predicted category probability data, the sample label data and the total time step; and adjusting the various parameters based on the loss value to obtain the trained neural network model.

[0073] It should be noted that the operations performed on the above-mentioned structural frames can be referred to in Figure 1 The related contents described are similar and will not be repeated here.

[0074] Next, refer to Figure 5 A data identification device according to an embodiment of the present disclosure is described. Figure 5 is a block diagram illustrating a data identification device 500 according to an embodiment of the present disclosure.

[0075] Reference Figure 5 According to an embodiment of the present disclosure, the data identification device 500 may include a data acquisition module 510 and an identification module 520 .

[0076] According to an embodiment of the present disclosure, the data acquisition module 510 may execute: acquiring input data to be recognized, where the input data includes image data and / or text data.

[0077] According to an embodiment of the present disclosure, the recognition module 520 may execute: inputting the input data to be recognized into a neural network model based on deep time alignment gradient enhancement to obtain corresponding predicted category probability data as a data recognition result.

[0078] Here, the neural network model based on deep time alignment gradient enhancement is trained using the above training method.

[0079] Figure 6 is a block diagram illustrating a computing device 600 according to an embodiment of the present disclosure.

[0080] Reference Figure 6, the computing device 600 according to an embodiment of the present disclosure may include a processor 610 and a memory 620. The processor 610 may include (but not limited to) a central processing unit (CPU), a digital signal processor (DSP), a microcomputer, a field programmable gate array (FPGA), a system on a chip (SoC), a microprocessor, an application-specific integrated circuit (ASIC), etc. The memory 620 may store computer executable instructions to be executed by the processor 610. The memory 620 includes a high-speed random access memory and / or a non-volatile computer-readable storage medium. When the processor 610 executes the computer executable instructions stored in the memory 620, the training method of the neural network based on deep time-aligned gradient enhancement as described above and the data recognition method as described above may be implemented.

[0081] According to an embodiment of the present disclosure, the training method and data recognition method of a neural network based on deep time alignment gradient enhancement can be written as a computer program / instruction to form a computer program product and stored on a computer-readable storage medium. When the computer program / instruction is executed by a processor, the training method of a neural network based on deep time alignment gradient enhancement and the data recognition method as described above can be implemented. When the instructions in the computer-readable storage medium are executed by a processor of an electronic device / server, the electronic device / server is enabled to execute the training method of a neural network based on deep time alignment gradient enhancement and the data recognition method as described above. Examples of computer-readable storage media include: read-only memory (ROM), random access programmable read-only memory (PROM), electrically erasable programmable read-only memory (EEPROM), random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), flash memory, non-volatile memory, CD-ROM, CD-R, CD+R, CD-RW, CD+RW, DVD-ROM, DVD-R, DVD+R, DVD-RW, DVD+RW, DVD-RAM, BD-ROM, BD-R, BD-R LTH, BD-RE, Blu-ray or optical disk storage, hard disk drive (HDD), solid state drive (SSD), card storage (such as, multimedia card, secure digital (SD) card or extreme digital (XD) card), magnetic tape, floppy disk, magneto-optical data storage device, optical data storage device, hard disk, solid state disk and any other device, any other device is configured to store computer program and any associated data, data files and data structures in a non-transitory manner and provide the computer program and any associated data, data files and data structures to a processor or computer so that the processor or computer can execute the computer program. In one example, the computer program and any associated data, data files and data structures are distributed on a networked computer system so that the computer program and any associated data, data files and data structures are stored, accessed and executed in a distributed manner by one or more processors or computers.

[0082] According to the embodiments of the present invention, the training method and device of the neural network based on deep time alignment gradient enhancement and the data recognition method and device can realize gradient enhancement and spatiotemporal deep supervision by adopting a convolutional network based on deep time alignment gradient enhancement and an auxiliary classifier network, solve the problem that the gradient may disappear or deviate during the training process and enhance the gradient flow, thereby improving the training efficiency and model performance of the neural network model.

[0083] On the other hand, by using the deviation measure between the membrane potential and the trigger threshold to dynamically adjust the proxy gradient function and balance the gradient calculation at different time steps, the gradient vanishing problem in the deep network of the model is solved.

[0084] On the other hand, according to the training method and device of the neural network based on deep time-aligned gradient enhancement and the data recognition method and device of the embodiments of the present disclosure, by introducing an auxiliary classifier network, supervision of the output of the network stage at multiple time steps is achieved, thereby enhancing the gradient flow and improving the training efficiency and network performance of the model.

[0085] Although some embodiments of the present disclosure have been disclosed and described, it will be appreciated by those skilled in the art that modifications and variations may be made to the embodiments without departing from the concept and spirit of the present disclosure, the scope of which is defined by the claims and their equivalents.

Claims

1. A training method for a neural network based on deep time-aligned gradient enhancement, characterized in that: The training method comprises: Acquire input sample data, where the input sample data includes sample image data and / or sample text data and corresponding sample label data; Inputting the input sample data into a neural network model based on deep time alignment gradient enhancement to obtain predicted category probability data corresponding to the category of the input sample data; According to the predicted category probability data and the sample label data, adjusting various parameters of the neural network model based on deep time alignment gradient enhancement to obtain a trained neural network model, The neural network model based on deep time alignment gradient enhancement includes an input layer, K stages and an output layer, the first stage to the K-1th stage each include a convolutional network based on deep time alignment gradient enhancement and an auxiliary classifier network, and the Kth stage includes a convolutional network based on deep time alignment gradient enhancement, Wherein, K is a positive integer greater than 1.

2. The training method according to claim 1, characterized in that: The following formula is used to calculate the gradient parameters in the convolutional network based on deep time-aligned gradient enhancement: ， in, ,and , in, t represents the time step, represents the gradient parameter, Indicates t The membrane potential at each time step is Indicates the preset trigger threshold. Indicates t The L2 norm metric at the time step, σ1 represents the L2 norm metric at the first time step, express The normalization factor of Represents preset hyperparameters.

3. The training method according to claim 1, characterized in that: The step of inputting the input sample data into a neural network model based on deep time alignment gradient enhancement to obtain predicted category probability data corresponding to the category of the input sample data comprises: Inputting the input sample data into the input layer, the first stage to the K-1th stage in a forward propagation manner to obtain K-1 predicted category probability vectors at each time step; The K-1th predicted category probability vector is input into the Kth stage and the output layer in a forward propagation manner to obtain predicted category probability data corresponding to the category of the input sample data at each time step.

4. The training method according to claim 1, characterized in that: The step of adjusting various parameters of the neural network model based on deep time alignment gradient enhancement according to the predicted category probability data and the sample label data to obtain a trained neural network model comprises: Get the total time steps used for model training; According to the predicted category probability data, the sample label data and the total time step, the loss value for adjusting each parameter is calculated based on the following formula: ， in, = ( ), in, represents the loss value, K represents the total number of stages and k The value range is [1, K -1], T represents the total time step, t Represents the time step and its value range is [1, T ], represents the cross entropy loss function, Indicated in t The next time step k The predicted category probability output of each stage, Indicated in t The predicted category probability data at time steps and the predicted category probability data before global average pooling, represents the sample label data, represents the network function of the auxiliary classifier network, Indicated in t The time step k The feature vector of each stage; Various parameters are adjusted based on the loss value to obtain a trained neural network model.

5. The training method according to claim 1, characterized in that: Each of the 1st to K-1th stages includes a deep temporal aligned gradient boosting based convolutional network and an auxiliary classifier network connected sequentially, and The auxiliary classifier network includes three convolutional layers, a global average pooling layer and a fully connected layer connected in sequence.

6. A data identification method, characterized in that: The data identification method comprises: Acquire input data to be recognized, wherein the input data includes image data and / or text data; The input data to be identified is input into a neural network model based on deep time alignment gradient enhancement to obtain corresponding predicted category probability data as a data identification result. Wherein, the neural network model based on deep time alignment gradient enhancement is trained using the training method described in any one of claims 1 to 5.

7. A training device for a neural network based on deep time-aligned gradient enhancement, characterized in that: The training device comprises: An acquisition module is configured to: acquire input sample data, wherein the input sample data includes sample image data and / or sample text data and corresponding sample label data; A prediction module is configured to: input the input sample data into a neural network model based on deep time alignment gradient enhancement to obtain predicted category probability data corresponding to the category of the input sample data; A training module is configured to: adjust various parameters of the neural network model based on deep time alignment gradient enhancement according to the predicted category probability data and the sample label data to obtain a trained neural network model, The neural network model based on deep time alignment gradient enhancement includes an input layer, K stages and an output layer, the first stage to the K-1th stage each include a convolutional network based on deep time alignment gradient enhancement and an auxiliary classifier network, and the Kth stage includes a convolutional network based on deep time alignment gradient enhancement, Wherein, K is a positive integer greater than 1.

8. A data identification device, characterized in that: The data identification device comprises: An acquisition module is configured to: acquire input data to be recognized, wherein the input data includes image data and / or text data; The recognition module is configured to: input the input data to be recognized into the neural network model based on deep time alignment gradient enhancement to obtain corresponding predicted category probability data as the data recognition result, Wherein, the neural network model based on deep time alignment gradient enhancement is trained using the training method described in any one of claims 1 to 5.

9. A computer program product, characterized in that The computer program product includes a computer program / instruction, which, when executed by a processor, implements the training method of a neural network based on deep time-aligned gradient enhancement as described in any one of claims 1 to 5 and the data recognition method as described in claim 6.

10. A computing device, characterized in that The computing device includes: at least one processor; at least one memory storing computer executable instructions, wherein the computer executable instructions, when executed by the at least one processor, cause the at least one processor to execute the training method of a neural network based on deep time-aligned gradient enhancement as described in any one of claims 1 to 5 and the data recognition method as described in claim 6.

Images