Model training method, device and equipment, medium and computer program product

Abstract

The invention provides a model training method, device and equipment, a medium and a computer program product, and relates to the technical field of artificial intelligence. The method comprises the steps that a pulse model trained according to a dynamic visual data set collected by a dynamic visual sensor and a predefined truncation function are acquired, and an initial quantization pulse model is acquired; according to the first moment of the input variable of the initial quantization pulse model, the credibility estimation value of the initial quantization pulse model is obtained, the initial quantization pulse model is updated, and a quantization pulse model for converting continuous data collected by the dynamic vision sensor into pulse sequence data is obtained. A truncation function is adopted to truncate the pulse model to obtain an initial quantization pulse model, and the prediction accuracy is improved while model memory compression is preliminarily achieved; the credibility estimation value is obtained through the first moment of the input variable, the nonlinear estimation has a closed-form solution, accurate estimation can be realized, and the time sequence data prediction consistency is further improved.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a model training method, apparatus, device, medium, and computer program product. Background Technology

[0002] With the rapid development of brain-inspired intelligence and event-driven perception technologies, dynamic vision systems, represented by Dynamic Vision Sensors (DVS), have shown significant application potential in resource-constrained scenarios such as intelligent robots, autonomous driving, unmanned systems, and edge computing due to their advantages such as high temporal resolution, low power consumption, and high dynamic range. Dynamic vision sensors output environmental change information in the form of asynchronous event streams. Their data exhibits significant characteristics such as temporal continuity, spatial sparsity, and non-framed nature, posing new challenges to traditional frame-based visual modeling methods.

[0003] Spiking Neural Networks (SNNs), using pulse sequences as their basic information carrier, are naturally well-suited to dynamic visual data in terms of time encoding, energy efficiency, and event-driven processing, and are considered an important modeling paradigm for dynamic visual perception. However, current SNNs still suffer from drawbacks such as slow inference speed, difficulty in storing high-precision numerical values ​​(making deployment on basic hardware difficult), low prediction accuracy, and low prediction consistency, which restrict their large-scale application in edge intelligence and embedded systems. Therefore, lightweight construction and consistency assurance of spiking neural networks are crucial issues for achieving high-performance computing and lightweight deployment in dynamic visual scenarios.

[0004] Quantumization is currently the main technique for achieving lightweight construction of spiking neural networks. SNN-Compression-ADMM achieves network pruning and model compression through ADMM optimization. ANN2SNN-Scaling achieves model compression by transforming a quantized ANN into an equivalent SNN. Experimental results show that the above methods significantly reduce model size and improve inference speed, but still have drawbacks such as high-precision numerical storage, decreased prediction accuracy, and low prediction consistency. Summary of the Invention

[0005] The purpose of this invention is to provide a model training method, apparatus, device, medium, and computer program product to solve the problems of high-precision numerical storage, decreased prediction accuracy, and low prediction consistency in the quantization method of spiking neural networks in dynamic visual scenes in the prior art.

[0006] To achieve the above objectives, embodiments of the present invention provide a model training method, comprising:

[0007] Obtain the impulse model trained based on the dynamic visual dataset, and a predefined truncation function; wherein the dynamic visual dataset is environmental change data collected by a dynamic visual sensor and recorded in the form of an asynchronous event stream;

[0008] Based on the pulse model and the truncation function, an initial quantized pulse model is obtained;

[0009] Based on the first moment of the input variables of the initial quantized pulse model, obtain the confidence estimate of the initial quantized pulse model;

[0010] The initial quantized pulse model is updated based on the confidence estimate to obtain a quantized pulse model; wherein, the quantized pulse model is used to convert continuous data collected by the dynamic vision sensor into pulse sequence data.

[0011] Optionally, the method, wherein obtaining the initial quantized pulse model based on the pulse model and the truncation function includes:

[0012] The first quantization parameter is obtained based on the truncation function and the first full-precision parameter of the pulse model;

[0013] Construct a quantized pulse model to be trained based on the first quantization parameters;

[0014] The first parameter update value of the quantized impulse model to be trained is obtained according to the first gradient; wherein, the first gradient is obtained according to the approximate derivative of the truncation function and the second gradient of the impulse model; the second gradient includes the alternative gradient of the impulse model;

[0015] The initial quantized pulse model is obtained by adjusting the quantized pulse model to be trained according to the second quantization parameter; wherein the second quantization parameter is obtained based on the update value of the first parameter and the truncation function.

[0016] Optionally, the method, wherein obtaining a confidence estimate of the initial quantized pulse model based on the first moment of the input variables of the initial quantized pulse model, includes:

[0017] The second moment of the variable before activation is obtained based on the first moment of the input variable;

[0018] The second moment is processed using an activation function, and the variables before activation are transformed into variables after activation.

[0019] Obtain the third moment of the activated variable, wherein the third moment includes the confidence estimate.

[0020] Optionally, the method, wherein updating the initial quantized pulse model based on the confidence estimate to obtain the quantized pulse model includes:

[0021] A consistency optimization problem is established based on the aforementioned credibility estimate;

[0022] The quantized pulse model is obtained by solving the consistency optimization problem.

[0023] Optionally, the method, wherein obtaining the quantized impulse model by solving the consistency optimization problem includes:

[0024] The first full-precision parameters of the impulse model are updated using gradient descent to obtain the second full-precision parameters.

[0025] The truncation threshold of the truncation function is updated based on the second full-precision parameter;

[0026] The second full-precision parameter is truncated using the truncation threshold and the truncation function to obtain the third quantized parameter;

[0027] The quantized pulse model is obtained based on the third quantization parameter.

[0028] Optionally, the method, wherein obtaining the impulse model trained on the dynamic visual dataset includes:

[0029] Construct an initial impulse model and obtain the dynamic visual dataset;

[0030] Define the loss function of the initial impulse model and establish the dynamic visual training optimization problem of the initial impulse model;

[0031] The second parameter update value of the initial impulse model is obtained by solving the dynamic visual training optimization problem by replacing the gradient.

[0032] The pulse model is obtained based on the updated value of the second parameter and the dynamic visual dataset.

[0033] To achieve the above objectives, embodiments of the present invention provide a model training apparatus, comprising:

[0034] The first acquisition module is used to acquire the impulse model trained based on the dynamic visual dataset, and a predefined truncation function; wherein, the dynamic visual dataset is environmental change data collected by a dynamic visual sensor and recorded in the form of an asynchronous event stream;

[0035] The second acquisition module is used to acquire an initial quantized pulse model based on the pulse model and the truncation function;

[0036] The third acquisition module is used to acquire the confidence estimate of the initial quantized pulse model based on the first moment of the input variables of the initial quantized pulse model.

[0037] The fourth acquisition module is used to update the initial quantized pulse model based on the confidence estimate and acquire the quantized pulse model; wherein the quantized pulse model is used to convert the continuous data collected by the dynamic vision sensor into pulse sequence data.

[0038] To achieve the above objectives, embodiments of the present invention provide a model training device, comprising: a processor, a memory, and a program or instructions stored in the memory and executable on the processor; wherein, when the processor executes the program or instructions, it implements the model training method as described above.

[0039] To achieve the above objectives, embodiments of the present invention provide a readable storage medium having a program or instructions stored thereon, wherein the program or instructions, when executed by a processor, implement the steps in the model training method described above.

[0040] To achieve the above objectives, embodiments of the present invention provide a computer program product, which includes computer instructions that, when executed by a processor, implement the steps of the model training method described above.

[0041] The beneficial effects of the above-described technical solution of the present invention are as follows:

[0042] In this embodiment of the invention, a predefined truncation function is used to truncate the impulse model trained on a dynamic visual dataset collected by a dynamic visual sensor to obtain an initial quantized impulse model. This initially achieves low-precision numerical storage and high sparsity of the impulse model, thereby significantly compressing the model memory while improving the prediction accuracy of dynamic visual data. The initial quantized impulse model is updated based on the confidence estimate obtained from the first moment of the input variables using the initial quantized impulse model, resulting in a quantized impulse model for converting continuous data into impulse sequence data. The confidence estimate obtained through the first moment of the input variables uses nonlinear estimation with a closed-form solution, enabling accurate confidence estimation and improving the consistency of time-series data prediction for data collected by the dynamic visual sensor. Attached Figure Description

[0043] Figure 1 This is a schematic diagram of the model training method described in an embodiment of the present invention;

[0044] Figure 2 This is a schematic diagram of the architecture of the model training method described in an embodiment of the present invention;

[0045] Figure 3This is a flowchart illustrating the process of obtaining a quantized pulse model using the model training method described in this embodiment of the invention.

[0046] Figure 4 This is a flowchart of the algorithm for reliability estimation and consistency optimization of the quantized impulse model in the model training method described in this embodiment of the invention;

[0047] Figure 5 This is a schematic diagram of the model training device described in an embodiment of the present invention. Detailed Implementation

[0048] To make the technical problems, technical solutions and advantages of the present invention clearer, a detailed description will be given below in conjunction with the accompanying drawings and specific embodiments.

[0049] It should be understood that the phrase "one embodiment" or "an embodiment" throughout the specification means that a specific feature, structure, or characteristic related to the embodiment is included in at least one embodiment of the invention. Therefore, "in one embodiment" or "in an embodiment" appearing throughout the specification do not necessarily refer to the same embodiment. Furthermore, these specific features, structures, or characteristics can be combined in any suitable manner in one or more embodiments.

[0050] In various embodiments of the present invention, it should be understood that the sequence number of each process described below does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0051] In addition, the terms "system" and "network" are often used interchangeably in this article.

[0052] In the embodiments provided by this invention, it should be understood that "B corresponding to A" means that B is associated with A, and B can be determined based on A. However, it should also be understood that determining B based on A does not mean that B is determined solely based on A; B can also be determined based on A and / or other information.

[0053] For ease of understanding, the following describes some aspects of the embodiments of the present invention:

[0054] like Figure 1 As shown, an embodiment of the present invention provides a model training method, which includes:

[0055] Step S10: Obtain the pulse model trained based on the dynamic visual dataset, and a predefined truncation function; wherein, the dynamic visual dataset is environmental change data collected by a dynamic visual sensor and recorded in the form of an asynchronous event stream;

[0056] It should be noted that, as Figure 2As shown, in step A1, the spur model construction parameters are initialized, and in step A2, the first stage of training takes place. By constructing a spur model, a supervised learning dataset for a dynamic visual scene (i.e., the dynamic visual dataset) is prepared. This dataset is typically collected by a dynamic visual sensor, recording environmental change information in the form of an asynchronous event stream, and has significant characteristics such as temporal continuity, spatial sparsity, and non-framed nature. The constructed spur model is trained using the dynamic visual dataset, and the full-precision parameters are calculated to obtain the trained spur model. The truncation function is also defined.

[0057] Step S20: Obtain the initial quantized pulse model based on the pulse model and the truncation function;

[0058] It should be noted that, in Figure 2 In step A4, a truncation function is defined to construct a quantized model. The corresponding quantized parameters are calculated using the truncation function from step S10 and the full-precision parameters of the impulse model, thereby constructing the corresponding quantized impulse model. In step A5, the second stage of training begins. The constructed quantized impulse model is trained using the dynamic visual dataset to obtain the initial quantized impulse model.

[0059] Step S30: Obtain the confidence estimate of the initial quantized pulse model based on the first moment of the input variables of the initial quantized pulse model;

[0060] It should be noted that, in Figure 2 In step A3, the confidence estimation of the impulse model and in step A6, the confidence estimation of the quantized model, are achieved by passing the first moment of the input variable, i.e. the first and second moments of the input variable in the initial quantized model, layer by layer, to output the first and second moments of the variable after the last layer of activation of the initial quantized impulse model. The first moment is used as the predicted value of the model, and the second moment is used as the confidence estimate of the model.

[0061] Step S40: Update the initial quantized pulse model according to the confidence estimate to obtain the quantized pulse model; wherein, the quantized pulse model is used to convert the continuous data collected by the dynamic vision sensor into pulse sequence data;

[0062] It should be noted that, in Figure 2 In step A7, the moment alignment establishes a consistent optimization problem. In step A9, the model converges or the maximum number of iterations is obtained, thereby acquiring the quantized pulse model.

[0063] In this embodiment, a predefined truncation function is used to truncate the pulse model trained on the dynamic visual dataset collected by the dynamic visual sensor to obtain an initial quantized pulse model. This initially achieves low-precision numerical storage and high sparsity of the pulse model, thereby significantly compressing the model memory while improving prediction accuracy. The initial quantized pulse model is updated based on the confidence estimate obtained from the first moment of the input variables using the initial quantized pulse model, resulting in a quantized pulse model used to convert continuous data into pulse sequence data. The confidence estimate obtained from the first moment of the input variables uses nonlinear estimation with a closed-form solution, which can achieve accurate estimation and thus improve prediction consistency.

[0064] Optionally, the method, wherein step S20 includes:

[0065] The first quantization parameter is obtained based on the truncation function and the first full-precision parameter of the pulse model;

[0066] Construct a quantized pulse model to be trained based on the first quantization parameters;

[0067] The first parameter update value of the quantized impulse model to be trained is obtained according to the first gradient; wherein, the first gradient is obtained according to the approximate derivative of the truncation function and the second gradient of the impulse model; the second gradient includes the alternative gradient of the impulse model;

[0068] The initial quantized pulse model is obtained by adjusting the quantized pulse model to be trained according to the second quantization parameter; wherein the second quantization parameter is obtained based on the update value of the first parameter and the truncation function.

[0069] In this embodiment, such as Figure 3As shown, in step B4, a truncation function is defined to construct a quantized model, and a quantized pulse model is constructed. A truncation function is also defined to perform quantum initialization of parameters and obtain the real-time first quantized parameters. In step B5, feedforward inference and loss are performed. Based on the first quantized parameters, the quantized pulse model to be trained is constructed, and feedforward inference is executed to establish a dynamic visual training optimization problem for the quantized pulse model. In step B3 (or B6), the substitution gradient and quantized parameters are calculated. The dynamic visual training optimization problem is solved by calculating the substitution gradient of the quantized pulse model, and the dynamic visual dataset is fed into the quantized pulse model to update the hyperparameters of the truncation function. The quantized parameters of the model are then calculated. The updated values ​​of the quantized pulse model parameters are calculated using gradient descent. The gradient is the product of the approximate derivative of the truncation function and the substitution gradient of the pulse model. That is, the first parameter update value of the quantized pulse model to be trained is obtained based on the first gradient, which is obtained based on the approximate derivative of the truncation function and the second gradient of the pulse model; the second gradient includes the substitution gradient of the pulse model. In step B7, feedforward inference and loss are performed. Feedforward inference is executed, and its loss value is calculated. Quantized parameters are calculated based on the parameter update value and the truncation function. The training data of the dynamic visual scene is fed into the quantized pulsation model. The above steps are repeated until the model training converges or the maximum number of training rounds is reached, indicating that data transfer is complete. The loop ends, training is completed, and the quantized pulsation model and its parameters are output. Specifically, the second quantized parameters are obtained based on the first parameter update value and the truncation function, and the second quantized parameters are used to adjust the quantized pulsation model to be trained, obtaining the initial quantized pulsation model.

[0070] In addition to being obtained through construction and training, the pulse model can also be obtained using pre-trained full-precision parameters. Deployment to the pulse model In, such as Figure 4 As shown, in step C1, the pre-training parameters are directly used to obtain the impulse model. An initial model is constructed based on the impulse model, and training is performed before the initial model converges to finally obtain the quantized impulse model. During this process, a truncation threshold is set. and truncation function Calculate the quantization weights of the pulse model. And perform the feedforward budget of the quantized impulse model and calculate its feedforward loss. (That is, obtain the first quantization parameter based on the truncation function and the first full-precision parameter of the impulse model). In step C2, update the quantization weights (that is, obtain the first parameter update value of the quantized impulse model to be trained based on the first gradient), and update the quantization parameter according to the following formula:

[0071] ;

[0072] in, The learning rate is represented by the gradient term, which consists of two parts: the substitution gradient of the impulse model and the approximate derivative of the cutoff function.

[0073] .

[0074] In step C3, the truncation function is updated based on the full-precision parameters. Update the cutoff threshold, i.e., the formula: ,in express The number of elements in the middle; in step C4, update the truncation threshold and use a grid search method within the search range. Solving for it internally, and updating it to obtain a new cutoff threshold. ,in The hyperparameters used to determine the search range are typically set to... In step C5, the quantization weights are updated according to the formula. The quantization parameters are updated, that is, the quantized impulse model to be trained is adjusted according to the second quantization parameter, which is obtained based on the update value of the first parameter and the truncation function.

[0075] In step C6, feedforward inference is performed with loss, using the updated quantized parameters. Perform feedforward inference and calculate the loss value to obtain the initial quantized impulse model.

[0076] The quantized construction of the pulsation model in this embodiment consists of three steps: setting a cutoff function, establishing a loss minimization problem, and calculating the alternative gradient. The loss minimization problem is solved by effectively approximating the cutoff function. Specifically: (1) Unlike the traditional pulsation model compression method, the quantized construction can achieve low-precision numerical storage and high sparsity, thereby achieving a significant compression of model memory (theoretically at least 16 times), a reduction in inference energy consumption (theoretically at least 2 times), an increase in inference speed (theoretically at least 4 times), and controlling the decrease in prediction accuracy to within 5%; (2) The quantized pulsation model is trained by the gradient descent algorithm. Its gradient consists of two parts: the approximate derivative of the cutoff function and the alternative gradient of the pulsation model. This training method is compatible with the traditional pulsation model training method using the alternative gradient, and can be trained directly using pre-trained parameters without training from scratch, which greatly improves training efficiency.

[0077] Optionally, the method, wherein step S30 includes:

[0078] The second moment of the variable before activation is obtained based on the first moment of the input variable;

[0079] The second moment is processed using an activation function, and the variables before activation are transformed into variables after activation.

[0080] Obtain the third moment of the activated variable, wherein the third moment includes the confidence estimate.

[0081] In this embodiment, such as Figure 4 As shown, in step C7, linear estimation is performed, a linear transformation is executed, and a linear estimation of confidence is performed. The first and second moments of the pre-activation variables (i.e., the second moments) are calculated using the first and second moments of the input variables (i.e., the first moments). In step C7, the pre-activation variables are transformed into the pre-activation variables. In step C8, the post-activation variables are transformed into the post-activation variables using a nonlinear transformation. In step C9, nonlinear estimation is performed, and a nonlinear estimation of confidence is performed. The first and second moments of the post-activation variables (i.e., the third moments) are calculated using the first and second moments of the pre-activation variables (i.e., the second moments). In step C10, it is determined whether the transfer is complete. During the model inference stage, steps C7 to C9 are repeated according to the inter-layer order until the last layer, i.e., the data transfer is determined to be complete. In step C11, the confidence estimate is obtained. The first moment of the post-activation variables is used as the predicted value of the model, and the second moment is used as the confidence estimate of the model. The loop ends, and the confidence estimate is output.

[0082] The confidence estimate is obtained in two steps: linear estimation and nonlinear estimation. Both estimation calculations have closed-form solutions and are performed simultaneously with the data transmission in the inference stage. Specifically: (1) In this embodiment of the invention, only the first and second moments of the variables in the deep learning model are transmitted. There is no need to calculate the true distribution or rely on the distribution assumption. (2) Unlike the assumption that each parameter follows a distribution in the traditional MP algorithm, the linear estimation in this embodiment of the invention adopts the assumption that the parameters of the same layer follow the same distribution and only performs linear operations, which can greatly reduce the computational complexity. (3) Unlike the approximate calculation of the parameter distribution in Bayesian algorithms, the nonlinear estimation in this embodiment of the invention has a closed-form solution and can achieve accurate estimation.

[0083] Optionally, the method, wherein step S40 includes:

[0084] A consistency optimization problem is established based on the aforementioned credibility estimate;

[0085] The quantized pulse model is obtained by solving the consistency optimization problem.

[0086] In this embodiment, the consistency optimization problem is established based on the confidence estimate, and the expression of the consistency optimization problem is as follows:

[0087] ;

[0088] in, It is the optimal cutoff threshold. To balance the hyperparameters, and and The formula is as follows:

[0089] ;

[0090] ;

[0091] Among them, matrix The parameters represent the full-precision parameters of the pulse model. For matrix The Line number Column elements, and if In the function middle Representing an interval In the function The middle represents the interval .

[0092] In step C12, the above steps are repeated until the model training converges or the set maximum number of training rounds is reached to determine that data transfer is complete. The loop ends, training is completed, and the quantized pulse model and the confidence estimate are output.

[0093] The consistency guarantee in this embodiment of the invention consists of five steps: model initialization, updating full-precision parameters, establishing a consistency optimization problem, solving the consistency optimization problem using a grid search method and updating the hyperparameters of the truncation function, and updating the quantization parameters. Specifically, the establishment and solution of the consistency optimization problem are as follows: (1) updating the model parameters and the hyperparameters of the truncation function using the gradient descent algorithm; (2) establishing a consistency optimization problem using moment alignment, balancing prediction accuracy and reliability; and (3) updating the hyperparameters of the truncation function and the quantization parameters using mature algorithms such as grid search. Unlike traditional optimization algorithms, the consistency guarantee algorithm proposed in this embodiment of the invention plays a crucial role in the training phase, significantly improving training efficiency.

[0094] Optionally, the method, wherein obtaining the quantized impulse model by solving the consistency optimization problem includes:

[0095] The first full-precision parameters of the impulse model are updated using gradient descent to obtain the second full-precision parameters.

[0096] The truncation threshold of the truncation function is updated based on the second full-precision parameter;

[0097] The second full-precision parameter is truncated using the truncation threshold and the truncation function to obtain the third quantized parameter;

[0098] The quantized pulse model is obtained based on the third quantization parameter.

[0099] In this embodiment, the optimizable parameters of the consistency optimization problem are the quantized parameters of the impulse model. The optimization objective is to ensure that the first and second moments of the impulse model using full-precision parameters and the impulse model using quantized parameters are consistent, thereby improving the prediction accuracy and output reliability of the impulse model on dynamic visual datasets. Figure 2 As shown, in step A8, the third stage of training updates the truncation function and quantized weights. The consistency optimization problem is solved using gradient descent to obtain model parameters. Since the gradient update value is full precision, the model parameters obtained in this step are full precision; that is, the first full precision parameters of the pulsation model are updated using gradient descent to obtain the second full precision parameters. The truncation threshold of the truncation function is updated based on the full precision parameters, i.e., the truncation threshold of the truncation function is updated based on the second full precision parameters. Using the truncation threshold and the truncation function, the full precision parameters of the pulsation model are truncated to obtain quantized parameters; that is, the second full precision parameters are truncated using the truncation threshold and the truncation function to obtain the third quantized parameters. In step A9, the model converges or reaches the maximum number of iterations, and the above steps are repeated until the termination condition in step A9 is met, training ends, and the quantized pulsation model obtained based on the third quantized parameters is output.

[0100] Optionally, the method, wherein step S10 includes:

[0101] Construct an initial impulse model and obtain the dynamic visual dataset;

[0102] Define the loss function of the initial impulse model and establish the dynamic visual training optimization problem of the initial impulse model;

[0103] The second parameter update value of the initial impulse model is obtained by solving the dynamic visual training optimization problem by replacing the gradient.

[0104] The pulse model is obtained based on the updated value of the second parameter and the dynamic visual dataset.

[0105] In this embodiment, firstly, in Figure 3In step B1, the spur model is initialized with parameters. A spur model is constructed, and its parameters are initialized with full precision, i.e., an initial spur model is constructed. If convergence is not achieved, feedforward inference and loss are performed, and the first stage of training is executed: a loss function is set, and an optimization problem for the spur model is established, i.e., the loss function of the initial spur model is set, and a dynamic visual training optimization problem for the initial spur model is established. In step B2, the substitution gradient and full-precision parameters are calculated. The substitution gradient of the spur model is calculated, i.e., the dynamic visual training optimization problem is solved using the substitution gradient, and the second parameter update value of the initial spur model is obtained. The dynamic visual dataset is fed to the spur model, and the full-precision parameters are calculated, i.e., the spur model is obtained based on the second parameter update value and the dynamic visual dataset.

[0106] The model training method proposed in this invention effectively achieves memory compression, inference acceleration, and energy consumption reduction. Experimental results show that, without introducing additional hardware acceleration, the model training method maintains prediction accuracy while compressing model memory by 15.2 times, reducing computational complexity by 2-4 times, and reducing energy consumption by 15 times, demonstrating high technical and practical value. It can ensure the quantization of the entire inference process for a large class of spiking models, including spiking neural networks, and is compatible with low-computing-power hardware such as ASICs, FPGAs, and RISC-V chips, thus having a wide range of application scenarios. The consistency optimization method in this invention improves the reliability and consistency of spiking neural networks. Experiments show that it increases the reliability of a range of spiking neural networks by at least 20%, achieving effective consistency assurance, and ensuring high consistency of environmental change data collected by dynamic vision sensors after conversion into pulse sequence data.

[0107] like Figure 5 As shown, to achieve the above objectives, embodiments of the present invention provide a model training apparatus, comprising:

[0108] The first acquisition module 501 is used to acquire a pulse model trained based on a dynamic visual dataset, and a predefined truncation function; wherein the dynamic visual dataset is environmental change data collected by a dynamic visual sensor and recorded in the form of an asynchronous event stream;

[0109] The second acquisition module 502 is used to acquire an initial quantized pulse model based on the pulse model and the truncation function;

[0110] The third acquisition module 503 is used to acquire the confidence estimate of the initial quantized pulse model based on the first moment of the input variables of the initial quantized pulse model.

[0111] The fourth acquisition module 504 is used to update the initial quantized pulse model according to the confidence estimate and acquire the quantized pulse model; wherein the quantized pulse model is used to convert the continuous data collected by the dynamic vision sensor into pulse sequence data.

[0112] Optionally, in the aforementioned apparatus, the second acquisition module 502 includes:

[0113] The first acquisition submodule is used to acquire the first quantization parameter based on the truncation function and the first full-precision parameter of the impulse model;

[0114] The first processing submodule is used to construct a quantized pulse model to be trained based on the first quantization parameters;

[0115] The second acquisition submodule is used to acquire the first parameter update value of the quantized impulse model to be trained based on the first gradient; wherein, the first gradient is acquired based on the approximate derivative of the truncation function and the second gradient of the impulse model; the second gradient includes the alternative gradient of the impulse model;

[0116] The third acquisition submodule is used to adjust the quantized pulse model to be trained according to the second quantization parameter to acquire the initial quantized pulse model; wherein the second quantization parameter is acquired according to the update value of the first parameter and the truncation function.

[0117] Optionally, in the aforementioned apparatus, the third acquisition module 503 includes:

[0118] The fourth acquisition submodule is used to acquire the second moment of the variable before activation based on the first moment of the input variable;

[0119] The second processing submodule is used to process the second moment using an activation function and to transform the pre-activation variable into the post-activation variable.

[0120] The fifth acquisition submodule is used to acquire the third moment of the activated variable, wherein the third moment includes the confidence estimate.

[0121] Optionally, in the aforementioned apparatus, the fourth acquisition module 504 includes:

[0122] The third processing submodule is used to establish a consistency optimization problem based on the confidence estimate.

[0123] The sixth acquisition submodule is used to obtain the quantized pulse model by solving the consistency optimization problem.

[0124] Optionally, in the aforementioned apparatus, the sixth acquisition submodule includes:

[0125] The first acquisition unit is used to update the first full-precision parameters of the pulse model using the gradient descent method, and to acquire the second full-precision parameters.

[0126] The first processing unit is used to update the truncation threshold of the truncation function according to the second full-precision parameter;

[0127] The second acquisition unit is used to truncate the second full-precision parameter using the truncation threshold and the truncation function to acquire the third quantized parameter;

[0128] The third acquisition unit is used to acquire the quantized pulse model based on the third quantization parameter.

[0129] Optionally, in the aforementioned apparatus, the first acquisition module 501 includes:

[0130] The fourth processing subunit is used to construct an initial impulse model and to acquire the dynamic visual dataset;

[0131] The fifth processing subunit is used to set the loss function of the initial impulse model and establish the dynamic visual training optimization problem of the initial impulse model;

[0132] The seventh acquisition subunit is used to solve the dynamic visual training optimization problem by substituting gradients and obtain the second parameter update value of the initial impulse model;

[0133] The eighth acquisition subunit is used to acquire the pulse model based on the updated value of the second parameter and the dynamic visual dataset.

[0134] It should be noted that the apparatus provided in this embodiment of the invention can implement all the method steps implemented in the above method embodiment and can achieve the same technical effect. Therefore, the parts and beneficial effects that are the same as those in the method embodiment will not be described in detail here.

[0135] To achieve the above objectives, embodiments of the present invention provide a model training device, comprising: a processor, a memory, and a program or instructions stored in the memory and executable on the processor; wherein, when the processor executes the program or instructions, it implements the model training method as described above.

[0136] To achieve the above objectives, embodiments of the present invention provide a readable storage medium having a program or instructions stored thereon, wherein the program or instructions, when executed by a processor, implement the steps in the model training method described above.

[0137] To achieve the above objectives, embodiments of the present invention provide a computer program product, which includes computer instructions that, when executed by a processor, implement the steps of the model training method described above.

[0138] It should be further noted that the terminals described in this specification include, but are not limited to, smartphones, tablets, etc., and many of the functional components described are referred to as modules in order to emphasize the independence of their implementation.

[0139] In this embodiment of the invention, the module can be implemented in software so that it can be executed by various types of processors. For example, an identified executable code module may include one or more physical or logical blocks of computer instructions, which may be constructed as objects, procedures, or functions. Nevertheless, the executable code of the identified module does not need to be physically located together, but may include different instructions stored in different bits, which, when logically combined, constitute the module and achieve the module's intended purpose.

[0140] In practice, an executable code module can be a single instruction or many instructions, and can even be distributed across multiple different code segments, different programs, and across multiple memory devices. Similarly, operational data can be identified within the module and can be implemented in any suitable form and organized within any suitable data structure. This operational data can be collected as a single dataset or distributed across different locations (including different storage devices), and can exist, at least in part, solely as electronic signals within the system or network.

[0141] When a module can be implemented using software, considering the current level of hardware technology, modules that can be implemented in software can be implemented using hardware circuits by those skilled in the art to achieve the corresponding functions, without considering cost. These hardware circuits include conventional very-large-scale integrated circuits (VLSI) or gate arrays, as well as existing semiconductors such as logic chips and transistors, or other discrete components. Modules can also be implemented using programmable hardware devices, such as field-programmable gate arrays, programmable array logic, and programmable logic devices.

[0142] The exemplary embodiments described above are with reference to the accompanying drawings. Many different forms and embodiments are feasible without departing from the spirit and teachings of the invention. Therefore, the invention should not be construed as limiting the exemplary embodiments set forth herein. Rather, these exemplary embodiments are provided to make the invention complete and convey the scope of the invention to those skilled in the art. In these drawings, component dimensions and relative dimensions may be exaggerated for clarity. The terminology used herein is for the purpose of describing particular exemplary embodiments only and is not intended to be limiting. As used herein, unless clearly indicated otherwise, the singular forms “a,” “an,” and “the” are intended to include all such forms. It will be further understood that the terms “comprising” and / or “including”, when used in this specification, indicate the presence of the stated features, integers, steps, operations, components, and / or elements, but do not exclude the presence or addition of one or more other features, integers, steps, operations, components, and / or groups thereof. Unless otherwise indicated, when stated, a range of values ​​includes the upper and lower limits of the range and any subranges in between.

[0143] The above description represents the preferred embodiments of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A model training method, characterized in that, include: Obtain the impulse model trained based on the dynamic visual dataset, and a predefined truncation function; wherein the dynamic visual dataset is environmental change data collected by a dynamic visual sensor and recorded in the form of an asynchronous event stream; Based on the pulse model and the truncation function, an initial quantized pulse model is obtained; Based on the first moment of the input variables of the initial quantized pulse model, obtain the confidence estimate of the initial quantized pulse model; The initial quantized pulse model is updated based on the confidence estimate to obtain a quantized pulse model; wherein, the quantized pulse model is used to convert continuous data collected by the dynamic vision sensor into pulse sequence data.

2. The method according to claim 1, characterized in that, Based on the pulse model and the truncation function, an initial quantized pulse model is obtained, including: The first quantization parameter is obtained based on the truncation function and the first full-precision parameter of the pulse model; Construct a quantized pulse model to be trained based on the first quantization parameters; The first parameter update value of the quantized impulse model to be trained is obtained according to the first gradient; wherein, the first gradient is obtained according to the approximate derivative of the truncation function and the second gradient of the impulse model; the second gradient includes the alternative gradient of the impulse model; The initial quantized pulse model is obtained by adjusting the quantized pulse model to be trained according to the second quantization parameter; wherein the second quantization parameter is obtained based on the update value of the first parameter and the truncation function.

3. The method according to claim 1, characterized in that, Based on the first moment of the input variables of the initial quantized pulse model, obtain the confidence estimate of the initial quantized pulse model, including: The second moment of the variable before activation is obtained based on the first moment of the input variable; The second moment is processed using an activation function, and the variables before activation are transformed into variables after activation. Obtain the third moment of the activated variable, wherein the third moment includes the confidence estimate.

4. The method according to claim 1, characterized in that, The initial quantized pulse model is updated based on the confidence estimate to obtain the quantized pulse model, including: A consistency optimization problem is established based on the aforementioned credibility estimate; The quantized pulse model is obtained by solving the consistency optimization problem.

5. The method according to claim 4, characterized in that, The quantized pulse model is obtained by solving the consistency optimization problem, including: The first full-precision parameters of the impulse model are updated using gradient descent to obtain the second full-precision parameters. The truncation threshold of the truncation function is updated based on the second full-precision parameter; The second full-precision parameter is truncated using the truncation threshold and the truncation function to obtain the third quantized parameter; The quantized pulse model is obtained based on the third quantization parameter.

6. The method according to claim 1, characterized in that, Obtain the impulse model trained on the dynamic vision dataset, including: Construct an initial impulse model and obtain the dynamic visual dataset; Define the loss function of the initial impulse model and establish the dynamic visual training optimization problem of the initial impulse model; The second parameter update value of the initial impulse model is obtained by solving the dynamic visual training optimization problem by replacing the gradient. The pulse model is obtained based on the updated value of the second parameter and the dynamic visual dataset.

7. A model training device, characterized in that, include: The first acquisition module is used to acquire the impulse model trained based on the dynamic visual dataset, and a predefined truncation function; wherein, the dynamic visual dataset is environmental change data collected by a dynamic visual sensor and recorded in the form of an asynchronous event stream; The second acquisition module is used to acquire an initial quantized pulse model based on the pulse model and the truncation function; The third acquisition module is used to acquire the confidence estimate of the initial quantized pulse model based on the first moment of the input variables of the initial quantized pulse model. The fourth acquisition module is used to update the initial quantized pulse model based on the confidence estimate and acquire the quantized pulse model; wherein the quantized pulse model is used to convert the continuous data collected by the dynamic vision sensor into pulse sequence data.

8. A model training device, comprising: A processor, a memory, and a program or instructions stored in the memory and executable on the processor; characterized in that, when the processor executes the program or instructions, it implements the model training method as described in any one of claims 1-6.

9. A readable storage medium having a program or instructions stored thereon, characterized in that, When the program or instructions are executed by the processor, they implement the steps in the model training method as described in any one of claims 1-6.

10. A computer program product, characterized in that, It includes computer instructions that, when executed by a processor, implement the steps of the model training method as described in any one of claims 1-6.

Images