Model inference method, processing system and computing chip

Abstract

The application provides a model inference method, a processing system and a computing chip, and relates to the technical field of artificial intelligence. The method comprises: performing quantization processing on model parameter information of a target neural network model to obtain a quantization result; in a process of performing model inference based on the quantization result, determining and accumulating an error generated by the quantization processing to obtain a cumulative quantization error; and performing compensation on an output of the target neural network model based on the cumulative quantization error to obtain a model inference result. The embodiment of the application can improve the accuracy of the output of the quantized target neural network model and improve the reliability of the model inference process.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, specifically to a model reasoning method, a processing system, and a computing chip. Background Technology

[0002] With the rise and development of third-generation neural network technologies, such as spiking neural networks (SNNs), which use pulse timing for information encoding, they have significant advantages such as event-driven, high sparsity, and low power consumption. Therefore, they are widely used in edge intelligence, autonomous driving, and wearable devices.

[0003] When deploying neural network models to hardware devices, traditional solutions mostly adopt a one-way process of "quantization-deployment" without considering the errors caused by quantization. This results in low accuracy of the model output results when the quantized model is deployed on the hardware, and low reliability of the model inference process. Summary of the Invention

[0004] In view of this, the embodiments of this application aim to provide a model inference method, processing system and computing chip to solve the problems of low accuracy of quantized model output and low reliability of model inference process in the prior art.

[0005] In a first aspect, one embodiment of this application provides a model inference method, comprising: quantizing the model parameter information of a target neural network model to obtain a quantization result; determining and accumulating the error generated by the quantization process during model inference based on the quantization result to obtain a cumulative quantization error; and compensating the output of the target neural network model based on the cumulative quantization error to obtain a model inference result.

[0006] In conjunction with the first aspect, in some implementations of the first aspect, the target neural network model includes a spiking neural network model, and the model parameter information includes at least two of the following: weights, membrane potential, and time step.

[0007] In conjunction with the first aspect, in certain implementations of the first aspect, when the model parameter information includes membrane potential, the model parameter information of the target neural network model is quantized to obtain a quantization result, including: for each neuron in the target neural network model, obtaining the decay coefficient corresponding to the neuron; based on the decay coefficient, determining the membrane potential quantization bit width corresponding to the neuron, wherein the decay coefficient and the membrane potential quantization bit width are positively correlated; and based on the membrane potential quantization bit width, quantizing the membrane potential of the neuron to obtain a membrane potential quantization result, the quantization result including the membrane potential quantization result.

[0008] In conjunction with the first aspect, in certain implementations of the first aspect, when the model parameter information includes time steps, the model parameter information of the target neural network model is quantized to obtain a quantization result, including: for each neuron layer in the target neural network model, determining the impulse firing rate corresponding to the neuron layer; based on the impulse firing rate, determining the time step compression ratio corresponding to the neuron layer, wherein the impulse firing rate and the time step compression ratio are negatively correlated; and quantizing the time step of the input impulse of the neuron layer according to the time step compression ratio to obtain a time quantization result, wherein the quantization result includes the time quantization result.

[0009] In conjunction with the first aspect, in some implementations of the first aspect, when the model parameter information also includes membrane potential, the model parameter information of the target neural network model is quantized to obtain quantization results, including: for each neuron in the target neural network model, obtaining the decay coefficient corresponding to the neuron; determining the quantization bit width of the membrane potential corresponding to the neuron based on the time step compression ratio corresponding to the neuron layer and the decay coefficient corresponding to the neuron; and quantizing the membrane potential of the neuron based on the quantization bit width of the membrane potential to obtain the membrane potential quantization result, wherein the quantization result includes the membrane potential quantization result.

[0010] In conjunction with the first aspect, in certain implementations of the first aspect, when the model parameter information includes weights, the model parameter information of the target neural network model is quantized to obtain quantization results, including: for each weight in the target neural network model, determining the importance metric corresponding to the weight; based on the importance metric, determining the quantization precision corresponding to the weight, wherein the importance metric and the quantization precision are positively correlated; quantizing the weights according to the quantization precision to obtain weight quantization results, the quantization results including weight quantization results.

[0011] In conjunction with the first aspect, in certain implementations of the first aspect, during the model inference process based on the quantization results, the error generated by the quantization process is determined and accumulated to obtain the cumulative quantization error. This includes: for each neuron and its corresponding weight in the target neural network model, determining the weight quantization error based on the weight quantization result corresponding to the neuron; determining the membrane potential quantization error based on the membrane potential quantization result corresponding to the neuron at each time step of the model neuron inference process; and performing error accumulation processing based on the membrane potential quantization error and the weight quantization error to obtain the cumulative quantization error.

[0012] In conjunction with the first aspect, in some implementations of the first aspect, the output of the target neural network model is compensated based on the cumulative quantization error to obtain the model inference result, including: for each neuron in the target neural network model, if the cumulative quantization error corresponding to the neuron reaches a preset compensation threshold, the membrane potential corresponding to the neuron is increased by the target compensation potential so that the neuron outputs a compensation pulse to obtain the model inference result.

[0013] In conjunction with the first aspect, in some implementations of the first aspect, the target compensation potential is the potential corresponding to the pulse firing threshold, and the preset compensation threshold is less than the pulse firing threshold.

[0014] Secondly, one embodiment of this application provides a processing system for model inference, including at least one processing unit. The processing unit is configured corresponding to neurons in a target neural network model. The processing unit includes: a quantization storage module for storing quantization results, which are obtained by quantizing the model parameter information of the target neural network model; an inference calculation module connected to the quantization storage module for performing model inference based on the quantization results to obtain model inference results; an error accumulation module connected to the inference calculation module for determining and accumulating the error generated by the quantization process during model inference based on the quantization results, obtaining and storing the accumulated quantization error; and an error compensation module connected to both the error accumulation module and the inference calculation module for compensating the output of the target neural network model based on the accumulated quantization error.

[0015] Thirdly, one embodiment of this application provides a computing chip that integrates a processing system for model inference as described in claim 10.

[0016] Fourthly, one embodiment of this application provides a model inference apparatus, comprising: a quantization module for quantizing the model parameter information of a target neural network model to obtain a quantization result; an accumulation module for determining and accumulating the error generated by the quantization process during model inference based on the quantization result to obtain a cumulative quantization error; and a compensation module for compensating the output of the target neural network model based on the cumulative quantization error to obtain a model inference result.

[0017] Fifthly, one embodiment of this application provides a computer-readable storage medium storing a computer program for performing the model inference method described in the first aspect.

[0018] In a sixth aspect, one embodiment of this application provides an electronic device, the electronic device comprising: a processor; a memory for storing processor-executable instructions; the processor being configured to execute the model inference method described in the first aspect.

[0019] In a seventh aspect, one embodiment of this application provides a computer program product including instructions that, when executed on an electronic device, cause the electronic device to implement the model reasoning method described in the first aspect.

[0020] In this application, by determining and accumulating the error generated by the quantization process in real time during the model inference process based on the quantization results of the model parameters, and by compensating the model output in real time based on the accumulated quantization error, an active monitoring and compensation mechanism for quantization error is introduced on the forward inference path. This effectively suppresses the propagation and amplification of quantization error during the inference process, corrects the output deviation caused by error accumulation, thereby improving the accuracy of the output of the target neural network model after quantization and enhancing the reliability of the model inference process. Attached Figure Description

[0021] The above and other objects, features, and advantages of this application will become more apparent from the more detailed description of the embodiments of this application in conjunction with the accompanying drawings. The drawings are provided to further illustrate the embodiments of this application and form part of the specification. They are used together with the embodiments of this application to explain this application and do not constitute a limitation thereof. In the drawings, the same reference numerals generally represent the same components or steps.

[0022] Figure 1 The diagram shown is a flowchart of a model reasoning method provided in an embodiment of this application.

[0023] Figure 2 The diagram shown is a flowchart of a model reasoning method provided in another embodiment of this application.

[0024] Figure 3 The diagram shown is a schematic representation of a processing system for model reasoning provided in an embodiment of this application.

[0025] Figure 4 The diagram shown is a schematic diagram of the structure of a model inference device provided in an embodiment of this application.

[0026] Figure 5 The diagram shown is a structural schematic of an electronic device provided in an embodiment of this application. Detailed Implementation

[0027] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0028] Furthermore, to better illustrate this application, numerous specific details are provided in the following detailed embodiments. Those skilled in the art should understand that this application can be implemented even without certain specific details. In some instances, methods and means well-known to those skilled in the art have not been described in detail in order to highlight the main points of this application.

[0029] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.

[0030] Furthermore, the terms “first,” “second,” “third,” and “fourth” are used only for distinguishing descriptions and should not be interpreted as indicating or implying relative importance.

[0031] With the rise and rapid development of third-generation neural network technologies, such as the spiking neural networks (SNNs) that have attracted much attention in recent years, they have gradually become a research hotspot due to their unique information processing methods. Specifically, this type of neural network uses pulse timing for information encoding, which endows it with a series of significant advantages, such as an event-driven working mechanism, a highly sparsity computing model, and low power consumption. These advantages have enabled spiking neural networks to be widely used in many cutting-edge fields, especially in edge intelligence, autonomous driving, and wearable devices, where their potential has been fully explored and demonstrated. However, as SNN model structures become increasingly complex (e.g., deep spiking convolutional networks and large-scale spiking Transformer architectures), the number of model parameters and computational overhead increases dramatically, posing a severe challenge to their deployment on resource-constrained neuromorphic chips.

[0032] Specifically, in deploying complex neural network models to actual hardware devices, traditional solutions typically employ a one-way "quantization-deployment" process. The core idea of ​​this approach is to reduce computational complexity and storage requirements by quantizing model parameters, thereby adapting to the resource constraints of the hardware. However, this process has a critical problem: it does not fully consider the quantization errors that may be introduced during the quantization process. While these errors may seem negligible in theory, they can significantly impact model performance in practical applications. Specifically, when the quantized model is deployed to hardware devices for inference tasks, the accuracy of the model's output often falls short of expectations due to the presence of quantization errors. For example, for SNN models, the Integrate-and-Fire mechanism has error propagation characteristics. This means that small errors generated during quantization are continuously accumulated and propagated during the SNN model's inference process, and are amplified over time, leading to a shift in the pulse firing timing and a decrease in the accuracy of the model's output. Furthermore, these quantization errors can significantly reduce the reliability of the model's inference process, thus affecting the overall system performance and stability.

[0033] To address the aforementioned technical problems, this application provides a model inference method, processing system, and computing chip. The model inference method includes: quantizing the model parameter information of a target neural network model to obtain a quantization result; determining and accumulating the error generated by the quantization process during model inference based on the quantization result to obtain a cumulative quantization error; and compensating the output of the target neural network model based on the cumulative quantization error to obtain a model inference result. In this application, by determining and accumulating the error generated by the quantization process in real time during model inference based on the model parameter quantization result, and compensating the model output in real time based on the obtained cumulative quantization error, an active monitoring and compensation mechanism for quantization error is introduced into the forward inference path. This effectively suppresses the propagation and amplification of quantization error during the inference process, corrects output deviations caused by error accumulation, thereby improving the accuracy of the output of the quantized target neural network model and enhancing the reliability of the model inference process.

[0034] Furthermore, the technical solutions of this application can be applied to any scenario where a neural network model is deployed on a terminal to perform computations using the neural network model. For example, in the field of edge intelligence, it can be used for low-power event camera vision processing in scenarios such as intelligent monitoring and autonomous driving; in the field of robotics, it can be used for real-time neuromorphic perception and decision-making, supporting tasks such as Simultaneous Localization and Mapping (SLAM), obstacle avoidance, and grasping; in the field of the Internet of Things, it can be used for edge intelligence for lifelong learning, enabling device adaptation and personalization; and in the field of brain-computer interfaces, it can be used for low-latency neural signal processing, supporting real-time decoding and feedback.

[0035] The following is combined with Figures 1 to 2 The model reasoning method provided in this application is described in detail.

[0036] Figure 1 The diagram shown is a flowchart illustrating a model inference method provided in an embodiment of this application. This method can be executed by an electronic device that deploys a target neural network model, a dedicated hardware accelerator, or a combination thereof. This electronic device may include smart devices such as computers, mobile phones, vehicles, and wearable devices.

[0037] like Figure 1 As shown, the model inference method may include the following steps.

[0038] S110 quantizes the model parameter information of the target neural network model to obtain the quantization result.

[0039] In some examples, the target neural network model can be any type of neural network model deployed to edge hardware electronic devices, such as a third-generation neural network model using an integral-exponential mechanism. Model parameter information can refer to various quantifiable parameter information that the target neural network model relies on when performing inference computations, such as the weights and biases connecting different neurons or network layers (i.e., neuron layers), the state variables inside neurons (such as membrane potentials in spiking neural networks), and the time steps involved in processing temporal information, etc. The specific content it contains may depend on the type of target neural network model.

[0040] In addition, quantization can refer to the process of converting model parameter information from a high-precision data representation to a low-precision data representation, thereby reducing the model's demand for storage and computing resources. For example, quantization may include, but is not limited to: converting floating-point numbers to fixed-point numbers, converting high-bit integers to low-bit integers, or downsampling and compressing time series data. The quantization result can refer to the low-precision data obtained after quantization of model parameter information and / or its related quantization parameters (such as scaling factors, compression ratios, etc.).

[0041] For example, when deploying a target neural network model to a target hardware electronic device and performing hardware inference, the model parameter information to be quantized (e.g., quantizing only weights, or quantizing both weights and membrane potential) and the target precision (e.g., INT8) can be determined based on the support capabilities of the target hardware electronic device, the application's requirements for inference speed and power consumption, and / or relevant information about the target neural network model itself. For the selected model parameter information, a uniform or non-uniform quantization function can be used to map the original high-precision model parameter information to the integer domain of the target precision.

[0042] In some embodiments, the target neural network model described above may include a spiking neural network model. Accordingly, the model parameter information may include at least two of the following: weights, membrane potential, and time step.

[0043] Since traditional neural network model quantization strategies only perform static compression on weights, while the information encoding of SNN models depends on the dynamic coupling of three dimensions: weights, membrane potential, and time step, if quantization is performed directly according to the traditional weight quantization strategy, ignoring the time accumulation characteristics of membrane potential and the sparsity of time step, quantization errors will accumulate in the time dimension, destroying the temporal encoding structure of SNN.

[0044] Based on this, in this embodiment, at least two of the weights, membrane potential, and time step can be quantized for the spiking neural network model to effectively handle the quantization error that is unique to the SNN model and accumulates in the spatiotemporal dimension.

[0045] Specifically, weight quantization can be used to quantify synaptic connection strength; membrane potential quantization can be used to quantify simulated state variables within neurons; and time step quantization can be used to compress the time step of the corresponding input pulse sequence in the neuron layer. In some embodiments, all three can be quantized simultaneously to construct a weight-membrane potential-time (WUT) three-domain coupled quantization model. In other embodiments, depending on specific hardware resources and accuracy requirements, the quantization and compensation process of this embodiment can be implemented only for two of them (such as weights and membrane potential, weights and time steps, and membrane potential and time steps), which can also achieve the technical effect of suppressing the accumulation of errors in specific dimensions.

[0046] S120, during the model inference process based on the quantization results, the error generated by the quantization process is determined and accumulated to obtain the cumulative quantization error.

[0047] In some examples, the error generated by quantization can refer to numerical representation errors caused by the reduced precision of the quantized data. For example, it may include rounding and / or truncation errors caused by quantization. Rounding errors can be caused by discretizing continuous values ​​to the nearest quantization level, and truncation errors can be caused by pruning out-of-range values. Specifically, this error can be the difference between the original high-precision parameter values ​​corresponding to the model parameter information and the quantized parameter values.

[0048] For example, one or more error accumulators can be maintained in parallel outside the main process of model inference computation. Whenever a quantized parameter (such as a weight) is used in the inference computation or an intermediate result is generated (such as membrane potential quantization), the quantization error introduced by that step is calculated synchronously. For instance, in convolution computation, when an INT8 quantized weight is read from memory for multiplication, its corresponding original FP32 weight can be read synchronously, and the difference between the two yields the error introduced by that weight in this use. Subsequently, based on the role of this error in the current computation (e.g., which input data the weight is multiplied by), it is accumulated into the error accumulator corresponding to the affected feature map or neuron. This process continues at each step of inference, so that the values ​​in the error accumulator dynamically reflect the propagation and accumulation of the current quantization error, resulting in the accumulated quantization error.

[0049] S130, based on the accumulated quantization error, compensates for the output of the target neural network model to obtain the model inference result.

[0050] In some examples, compensation can be a forward, proactive correction operation on the output of the target neural network model to fully or partially offset the negative impact of accumulated quantization error on the final output. Furthermore, the object of compensation (i.e., the output of the target neural network model) can include the final output of the target neural network model, or intermediate outputs of network layers or neurons within the network (such as the spiking outputs of neurons in a spiking neural network model). The timing and intensity of compensation can be determined based on the accumulated quantization error.

[0051] For example, one or more compensation thresholds can be set for network layers or neurons that require compensation. During model inference, the accumulated quantization error stored in the error accumulator is compared with these compensation thresholds in real time or periodically. The compensation mechanism is triggered when the absolute value or the value of a specific component of the accumulated quantization error reaches or exceeds the compensation threshold. The specific form of compensation is varied. For example, for networks with continuous output values ​​(such as CNNs), a correction amount related to the accumulated quantization error can be directly added to the final output score; for networks with discrete events or impulses as outputs (such as SNNs), a compensation signal (such as increasing the membrane potential) can be injected into the internal state of the neuron to induce or inhibit an event output or impulse firing. After compensation is performed, the corresponding accumulated quantization error value can be selectively cleared or partially reduced.

[0052] In this embodiment, by determining and accumulating the error generated by the quantization process in real time during the model inference process based on the quantization results of the model parameters, and by compensating the model output in real time based on the accumulated quantization error, an active monitoring and compensation mechanism for quantization error is introduced on the forward inference path. This effectively suppresses the propagation and amplification of quantization error during the inference process, corrects the output deviation caused by error accumulation, thereby improving the accuracy of the output of the target neural network model after quantization and enhancing the reliability of the model inference process.

[0053] Furthermore, when the target neural network model is a spiking neural network model, traditional quantization strategies typically use fixed bit width or precision for quantization, without considering the differences in membrane potential decay coefficients between different neurons. For example, long-term memory neurons are sensitive to quantization errors and require high precision, while fast decay neurons allow for low precision, which may lead to waste of hardware resources or loss of key information, making it difficult to balance precision and compression rate.

[0054] Therefore, to address the aforementioned issues, embodiments of this application employ different adaptive quantization strategies for different model parameter information. For example, for membrane potential, a dynamic bit width allocation strategy based on the attenuation coefficient can be used for quantization; for time steps, an adaptive compression strategy based on pulse density can be used for quantization; and for weights, an importance-based hierarchical quantization strategy can be used for quantization.

[0055] The adaptive quantization strategy used for each model parameter is described in detail below.

[0056] In some embodiments, when the model parameter information includes membrane potential, step S110 may specifically include: for each neuron in the target neural network model, obtaining the attenuation coefficient corresponding to the neuron; determining the membrane potential quantization bit width corresponding to the neuron based on the attenuation coefficient, wherein the attenuation coefficient and the membrane potential quantization bit width are positively correlated; and quantizing the membrane potential of the neuron based on the membrane potential quantization bit width to obtain the membrane potential quantization result, wherein the quantization result includes the membrane potential quantization result.

[0057] In some examples, for SNN models, the neurons employ a Leaky Integrate-and-Fire (LIF) mechanism, with a decay coefficient... This reflects the leakage rate of the neuronal membrane potential and determines its memory duration. Attenuation coefficient. A larger value (i.e., closer to 1) indicates a stronger neuronal memory capacity (i.e., slower decay). Small quantization errors can be amplified over long integration times, making the neuron more sensitive to errors and requiring higher quantization precision (i.e., a wider membrane potential quantization bit width). Conversely, a smaller value indicates a slower decay coefficient. The smaller the value, the faster the neuron state updates (i.e., fast decay), and therefore it is less sensitive to historical errors and can tolerate lower quantization precision (i.e., smaller membrane potential quantization bit width).

[0058] For example, a piecewise function can be used to implement the attenuation coefficient. Quantization bit width to membrane potential The mapping. For example, if , can be set ;like , can be set ;like , can be set The bit width is quantized based on the membrane potential corresponding to each neuron. The quantization step size can be calculated using the following formula (1). .

[0059] in, The pulse firing threshold is set based on this quantization step size. The membrane potential of each neuron can then be calculated using the following formula (2). Quantify it.

[0060] in, membrane potential The corresponding quantization value, that is, the membrane potential quantization result.

[0061] It should be noted that the above mapping relationship between the attenuation coefficient and the bit width is only an example. In actual applications, it can be adjusted according to model statistics and hardware constraints, such as using a 5 / 7 / 9-bit bit width or other segmentation rules. As long as the principle of "positive correlation between attenuation coefficient and membrane potential quantization bit width" is followed, it falls within the protection scope of this application.

[0062] In this way, by quantifying the membrane potential of error-sensitive "long-term memory" neurons with high precision, and quantifying the membrane potential of error-insensitive "rapid decay" neurons with lower precision, hardware resources are saved, thereby ensuring that, under the condition of limited hardware resources, the precision is allocated to the most critical neurons to record their membrane potential state.

[0063] In other embodiments, when the model parameter information includes time steps, step S110 may specifically include: determining the pulse firing rate corresponding to each neuron layer in the target neural network model; determining the time step compression ratio corresponding to the neuron layer based on the pulse firing rate, wherein the pulse firing rate and the time step compression ratio are negatively correlated; quantizing the time step of the input pulses of the neuron layer according to the time step compression ratio to obtain a time quantization result, wherein the quantization result includes the time quantization result.

[0064] In some examples, the pulse firing rate The pulse firing rate characterizes the activity level of the neuronal layer over time. The number of input pulses can be determined based on the ratio of the number of input pulses to the number of original time steps for the corresponding neuron layer. For example, if the input pulse sequence is [1, 1, 0, 1], then the number of input pulses is 3 and the number of original time steps is 4. .in addition, The lower the value, the more silent the neuron layer is, indicating a large number of idle cycles. For such sparse layers, a high proportion of time step compression can be applied, that is, a higher time step compression ratio can be used to skip idle cycles without pulses. Conversely, for dense layers, the original time steps should be maintained or only a lower proportion of time step compression should be applied (i.e., a lower time step compression ratio) to preserve temporal information as much as possible.

[0065] For example, a piecewise function can be used to implement the change from the pulse firing rate. Mapping to the compression ratio at each time step. For example, if... Then the time step compression ratio can be set to 8:1, which means the original time step compression ratio can be reduced to 8:1. Compress to ;like Then the time step compression ratio can be set to 4:1, which means the original time step compression ratio can be reduced to 4:1. Compress to ;like Then the time step compression ratio can be set to 2:1, which means the original time step compression ratio can be reduced to 2:1. Compress to ;like Then the time step compression ratio can be set to 1:1, that is, the original time step compression ratio is not adjusted. Compress while preserving the original time step. The specific values ​​of the mapping relationship between the pulse firing rate and the time step compression ratio can be adjusted according to the actual network and performance objectives. As long as the principle of "negative correlation between pulse firing rate and time step compression ratio" is followed, they all fall within the scope of protection of this application.

[0066] After determining the time step compression ratio, the original time steps can be quantized according to the time step compression ratio. For example, if the time step compression ratio is 2:1 and the original number of time steps is 4, the original time steps can be compressed to 2, thus obtaining a time quantization result of 2 time steps. Then, the calculation is performed in the neuron layer according to the pulse sequence of 2 time steps.

[0067] Additionally, it should be noted that before the output pulse sequence of the current neuron layer is used as the input pulse sequence to the next neuron layer, the pulse sequence after time step compression can be reconstructed to restore the pulse sequence of the original time step. The reconstruction methods include, but are not limited to, linear interpolation reconstruction.

[0068] Thus, by employing the aforementioned feature of dynamically determining the time step compression ratio based on the pulse firing rate, this embodiment can directly reduce the number of clock cycles required for the model to process during hardware inference. Skipping idle cycles for sparse layers significantly reduces the model's dynamic power consumption and inference latency. Furthermore, due to the existence of subsequent compensation mechanisms, errors that may be introduced by time step compression can be corrected to a certain extent.

[0069] Furthermore, when time-step quantization and membrane potential quantization coexist, there is a coupling relationship between the two. Time-step compression is equivalent to concentrating the input effects of multiple original steps into fewer hardware steps, which may lead to an increase in the range of membrane potential variations. If the membrane potential quantization bit width remains unchanged, new quantization errors or saturation may be introduced due to insufficient dynamic range.

[0070] To address the aforementioned issues, embodiments of this application also provide a coupling quantization strategy. Based on this, in other embodiments, when the model parameter information also includes membrane potential, the model parameter information of the target neural network model is quantized to obtain a quantization result, including: for each neuron in the target neural network model, obtaining the decay coefficient corresponding to the neuron; determining the quantization bit width of the membrane potential corresponding to the neuron based on the time step compression ratio corresponding to the neuron layer and the decay coefficient corresponding to the neuron; and quantizing the membrane potential of the neuron based on the quantization bit width to obtain a membrane potential quantization result, the quantization result including the membrane potential quantization result.

[0071] In some examples, the target neural network model (such as an SNN model) may include at least one neuron layer, and a neuron layer may include at least one neuron.

[0072] For example, when the model parameter information includes both membrane potential and time step, the time step compression ratio of the neuron layer to which each neuron is located and its own decay coefficient can be considered when determining the quantization bit width of the membrane potential.

[0073] In some specific examples, for each neuron, in determining its decay coefficient and the time step compression ratio of its neuron layer Then, the quantization bit width of the membrane potential corresponding to the neuron can be determined according to the following formula (3). .

[0074] in, As the reference width, It is the equivalent cumulative factor reflecting the dynamic range expansion factor. This is a safety margin. Additionally, the equivalent cumulative factor can be determined using the following formula (4). .

[0075] In addition, the reference width can be determined according to the following formula (5). .

[0076] in, This represents the maximum input weight or number of pulses per step. The minimum precision (decimal step size) is the target.

[0077] In addition, since the hardware actually uses fixed-point numbers... The format (also known as Q format) is used to represent the bit width, and the total bit width or the number of decimal places is usually fixed (e.g., keeping the number of decimal places n fixed, i.e., ensuring...). The integer part m remains unchanged. (Dynamically expandable), therefore, in engineering implementation, different... The recommended quantization bit width for membrane potential under the combination is used to form a parameter mapping table, which is then used during membrane potential quantization processing based on the determined bit width. and By looking up the table, the Q format of the final membrane potential quantization bit width can be determined.

[0078] For example, suppose =8, =0.0625, =1, =9. The parameter mapping table shown in Table 1 below can be calculated using the above formula.

[0079] Table 1 As shown in Table 1 above, when =0.99 and When =16, relative to the original time step ( =1) The bit width needs to be increased by 5 bits; but the same If If the value is 0.90, then the bit width only needs to be increased by 2 bits. Therefore, it can be seen that... The closer it is to 1 (slow decay), the stronger the cumulative effect of time compression, and the larger the required bit width increment.

[0080] The quantization bit width of the membrane potential was determined through comprehensive analysis. Then, the membrane potential of the neuron can be quantified according to the aforementioned formulas (1) and (2), which will not be elaborated here.

[0081] In this way, by comprehensively determining the quantization bit width of the membrane potential based on the time step compression ratio corresponding to the neuron layer where the neuron is located and the decay coefficient corresponding to the neuron, the quantization accuracy of the membrane potential can adaptively match the equivalent input signal accumulation effect caused by time step compression. This ensures that while pursuing a high compression ratio to reduce computational latency and power consumption, the representation of the membrane potential state still maintains sufficient accuracy, thereby improving the robustness and overall accuracy of the joint quantization strategy of the time domain and the membrane potential domain.

[0082] In addition to the quantization processing in the time domain and membrane potential domain mentioned above, quantization processing can also be performed in the weight domain to achieve joint quantization processing across the three domains. Based on this, in some other embodiments, when the model parameter information includes weights, step S110 may specifically include: for each weight in the target neural network model, determining an importance metric corresponding to the weight; determining the quantization precision corresponding to the weight based on the importance metric, wherein the importance metric and the quantization precision are positively correlated; and quantizing the weights according to the quantization precision to obtain the weight quantization result, the quantization result including the weight quantization result.

[0083] In some examples, weights can be the weight values ​​of synapses connecting neurons in the model. Importance metrics can be measures used to evaluate the contribution of weights to the final model output. This importance metric can be determined based on the weight magnitude and the weight gradient norm during model training, for example, by using the following formula (6) to determine the weights in the weight tensor corresponding to each neuron layer. Corresponding importance metric .

[0084] in, For loss function, For the loss function L, the weights The partial derivatives of the weights. Additionally, it should be noted that the method for calculating importance metrics is not limited to the examples above; any metric that reflects the contribution of weights to model performance (such as sensitivity analysis based on the Hessian matrix) can be applied here.

[0085] For example, a piecewise function can be used to map importance metrics to quantization precision. Specifically, after calculating the importance metric for each weight in the weight tensor corresponding to the neuron layer, the maximum importance metric can be determined. And set an importance threshold ratio. (e.g., 0.3). If This indicates that the weight is highly important and should be quantized with high precision (e.g., the quantization precision should be set to INT8); if If the importance of a weight is low, then low-precision quantization (e.g., setting the quantization precision to INT4) can be used. For high-precision quantization methods, such as INT8, 256 quantization levels can be included, covering 99% of the weight distribution. For low-precision quantization methods, such as INT4, 16 quantization levels can be included, and K-means clustering is used to generate a non-uniform codebook to better fit the weight distribution. Furthermore, the mapping relationship between the above importance metric and quantization precision is only an example; in practical applications, it can be adjusted as needed, such as using other quantization precisions or other segmentation rules. As long as the principle of "positive correlation between importance metric and quantization precision" is followed, it falls within the scope of protection of this application.

[0086] In this way, by employing an importance-aware quantization strategy to quantize the weights, high-precision quantization is used for weights with high importance to preserve key features, while low-precision quantization is used for weights with low importance to compress the model size. This allows for the overall reduction in model size while prioritizing the protection of key feature information that significantly impacts the model output. This enables higher model accuracy compared to uniform quantization weights under the same average bit width, or a higher model compression rate under the same accuracy requirements.

[0087] It should be noted that the embodiments of this application may include the above-described quantization processing schemes for any one or any combination of weights, membrane potentials and time steps, and are not limited herein.

[0088] Furthermore, based on the implementation of joint quantization across three domains, the determination and accumulation of quantization errors during model inference can be achieved in the following manner. Accordingly, in some embodiments, step S120 may specifically include: for each neuron and its corresponding weight in the target neural network model, determining the weight quantization error based on the weight quantization result corresponding to the neuron; determining the membrane potential quantization error at each time step of the model neuron inference process based on the membrane potential quantization result corresponding to the neuron; and performing error accumulation processing based on the membrane potential quantization error and the weight quantization error to obtain the cumulative quantization error.

[0089] In some examples, determining the weight quantization error could involve calculating the difference between the high-precision original value and the quantized value of the weights before they are loaded or used. Similarly, determining the membrane potential quantization error could involve calculating the difference between the updated high-precision original value and the quantized value of the membrane potential at each time step when the neuron's membrane potential is updated.

[0090] For example, after determining the weight quantization error and the membrane potential quantization error, the total quantization error generated by the quantization process can be determined comprehensively based on the weight quantization error and the membrane potential quantization error, and then the total quantization error can be accumulated based on the time step. For example, the total quantization error corresponding to each time step can be determined comprehensively according to the following formula (7). .

[0091] in, The input pulse from the previous time step. For weight quantization error, For membrane potential quantization error, For weight compensation coefficient, This is the membrane potential compensation coefficient. and It can be set according to actual needs. The total quantified error at the previous time step... Based on this, the total quantization error at the current time step is accumulated. The cumulative quantization error can then be obtained. .

[0092] In this way, by calculating the quantization error in real time after weight quantization and membrane potential quantization at each time step, and accumulating these quantization errors by time step and network connection, the dynamic propagation and accumulation process of quantization error in time and space (across neurons) can be accurately tracked, providing a reliable and real-time data foundation for subsequent accurate compensation based on accumulated error.

[0093] Furthermore, in order to transform the accumulated quantization error into an effective compensation action, embodiments of this application can perform error compensation based on a forward compensation mechanism and by outputting compensation pulses. Based on this, in some embodiments, step S130 may specifically include: for each neuron in the target neural network model, if the accumulated quantization error corresponding to the neuron reaches a preset compensation threshold, then the membrane potential corresponding to the neuron is increased by a target compensation potential, so that the neuron outputs a compensation pulse to obtain the model inference result.

[0094] In some examples, the preset compensation threshold can be a pre-set threshold for triggering error compensation operations. This preset compensation threshold can be set according to the actual application scenario of the model, and its specific value can be related to the neuron's pulse firing threshold. Additionally, the target compensation potential can be a potential that enables the neuron to output at least one compensation pulse, such as the potential corresponding to the pulse firing threshold. The pulse firing threshold can be the potential threshold for the neuron to output a pulse; that is, when the neuron's membrane potential reaches this pulse firing threshold, it can output a pulse.

[0095] For example, at each time step of the model inference process, the cumulative quantization error of each neuron in the model can be calculated. With preset compensation threshold If a comparison is made, Then, by injecting the target compensation potential into the membrane potential accumulator, the membrane potential corresponding to the neuron is increased by the target compensation potential, prompting the neuron to output the compensation pulse in addition to the original output. It should be noted that if the neuron is the neuron of the last layer of the model, its original output pulse and compensation pulse can be used as the final output of the model, that is, as the model inference result; otherwise, the compensation pulse can be input to the neurons of the next layer along with the original output pulse, as the input pulse of the neurons of the next layer.

[0096] Specifically, compensation pulse The distribution logic can be shown in the following formula (8).

[0097] in, It is a step function. To accumulate quantization error, This is the preset compensation threshold.

[0098] In addition, after outputting the compensation pulse, the accumulated quantization error can be... Reset to This allows the quantization error to be accumulated in subsequent time steps.

[0099] In this way, by setting a preset compensation threshold, compensation is only triggered when the accumulated quantization error is large enough to affect the pulse firing decision, avoiding unnecessary frequent operations. In addition, converting compensation into the action of injecting a fixed potential into the membrane potential is a hardware-friendly implementation method that directly corrects the internal state of the neuron and simulates pulses that may be delayed or lost due to quantization errors, thereby effectively preventing pulse timing deviations.

[0100] Furthermore, to ensure the stability and effectiveness of the compensation action, the target compensation potential and the preset compensation threshold need to be set appropriately. Therefore, in some embodiments, the target compensation potential is the potential corresponding to the pulse firing threshold, and the preset compensation threshold is less than the pulse firing threshold.

[0101] In some examples, the value of the target compensation potential can be equal to the pulse firing threshold; that is, for the pulse firing threshold... ,like This can increase the membrane potential corresponding to the neuron. Thus, when calculating the output pulse corresponding to the neuron at the current time step, even if the original membrane potential of the neuron fails to reach the pulse firing threshold due to accumulated quantization errors, the membrane potential of the neuron has increased due to the compensation action. Therefore, a pulse can still be output to correct the output.

[0102] In other examples, the preset compensation threshold can be set to any value less than the pulse firing threshold, for example, 0.25 times the pulse firing threshold. =1, then On the one hand, this setting ensures the sensitivity of the compensation mechanism, so that when the membrane potential falls slightly below the discharge threshold due to accumulated quantization errors (e.g., At that time, although the pulse firing condition was not met, the accumulated error... It may have reached or exceeded 0.25, at which point compensation is triggered, and injection occurs. The target compensation potential enables the membrane potential to jump from 0.99 to 1.99, thereby immediately meeting the pulse delivery condition, outputting a compensation pulse, and resetting the membrane potential, effectively preventing the situation where the pulse cannot be delivered due to being "just short". On the other hand, this setting also avoids triggering compensation when the error is very small, which could lead to overcompensation or waste of resources.

[0103] It should be noted that the ratio between the preset compensation threshold and the pulse firing threshold is not limited to 1 / 4 as described in the example above. It can also be adjusted according to the network error distribution statistics and the actual application scenario of the model, for example, between 1 / 10 and 1 / 2.

[0104] In this way, by setting the target compensation potential to the potential corresponding to the pulse firing threshold and setting the preset compensation threshold to be less than the pulse firing threshold, the stability and effectiveness of the compensation action can be ensured.

[0105] Based on the above embodiments, the SNN model will be used as an example below, combined with... Figure 2 To better illustrate the model reasoning method provided in this application, a specific example will be given.

[0106] Figure 2 The diagram shown is a flowchart of a model reasoning method provided in another embodiment of this application.

[0107] like Figure 2 As shown, the overall process of this model inference method can be divided into four main stages: model analysis, three-domain quantization, error compensation, and hardware inference. Each main stage is described in detail below.

[0108] During the model analysis phase, the original SNN model can be imported to analyze the network topology and extract weights, membrane potentials, time steps, and neuron dynamics parameters. Specifically, this can serve as the entry point, receiving a floating-point precision (FP32) SNN model. Its core principle is to traverse the computation graph, identifying LIF neuron nodes and synaptic connections. Key extracted parameters may include: synaptic weight tensor W, neuron membrane potential decay coefficient α, impulse firing threshold θ, original time step T, and gradient information from the training process (used for weight importance assessment).

[0109] In the three-domain quantization stage, the weights, membrane potentials and time steps can be jointly quantized separately. The specific processing procedure can be referred to the aforementioned related embodiments, and will not be repeated here.

[0110] During the error compensation phase, the cumulative quantization error can be calculated based on the quantization error obtained from the weight domain and the membrane potential domain, and compared with the preset compensation threshold. Then, when the preset compensation threshold is reached, a compensation pulse is generated to inject the corresponding compensation potential into the membrane potential of the LIF neuron.

[0111] During the hardware inference phase, a time controller can be configured according to the compressed time step to control the inference process of the LIF neuron. The current accumulated potential state of the LIF neuron membrane potential is compared with the pulse firing threshold. When the membrane potential reaches the pulse firing threshold, a pulse is output.

[0112] It should be noted that for any parts not covered above, please refer to the aforementioned related embodiments, which will not be elaborated upon further here.

[0113] In this embodiment, through three-domain joint optimization, the accuracy loss can be controlled to within 1% under mixed high and low precision. Furthermore, by using an error compensation mechanism to convert quantization errors into compensation pulses, the accumulation of errors in the time dimension is effectively suppressed, preventing pulse loss. In addition, time-domain quantization utilizes pulse sparsity, reducing the number of on-chip memory accesses and significantly lowering memory bandwidth usage and dynamic power consumption.

[0114] Based on the above embodiments, the model reasoning method provided in this application will be illustrated below using an industrial edge defect detection scenario as an example.

[0115] For example, in the scenario of surface defect detection in industrial intelligent manufacturing, the aforementioned target neural network model can be an industrial detection model. This model can be deployed on a lightweight neuromorphic edge chip with an SRAM storage capacity of ≤256KB, a single-frame inference power consumption limit of 500mW, and no external DDR cache, representing an extremely resource-constrained hardware environment. Specifically, this model can be an SNN model, such as a 10-layer deep SNN model using LIF neurons. In this model structure, shallow neurons primarily respond to background texture noise, exhibiting extremely high pulse sparsity; deep neurons continuously accumulate weak defect features, and the membrane potential possesses long-term accumulation characteristics. Traditional model inference methods in this scenario will encounter problems such as frame-by-frame accumulation of deep membrane potential errors, defect pulse emission offset, and missed detection of microcracks.

[0116] Based on this, in the above application scenarios, the model inference method provided in this application may include, for example,: quantizing the model parameter information of the industrial detection model to obtain a quantization result; determining and accumulating the error generated by the quantization process in the process of using the industrial detection model to perform object detection on the acquired target image based on the quantization result, to obtain a cumulative quantization error; and compensating the output of the industrial detection model based on the cumulative quantization error to obtain an object detection result.

[0117] In some examples, the acquisition device used to acquire the target image can be an industrial event camera, the target image can be an image of a metal component, and the detection object of the industrial inspection model can be micro-defects such as cracks, scratches, and pinholes on the surface of the metal component in the target image. For example, the industrial inspection model generates a pulse event when pixel brightness changes abruptly in the target image, outputting a binary pulse sequence, which can be the object detection result output by the industrial inspection model.

[0118] Specifically, in the model analysis phase, parameters can be analyzed and extracted from the industrial detection model. Extracted parameters may include: weight tensor, neuron decay coefficient, neuron firing threshold, original time step, and pulse firing rate. For the weight tensor, the convolution kernel weight range is [-1.0, 1.0], with a large gradient norm for convolution kernels at defect edges and sparse gradients for background convolution kernels. For the neuron decay coefficient α, in shallow layers 1-3, α ∈ [0.25, 0.45], belonging to fast leakage neurons; in deep layers 4-10, α ∈ [0.92, 0.96], belonging to long-term memory neurons. The neuron firing threshold can be uniformly set to θ = 1.0. For the original time step... It can be 16. Regarding the pulse firing rate... Background area Defect texture area .

[0119] In the three-domain quantization stage, weights, membrane potentials, and time steps can be jointly quantized separately. For example, for weight domain quantization, convolutional kernels sensitive to defect edges have large weight amplitudes and high gradient norms, indicating high importance, and are quantized using INT8 high-precision quantization; low-importance weights in smooth background regions are quantized using INT4, compressing the storage bit width. For membrane potential domain quantization, the membrane potential register bit width can be dynamically allocated according to the neuron decay coefficient α. For example, shallow noise neurons with α < 0.5 are configured with a 4-bit membrane potential bit width, while deep defect feature neurons with α > 0.9 are configured with an 8-bit high bit width. Truncation quantization is performed on the membrane potential at each inference time step. The membrane potential changes in deep, weak defects are small, and the high bit width is used to retain subtle potential increments, avoiding quantization smoothing out microcrack features. In addition, for the time domain, background regions... Time steps compressed to / 8=2, the system can actively shut off idle clock cycles and skip pulseless inference; defective texture dense layer The original 16 time steps are kept uncompressed to ensure the spatiotemporal continuity of the defects.

[0120] During the error compensation stage, the cumulative quantization error can be calculated based on the quantization error obtained from the weight domain and membrane potential domain. This cumulative error is then compared with a preset compensation threshold (e.g., 0.25 in this scenario). When the preset compensation threshold is reached, a compensation pulse is generated to inject a corresponding compensation potential into the membrane potential of the LIF neuron. In this scenario, the single-step quantization error amplitude of the shallow background neuron layer is less than 0.01, and the cumulative amount remains below the compensation threshold for an extended period, so the comparator does not trigger a compensation pulse. After 8-16 consecutive time steps of iteration in the deep neuron layer, the weight error and membrane potential error continue to accumulate, and the cumulative error in the microcrack area of ​​the workpiece can reach 0.31, exceeding the 0.25 compensation threshold. At this point, a compensation pulse is generated and directly injected into the membrane potential accumulator to compensate for the potential deficit caused by quantization, preventing the membrane potential from being stuck in the sub-threshold range and causing missed defect detection.

[0121] During the hardware inference phase, a time controller can be configured based on the compressed time step to control the inference process of each LIF neuron layer in the model. The current accumulated membrane potential of the LIF neuron is compared with a pulse firing threshold; when the membrane potential reaches the threshold, a pulse is output. Using the model inference method provided in this application, the false negative rate for minute defects can be reduced from 6.8% to 1.6%, and inference power consumption can be reduced by 38%.

[0122] In addition to the above-mentioned application scenarios, the model inference method provided in this application can also be used in physiological monitoring scenarios.

[0123] For example, in wearable device-based human physiological monitoring scenarios, the aforementioned target neural network model can be a physiological monitoring model. This model can be deployed on a micro-physiological monitoring chip in a smart bracelet, with a hardware power consumption limit of ≤100mW, a single-chip storage capacity of 128KB, no heat dissipation structure, and requires continuous operation with ultra-low power consumption. Specifically, this model can be an SNN model, such as a Transformer SNN, used to identify abnormal physiological states such as premature beats, bradycardia, and arrhythmia. In this model structure, neurons generally possess medium- to long-term memory characteristics, and the membrane potential drifts slowly. Quantization errors are easily accumulated over monitoring time, causing waveform distortion. Traditional model inference methods in this scenario can lead to problems such as misjudgment of physiological inflection points.

[0124] Based on this, in the above application scenarios, the model inference method provided in this application may include, for example,: quantizing the model parameter information of the physiological monitoring model to obtain quantization results; determining and accumulating the error generated by the quantization process during the process of using the physiological monitoring model to detect anomalies in the collected physiological signals based on the quantization results, to obtain cumulative quantization error; and compensating the output of the physiological monitoring model based on the cumulative quantization error to obtain physiological state detection results.

[0125] In some examples, the physiological signals collected above can be physiological pulse signals such as human electrocardiogram pulses and blood oxygen fluctuation pulses, which have temporal characteristics such as low frequency, continuity, and slight drift. Accordingly, the physiological monitoring model can be used to identify abnormal physiological states such as premature beats, bradycardia, and arrhythmia. For example, when the physiological monitoring model detects an abnormal signal, it generates a pulse event and outputs a pulse sequence, which can be the physiological state detection result output by the physiological monitoring model.

[0126] It should be noted that the specific execution process of the model inference method in this scenario is similar to the corresponding process described above. For details, please refer to the relevant parts of the aforementioned embodiments, which will not be repeated here.

[0127] Furthermore, for easily misdiagnosed signals such as borderline premature beats and mild arrhythmias, the original floating-point membrane potential change trend can be restored during the quantification compensation process, avoiding health misjudgments caused by quantification distortion. In this scenario, using the model inference method provided in this application, the continuous monitoring battery life of wearable devices can be increased by 25%, and the physiological state recognition error can be controlled within 0.8%.

[0128] The above text combined Figure 1 and Figure 2 The present application describes in detail the embodiments of the model reasoning method. The following is a combination of... Figure 3The processing system for model reasoning described in this application is described in detail. It should be understood that the description of the model reasoning method embodiments corresponds to the description of the processing system embodiments for model reasoning; therefore, any parts not described in detail can be referred to the foregoing method embodiments.

[0129] Figure 3 The diagram shown is a schematic representation of a processing system for model inference provided in an embodiment of this application. Figure 3 As shown, the processing system for model inference provided in this application embodiment includes at least one processing unit 300, which is configured to correspond to neurons in the target neural network model. Specifically, the processing unit may include: The quantization storage module 310 is used to store quantization results, which are obtained by quantizing the model parameter information of the target neural network model. The inference calculation module 320 is connected to the quantization storage module 310 and is used to perform model inference based on the quantization results to obtain the model inference results. The error accumulation module 330 is connected to the inference calculation module 320 and is used to determine and accumulate the error generated by the quantization process during the model inference process based on the quantization results to obtain and store the accumulated quantization error. The error compensation module 340 is connected to the error accumulation module 330 and the inference calculation module 320 respectively and is used to compensate the output of the target neural network model based on the accumulated quantization error.

[0130] In some examples, the quantization storage module 310 may be a storage register, which may include a weight register and a configuration register. The weight register can be used to store the weight quantization result, and the configuration register can be used to store relevant configuration information such as the neuron membrane potential decay coefficient, the pulse firing threshold, the membrane potential quantization bit width, and the preset compensation threshold.

[0131] In some examples, the inference computation module 320 may include, for example, an inference computation unit, a membrane potential accumulator, a master comparator, and a pulse generator. The inference computation unit can be used to perform multiply-accumulate calculations based on quantization results; the membrane potential accumulator can be used to store the current membrane potential U of the neuron, supporting a configurable bit width (e.g., 4 / 6 / 8-bit); the master comparator can be used to compare the membrane potential with a pulse firing threshold; and the pulse generator can be used to generate output pulses and / or compensation pulses based on the comparison results.

[0132] In some examples, the error accumulation module 330 may include, for example, an error accumulator, which may specifically be an error accumulation register for calculating and storing the accumulated quantization error, with a fixed bit width of 8 bits.

[0133] In some examples, the error compensation module 340 may include, for example, a compensation comparator for comparing the accumulated quantization error with a preset compensation threshold, and outputting a compensation potential to the pulse generator in the inference calculation module 320 when the accumulated quantization error reaches the preset compensation threshold, so that the generator outputs a compensation pulse.

[0134] For example, one processing unit 300 as described above can be set for each neuron in the model. For a target neural network model containing multiple neurons, multiple processing units 300 can be set accordingly.

[0135] Thus, by configuring a processing unit for each neuron and incorporating dedicated hardware modules (such as error accumulation module 330 and error compensation module 340) within that unit to support error accumulation and compensation, the aforementioned model inference method can be executed efficiently, enabling high-precision, low-power neural network quantization inference at edge terminals. Furthermore, the error compensation mechanism is implemented through hardware modules, eliminating the need for additional software overhead and reducing software development costs.

[0136] The above text combined Figure 3 This application provides a detailed description of embodiments of the processing system for model inference provided in this application. Additionally, embodiments of this application also provide a computing chip that can integrate the processing system for model inference provided in any embodiment of this application. For example, multiple processing units configured for a target neural network model containing multiple neurons can be arranged into a processing unit array and integrated onto a single computing chip to form a complete neuromorphic hardware accelerator.

[0137] The following is combined with Figure 4 This application provides a detailed description of embodiments of the model inference apparatus. It should be understood that the descriptions of the model inference method embodiments correspond to the descriptions of the model inference apparatus embodiments; therefore, any parts not described in detail can be found in the preceding method embodiments.

[0138] Figure 4 The diagram shown is a structural schematic of a model inference device provided in an embodiment of this application. Figure 4 As shown, the model inference device 400 provided in this application embodiment includes: The quantization module 410 is used to quantize the model parameter information of the target neural network model to obtain the quantization result; The accumulation module 420 is used to determine and accumulate the error generated by the quantization process during the model inference process based on the quantization results, so as to obtain the cumulative quantization error. The compensation module 430 is used to compensate the output of the target neural network model based on the cumulative quantization error to obtain the model inference result.

[0139] In some embodiments of this application, the target neural network model includes a spiking neural network model, and the model parameter information includes at least two of the following: weights, membrane potential, and time step.

[0140] In some embodiments of this application, the quantization module 410 is further configured to: obtain the attenuation coefficient corresponding to each neuron in the target neural network model; determine the membrane potential quantization bit width corresponding to the neuron based on the attenuation coefficient, wherein the attenuation coefficient and the membrane potential quantization bit width are positively correlated; and quantize the membrane potential of the neuron based on the membrane potential quantization bit width to obtain the membrane potential quantization result, wherein the quantization result includes the membrane potential quantization result.

[0141] In some embodiments of this application, the quantization module 410 is further configured to: determine the pulse firing rate corresponding to each neuron layer in the target neural network model; determine the time step compression ratio corresponding to the neuron layer based on the pulse firing rate, wherein the pulse firing rate and the time step compression ratio are negatively correlated; and quantize the time step of the input pulse of the neuron layer according to the time step compression ratio to obtain a time quantization result, wherein the quantization result includes the time quantization result.

[0142] In some embodiments of this application, the quantization module 410 is further configured to: obtain the decay coefficient corresponding to each neuron in the target neural network model; determine the membrane potential quantization bit width corresponding to the neuron based on the time step compression ratio corresponding to the neuron layer where the neuron is located and the decay coefficient corresponding to the neuron; and quantize the membrane potential of the neuron based on the membrane potential quantization bit width to obtain the membrane potential quantization result, wherein the quantization result includes the membrane potential quantization result.

[0143] In some embodiments of this application, the quantization module 410 is further configured to: determine an importance metric value corresponding to each weight in the target neural network model; determine the quantization precision corresponding to the weight based on the importance metric value, wherein the importance metric value and the quantization precision are positively correlated; and quantize the weights according to the quantization precision to obtain a weight quantization result, wherein the quantization result includes the weight quantization result.

[0144] In some embodiments of this application, the accumulation module 420 is further configured to: determine the weight quantization error for each neuron and its corresponding weight in the target neural network model based on the weight quantization result corresponding to the neuron; determine the membrane potential quantization error at each time step of the inference process of the model neuron based on the membrane potential quantization result corresponding to the neuron; and perform error accumulation processing based on the membrane potential quantization error and the weight quantization error to obtain the cumulative quantization error.

[0145] In some embodiments of this application, the compensation module 430 is further configured to: for each neuron in the target neural network model, if the cumulative quantization error corresponding to the neuron reaches a preset compensation threshold, increase the membrane potential corresponding to the neuron to a target compensation potential so that the neuron outputs a compensation pulse and obtains the model inference result.

[0146] In some embodiments of this application, the target compensation potential is the potential corresponding to the pulse firing threshold, and the preset compensation threshold is less than the pulse firing threshold.

[0147] Below, for reference Figure 5 This describes an electronic device according to embodiments of the present application. Figure 5 The diagram shown is a structural schematic of an electronic device provided in an exemplary embodiment of this application.

[0148] like Figure 5 As shown, the electronic device 500 includes one or more processors 501 and memory 502.

[0149] The processor 501 may be a central processing unit (CPU) or other form of processing unit with data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device 500 to perform desired functions.

[0150] The memory 502 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, random access memory (RAM) and / or cache memory. The non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 501 may execute the program instructions to implement the model inference methods of the various embodiments of this application described above and / or other desired functions. The computer-readable storage medium may also store various contents such as model parameter information, quantization results, and accumulated quantization errors.

[0151] In one example, the electronic device 500 may also include an input device 503 and an output device 504, which are interconnected via a bus system and / or other forms of connection mechanism (not shown).

[0152] The input device 503 may include, for example, a keyboard, a mouse, etc.

[0153] The output device 504 can output various information to the outside, including model parameter information, quantization results, accumulated quantization error, etc. The output device 504 may include, for example, a display, a speaker, a printer, and a communication network and its connected remote output devices, etc.

[0154] Of course, for the sake of simplicity, Figure 5 Only some of the components of the electronic device 500 relevant to this application are shown in this illustration; components such as buses, input / output interfaces, etc., are omitted. In addition, the electronic device 500 may include any other suitable components depending on the specific application.

[0155] In addition to the methods and devices described above, embodiments of this application may also be computer program products, which include computer program instructions that, when executed by a processor, cause the processor to perform the steps in the model reasoning methods according to various embodiments of this application described above.

[0156] The computer program product can be written in any combination of one or more programming languages ​​to perform the operations of the embodiments of this application. The programming languages ​​include object-oriented programming languages ​​such as Java and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the user's computing device, partially on the user's computing device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0157] Furthermore, embodiments of this application may also be computer-readable storage media storing computer program instructions thereon, which, when executed by a processor, cause the processor to perform the steps in the model reasoning methods according to various embodiments of this application described above.

[0158] The computer-readable storage medium may be any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof.

[0159] The basic principles of this application have been described above with reference to specific embodiments. However, it should be noted that the advantages, benefits, and effects mentioned in this application are merely examples and not limitations, and should not be considered as essential features of each embodiment of this application. Furthermore, the specific details disclosed above are for illustrative and facilitative purposes only, and are not limitations. These details do not limit the application to the necessity of employing the aforementioned specific details for implementation.

[0160] The block diagrams of devices, apparatuses, devices, and systems involved in this application are merely illustrative examples and are not intended to require or imply that they must be connected, arranged, or configured in the manner shown in the block diagrams. As those skilled in the art will recognize, these devices, apparatuses, devices, and systems can be connected, arranged, and configured in any manner. Words such as “comprising,” “including,” “having,” etc., are open-ended terms meaning “including but not limited to,” and are used interchangeably with them. The terms “or” and “and” as used herein refer to the terms “and / or,” and are used interchangeably with them unless the context clearly indicates otherwise. The term “such as” as used herein refers to the phrase “such as but not limited to,” and is used interchangeably with it.

[0161] It should also be noted that in the apparatus, equipment, and methods of this application, the components or steps can be disassembled and / or recombined. These disassemblies and / or recombinations should be considered as equivalent solutions of this application.

[0162] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use this application. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein can be applied to other aspects without departing from the scope of this application. Therefore, this application is not intended to be limited to the aspects shown herein, but rather to be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0163] The above description has been given for purposes of illustration and description. Furthermore, this description is not intended to limit the embodiments of this application to the forms disclosed herein. Although numerous exemplary aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, alterations, additions, and sub-combinations thereof.

Claims

1. A model reasoning method, characterized in that, include: The model parameter information of the target neural network model is quantized to obtain the quantization result; During the model inference process based on the quantization results, the error generated by the quantization process is determined and accumulated to obtain the cumulative quantization error. Based on the accumulated quantization error, the output of the target neural network model is compensated to obtain the model inference result.

2. The method according to claim 1, characterized in that, The target neural network model includes a spiking neural network model, and the model parameter information includes at least two of the following: weights, membrane potential, and time step.

3. The method according to claim 2, characterized in that, When the model parameter information includes the membrane potential, the quantization processing of the model parameter information of the target neural network model to obtain the quantization result includes: For each neuron in the target neural network model, obtain the decay coefficient corresponding to that neuron; Based on the attenuation coefficient, the membrane potential quantization bit width corresponding to the neuron is determined, wherein the attenuation coefficient and the membrane potential quantization bit width are positively correlated; Based on the membrane potential quantization bit width, the membrane potential of the neuron is quantized to obtain the membrane potential quantization result, which includes the membrane potential quantization result.

4. The method according to claim 2, characterized in that, When the model parameter information includes the time step, the quantization processing of the model parameter information of the target neural network model to obtain the quantization result includes: For each neuron layer in the target neural network model, determine the impulse firing rate corresponding to that neuron layer; Based on the pulse firing rate, the time step compression ratio corresponding to the neuron layer is determined, wherein the pulse firing rate and the time step compression ratio are negatively correlated; According to the time step compression ratio, the time step of the input pulse of the neuron layer is quantized to obtain a time quantization result, which includes the time quantization result.

5. The method according to claim 4, characterized in that, When the model parameter information also includes the membrane potential, the quantization processing of the model parameter information of the target neural network model to obtain the quantization result includes: For each neuron in the target neural network model, obtain the decay coefficient corresponding to that neuron; Based on the time step compression ratio corresponding to the neuron layer where the neuron is located and the attenuation coefficient corresponding to the neuron, the quantization bit width of the membrane potential corresponding to the neuron is determined. Based on the membrane potential quantization bit width, the membrane potential of the neuron is quantized to obtain the membrane potential quantization result, which includes the membrane potential quantization result.

6. The method according to claim 2, characterized in that, When the model parameter information includes the weights, the quantization processing of the model parameter information of the target neural network model to obtain the quantization result includes: For each weight in the target neural network model, determine the importance metric value corresponding to that weight; Based on the importance metric, the quantization precision corresponding to the weight is determined, wherein the importance metric and the quantization precision are positively correlated; The weights are quantized according to the quantization precision to obtain the weight quantization result, which includes the weight quantization result.

7. The method according to any one of claims 1 to 6, characterized in that, In the process of model inference based on the quantization results, determining and accumulating the error generated by the quantization process to obtain the cumulative quantization error includes: For each neuron and its corresponding weight in the target neural network model, the weight quantization error is determined based on the weight quantization result of the weight corresponding to the neuron. At each time step of the model neuron inference process, the membrane potential quantization error is determined based on the membrane potential quantization result corresponding to the neuron. The cumulative quantization error is obtained by performing error accumulation processing based on the membrane potential quantization error and the weight quantization error.

8. The method according to any one of claims 1 to 6, characterized in that, The step of compensating the output of the target neural network model based on the accumulated quantization error to obtain the model inference result includes: For each neuron in the target neural network model, if the cumulative quantization error corresponding to the neuron reaches a preset compensation threshold, the membrane potential corresponding to the neuron is increased by the target compensation potential so that the neuron outputs a compensation pulse to obtain the model inference result.

9. The method according to claim 8, characterized in that, The target compensation potential is the potential corresponding to the pulse firing threshold, and the preset compensation threshold is less than the pulse firing threshold.

10. A processing system for model reasoning, characterized in that, It includes at least one processing unit, which is configured to correspond to neurons in the target neural network model, and the processing unit includes: A quantization storage module is used to store quantization results, which are obtained by quantizing the model parameter information of the target neural network model. The inference computation module is connected to the quantization storage module and is used to perform model inference based on the quantization result to obtain the model inference result. An error accumulation module, connected to the inference calculation module, is used to determine and accumulate the error generated by the quantization process during model inference based on the quantization result, and to obtain and store the accumulated quantization error. An error compensation module is connected to both the error accumulation module and the inference calculation module, and is used to compensate the output of the target neural network model based on the accumulated quantization error.

11. A computing chip, characterized in that, The computing chip integrates a processing system for model inference as described in claim 10.

12. A computer-readable storage medium, characterized in that, The storage medium stores a computer program for executing the model reasoning method according to any one of claims 1 to 9.

13. An electronic device, characterized in that, include: processor; Memory used to store the processor's executable instructions; The processor is used to execute the model reasoning method according to any one of claims 1 to 9.

14. A computer program product, characterized in that, The computer program product includes instructions that, when executed on an electronic device, cause the electronic device to implement the model reasoning method of any one of claims 1 to 9.

Images