A neural network accelerator based on dynamic sparse pruning

Abstract

The application belongs to the technical field of data processing, and specifically discloses a neural network accelerator based on dynamic sparse pruning, which realizes the optimized distribution of the accelerator computing task by collecting the running speed and the computing speed of the accelerator in real time during the running of the neural network, and dynamically adjusting the weight index and the node range of the computing task when detecting that the computing speed is lower than the running speed, can maximize the guarantee of the synchronization of the running speed and the computing speed under the condition of the shortage of computing resources, and at the same time, will not significantly affect the identification of the nodes needing pruning, and at the same time, when the weight calculation based on the node initially identifies that the weight is lower than the threshold value, the historical weight calculation result of the corresponding node is called to analyze the redundancy trend, thereby the pruning demand is judged, the occurrence rate of false pruning is reduced to a certain extent, the accuracy of pruning is improved, unnecessary pruning and recovery operations can be avoided, the waste of computing resources is reduced, and the resource utilization rate is improved.

Description

Technical Field

[0001] This invention belongs to the field of data processing technology, and specifically discloses a neural network accelerator based on dynamic sparse pruning. Background Technology

[0002] Neural networks are computational models that simulate the structure of neurons in the human brain. They process input data through multiple layers of nonlinear transformations, enabling them to learn and capture complex patterns and relationships. They are widely used in tasks such as image recognition and natural language processing. During operation, due to factors such as redundant connections, optimization objectives, and computational resource limitations, unimportant nodes often appear in neural networks. To improve inference speed and maintain model accuracy, these unimportant nodes need to be removed through pruning. This requires calculating weights for the nodes in the neural network during operation and promptly identifying unimportant nodes; this process is typically accomplished efficiently using accelerators.

[0003] When using accelerators to calculate node weights, all nodes are typically computed comprehensively to ensure no important nodes are overlooked. However, this comprehensive computation approach is redundant in terms of resource utilization, increasing the computational load on the accelerator. Especially when computational resources are limited, this high computational load often leads to a mismatch between the accelerator's computational speed and the neural network model's running speed. When the accelerator's computational speed is slower than the model's running speed, the accelerator becomes the performance bottleneck of the entire system, making it difficult to identify and remove unimportant nodes in a timely manner. This further affects the timeliness and effectiveness of pruning, thus limiting the overall running efficiency of the model.

[0004] Furthermore, when performing pruning based on node weight calculations, pruning is typically executed immediately upon detecting a weight falling below a set threshold. However, this immediate pruning operation does not consider whether the occurrence of the current weight falling below the threshold is accidental. If a node has consistently maintained a high weight without redundancy, and its weight only falls below the threshold at this moment, pruning it may lead to incorrect pruning. Incorrectly pruned nodes will increase unnecessary recovery time during subsequent restoration, affecting the effectiveness of the pruning operation and the stability of the model. In addition, frequent pruning and restoration operations may interfere with the model's training process, affecting the model's convergence speed and final performance. Summary of the Invention

[0005] In view of this, the present invention aims to propose a neural network accelerator based on dynamic sparse pruning, which effectively solves the problems mentioned in the background art by optimizing the allocation of computational tasks in the weight calculation of neural network nodes in the accelerator and adding a comprehensive evaluation of pruning execution.

[0006] The objective of this invention can be achieved through the following technical solution: This invention proposes a neural network accelerator based on dynamic sparse pruning, which includes the following modules: a running speed detection module, used to detect the running speed of the model in real time during the operation of the neural network.

[0007] The computation speed detection module is used to detect the computation speed according to the detection frequency of the model running speed when the accelerator performs weight calculations on the nodes in each layer of the neural network.

[0008] The comparative analysis module is used to compare the running speed of the simultaneously detected model with the computing speed of the accelerator.

[0009] The initial adjustment module for weight calculation is used to obtain the node that currently requires weight calculation when the accelerator's calculation speed is lower than the model's running speed, and then perform partial weight calculation for the current node.

[0010] The weight adjustment comparison and analysis module is used to compare the running speed of the simultaneously detected model with the computing speed of the accelerator after calculating some weights at the current node.

[0011] The key node weight calculation module is used to retrieve historical pruning records when the accelerator's computing speed is still lower than the model's running speed, thereby filtering out key nodes and then calculating the weights of the key nodes.

[0012] The required pruning node screening module is used to identify redundant nodes in the weight calculation of nodes in each layer of the neural network, construct a historical weight time series set of redundant nodes for redundancy trend analysis, and thereby screen out the required pruning nodes.

[0013] The pruning module is used to perform pruning operations on the selected nodes that require pruning.

[0014] Compared with the prior art, the beneficial effects of the present invention are as follows: 1. The present invention achieves optimized allocation of accelerator computing tasks by detecting the running speed and the accelerator's computing speed in real time during the operation of the neural network, and dynamically adjusting the weight index and the computing tasks of the node range when the computing speed is detected to be lower than the running speed. This method can maximize the synchronization of running speed and computing speed under the condition of tight computing resources, while not significantly affecting the identification of nodes that need to be pruned. It can not only improve the overall performance and real-time performance of the system, but also reduce the computing overhead and enhance the robustness and adaptability of the system.

[0015] 2. When the weight of a node is initially identified as being below a threshold in the node-based weight calculation, the present invention retrieves the historical weight calculation results of the corresponding node for redundancy trend analysis, thereby judging the pruning requirement. On the one hand, this reduces the occurrence rate of false pruning to a certain extent, improves the accuracy of pruning, and ensures that the model can still maintain high accuracy and performance after pruning. On the other hand, it avoids unnecessary pruning and restoration operations, reduces the waste of computing resources, and improves resource utilization efficiency. Attached Figure Description

[0016] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 This is a schematic diagram showing the connections between the various modules of the system in this invention.

[0018] Figure 2 This is a schematic diagram of the preferred weight indicator screening in this invention. Detailed Implementation

[0019] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0020] See Figure 1 As shown, this invention proposes a neural network accelerator based on dynamic sparse pruning, including a running speed detection module, a computation speed detection module, a comparison analysis module, a weight calculation initial adjustment module, a weight adjustment comparison analysis module, a key node weight calculation module, a required pruning node screening module, and a pruning operation module. The running speed detection module and the computation speed detection module are both connected to the comparison analysis module. The comparison analysis module is connected to the weight calculation initial adjustment module, the weight calculation initial adjustment module is connected to the weight adjustment comparison analysis module, the weight adjustment comparison analysis module is connected to the key node weight calculation module, the key node weight calculation module is connected to the required pruning node screening module, and the required pruning node screening module is connected to the pruning operation module.

[0021] The running speed detection module is used to detect the running speed of the model in real time during the operation of the neural network.

[0022] It's important to know that model running speed is usually measured in the number of samples processed per second or the number of inferences performed per second.

[0023] The computation speed detection module is used to detect the computation speed according to the detection frequency of the model running speed when the accelerator performs weight calculations on the nodes in each layer of the neural network.

[0024] It's important to know that computing speed is measured in the number of weights calculated per second or the number of calculations performed per second.

[0025] It should be noted that the accelerator's computing speed is detected at the same frequency as the model's running speed in order to compare the speed difference between the two in real time by synchronously detecting the model's running speed and the accelerator's computing speed, so as to detect the accelerator's computing speed drop in a timely manner and thus quickly locate and solve the problem.

[0026] In the specific operation of the above scheme, the model running speed and accelerator computing speed can be detected by using performance monitoring tools or custom monitoring code.

[0027] The comparative analysis module is used to compare the running speed of the simultaneously detected model with the computing speed of the accelerator.

[0028] See Figure 2 As shown, the initial weight calculation and adjustment module is used to obtain the node for which the current weight calculation is required when the accelerator calculation speed is lower than the model running speed, and then perform partial weight calculation for the current node. The specific operation is as follows: count the number of weight indicators used in the node weight calculation, retrieve historical pruning records, and extract the pruning-oriented weight indicators from them.

[0029] It's important to note that multiple weight metrics are used when calculating weights for nodes at different layers of a neural network. For example, weight metrics include absolute weight values, weight gradients, and node activation values. Absolute weight values ​​are the absolute values ​​of node weights; nodes with absolute weight values ​​close to zero are generally considered unimportant. Weight gradients are the gradient values ​​of node weights, representing the rate at which weights change during training, reflecting the node's importance. Nodes with larger gradients typically have a greater impact on the model's loss function and are therefore more important. Node activation values ​​are the activation values ​​of nodes during forward propagation, reflecting the node's activity level in the actual input data; nodes with high activation values ​​are generally more important. Using multiple weight metrics for node weight calculation allows for a more comprehensive assessment of node importance in the neural network, avoiding mispruning due to the randomness of a single weight.

[0030] It's worth noting that in node weight calculations based on multiple weighting indicators, when the analysis results show that a node's weight is below a set threshold and pruning is required, the system automatically generates a pruning record. This record details the pruned node and the weighting indicator that triggered the pruning. The weighting indicator that triggered the pruning refers to the prominent weighting indicator that plays a decisive role in the pruning decision during the comprehensive weight calculation. Because different weighting indicators have different sensitivities to node importance assessment during comprehensive weight calculation, it's unlikely that all weighting indicators will have an equal impact on pruning. Therefore, when the comprehensive weight calculation results indicate that pruning is necessary, a weighting indicator that triggers the pruning will appear.

[0031] For details on how to obtain the pruning-directing weight indicator, please refer to the implementation example: In the node weight calculation based on multiple weight indicators, there will be multiple weight values. By multiplying the multiple weight values, a comprehensive weight is obtained and compared with the comprehensive weight threshold set by the system. When the comprehensive weight is lower than the comprehensive weight threshold, the weight values ​​are arranged in ascending order. Then, the smallest weight value is extracted from the arrangement result and removed. The removed weight value is used to calculate the comprehensive weight. When the recalculated comprehensive weight is compared with the comprehensive weight threshold, if the recalculated comprehensive weight reaches the comprehensive weight threshold, the weight indicator corresponding to the removed weight value is used as the pruning-directing weight indicator.

[0032] In the above embodiments, a comprehensive weight is obtained by multiplying multiple weight values. This method amplifies the differences between weight values, helping to distinguish between important and unimportant nodes. By comparing it with a set comprehensive weight threshold, it can be clearly determined whether a node needs pruning. When pruning is required, the weight values ​​are arranged in ascending order to clearly identify the smallest weight value. The comprehensive weight is recalculated by removing the smallest weight value. When the recalculated result reaches the comprehensive weight threshold, the weight indicator corresponding to the smallest weight value is considered a pruning-oriented weight indicator. This is because a small weight value in the cumulative calculation will significantly reduce the comprehensive weight, and the cumulative result of the remaining weight values ​​after removing the smallest weight value will significantly increase. If this new cumulative result reaches the comprehensive weight threshold, it indicates that the smallest weight value has a decisive impact on the comprehensive weight. The calculation process of obtaining the pruning-oriented weight indicator by multiplying weight values ​​is simple, easy to implement, and efficient.

[0033] It is important to note that when using the above method to obtain the pruning-directed weight indicators, it is necessary to avoid directly multiplying the different weight indicators due to their different numerical ranges. This may result in excessively large numerical differences and affect the accuracy of the overall weight. It is advisable to standardize the weight values ​​so that different weight values ​​can be compared within the same range.

[0034] In another embodiment of the above operation, the pruning-directing weight indicators can be obtained by setting thresholds for different weight indicators. These thresholds can be determined based on experience and experiments. Each weight value is compared with the threshold set for the corresponding weight indicator, and the weight indicator with the largest difference from the set threshold is selected as the pruning-directing weight indicator. This method of obtaining the weight indicator can clearly identify the weight indicator with the greatest impact on the pruning decision by calculating the difference, and the difference measurement provides interpretability for the pruning decision, making it easier to understand and debug. However, it is necessary to pay attention to the reasonable setting of the thresholds for different weight indicators to avoid the impact of unreasonable threshold settings on the results.

[0035] In summary, the specific method for obtaining pruning weight indicators can be selected according to usage requirements, and is not limited to the implementation methods listed above.

[0036] By comparing the pruning direction weight indicators corresponding to each historical pruning record, the frequency of occurrence of each pruning direction weight indicator is summarized.

[0037] The node weight calculation logs are retrieved and categorized according to the same weight indicators to form several node weight calculation logs categorized by weight indicators. Then, resource consumption indicators are extracted from the node weight calculation logs.

[0038] It should be added that a detailed node weight calculation log is usually generated during each node weight calculation process to provide comprehensive recording and analysis tools. The weight calculation log includes the node name, weight indicators, resource consumption indicators, etc. Specifically, it records the name of each node for easy tracking and location; weight indicators: records the weight indicator value of each node; resource consumption indicators: records the resources used during the calculation process.

[0039] As an example of the above operations, resource consumption metrics include, but are not limited to, CPU utilization, memory usage, and disk usage.

[0040] The resource consumption index is calculated from the resource consumption indicators in the weight calculation logs of each node. The specific calculation is as follows: The resource consumption indicators in the weight calculation logs of each node are normalized to obtain the normalized value of each resource consumption indicator.

[0041] The normalization process described above is to enable resource consumption indicators of different units to be compared on the same scale. In specific example operations, the normalization process can adopt max-min normalization or standardization.

[0042] The resource consumption index corresponding to each node's weighted log is obtained by weighting and averaging the normalized values ​​of various resource consumption indicators.

[0043] In the above weighted average calculation, the weight values ​​of each resource consumption indicator can be initially set by the system. In the example where the source consumption indicators are CPU utilization, memory usage, and disk usage, the weight values ​​corresponding to CPU utilization, memory usage, and disk usage are 0.6, 0.2, and 0.2, respectively.

[0044] The average resource consumption index of each log entry in the weight calculation log of various weight indicators is used to calculate the unit resource consumption index corresponding to each weight indicator, that is, the resource consumption index of each weight indicator in each weight calculation process.

[0045] Arrange the pruning indicators according to their frequency of occurrence from highest to lowest to obtain the pruning tendency ranking results for each type of weighted indicator.

[0046] The resource consumption ranking results of various weighted indicators are obtained by arranging them in ascending order of unit resource consumption index.

[0047] It is important to emphasize that the higher the frequency of a certain weight indicator in the historical pruning records, the stronger the pruning tendency of that weight indicator, and the more suitable it is for priority calculation; the greater the resource consumption of a certain weight indicator shown in the node calculation log, the higher the resource demand of that weight indicator during the calculation process, and the less suitable it is for priority calculation.

[0048] Extract the sorting number from the above sorting results respectively, and add the two sorting numbers to obtain the comprehensive sorting number of each weight indicator. Then, take the weight indicator with the smallest comprehensive sorting number as the preferred weight indicator.

[0049] It's important to understand that since the pruning tendency ranking is in descending order of frequency, a smaller ranking number for the weight indicator within pruning tendency improves pruning accuracy and reduces the risk of incorrect pruning. Conversely, the resource consumption ranking is in ascending order of unit resource consumption index. Therefore, a smaller ranking number for the weight indicator within resource consumption indicates lower resource requirements during calculation. Choosing such a weight indicator optimizes computational resource utilization and improves computational efficiency. By adding the ranking numbers of both and selecting the weight indicator with the smallest overall ranking number as the preferred weight indicator, both pruning tendency and resource consumption are considered comprehensively even when the accelerator's computation speed is lower than the model's running speed. This ensures that the preferred weight indicator performs well in both pruning and computational resource consumption.

[0050] The current node is weighted according to the preferred weight criteria.

[0051] This invention achieves partial weight calculation for nodes by selecting preferred weight indicators from multiple weight indicators when the accelerator's computing speed is lower than the model's running speed. This reduces unnecessary calculations, speeds up the computing process, and allows the computing speed to synchronize with the model's running speed as quickly as possible.

[0052] The weight adjustment comparison analysis module is used to continue comparing the running speed of the model with the calculation speed of the accelerator after calculating some weights at the current node.

[0053] The key node weight calculation module is used to retrieve historical pruning records when the accelerator's calculation speed is still lower than the model's running speed, thereby filtering out key nodes and then calculating the weights of the key nodes.

[0054] In the preferred implementation of the above scheme, retrieving historical pruning records also includes: extracting pruning nodes from the retrieved historical pruning records, and then classifying the historical pruning records corresponding to the same pruning node to construct a set of historical pruning records for each node.

[0055] It should be understood that the node pruning mentioned in this invention is dynamic pruning, that is, nodes are dynamically removed and restored during the training process. In other words, the pruned nodes can be restored and are reversible.

[0056] In the further optimization and implementation of the above scheme, the key node is the following screening process: count the number of historical pruning records in the historical pruning record set categorized by each node.

[0057] Extract the weight difference before and after pruning from historical pruning records, and then calculate the average pruning weight difference for each node by averaging the weight differences before and after pruning in the historical pruning records.

[0058] It's important to note that the weight difference before and after pruning refers to the weight after pruning minus the weight before pruning, i.e., the change in pruning weights. Generally, pruning strategies typically select connections with smaller weights to remove. This is because connections with smaller weights contribute less to the model's output, and removing these connections has a smaller impact on model performance. Therefore, the weight values ​​before pruning are larger, while the weight values ​​after pruning become smaller or are even set to zero, making the weight difference after pruning usually a positive value.

[0059] It should be noted that nodes with significant changes in pruning weights may have a greater impact on model performance, and pruning these nodes may lead to a significant decrease in model performance. Nodes with smaller changes in weights may have a smaller impact on model performance, and pruning these nodes may be a priority.

[0060] The pruning tendency of each node is calculated by combining the number of historical pruning records categorized by node with the average pruning weight difference. The formula for calculating the pruning tendency is as follows: In the formula, PT represents the pruning tendency, m represents the number of historical pruning records for node classification, and M represents the total number of historical pruning records retrieved. This represents the average pruning weight difference between nodes.

[0061] As shown in the above formula, the more historical pruning records a node has, the more unimportant it has been considered in previous pruning processes. A smaller average pruning weight difference indicates that the node's weight has stabilized after multiple training and pruning processes, suggesting that its weight has converged and further training will not significantly change it. The greater the node's tendency to be pruned, the more weight calculation is needed.

[0062] The pruning tendency of each node is compared with the configured limit pruning tendency, which is initially configured by the system to assist in the selection of key nodes. For example, the limit pruning tendency is 70%, and nodes that reach the limit pruning tendency are extracted as key nodes.

[0063] This invention significantly reduces computational load and alleviates the computational burden on the accelerator when the accelerator's computational speed is still lower than the model's running speed after the node weight calculation. Instead of performing weight calculation on all nodes, it selects key nodes from among many nodes by using historical pruning records for weight calculation. This allows the computational speed to be synchronized with the model's running speed as soon as possible, while also helping to concentrate computational resources on key nodes and ensure that resources are used for the most important computational tasks.

[0064] The required pruning node screening module is used to construct a historical weight time series set of redundant nodes when redundant nodes are identified in the weight calculation of nodes in each layer of the neural network, and to perform redundancy trend analysis, thereby screening out the required pruning nodes.

[0065] Specifically, the process for identifying redundant nodes is as follows: the weight value obtained by each node in the corresponding preferred weight indicator calculation is compared with the weight threshold set by pruning. If the weight value calculated by a node is less than the weight threshold set by pruning, the node is recorded as a redundant node, and the current time is recorded as the redundant time.

[0066] It should be added that the ultimate goal of the accelerator in calculating node weights is to identify redundant nodes and prune them based on the weight calculation results. Therefore, redundant nodes are calculated not only based on the preferred weight indicators when the accelerator's calculation speed is lower than the model's running speed, but also based on multiple weight indicators when the accelerator's calculation speed is not lower than the model's running speed. When the accelerator's calculation speed is lower than the model's running speed, redundant nodes can be obtained by comparing the calculated preferred weight indicators of each node with the threshold of the corresponding weight indicator. When the accelerator's calculation speed is not lower than the model's running speed, redundant nodes can be obtained by comparing the comprehensive weight of each node with the set comprehensive weight threshold.

[0067] More specifically, the historical weight time series set of redundant nodes is constructed as follows: retrieve the node weight calculation log and extract the weight calculation log of the redundant node before the redundancy time.

[0068] It should be noted that the node weight calculation logs retrieved above refer to the weight calculation logs of redundant nodes.

[0069] It should also be noted that the weight calculation logs of redundant nodes before the redundancy time contain all weight calculation logs from the last pruning operation to the current identification as redundant.

[0070] The calculation time is extracted from the weight calculation logs corresponding to the redundant nodes, and the weight calculation logs are arranged in chronological order of calculation time. The calculation results of the preferred weight indicators are extracted from the weight calculation logs in the order of arrangement to form the historical weight time series set of the redundant nodes.

[0071] It is important to emphasize that the weight calculation logs corresponding to the extracted redundant nodes may contain cases where the accelerator calculation speed is lower than the model running speed, or cases where the accelerator calculation speed is not lower than the model running speed. Therefore, the weight calculations extracted from the weight calculation logs may contain either preferred weight indicators or comprehensive weight indicator calculations. If both are comprehensive weight indicator calculations, then the weight calculation results extracted from the weight calculation logs will be comprehensive weight indicator calculation results. If preferred weight indicator calculations exist, regardless of whether comprehensive weight indicator calculations exist, the weight calculation results extracted from the weight calculation logs will be the preferred weight indicator calculation results.

[0072] To be more specific, the redundancy trend analysis is performed as follows: Construct a coordinate system with the calculation time as the horizontal axis and the calculation results of the preferred weight indicator as the vertical axis. Mark several points in the constructed coordinate system for the historical weight time series set of redundant nodes to form the weight change curve of the redundant nodes.

[0073] In the weight change curve of redundant nodes, inflection points are marked, and the weight change curve is divided into a descending segment, a horizontal segment, and an ascending segment based on the marked inflection points.

[0074] It should be noted that the inflection point is the turning point on the weight change curve, which can be used to divide different segments (decreasing segment, horizontal segment, and increasing segment). Specifically, the inflection point can be marked by calculating the slope, where the increasing segment is the segment where the weight value continuously increases, the decreasing segment is the segment where the weight value continuously decreases, and the horizontal segment is the segment where the weight value remains unchanged.

[0075] Adjacent descending segment groups are marked from the segments divided by the weight change curve. The number of non-descending segments between adjacent descending segment groups is counted, and the descending continuity is calculated accordingly. Specifically, the number of non-descending segments between adjacent descending segment groups is divided by the total number of descending, horizontal, and ascending segments. An exponential function is constructed using the reciprocal of the natural constant as the base and the result of the division as the exponent to obtain the descending continuity. The fewer the number of non-descending segments between them, the greater the descending continuity. If there are multiple adjacent descending segment groups, the average descending continuity of all adjacent descending segment groups is taken as the descending continuity.

[0076] The percentage of descending segments is statistically analyzed. Specifically, the number of descending segments is counted and divided by the total number of descending, horizontal, and ascending segments. This value is then combined with the descending continuity to calculate a weighted average, resulting in the redundancy trend index of the redundant nodes.

[0077] Since the proportion of the descent segment reflects the overall redundancy trend, while the continuity of the descent reflects the redundancy trend of local details, the weights of the proportion of the descent segment and the continuity of the descent can be set to 0.6 and 0.4 respectively when calculating the redundancy trend by weighted average, so as to give greater importance to the proportion of the descent segment.

[0078] To be further specific, the process of selecting the required pruning nodes is as follows: the redundancy trend index of the redundant node is compared with the standard redundancy trend index set by the system. If the redundancy trend index of the redundant node reaches the standard redundancy trend index, then the redundant node is selected as the required pruning node.

[0079] The purpose of setting the redundancy trend index in the system mentioned above is to provide a quantitative boundary to assist in the selection of nodes for pruning requirements.

[0080] This invention calculates a redundancy trend index by combining multiple indicators (the proportion of the falling segment and the continuity of the falling segment), which identifies whether the weights show a downward trend, thereby more accurately identifying the nodes that truly need pruning and avoiding the mistaken pruning of important nodes.

[0081] The pruning operation module is used to perform pruning operations on the selected nodes that require pruning.

[0082] In further innovative implementations, the pruning operation module also includes: when all nodes in a certain layer of the neural network are analyzed at the same time and are found to be nodes that need to be pruned, the redundancy trend index of each node in that layer is compared, the node with the smallest redundancy trend index is selected for retention, and the other nodes are pruned.

[0083] It's important to understand that this approach ensures the model's connectivity and basic performance. Completely pruning a layer can significantly reduce the model's accuracy. Retaining a node ensures the model's connectivity, prevents the model from malfunctioning due to complete pruning, and helps maintain the model's generalization ability, preventing overfitting caused by excessive pruning.

[0084] The above content is merely an example and illustration of the concept of the present invention. Those skilled in the art can make various modifications or additions to the specific embodiments described, or use similar methods to replace them, as long as they do not deviate from the concept of the invention or exceed the scope defined by the present invention, and all such modifications and additions should fall within the protection scope of the present invention.

Claims

1. A neural network accelerator based on dynamic sparse pruning, characterized in that... It includes the following modules: The running speed detection module is used to detect the model running speed in real time during the operation of the neural network; The computation speed detection module is used to detect the computation speed according to the detection frequency of the model running speed when the accelerator performs weight calculations on the nodes in each layer of the neural network. The comparative analysis module is used to compare the running speed of the simultaneously detected models with the computing speed of the accelerator. The initial adjustment module for weight calculation is used to obtain the node that currently requires weight calculation when the accelerator calculation speed is lower than the model running speed, and then perform partial weight calculation for the current node. The weight adjustment comparison and analysis module is used to continue comparing the running speed of the simultaneously detected model with the computing speed of the accelerator after calculating some weights of the current node. The key node weight calculation module is used to retrieve historical pruning records when the accelerator's computing speed is still lower than the model's running speed, thereby filtering out key nodes and then calculating the weights of the key nodes. The required pruning node screening module is used to identify redundant nodes in the weight calculation of nodes in each layer of the neural network, construct a historical weight time series set of redundant nodes for redundancy trend analysis, and thereby screen out the required pruning nodes. The pruning module is used to perform pruning operations on the selected nodes that require pruning.

2. The neural network accelerator based on dynamic sparse pruning as described in claim 1, characterized in that: The process for calculating the partial weight of the current node is as follows: The number of weight indicators used in the calculation of node weights is counted, and historical pruning records are retrieved to extract the pruning-related weight indicators. By comparing the pruning direction weight indicators corresponding to each historical pruning record, the frequency of occurrence of each pruning direction weight indicator is summarized. Retrieve node weight calculation logs and classify them according to the same weight indicators to form several node weight calculation logs categorized by weight indicators. Then extract resource consumption indicators from the node weight calculation logs. Calculate the resource consumption index from the resource consumption indicators in the weight calculation log of each node. The unit resource consumption index corresponding to each weight indicator is obtained by averaging the resource consumption index of each log in the weight calculation log of the various weight indicators. Arrange the pruning indicators according to their frequency of occurrence from highest to lowest to obtain the pruning tendency ranking results for each type of weighted indicator; The resource consumption ranking results of various weighted indicators are obtained by arranging them in ascending order of unit resource consumption index. Extract the sorting number from the above sorting results respectively, and add the two sorting numbers to obtain the comprehensive sorting number of each weight indicator. Then, take the weight indicator with the smallest comprehensive sorting number as the preferred weight indicator. The current node is weighted according to the preferred weight criteria.

3. A neural network accelerator based on dynamic sparse pruning as described in claim 2, characterized in that: The resource consumption index is calculated as follows: The resource consumption indicators in the weight calculation logs of each node are normalized to obtain the normalized values ​​of each resource consumption indicator. The resource consumption index corresponding to each node's weighted log is obtained by weighting and averaging the normalized values ​​of various resource consumption indicators.

4. A neural network accelerator based on dynamic sparse pruning as described in claim 1, characterized in that: Retrieving historical pruning records also includes the following operations: Pruning nodes are extracted from the retrieved historical pruning records, and then the historical pruning records corresponding to the same pruning node are classified to construct a set of historical pruning records for each node.

5. A neural network accelerator based on dynamic sparse pruning as described in claim 4, characterized in that: The key nodes are selected through the following process: Count the number of historical pruning records in the historical pruning record set categorized by each node; Extract the weight difference before and after pruning from the historical pruning records, and then classify each node by averaging the weight difference before and after pruning in the historical pruning records to obtain the average pruning weight difference for each node. The pruning tendency of each node is calculated by combining the number of historical pruning records categorized by node with the average pruning weight difference. The formula for calculating the pruning tendency is as follows: In the formula, PT represents the pruning tendency, m represents the number of historical pruning records for node classification, and M represents the total number of historical pruning records retrieved. This represents the average difference in pruning weights between nodes; The pruning tendency of each node is compared with the configured limit pruning tendency, and the nodes that reach the limit pruning tendency are extracted as key nodes.

6. A neural network accelerator based on dynamic sparse pruning as described in claim 2, characterized in that: The process of identifying redundant nodes is as follows: The weight values ​​obtained by each node in the corresponding preferred weight indicator calculation are compared with the weight threshold set by pruning. If the weight value calculated by a node is less than the weight threshold set by pruning, the node is recorded as a redundant node and the current time is recorded as the redundant time.

7. A neural network accelerator based on dynamic sparse pruning as described in claim 6, characterized in that: The historical weight time series set of redundant nodes is constructed as follows: Retrieve the node weight calculation logs and extract the weight calculation logs of redundant nodes before the redundancy time. The calculation time is extracted from the weight calculation logs corresponding to the redundant nodes, and the weight calculation logs are arranged in chronological order of calculation time. The calculation results of the preferred weight indicators are extracted from the weight calculation logs in the order of arrangement to form the historical weight time series set of the redundant nodes.

8. A neural network accelerator based on dynamic sparse pruning as described in claim 7, characterized in that: The redundancy trend analysis is described in the following process: A coordinate system is constructed with computation time as the horizontal axis and the computation results of the preferred weight indicator as the vertical axis. Several points are marked in the constructed coordinate system for the historical weight time series set of redundant nodes to form the weight change curve of the redundant nodes. In the weight change curve of redundant nodes, inflection points are marked, and the weight change curve is divided into a descending segment, a horizontal segment, and an ascending segment based on the marked inflection points. Mark adjacent descending segments from the segments divided by the weight change curve, and count the number of non-descending segments between adjacent descending segments to calculate the descending continuity. The percentage of the descent segment is statistically analyzed, and a weighted average is calculated based on this percentage and the descent continuity to obtain the redundancy trend index of the redundant nodes.

9. A neural network accelerator based on dynamic sparse pruning as described in claim 8, characterized in that: The process of selecting nodes that require pruning is as follows: The redundancy trend index of the redundant node is compared with the redundancy trend index set by the system. If the redundancy trend index of the redundant node reaches the redundancy trend index, then the redundant node is designated as a node to be pruned.

10. A neural network accelerator based on dynamic sparse pruning as described in claim 1, characterized in that: The pruning operation module also includes the following process: When all nodes in a certain layer of the neural network are identified as nodes requiring pruning at the same time, the redundancy trend index of each node in that layer is compared. The node with the lowest redundancy trend index is selected for retention, while the other nodes are pruned.

Images