Neural network accelerator based on dynamic sparse pruning

Abstract

The invention belongs to the technical field of data processing, and particularly discloses a neural network accelerator based on dynamic sparse pruning, which is characterized in that the operation speed and the calculation speed of the accelerator are acquired in real time in the operation process of a neural network, and the weight index and the calculation task of the node range are dynamically adjusted when the calculation speed is detected to be lower than the operation speed. According to the method, the optimal allocation of the calculation tasks of the accelerator is realized, the synchronization of the running speed and the calculation speed can be ensured to the greatest extent under the condition of shortage of calculation resources, and meanwhile, the identification of nodes needing to be pruned cannot be obviously influenced; meanwhile, when the weight is recognized to be lower than a threshold value for the first time based on node weight calculation, historical weight calculation results of the corresponding nodes are called for redundancy trend analysis, pruning requirement judgment is carried out, the occurrence rate of wrong pruning is reduced to a certain degree, pruning accuracy is improved, unnecessary pruning and recovery operation can be avoided, and the pruning efficiency is improved. Waste of computing resources is reduced, and the resource utilization rate is increased.

Description

Technical Field

[0001] The invention belongs to the technical field of data processing, and specifically discloses a neural network accelerator based on dynamic sparse pruning. Background Art

[0002] A neural network is a computational model that simulates the structure of neurons in the human brain. It processes input data through multiple layers of nonlinear transformations, can learn and capture complex patterns and relationships, and is widely used in tasks such as image recognition and natural language processing. During operation, due to factors such as redundant connections, optimization goals, and computing resource limitations, unimportant nodes often appear in neural networks. In order to increase the speed of inference and maintain model accuracy, these unimportant nodes need to be removed through pruning. To this end, it is necessary to calculate the weights of the nodes in the neural network during operation and identify unimportant nodes in a timely manner. This process is usually completed efficiently with the help of accelerators.

[0003] In order to ensure that no important nodes are missed when calculating node weights with the help of an accelerator, all nodes are usually fully calculated. However, this comprehensive calculation method has redundancy in resource utilization and increases the computing load of the accelerator. Especially when computing resources are limited, this high-load calculation often causes the accelerator's computing speed to be out of sync with the running speed of the neural network model. When the accelerator's computing speed is lower than the running speed of the model, the accelerator will become the performance bottleneck of the entire system, making it difficult to identify and remove unimportant nodes in a timely manner, further affecting the timeliness and effectiveness of pruning, thereby limiting the overall operating efficiency of the model.

[0004] In addition, when performing pruning operations based on the results of node weight calculation, pruning is usually performed immediately when the weight is detected to be lower than the set threshold. However, this immediate pruning operation does not take into account whether the current weight is lower than the threshold by chance. If a node has always maintained a high weight in the early stage and has no redundant trend, and only at the current moment the weight is lower than the threshold, pruning at this time may cause mispruning. The mispruned node will increase unnecessary recovery time when it is subsequently restored, affecting the effectiveness of the pruning operation and the stability of the model. In addition, frequent pruning and recovery operations may interfere with the model training process, affecting the convergence speed and final performance of the model. Summary of the invention

[0005] In view of this, the present invention aims to propose a neural network accelerator based on dynamic sparse pruning, which effectively solves the problems raised in the above background technology by optimizing the computing task allocation in the accelerator's weight calculation of neural network nodes and adding a comprehensive evaluation of pruning execution.

[0006] The purpose of the present invention can be achieved through the following technical solutions: The present invention proposes a neural network accelerator based on dynamic sparse pruning, comprising the following modules: an operation speed detection module, which is used to detect the model operation speed in real time during the operation of the neural network.

[0007] The calculation speed detection module is used to detect the calculation speed according to the detection frequency of the model operation speed when the accelerator calculates the weights of the nodes in each layer of the neural network.

[0008] The comparative analysis module is used to compare the running speed of the simultaneously detected model with the accelerator calculation speed.

[0009] The weight calculation initial adjustment module is used to obtain the node of the current required weight calculation as the current node when the accelerator calculation speed is lower than the model operation speed, and then perform partial weight calculation of the current node.

[0010] The weight adjustment comparison and analysis module is used to continue to compare the running speed of the simultaneously detected model with the accelerator calculation speed after partial weight calculation of the current node.

[0011] The key node weight calculation module is used to retrieve historical pruning records when the accelerator calculation speed is still lower than the model operation speed, screen out key nodes based on the records, and then perform weight calculation on the key nodes.

[0012] The demand pruning node screening module is used to construct a historical weight time series set of redundant nodes for redundant trend analysis when redundant nodes are identified in the weight calculation of nodes in each level of the neural network, thereby screening out the demand pruning nodes.

[0013] The pruning operation module is used to perform pruning operations on the filtered demand pruning nodes.

[0014] Compared with the prior art, the beneficial effects of the present invention are as follows: 1. The present invention realizes the optimal allocation of accelerator computing tasks by detecting the running speed and the computing speed of the accelerator in real time during the operation of the neural network, and dynamically adjusting the weight index and the computing tasks of the node range when it is detected that the computing speed is lower than the running speed. This method can maximize the synchronization of the running speed and the computing speed when computing resources are tight, and will not significantly affect the identification of nodes that need to be pruned. It can not only improve the overall performance and real-time performance of the system, but also reduce computing overhead and enhance the robustness and adaptability of the system.

[0015] 2. When the weight calculation based on the node first identifies that the weight is lower than the threshold, the present invention retrieves the historical weight calculation results of the corresponding node to perform redundant trend analysis, thereby judging the pruning demand. On the one hand, it reduces the occurrence rate of false pruning to a certain extent, improves the accuracy of pruning, and ensures that the model can still maintain a high accuracy and performance after pruning. On the other hand, it avoids unnecessary pruning and recovery operations, reduces the waste of computing resources, and improves resource utilization efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for describing the embodiments are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0017] Figure 1 It is a schematic diagram of the connection of various modules of the system in the present invention.

[0018] Figure 2 This is a schematic diagram of the preferred weighted indicator screening in the present invention. DETAILED DESCRIPTION

[0019] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0020] See also Figure 1 As shown, the present invention proposes a neural network accelerator based on dynamic sparse pruning, including an operation speed detection module, a calculation speed detection module, a comparison analysis module, a weight calculation initial adjustment module, a weight adjustment comparison analysis module, a key node weight calculation module, a demand pruning node screening module, and a pruning operation module, wherein the operation speed detection module and the calculation speed detection module are both connected to the comparison analysis module, the comparison analysis module is connected to the weight calculation initial adjustment module, the weight calculation initial adjustment module is connected to the weight adjustment comparison analysis module, the weight adjustment comparison analysis module is connected to the key node weight calculation module, the key node weight calculation module is connected to the demand pruning node screening module, and the demand pruning node screening module is connected to the pruning operation module.

[0021] The running speed detection module is used to detect the model running speed in real time during the operation of the neural network.

[0022] It is important to know that model running speed is usually measured in the number of samples processed per second or the number of inferences performed per second.

[0023] The calculation speed detection module is used to detect the calculation speed according to the detection frequency of the model operation speed when the accelerator performs weight calculations on the nodes in each level of the neural network.

[0024] It is important to know that computational speed is measured in terms of the number of weights calculated per second or the number of calculations performed per second.

[0025] It should be pointed out that the purpose of detecting the computing speed of the accelerator according to the detection frequency of the model running speed is to compare the speed difference between the model running speed and the computing speed of the accelerator in real time by synchronously detecting the model running speed and the computing speed of the accelerator, so as to promptly discover the decrease in the computing speed of the accelerator and quickly locate and solve the problem.

[0026] In the specific operation of the above solution, the detection of the model running speed and the accelerator computing speed can be performed using a performance monitoring tool or a custom monitoring code.

[0027] The comparison and analysis module is used to compare the running speed of the model detected simultaneously with the calculation speed of the accelerator.

[0028] See also Figure 2 As shown, the weight initial calculation adjustment module is used to obtain the node of the current demand weight calculation as the current node when the accelerator calculation speed is lower than the model operation speed, and then perform partial weight calculation of the current node. The specific operations are as follows: count the number of weight indicators used in the node weight calculation, and call the historical pruning records to extract the pruning direction weight indicators.

[0029] It should be noted that more than one weight indicator is used when calculating the weights of nodes at each level of the neural network. Exemplarily, weight indicators include absolute weight values, weight gradients, node activation values, etc., where the absolute weight value is the absolute value of the node weight, and nodes with absolute weight values ​​close to zero are generally considered to be unimportant. The weight gradient is the gradient value of the node weight, that is, the speed at which the weight changes during the training process, reflecting the importance of the node during the training process. Nodes with large gradients usually have a greater impact on the model loss function, and the greater the importance of the node, the node activation value is the activation value of the node in the forward propagation, reflecting the activity of the node on the actual input data, and nodes with high activation values ​​are generally more important. By using multiple weight indicators to calculate node weights, the importance of nodes in the neural network can be more comprehensively evaluated to avoid mis-clipping due to the randomness of a single weight.

[0030] It should be added that in the node weight calculation based on multiple weight indicators, when the analysis results show that the node weight is lower than the set threshold and needs to be pruned, the system will automatically generate a pruning record, which will record in detail the pruned nodes and the pruning-directed weight indicators. The pruning-directed weight indicator refers to the prominent weight indicator that triggers pruning. The prominent weight indicator here refers to the specific weight indicator that plays a decisive role in the pruning decision in the comprehensive weight calculation. Since different weight indicators have different sensitivities in the evaluation of node importance in the comprehensive weight calculation, it is unlikely that all weight indicators will have the same effect on pruning. Based on this situation, the pruning-directed weight indicator will appear when the comprehensive weight calculation results show that pruning is needed.

[0031] For details on how to obtain the pruning direction weight indication, please refer to the embodiment: there will be multiple weight values ​​in the node weight calculation based on multiple weight indications. The comprehensive weight is obtained by multiplying the multiple weight values ​​and compared with the comprehensive weight threshold set by the system. When the comprehensive weight is lower than the comprehensive weight threshold, the weight values ​​are arranged in order from small to large, and then the minimum weight value is extracted from the arrangement result for elimination, and the eliminated weight value is subjected to comprehensive weight calculation. Therefore, when the recalculated comprehensive weight is compared with the comprehensive weight threshold, if the recalculated comprehensive weight reaches the comprehensive weight threshold, the weight indication corresponding to the eliminated weight value is used as the pruning direction weight indication.

[0032] In the above embodiment, a comprehensive weight is obtained by multiplying multiple weight values. This method can amplify the difference between weight values, which helps to distinguish important nodes from unimportant nodes. By comparing with the set comprehensive weight threshold, it can be clearly determined whether the node needs to be pruned. When it is determined that pruning is required, the weight values ​​are arranged in order from small to large, and the smallest weight value can be clearly identified. The comprehensive weight is recalculated by eliminating the smallest weight value. When the recalculated result reaches the comprehensive weight threshold, it can be considered that the weight indication corresponding to the smallest weight value is the pruning-oriented weight indication. This is because a small weight value in the cumulative calculation will significantly reduce the comprehensive weight, and the cumulative result of the remaining weight values ​​after eliminating the minimum weight value will significantly increase. If this new cumulative result reaches the comprehensive weight threshold, it means that the influence of the minimum weight value on the comprehensive weight is decisive. The calculation process of obtaining the pruning-oriented weight indication using weight value cumulative multiplication is simple, easy to implement and efficient.

[0033] It should be noted that when using the above method to obtain the pruning direction weight indication, it is necessary to avoid the different numerical ranges of different weight indications. Direct multiplication may cause excessive numerical differences and affect the accuracy of the comprehensive weight. The weight value can be standardized so that different weight values ​​can be compared within the same range.

[0034] In another embodiment of the above operation, the pruning direction weight indication is obtained and thresholds can be set for different weight indications. These thresholds can be determined based on experience and experiments. Each weight value is compared with the threshold set for the corresponding weight indication, and the weight indication with a large difference from the set threshold is selected as the pruning direction weight indication. This acquisition method can clearly identify the weight indication that has the greatest impact on the pruning decision by calculating the difference, and the difference measurement provides interpretability of the pruning decision, which is easy to understand and debug. However, it should be noted that the setting of different weight indication thresholds should be reasonable to avoid the impact of unreasonable threshold settings on the results.

[0035] In summary, it can be seen that the pruning direction weight indication acquisition method can be selected according to the use requirements, and is not limited to the above-listed implementation methods.

[0036] By comparing the pruning direction weight indicators corresponding to each historical pruning record, the occurrence frequency of each pruning direction weight indicator is summarized.

[0037] The node weight calculation log is retrieved and classified according to the same weight indicator to form a number of node weight calculation logs classified by various weight indicators, and then the resource consumption index is extracted from the node weight calculation log.

[0038] It should be added that a detailed node weight calculation log is usually generated during each node weight calculation process in order to provide a comprehensive recording and analysis tool. The weight calculation log will include node name, weight indicator, resource consumption indicator, etc., and specifically record the name of each node for easy tracking and positioning; weight indicator: record the weight indicator value of each node; resource consumption indicator: record the resources occupied during the calculation process.

[0039] As an example of the above operation, the resource consumption indicator includes but is not limited to CPU usage, memory usage, disk usage, etc.

[0040] The resource consumption index in each node weight calculation log is calculated as a resource consumption index. The specific calculation is as follows: the resource consumption index in each node weight calculation log is normalized to obtain a normalized value of each resource consumption index.

[0041] The normalization process described above is to enable resource consumption indicators of different units to be compared on the same scale. In the specific example operation, the normalization process may adopt maximum and minimum normalization, or standardization.

[0042] The normalized values ​​of various resource consumption indicators are weighted averaged to obtain the resource consumption index corresponding to each node weight calculation log.

[0043] The weight values ​​of various resource consumption indicators in the above weighted average calculation can be initially set for the system. In the example where the source consumption indicators are CPU usage, memory usage, and disk usage, the corresponding weight values ​​of CPU usage, memory usage, and disk usage are 0.6, 0.2, and 0.2.

[0044] The resource consumption index of each log in the weight calculation log of each weight indicator classification node is averaged to obtain the unit resource consumption index corresponding to each weight indicator, that is, the resource consumption index of each weight indicator in each weight calculation process.

[0045] Arrange the pruning orientation weight indicators in descending order according to the frequency of occurrence, and obtain the pruning tendency arrangement results of various weight indicators.

[0046] Arrange various weight indicators in order from small to large according to the unit resource consumption index to obtain the resource consumption arrangement results of various weight indicators.

[0047] It should be emphasized that the higher the frequency of a certain weight indicator appearing in the historical pruning records, the stronger the pruning tendency of the weight indicator is, and the more suitable it is for priority calculation; the greater the resource consumption of a certain weight indicator shown in the node calculation log, the higher the demand for resources of the weight indicator during the calculation process, and the less suitable it is for priority calculation.

[0048] The ranking numbers are extracted from the above arrangement results respectively, and the ranking numbers of the two are added together to obtain the comprehensive ranking number of each type of weight indicator, and then the weight indicator with the smallest comprehensive ranking number is taken as the preferred weight indicator.

[0049] It should be understood that since the pruning tendency arrangement results are arranged in descending order according to the frequency of occurrence, the smaller the ranking number of the weight indicator in the pruning tendency, the selection of such a weight indicator can improve the accuracy of pruning and reduce the risk of mispruning, while the resource consumption arrangement results are arranged in ascending order according to the unit resource consumption index, so the smaller the ranking number of the weight indicator in resource consumption, the lower the demand for resources for the weight indicator during the calculation process. Selecting such a weight indicator can optimize the utilization of computing resources and improve computing efficiency. By adding the ranking numbers of the two and selecting the weight indicator with the smallest comprehensive ranking number as the preferred weight indicator, the two key factors of pruning tendency and resource consumption can be comprehensively considered when the computing speed of the accelerator is lower than the running speed of the model, ensuring that the preferred weight indicator performs well in both pruning and computing resource consumption.

[0050] The weight of the current node is calculated according to the preferred weight indication.

[0051] The present invention realizes partial weight calculation of nodes by selecting preferred weight indicators from multiple weight indicators for weight calculation when the accelerator calculation speed is lower than the model operation speed, which can reduce unnecessary calculations, speed up the calculation speed, and synchronize the calculation speed with the model operation speed as soon as possible.

[0052] The weight adjustment comparison and analysis module is used to continue to compare the running speed of the simultaneously detected model with the accelerator calculation speed after partial weight calculation of the current node.

[0053] The key node weight calculation module is used to retrieve historical pruning records when the accelerator calculation speed is still lower than the model operation speed, thereby screening out key nodes and then performing weight calculation of the key nodes.

[0054] In a preferred implementation of the above solution, retrieving historical pruning records further includes: extracting pruning nodes from the retrieved historical pruning records, and then classifying the historical pruning records corresponding to the same pruning nodes to construct a historical pruning record set in which each node is classified.

[0055] It should be understood that the node pruning mentioned in the present invention belongs to dynamic pruning, that is, nodes are dynamically removed and restored during the training process. In other words, the pruned nodes can be restored and are reversible.

[0056] In a further preferred implementation of the above scheme, the key nodes are screened as follows: the number of historical pruning records in the historical pruning record set classified by each node is counted.

[0057] The weight difference before and after pruning is extracted from the historical pruning records, and then the weight difference before and after pruning in the historical pruning records of each node is classified and the average pruning weight difference corresponding to each node is calculated.

[0058] It should be added that the weight difference before and after pruning refers to the weight after pruning minus the weight before pruning, that is, the pruning weight change. Generally speaking, pruning strategies usually choose to remove connections with smaller weights. This is because connections with smaller weights contribute less to the output of the model, and removing these connections has less impact on the performance of the model. Therefore, the weight value before pruning is larger, while the weight value after pruning will become smaller or directly set to zero, which makes the weight difference after pruning usually positive.

[0059] It needs to be explained that nodes with larger pruning weight changes may have a greater impact on model performance. These nodes may cause a significant drop in model performance after pruning. Nodes with smaller weight changes may have less impact on model performance. These nodes have less impact on model performance after pruning and can be given priority for pruning.

[0060] The number of historical pruning records classified for each node is combined with the average pruning weight difference to calculate the pruning tendency of each node, where the pruning tendency calculation formula is: Where PT represents the pruning tendency, m represents the number of historical pruning records of node classification, and M represents the total number of historical pruning records retrieved. Represents the average pruning weight difference of the node.

[0061] From the above formula, we can see that the more historical pruning records a node has, the more likely it is that the node has been considered unimportant in the past multiple pruning processes. The smaller the average pruning weight difference, the more likely it is that the node's weight has stabilized during multiple training and pruning processes. This indicates that the node's weight has converged and further training will not change its weight much. The greater the tendency of the node to be pruned, the more weight calculation is needed.

[0062] The pruning tendency of each node is compared with the configured limited pruning tendency, where the limited pruning tendency is the initial configuration of the system and the configuration purpose is to assist in the screening of key nodes. For example, the limited pruning tendency is 70%, and the nodes that reach the limited pruning tendency are extracted as key nodes.

[0063] The present invention can significantly reduce the amount of calculation, alleviate the calculation burden of the accelerator, synchronize the calculation speed with the model running speed as soon as possible, and help concentrate computing resources on key nodes to ensure that resources are used for the most important computing tasks.

[0064] The demand pruning node screening module is used to construct a historical weight time series set of redundant nodes to perform redundancy trend analysis when redundant nodes are identified in the weight calculation of nodes in each level of the neural network, thereby screening out the demand pruning nodes.

[0065] Specifically, the redundant nodes are identified as follows: the weight value obtained by each node in the calculation of the corresponding preferred weight indicator is compared with the weight threshold set by pruning. If the weight value calculated by a node is less than the weight threshold set by pruning, the node is recorded as a redundant node, and the current time is recorded as the redundant time.

[0066] It should be added that the ultimate goal of the accelerator's node weight calculation is to identify redundant nodes for pruning based on the weight calculation results. Therefore, redundant nodes are not only calculated based on the preferred weight indicators when the accelerator calculation speed is lower than the model running speed, but also calculated based on multiple weight indicators when the accelerator calculation speed is not lower than the model running speed. When the accelerator calculation speed is lower than the model running speed, redundant nodes can be obtained by comparing the preferred weight indicators of each node with the threshold of the corresponding weight indicator. When the accelerator calculation speed is not lower than the model running speed, redundant nodes can be obtained by comparing the comprehensive weights of each node with the set comprehensive weight threshold.

[0067] More specifically, the historical weight time series set of redundant nodes is constructed by the following operations: calling the node weight calculation log, and extracting the weight calculation log of the redundant node before the redundant time.

[0068] It should be pointed out that the node weight calculation log retrieved above refers to the weight calculation log of the redundant node.

[0069] It should also be pointed out that the weight calculation log of the redundant node before the redundant time contains all weight calculation logs from the last pruning operation to the current identification as redundant.

[0070] The calculation time is extracted from the weight calculation log corresponding to the extracted redundant node, and the weight calculation log is arranged in the order of calculation time, and the calculation results of the preferred weight indication are extracted from the weight calculation log in the arrangement order to form the historical weight time series set of the redundant node.

[0071] It should be emphasized that in the weight calculation log corresponding to the extracted redundant nodes, there may be a situation where the accelerator calculation speed is lower than the model running speed, or there may be a situation where the accelerator calculation speed is not lower than the model running speed. Therefore, the weight calculation extracted from the weight calculation log includes preferred weight indication and comprehensive weight indication calculation. At this time, if they are all comprehensive weight indication calculations, the weight calculation results extracted from the weight calculation log are all comprehensive weight indication calculation results. If there is a preferred weight indication calculation, regardless of whether there is a comprehensive weight indication calculation, the weight calculation results extracted from the weight calculation log are all preferred weight indication calculation results.

[0072] To be more specific, the redundant trend analysis refers to the following process: a coordinate system is constructed with calculation time as the horizontal axis and the calculation result of the preferred weight indicator as the vertical axis, and several points are marked in the constructed coordinate system for the historical weight time series set of the redundant nodes to form a weight change curve of the redundant nodes.

[0073] Inflection points are marked in the weight change curve of the redundant nodes, and the weight change curve is divided into a descending segment, a horizontal segment and an ascending segment based on the marked inflection points.

[0074] It should be noted that the inflection point is the turning point on the weight change curve, which can be used to divide different sections (declining section, horizontal section and rising section). Specifically, the inflection point can be marked by slope calculation, where the rising section is the section where the weight value continues to rise, the descending section is the section where the weight value continues to fall, and the horizontal section is the section where the weight value remains the same.

[0075] From the segments divided by the weight change curve, mark the adjacent descending segment groups, and count the number of non-descending segments in the adjacent descending segment groups, thereby calculating the descending continuity. Specifically, the number of non-descending segments in the adjacent descending segment groups can be divided by the total number of descending segments, horizontal segments, and ascending segments, and the reciprocal of the natural constant is used as the base, and the exponential function is constructed with the division result as the exponent to obtain the descending continuity. The fewer the number of non-descending segments in the interval, the greater the descending continuity. If there are multiple adjacent descending segment groups, the average of the descending continuity of each adjacent descending segment group is taken as the descending continuity.

[0076] The proportion of the descending segments is counted. Specifically, the number of descending segments can be counted, divided by the total number of descending segments, horizontal segments, and ascending segments, and then the weighted average is calculated in combination with the descending continuity to obtain the redundant trend index of the redundant node.

[0077] In the above description, since the proportion of the descending segment reflects the overall redundancy trend, and the descending continuity reflects the redundancy trend of local details, the weights of the descending segment proportion and the descending continuity can be set to 0.6 and 0.4 when calculating the redundant trend by weighted average, so as to achieve a higher emphasis on the descending segment proportion.

[0078] To be more specific, the screening out of demand pruning nodes is implemented as follows: comparing the redundancy trend index of the redundant node with the standard redundancy trend index set by the system, if the redundancy trend index of the redundant node reaches the standard redundancy trend index, the redundant node is used as the demand pruning node.

[0079] The purpose of setting the redundancy trend index in the above system is to provide a quantitative limit to assist in the screening of nodes that require pruning.

[0080] The present invention calculates the redundant trend index by combining multiple indicators (the proportion of the descending segment and the descending continuity) to identify whether the weight shows a descending trend, thereby more accurately identifying the nodes that really need to be pruned and avoiding the mistaken pruning of important nodes.

[0081] The pruning operation module is used to perform pruning operations on the screened demand pruning nodes.

[0082] In further innovative implementations, the pruning operation module also includes: when all nodes in a certain layer of the neural network are analyzed at the same time as nodes that need to be pruned, the redundant trend index of each node in the layer is compared, and the node with the smallest redundant trend index is selected to be retained, and the other nodes are pruned.

[0083] It should be understood that the above is done to ensure the connectivity and basic performance of the model. Completely pruning a layer may significantly reduce the accuracy of the model. Retaining a node can ensure the connectivity of the model and prevent the model from not working properly due to completely pruning a layer. At the same time, it helps to maintain the generalization ability of the model and prevent over-pruning from causing overfitting of the model.

[0084] The above contents are merely examples and explanations of the concept of the present invention. Those skilled in the art may make various modifications or additions to the specific embodiments described or replace them in a similar manner. As long as they do not deviate from the concept of the invention or exceed the scope defined by the present invention, they shall all fall within the protection scope of the present invention.

Claims

1. A neural network accelerator based on dynamic sparse pruning, characterized in that , including the following modules: The running speed detection module is used to detect the running speed of the model in real time during the operation of the neural network; A calculation speed detection module is used to detect the calculation speed according to the detection frequency of the model running speed when the accelerator performs weight calculations on the nodes in each layer of the neural network; A comparative analysis module is used to compare the running speed of the model under simultaneous detection with the calculation speed of the accelerator; The weight calculation initial adjustment module is used to obtain the node of the current required weight calculation as the current node when the accelerator calculation speed is lower than the model operation speed, and then perform partial weight calculation of the current node; The weight adjustment comparison analysis module is used to continue to compare the running speed of the model being tested simultaneously with the calculation speed of the accelerator after calculating the partial weight of the current node; The key node weight calculation module is used to retrieve the historical pruning records when the accelerator calculation speed is still lower than the model operation speed, and select the key nodes based on the records, and then calculate the weights of the key nodes; The demand pruning node screening module is used to construct a historical weight time series set of redundant nodes to perform redundancy trend analysis when redundant nodes are identified in the weight calculation of nodes in each level of the neural network, thereby screening out the demand pruning nodes; The pruning operation module is used to perform pruning operations on the filtered demand pruning nodes.

2. A neural network accelerator based on dynamic sparse pruning as claimed in claim 1, characterized in that: The partial weight calculation of the current node is performed as follows: Count the number of weight indicators used in node weight calculation, and retrieve historical pruning records to extract pruning direction weight indicators; By comparing the pruning direction weight indicators corresponding to each historical pruning record, the occurrence frequency of each pruning direction weight indicator is summarized; Retrieving the node weight calculation logs, and classifying them according to the same weight indicators to form several node weight calculation logs classified by various weight indicators, and then extracting resource consumption indicators from the node weight calculation logs; Calculate the resource consumption index of each node weight calculation log; The resource consumption index of each log in the weight calculation log of each weight indicator classification node is averaged to obtain the unit resource consumption index corresponding to each weight indicator; Arrange the pruning orientation weight indicators in descending order according to the frequency of occurrence, and obtain the pruning tendency arrangement results of various weight indicators; Arrange various weight indicators in order from small to large according to the unit resource consumption index to obtain the resource consumption arrangement results of various weight indicators; Extract the ranking numbers from the above arrangement results respectively, and add the ranking numbers of the two to obtain the comprehensive ranking number of each type of weight indicator, and then take the weight indicator with the smallest comprehensive ranking number as the preferred weight indicator; The weight of the current node is calculated according to the preferred weight indication.

3. A neural network accelerator based on dynamic sparse pruning as claimed in claim 2, characterized in that: The resource consumption index is calculated as follows: Normalize the resource consumption indicators in each node weight calculation log to obtain the normalized value of each resource consumption indicator; The normalized values ​​of various resource consumption indicators are weighted averaged to obtain the resource consumption index corresponding to each node weight calculation log.

4. The neural network accelerator based on dynamic sparse pruning according to claim 1, characterized in that: The retrieving of historical pruning records also includes the following operations: The pruning nodes are extracted from the retrieved historical pruning records, and the historical pruning records corresponding to the same pruning nodes are classified to construct a historical pruning record set for each node classification.

5. A neural network accelerator based on dynamic sparse pruning as claimed in claim 4, characterized in that: The key nodes are screened as follows: Count the number of historical pruning records in the historical pruning record set classified by each node; Extract the weight difference before and after pruning from the historical pruning records, and then classify each node and calculate the average weight difference before and after pruning in the historical pruning records to obtain the average pruning weight difference corresponding to each node; The number of historical pruning records classified for each node is combined with the average pruning weight difference to calculate the pruning tendency of each node, where the pruning tendency calculation formula is: Where PT represents the pruning tendency, m represents the number of historical pruning records of node classification, and M represents the total number of historical pruning records retrieved. Represents the average pruning weight difference of the node; The pruning tendency of each node is compared with the configured limited pruning tendency, and the nodes that reach the limited pruning tendency are extracted as key nodes.

6. A neural network accelerator based on dynamic sparse pruning as claimed in claim 2, characterized in that: The process of identifying redundant nodes is as follows: The weight value of each node obtained in the calculation of the corresponding preferred weight indicator is compared with the weight threshold set by pruning. If the weight value calculated by a node is less than the weight threshold set by pruning, the node is recorded as a redundant node and the current time is recorded as the redundant time.

7. A neural network accelerator based on dynamic sparse pruning as claimed in claim 6, characterized in that: The construction of the historical weight time series set of redundant nodes is performed as follows: Retrieve the node weight calculation log, and extract the weight calculation log of the redundant node before the redundant time; The calculation time is extracted from the weight calculation log corresponding to the extracted redundant node, and the weight calculation log is arranged in the order of calculation time, and the calculation results of the preferred weight indication are extracted from the weight calculation log in the arrangement order to form the historical weight time series set of the redundant node.

8. A neural network accelerator based on dynamic sparse pruning as claimed in claim 7, characterized in that: The redundant trend analysis is described in the following process: A coordinate system is constructed with the calculation time as the horizontal axis and the calculation result of the preferred weight indicator as the vertical axis. A number of points are marked in the constructed coordinate system for the historical weight time series set of the redundant nodes to form a weight change curve of the redundant nodes. Marking inflection points in the weight change curve of the redundant node, and dividing the weight change curve into a descending segment, a horizontal segment and an ascending segment based on the marked inflection points; Marking adjacent descending segment groups from the segments divided by the weight change curve, and counting the number of non-descending segments between the adjacent descending segment groups, thereby calculating the descending continuity; The proportion of the descending segment is counted, and the weighted average is calculated based on the descending continuity to obtain the redundant trend index of the redundant node.

9. A neural network accelerator based on dynamic sparse pruning as claimed in claim 8, characterized in that: The screening out of the demand pruning nodes is implemented as follows: The redundancy trend index of the redundant node is compared with the standard redundancy trend index set by the system. If the redundancy trend index of the redundant node reaches the standard redundancy trend index, the redundant node is used as the demand pruning node.

10. The neural network accelerator based on dynamic sparse pruning according to claim 1, characterized in that: The pruning operation module also includes the following process: When all nodes in a certain layer of the neural network are analyzed at the same time as nodes that need to be pruned, the redundancy trend index of each node in the layer is compared, and the node with the smallest redundancy trend index is selected to be retained, and the other nodes are pruned.

Images