Control method and device, equipment and storage medium executed by neural network

Abstract

The application discloses a control method and device of a neural network execution, equipment and a storage medium, relates to the technical field of neural network inference, and comprises the following steps: acquiring an activated neuron set of a current neural network layer and corresponding weight data of the activated neuron set; performing calculation of the current neural network layer based on the activated neuron set and the weight data; in the process of performing calculation, determining an activated neuron set of a next neural network layer based on an intermediate result of calculation, and accordingly, pre-fetching weight data corresponding to the activated neuron set of the next neural network layer. According to the application, the activated neuron set of the next layer is determined in advance and the corresponding weight data is pre-fetched based on the intermediate result in the calculation process of the current layer, so that only the weight data related to the activated neuron needs to be loaded, the data access amount is reduced, and efficient inference is realized under the condition of limited bandwidth.

Description

Technical Field

[0001] This application relates to the field of neural network inference technology, and in particular to a control method, apparatus, device, and storage medium for neural network execution. Background Technology

[0002] In neural network inference, computation is typically performed layer by layer, with each layer's computation depending on all the weight data of that layer. As the size of neural network models continues to increase, the number of model parameters grows significantly, requiring frequent loading of large amounts of weight data from external storage during inference, leading to a sharp increase in storage bandwidth demands.

[0003] Existing methods generally address the above problems by using high-bandwidth memory to increase data supply capacity, reducing model size to decrease computation and access volume, or model compression to reduce data volume. However, in power-constrained devices, high-bandwidth memory is power-intensive and difficult to apply; reducing model size will affect model performance; and model compression still cannot fundamentally reduce the frequency of access to weight data.

[0004] Therefore, how to reduce the amount of data accessed during neural network inference under bandwidth-constrained conditions is a key technical problem that urgently needs to be solved.

[0005] The above content is only used to help understand the technical solution of this application and does not represent an admission that the above content is prior art. Summary of the Invention

[0006] The main objective of this application is to provide a control method, apparatus, device, and storage medium for neural network execution, aiming to solve the technical problem of reducing the amount of data accessed during neural network inference under bandwidth-limited conditions.

[0007] To achieve the above objectives, this application proposes a control method for neural network execution, the method comprising: Obtain the set of activated neurons in the current neural network layer and the weight data corresponding to the set of activated neurons in the current neural network layer; Based on the set of activated neurons in the current neural network layer and the weight data corresponding to the set of activated neurons in the current neural network layer, the calculation of the current neural network layer is performed. During the calculation of the current neural network layer, based on the intermediate results of the calculation of the current neural network layer, the set of activated neurons for the next neural network layer is determined, and based on the set of activated neurons for the next neural network layer, the weight data corresponding to the set of activated neurons for the next neural network layer is pre-fetched.

[0008] In one embodiment, the step of performing calculations for the current neural network layer based on the set of activated neurons and the weight data corresponding to the set of activated neurons in the current neural network layer includes: The computation of the current neural network layer is divided into multiple computation blocks; Based on the set of activated neurons in the current neural network layer, each computation block is enabled or skipped; The computation of the current neural network layer is performed based on the weight data corresponding to the set of activated neurons in the enabled computation block.

[0009] In one embodiment, the step of determining the set of activated neurons for the next neural network layer based on the intermediate results calculated by the current neural network layer includes: When the computation of the current neural network layer reaches the preset computation progress, the intermediate results that the current neural network layer has calculated are extracted as a partial activation state. Based on the partial activation states, predict the set of activated neurons in the next neural network layer.

[0010] In one embodiment, the step of predicting the set of activated neurons in the next neural network layer based on the partial activation states includes: Based on the aforementioned partial activation states, an initial set of activated neurons is determined; Based on the correlation between each neuron in the next neural network layer that is not included in the initial set of activated neurons and the initial set of activated neurons, select some neurons whose correlation meets a preset condition and add them to the initial set of activated neurons to obtain the set of activated neurons for the next neural network layer.

[0011] In one embodiment, after the step of determining the initial set of activated neurons based on the partial activation states, the method further includes: Obtain historical prediction error compensation information of the neural network; The initial set of activated neurons is updated based on the historical prediction error compensation information.

[0012] In one embodiment, the step of predicting the set of activated neurons in the next neural network layer based on the partial activation states includes: The neurons of the next neural network layer are grouped to obtain neuron groups; Based on the partial activation states, activation prediction is performed on each neuron group to obtain the set of activated neurons for each neuron group. The sets of activated neurons in each of the aforementioned neuron groups are merged to obtain the set of activated neurons in the next neural network layer.

[0013] In one embodiment, the step of pre-fetching the weight data corresponding to the set of activated neurons in the next neural network layer based on the set of activated neurons in the next neural network layer includes: The index of each activated neuron in the set of activated neurons in the next neural network layer is converted into the physical address of the corresponding weight data. The physical addresses are sorted and merged to generate a continuous memory access sequence; According to the storage access sequence, the weight data corresponding to the activated neurons of the next neural network layer are prefetched from the external storage.

[0014] Furthermore, to achieve the above objectives, this application also proposes a control device for neural network execution, the control device for neural network execution comprising: The data module is used to obtain the set of activated neurons in the current neural network layer and the weight data corresponding to the set of activated neurons in the current neural network layer. The calculation module is used to perform calculations for the current neural network layer based on the set of activated neurons and the weight data corresponding to the set of activated neurons in the current neural network layer. The weight prefetching module is used to determine the set of activated neurons for the next neural network layer based on the intermediate results of the current neural network layer calculation during the execution of the current neural network layer calculation, and to prefetch the weight data corresponding to the set of activated neurons for the next neural network layer based on the set of activated neurons for the next neural network layer.

[0015] Furthermore, to achieve the above objectives, this application also proposes a control device for neural network execution, the device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the control method for neural network execution as described above.

[0016] In addition, to achieve the above objectives, this application also proposes a storage medium, which is a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, it implements the steps of the control method for neural network execution as described above.

[0017] In addition, to achieve the above objectives, this application also provides a computer program product, which includes a computer program that, when executed by a processor, implements the steps of the control method for neural network execution as described above.

[0018] This application provides a control method for neural network execution. First, the application obtains the set of activated neurons and their corresponding weight data of the current neural network layer. The calculation of the current layer is performed only based on the weight data corresponding to the neurons in the set of activated neurons, instead of loading all weights. During the calculation of the current layer, the set of activated neurons of the next neural network layer is determined based on the intermediate results of the current layer calculation, and the weight data corresponding to the set of activated neurons in the next neural network layer is pre-fetched accordingly. This ensures that the decision on the demand for neurons in the next layer and the loading of weight data are completed during the calculation of the current layer, rather than waiting for the calculation of the current layer to be completed before processing.

[0019] Therefore, this application achieves advance decision-making on the demand for neurons in the next layer by determining the set of activated neurons in the next layer in advance based on intermediate results during the current layer computation. This transforms the serial dependency relationship of "completing the current layer computation → determining the demand for the next layer → loading the weights of the next layer" in the traditional method into a parallel relationship of "determining the demand for the next layer and pre-fetching weights during the current layer computation". At the same time, by loading only the weight data corresponding to the set of activated neurons, the actual amount of data transmitted is reduced from full weights to sparse weights, thereby reducing the amount of data access. This achieves efficient inference under bandwidth-constrained conditions and solves the technical problem of insufficient inference efficiency of large-scale neural networks under bandwidth-constrained conditions. Attached Figure Description

[0020] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0021] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 A flowchart illustrating the control method for neural network execution in this application, provided in Embodiment 1. Figure 2 A partial flowchart is provided for Embodiment 2 of the control method for neural network execution in this application; Figure 3 A partial flowchart is provided for Embodiment 2 of the control method for neural network execution in this application; Figure 4 This is a schematic diagram of the module structure of the control device for neural network execution according to an embodiment of this application; Figure 5 This is a schematic diagram of the device structure of the hardware operating environment involved in the control method for neural network execution in the embodiments of this application.

[0023] The purpose, features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0024] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of this application and are not intended to limit this application.

[0025] To better understand the technical solution of this application, a detailed description will be provided below in conjunction with specific implementation methods.

[0026] In neural network inference, computation is typically performed layer by layer. Taking a model with multiple neural network layers as an example, each layer contains multiple neurons, and layers are connected by weight data, which is usually stored in external storage (such as DRAM, Dynamic Random Access Memory). In traditional methods, all weight data for each layer must be loaded from external storage before computation. As the size of the neural network model increases, the number of model parameters grows significantly, leading to increasingly prominent storage bandwidth pressure. While existing solutions can improve data supply capacity using high-bandwidth memory, high-bandwidth memory is power-intensive and difficult to apply in power-constrained devices. Reducing the model size or model compression can reduce the data volume to some extent, but the compressed data volume is still enormous, and storage bandwidth pressure remains. Furthermore, in traditional methods, each layer must wait for the current layer's computation to complete before determining which neurons are needed for the next layer and starting to load weights. This sequential dependency means that the weight data loading time cannot be covered by the computation time, further exacerbating inference latency. Therefore, the problem with existing technologies is that they still rely on the complete model execution approach and do not change the way data is accessed during neural network execution.

[0027] This application provides a control method for neural network execution. Its core idea is to transform the traditional full-model execution method into a selective execution method based on activation prediction. Instead of loading all weights, it predicts in advance which neurons will be activated in the next layer, loads the weights associated with these neurons, and performs the corresponding sparse computation. Simultaneously, by scheduling the prediction, data loading, and computation processes, the execution times of these three processes overlap, thereby hiding data access latency, reducing data access volume, and improving overall system performance.

[0028] It should be noted that the execution subject of the embodiments of this application can be a computing device with data processing and storage access functions, such as a neural network processor, an AI accelerator, or a system-on-a-chip. The following description will use a neural network processor as an example.

[0029] Example 1 This embodiment provides a control method for neural network execution. This method does not process a single layer, but rather iterates layer by layer throughout the neural network inference process—the set of activated neurons and weight data required for the current layer's calculation come from the results pre-fetched for the current layer in the previous iteration; the set of activated neurons and pre-fetched weight data for the next layer are generated during the current layer's calculation and then used as input for the next iteration.

[0030] In this embodiment, as Figure 1 As shown, the control method executed by the neural network includes steps S10 to S30: Step S10: Obtain the set of activated neurons in the current neural network layer and the weight data corresponding to the set of activated neurons in the current neural network layer.

[0031] It should be noted that the set of activated neurons refers to the set of indices of neurons activated during inference in the current neural network layer. Activated neurons are those whose output values ​​are not zero or exceed a preset activation threshold; the outputs of these neurons will serve as input for the next layer's computation. For example, the set of activated neurons can be stored as a list of indices or as a binary vector—the vector length equal to the total number of neurons in the current layer, with 1 for activation and 0 for deactivation. During inference, only neurons corresponding to the indices in the set of activated neurons participate in the computation of the current layer; neurons not included in the set do not participate in the computation.

[0032] It's important to note that weight data refers to the connection weight parameters between each activated neuron in the current neural network layer and all neurons in the next layer. These weight parameters are stored in external storage. Since the number of neurons in each layer of a neural network model can reach thousands or even tens of thousands, the total weight data is enormous. In traditional methods, all weight data for each layer needs to be loaded before computation. However, this step only needs to load the weight data corresponding to each neuron in the set of activated neurons. The amount of data loaded is the number of activated neurons multiplied by the number of weight parameters corresponding to each neuron, significantly less than the total amount of data loaded, thus reducing the demand for storage bandwidth.

[0033] It should be noted that for the first layer of the neural network, the set of activated neurons can be preset to all neurons, that is, initially all neurons in the first layer are considered to be in an activated state, and the corresponding weight data is all the weight data of the first layer. For each layer after the first layer, the set of activated neurons and the weight data are obtained by the determination and prefetching operations performed for that layer in the previous loop, see the explanation of step S30. Specifically, in the previous round when processing the previous layer, the determination of the set of activated neurons and the prefetching of the corresponding weight data of the current layer were completed in advance during the calculation of that layer, so this round can directly retrieve them from the on-chip cache without waiting.

[0034] Step S20: Calculate the current neural network layer based on the set of activated neurons in the current neural network layer and the weight data corresponding to the set of activated neurons in the current neural network layer.

[0035] It should be noted that the computation performed here is sparse computation, meaning that computational operations are only performed on neurons in the set of activated neurons. Neurons not included in the set of activated neurons do not participate in this computation, and their corresponding weight data is not loaded or used. Specifically, the computation unit determines which neurons in the current layer have valid outputs based on the set of activated neurons, and only performs multiplication and accumulation operations on the output values ​​of these valid neurons with their corresponding weight data. The specific process of multiplication and accumulation is as follows: for each neuron in the set of activated neurons, the output value of that neuron is taken out, multiplied one by one with each weight value in the corresponding weight data, and the product result is accumulated into the input accumulator of each neuron in the next layer. After all activated neurons have been calculated, the value in the input accumulator of each neuron in the next layer is the output result of the current layer, that is, the input activation value of the next layer. Since the multiplication and accumulation operations of unactivated neurons are skipped, the total computational load is reduced from "the total number of neurons in the current layer multiplied by the total number of neurons in the next layer" to "the number of activated neurons multiplied by the total number of neurons in the next layer", and the computational load is reduced accordingly according to the activation ratio.

[0036] It should be noted that when performing the calculation of the current neural network layer in step S20, the process can be further refined by enabling or skipping calculation blocks. Specific details are explained in Example 2.

[0037] Step S30: During the calculation of the current neural network layer, based on the intermediate results of the calculation of the current neural network layer, determine the set of activated neurons for the next neural network layer, and based on the set of activated neurons for the next neural network layer, pre-fetch the weight data corresponding to the set of activated neurons for the next neural network layer.

[0038] It's important to note that intermediate results refer to the output values ​​of neurons that have already completed computation during the current layer's calculation process. The current layer's computation is not instantaneous—as the computational unit processes the neurons in the current layer sequentially or in blocks, the output values ​​of neurons that have completed computation earlier are already available, while subsequent neurons are still in the process of computation. These output values ​​of neurons that have completed computation constitute the intermediate results. As the current layer's computation progresses, more and more neurons complete computation, and the information contained in the intermediate results becomes increasingly richer.

[0039] It's important to note that both determining the set of activated neurons for the next layer and prefetching weight data are completed during the current layer's computation, rather than waiting for the current layer's computation to finish. This way, the weight data loading time is covered by the remaining computation time of the current layer, thus hiding data access latency. The method for determining the set of activated neurons for the next layer involves processing intermediate results using a lightweight prediction model to output the activation probability of each neuron in the next layer, thereby determining which neurons will be activated. After determining the set of activated neurons for the next layer, the prefetching operation immediately begins—based on the index of each neuron in the set, the corresponding weight data is located and read from external storage, and preloaded into the on-chip cache for direct use in the next step S10. The earlier the intermediate results are extracted, the longer the time window for the prefetching operation, but this also requires sufficient information in the intermediate results to support accurate prediction. Therefore, a balance needs to be struck between information sufficiency and the prefetching time window.

[0040] In this embodiment, through steps S10 to S30, the set of activated neurons for the next layer is determined in advance based on intermediate results during the current layer's computation, and the corresponding weight data is prefetched. This ensures that the decision-making and weight loading for the next layer's requirements are initiated before the current layer's computation is completed, thus achieving parallelization of computation and memory access. At the same time, only the weight data corresponding to the set of activated neurons needs to be loaded, rather than the full set of weights. The actual data transmission volume is reduced from the full set of weights to sparse weights, achieving efficient inference under bandwidth-constrained conditions and solving the technical problem of excessive data access volume during neural network inference under bandwidth-constrained conditions.

[0041] Example 2 Based on the first embodiment of this application, in the second embodiment of this application, the same or similar content as the first embodiment can be referred to the above description, and will not be repeated hereafter.

[0042] In one feasible implementation, such as Figure 2 As shown, the computation of the current neural network layer in step S20 can be achieved through steps S21 to S23: Step S21: Divide the computation of the current neural network layer into multiple computation blocks.

[0043] It should be noted that the division of computational blocks can be based on the topology of the neural network layers or the number of neurons. The division is done at the layer level of the neural network model, with each computational unit corresponding to one or more layers. When multiple layers are merged for computation, they correspond to multiple layers. Specific criteria for division include: for sequentially connected layers, such as the layer-by-layer connections of a fully connected layer, each layer corresponds to one computational unit; for layers with branching or skipping connections, such as residual connections or Inception modules, the layers before the branch merging point are divided into the same computational unit. When the number of neurons in a layer is too small, such as less than a certain threshold (typically 64), it is merged with adjacent layers into one computational unit to avoid fragmentation of computational resources. For example, a fully connected layer with 1024 neurons can be divided into 16 computational blocks, each containing computational tasks for 64 neurons. Each computational block corresponds to a set of computational hardware units that can be independently enabled or skipped, such as vector processing units. Computations within a block are executed continuously, and skipping between blocks is possible as needed. The purpose of dividing computation into computation blocks is to enable or skip computations on a block-by-block basis, which allows for more coarse-grained control over the allocation of computing resources. When all neurons in a computation block are not activated, the entire computation block can be skipped directly without having to judge each neuron individually, thus reducing control overhead.

[0044] Step S22: Enable or skip each computation block based on the set of activated neurons in the current neural network layer.

[0045] It should be noted that each computation block is traversed, and it is determined whether at least one neuron in the block belongs to the set of active neurons. If at least one neuron in the block is in the set of active neurons, the block is activated, marked as pending execution, and the corresponding hardware unit is allocated computing resources. If none of the neurons in the block are in the set of active neurons, the block is skipped, no computing resources are allocated, and the corresponding hardware unit can remain idle or enter a low-power mode. This skipping decision is made before computation begins, eliminating the need for neuron-by-neuron checks during computation, thus reducing the complexity of the control logic and runtime overhead.

[0046] Step S23: Based on the weight data in the enabled computation block that corresponds to the set of activated neurons in the current neural network layer, perform the computation of the current neural network layer.

[0047] It's important to note that for activated computation blocks, multiplication and accumulation operations are performed only on the neurons within that block that belong to the set of activated neurons. Inactive neurons within the block are not involved in the computation. For skipped computation blocks, no computation is performed at all, and the weight data corresponding to the neurons in those blocks is not loaded or used. Thus, computational resources are allocated only to the neurons that actually need to perform computations, while the hardware units corresponding to unselected computation blocks can remain idle or in a low-power state, reducing invalid computations and wasted computational resources. This two-layer sparse computing mechanism, combining block-level skipping and neuron-level skipping, maximizes computational efficiency while maintaining computational correctness.

[0048] In this embodiment, by dividing computation into computation blocks and enabling or skipping them at the block level according to the set of activated neurons, computation skipping occurs not only at the level of a single neuron but also at the level of computation blocks, further reducing invalid computation and waste of computational resources.

[0049] In another feasible implementation, determining the set of activated neurons for the next neural network layer based on the intermediate results in step S30 can be achieved through steps S31 to S32: Step S31: When the computation of the current neural network layer reaches the preset computation progress, extract the intermediate results that have been calculated by the current neural network layer as a partial activation state.

[0050] It should be noted that the preset computation progress refers to the proportion of completed neuron computations out of the total number of neurons in the current layer. This proportion needs to be balanced between prediction accuracy and the prefetching time window—if the proportion is too low, the number of completed neurons is small, the partial activation states contain insufficient information, and prediction accuracy is poor; if the proportion is too high, the time window for prefetching is too short, and it may not be possible to prefetch weight data before the current layer's computation is complete, thus failing to hide data access delays. For example, the preferred range for the preset computation progress is 50% to 90%, more preferably 70%. Taking 70% as an example, assuming the current layer has 1000 neurons, when the computation of the 700th neuron is completed, the retrieval operation is triggered, and the retrieved content is the output values ​​of the first 700 completed neurons. The output values ​​of these 700 neurons constitute partial activation states, reflecting the activation patterns of approximately 70% of the neurons in the current layer. Although 30% of the neurons have not yet completed computation, 70% of the activation patterns already contain enough information to predict the set of activated neurons in the next layer—because adjacent neurons in a neural network often have similar activation patterns, 70% sampling is sufficient to infer the overall activation distribution. Meanwhile, the remaining 30% of computation time provides a time window for prefetching operations, allowing the loading of weight data to overlap with the remaining computation time.

[0051] It should be noted that the monitoring of computation progress can be achieved by inserting progress checkpoints during the computation process. When the computation unit processes the neuron corresponding to the preset progress, a progress check is triggered, and the output values ​​of the completed neurons are extracted and assembled into a partial activation state. When using the computation block division method in steps S21 to S23, the progress checkpoint can be set after a certain computation block is completed—for example, when all neurons in the k-th computation block have been computed, check whether the cumulative proportion of completed neurons has reached the preset computation progress; if so, trigger the extraction operation.

[0052] Step S32: Based on the partial activation states, predict the set of activated neurons for the next neural network layer.

[0053] It should be noted that the specific method of prediction is as follows: Partial activation states are input into the prediction model. The prediction model is a lightweight model trained offline, such as a small fully connected network, decision tree, or lookup table. The reason for choosing a lightweight model is that the prediction process needs to be completed before the current layer finishes computation, and the prediction latency must be controlled within the remaining computation time of the current layer. Therefore, the computational complexity of the prediction model must be much lower than the computational complexity of the current layer itself. The training data for this prediction model consists of sample pairs of "partial activation states of the current layer to the actual set of activated neurons in the next layer." During training, the complete network model is used to perform inference on a large amount of input data, recording the partial activation states of each layer when the computation progress reaches a preset proportion, and the actual set of activated neurons in the next layer after the computation of that layer is completed, forming the training samples. The training objective is to minimize the difference between the predicted activation set and the actual activation set, for example, using cross-entropy loss or F1 score as the loss function. After training, the prediction model is deployed in the prediction unit, performing predictions online with low latency and low computational overhead during inference.

[0054] During inference, the prediction model receives partial activation states as input, calculates the activation probability of each neuron in the next layer through forward propagation, and outputs a probability vector. The length of the vector is equal to the total number of neurons in the next layer, and each element is the predicted activation probability of the corresponding neuron. Then, neurons with activation probabilities greater than a preset probability threshold (e.g., 0.5) are selected, and their indices are used to form the set of activated neurons for the next neural network layer. The latency of this prediction process is typically tens to hundreds of clock cycles, far lower than the clock cycles required for the remaining computation in the current layer (for example, at 70% completion, the remaining 30% computation typically requires thousands or even tens of thousands of clock cycles). Therefore, prediction can be completed comfortably before the current layer finishes computation.

[0055] It should be noted that as the inference proceeds layer by layer, after each layer's calculation is completed, the predicted set of activated neurons can be compared with the actual set of activated neurons calculated for that layer, and the prediction accuracy can be calculated—that is, the number of correctly predicted neurons divided by the total number of neurons. If the prediction accuracy is lower than a preset threshold, online fine-tuning or retraining of the prediction model can be triggered to maintain prediction accuracy.

[0056] In this embodiment, by setting a preset calculation progress and extracting a portion of the activation states for prediction when the progress is reached, the determination of the set of activated neurons in the next layer does not need to wait for the current layer to complete all calculations, thus realizing advance decision-making and providing a more ample time window for subsequent prefetching operations.

[0057] In another feasible implementation, after obtaining the initial set of activated neurons based on probability threshold filtering in step S32 above, considering the risk that threshold filtering may miss some neurons that will actually be activated (i.e., insufficient recall), the set can be expanded to improve the coverage of predictions. The expansion operation can be specifically implemented through steps S41 to S42: Step S41: Determine the initial set of activated neurons based on the partial activation states.

[0058] It should be noted that this step is basically the same as step S32: inputting a portion of the activation states into the prediction model to obtain the activation probability of each neuron in the next layer, and selecting neurons with activation probabilities greater than a preset probability threshold (e.g., 0.5) to form an initial set of activated neurons. This initial set contains neurons that the prediction model believes will be activated with high confidence; it is the core result of the prediction and the basis for subsequent expansion operations.

[0059] Step S42: Based on the correlation between each neuron in the next neural network layer that is not included in the initial set of activated neurons and the initial set of activated neurons, select some neurons whose correlation meets the preset conditions and add them to the initial set of activated neurons to obtain the set of activated neurons for the next neural network layer.

[0060] It should be noted that correlation measures the degree of activation similarity between neurons not included in the initial set and those in the initial set. A higher correlation indicates that the unselected neuron is more likely to be activated along with the neurons in the initial set. Correlation can be calculated in several ways, and one or more combinations can be chosen depending on the specific application scenario.

[0061] The first approach calculates correlation based on network topological proximity. Within the same neural network layer, adjacent neurons often exhibit similar activation patterns—because they typically receive input from similar regions in the previous layer. Based on this principle, the spatial distance (e.g., Euclidean distance in a 2D feature map) between neurons not included in the initial set and their topologically closest counterparts can be calculated. This distance is used as a correlation index; the smaller the distance, the higher the correlation. Neurons with distances less than a preset threshold are added to the initial set. This approach is suitable for feature layers with spatial locality in convolutional neural networks.

[0062] The second approach calculates the correlation based on a secondary threshold of activation probability. In step S41, only neurons with activation probabilities greater than a preset probability threshold (e.g., 0.5) are selected. For neurons whose activation probabilities are close to 0.5 (e.g., between 0.3 and 0.5), their actual activation probability is not low. The activation probability values ​​of these neurons can be directly used as the correlation index, and neurons with activation probabilities greater than a lower extended threshold (e.g., 0.3) are added to the initial set. This approach is simple to implement, directly connects to the output of the prediction model, and requires no additional calculations.

[0063] The third approach calculates the correlation based on historical co-activation statistics. During historical runs, the frequency of co-activation of each pair of neurons is recorded—if a neuron has historically been frequently activated simultaneously with certain neurons in the initial set, its correlation is considered high. This co-activation frequency statistic can be used to select neurons that frequently co-occur with neurons in the initial set but are not currently included.

[0064] After selecting a subset of neurons that meet the correlation criteria, these neurons are added to the initial set of activated neurons to form an expanded set of activated neurons. This expanded set retains both the high-confidence initial prediction neurons (obtained through probability thresholding) and candidate neurons that might have been missed (obtained through correlation expansion). This improves the recall and robustness of predictions while maintaining high prediction accuracy, and reduces the risk of subsequent calculations skipping correct neurons due to missed predictions.

[0065] In this embodiment, by determining the initial set of activated neurons and then selecting supplementary neurons based on correlation, the final set of activated neurons maintains high prediction accuracy while increasing coverage and reducing the risk of computational accuracy issues caused by prediction omissions.

[0066] In another feasible implementation, after determining the initial set of activated neurons in step S41, the initial set can be corrected using prediction error information accumulated during historical runs to further improve prediction accuracy. The correction operation can be specifically implemented through steps S51 to S52: Step S51: Obtain historical prediction error compensation information of the neural network.

[0067] It should be noted that the historical prediction error compensation information comes from the comparative statistics of each prediction result and the actual activation result in historical runs. Specifically, after each layer of calculation, the system compares the set of activated neurons predicted in step S32 or S42 with the set of actually activated neurons determined based on the actual output value after the completion of that layer's calculation, neuron by neuron. For neurons predicted as inactive but actually activated (missed prediction), the index of the neuron and the number of missed predictions are recorded; for neurons predicted as activated but actually not activated (misprediction), the index of the neuron and the number of mispredictions are recorded. After multiple inference runs, each neuron accumulates a certain amount of historical error statistics—that is, the frequency with which the neuron was missed or mispredicted in history. This statistical information constitutes the historical prediction error compensation information. The historical prediction error compensation information can reveal the systematic bias of the prediction model—some neurons, due to their special activation patterns, may be repeatedly missed or misjudged by the prediction model, and this pattern bias can be corrected through the compensation mechanism.

[0068] Step S52: Update the initial set of activated neurons based on historical prediction error compensation information.

[0069] It should be noted that the specific update method is as follows: Each neuron in the initial set of activated neurons is iterated over, and its misprediction frequency in the historical prediction error compensation information is checked. If the misprediction frequency exceeds a preset misprediction threshold, it indicates that the prediction model tends to overestimate the activation probability of that neuron, and it should be removed from the initial set of activated neurons. Simultaneously, each neuron not included in the initial set of activated neurons is iterated over, and its omission frequency in the historical prediction error compensation information is checked. If the omission frequency exceeds a preset omission threshold, it indicates that the prediction model tends to underestimate the activation probability of that neuron, and it should be added to the initial set of activated neurons even if its current activation probability is slightly lower than the preset probability threshold. The updated initial set of activated neurons replaces the original initial set, serving as the basis for the subsequent correlation expansion step (i.e., step S42). In this way, the bias information accumulated by the prediction model in successive runs is fed back into the prediction results, achieving continuous improvement in prediction accuracy.

[0070] In this embodiment, the initial set of activated neurons is corrected by introducing historical prediction error compensation information, so that the prediction results can be adaptively adjusted according to the prediction deviation in actual operation. As the number of runs increases, the prediction accuracy gradually improves, avoiding the systematic omission or misprediction problems that may occur under the fixed threshold screening method.

[0071] In another feasible implementation, the prediction of the set of activated neurons for the next neural network layer in step S32 can also employ a strategy based on neuron grouping and stepwise prediction, as implemented through steps S61 to S63: Step S61: Group the neurons of the next neural network layer to obtain neuron groups.

[0072] It should be noted that grouping involves dividing all neurons in the entire layer into several non-overlapping subsets according to certain rules. The purpose of grouping is to decompose a large-scale prediction problem into multiple smaller-scale prediction sub-problems. Directly performing activation prediction on thousands or even tens of thousands of neurons at once would require the prediction model to process high-dimensional inputs and high-dimensional outputs simultaneously, resulting in significant computational overhead. However, by grouping neurons, coarse-grained group-level prediction can be performed first to filter out candidate groups, and then fine-grained neuron-level prediction can be performed on the candidate groups, thereby reducing unnecessary computation.

[0073] Grouping can be based on several strategies. One approach is to group neurons uniformly according to their index order, for example, dividing 1024 neurons into 16 groups of 64 neurons each. This method is the simplest to implement and does not rely on additional information. Another approach is to group neurons according to their spatial location in the network topology. For example, for convolutional feature layers, neurons within the same local spatial region can be grouped together, leveraging spatial locality to improve intra-group prediction consistency. Yet another approach is to cluster neurons based on their activation frequency throughout historical operations, grouping neurons with similar activation frequencies into the same group—neurons with high activation frequencies often have similar activation triggering conditions, and placing them in the same group can improve intra-group prediction accuracy. Regardless of the grouping strategy used, the grouping is completed and fixed during model deployment and does not change during inference.

[0074] Step S62: Based on the partial activation state, perform activation prediction for each neuron group to obtain the set of activated neurons for each neuron group.

[0075] It's important to note that the prediction isn't performed all at once for all neurons in the layer, but rather progressively from the group level to the neuron level. First, group-level prediction is performed: partial activation states are input into the group-level predictor, which outputs the result indicating whether an active neuron exists within each group. The group-level predictor can be a lightweight model with a structure similar to the full-layer prediction model but with an output dimension equal to the number of groups, or it can be based on rules derived from historical activation statistics of neurons in each group (e.g., if a group has never had an active neuron in a similar partial activation state historically, it is directly judged as having no activation). Groups determined by the group-level prediction to have no active neurons are skipped, saving computational overhead. For groups determined by the group-level prediction to have active neurons, neuron-level prediction is then performed: partial activation states and the group's identifier information are input into the neuron-level predictor, which outputs the activation probability of each neuron within that group. The set of active neurons for that group is then selected based on a probability threshold. Since the output dimension of the neuron-level predictor is only the number of neurons within that group, much smaller than the total number of neurons in the entire layer, the computational cost of a single prediction is significantly reduced. Neuron-level predictions between groups can be executed serially or in parallel—serial execution saves predictor hardware resources, while parallel execution shortens prediction latency.

[0076] Step S63: Merge the sets of activated neurons from each neuron group to obtain the set of activated neurons for the next neural network layer.

[0077] It should be noted that the merging method involves taking the union of the activation neuron indices of the predicted outputs from each group. Since the groups are non-overlapping, the activation neuron indices of the outputs from each group will naturally not be repeated; therefore, the merging operation is essentially a simple concatenation.

[0078] In this embodiment, by grouping neurons and employing a hierarchical prediction approach from the group level to the neuron level, the large-scale activation prediction is decomposed into hierarchical sub-problems—first, groups without activation are quickly eliminated at the coarse-grained level, and then candidate groups are accurately predicted at the fine-grained level. This hierarchical strategy makes the computational cost of prediction grow approximately linearly with the total number of neurons rather than quadratically, significantly reducing prediction overhead and latency when the number of neurons is large (e.g., tens of thousands).

[0079] In another feasible implementation, such as Figure 3 As shown, the prefetching of weight data for the next neural network layer in step S30 can be achieved through steps S71 to S73: Step S71: Convert the index of each activated neuron in the set of activated neurons in the next neural network layer into the physical address of the corresponding weight data.

[0080] It should be noted that the weight data corresponding to each neuron has a definite physical address in external storage. For the next layer of the neural network, each neuron corresponds to a set of weight parameters—that is, the connection weights between this neuron and all neurons in the next layer. This set of weight parameters occupies a contiguous physical address space in external storage. The conversion from neuron index to physical address is based on a pre-established mapping relationship, which is generated and stored in the system by the model compilation tool during model deployment. The process of establishing the mapping relationship is as follows: During the model deployment phase, the model compilation tool writes the weight data of the entire model into external storage according to a preset storage layout and records the starting physical address and data length of the weight data corresponding to each neuron, forming a one-to-one correspondence table between neuron indexes and physical addresses. During inference, the index of any neuron can be quickly converted into the physical address of its weight data by looking up the table. For example, for the activated neuron with index i in the next layer, looking up the table yields its weight data starting physical address as addr_i and data length as len_i, which is usually the total number of bytes of the connection weight parameters between this neuron and all neurons in the next layer.

[0081] Step S72: Sort and merge the physical addresses to generate a continuous memory access sequence.

[0082] It should be noted that the physical address sets obtained in step S71 are usually disordered in terms of address values—because the predicted set of activated neurons is logically sparse, the indices of each activated neuron are not contiguous, and their corresponding physical addresses are also not contiguous. If read requests are initiated directly according to these disordered addresses one by one, the external storage controller will perform a large number of random access operations. Each random access requires going through a complete process such as row activation and column gating, resulting in extremely low bandwidth utilization. Therefore, it is necessary to sort and merge the physical addresses to change the access mode from random access to contiguous access.

[0083] The sorting process involves arranging all the physical addresses obtained in step S71 in ascending order of their address values. After sorting, the originally chaotic physical addresses become ordered, providing a basis for subsequent identification of contiguous address segments that can be merged.

[0084] Merging refers to scanning the sorted sequence of physical addresses to identify multiple physical addresses that are adjacent or located within the same memory page (e.g., the same row address range in DRAM), and merging them into a larger access request. The rationale for merging is that if multiple physical addresses are contiguous in storage space or located within the same memory page, the external storage controller can read multiple data blocks consecutively after a single row activation, avoiding the overhead of multiple row activations. For example, suppose there are two address ranges after sorting: the first is from addr_a to addr_a+len_a, and the second is from addr_b to addr_b+len_b, where addr_b is exactly equal to addr_a+len_a (i.e., contiguous). These two ranges can be merged into a single consecutive access request from addr_a to addr_b+len_b. If there is a small gap between two ranges but they are within the same memory page, they can also be merged into a single request; the data in the gap can be skipped during reading.

[0085] After sorting and merging, what is obtained is a sequence of one or more consecutive storage access requests—a storage access sequence. Each access request in the storage access sequence contains a starting physical address and a consecutive read length, with the addresses of adjacent access requests being as consecutive or approximately consecutive as possible. This allows subsequent prefetch operations to be executed efficiently in batch read mode.

[0086] Step S73: Based on the storage access sequence, prefetch the weight data corresponding to the activated neurons of the next neural network layer from the external storage.

[0087] It should be noted that, since the items in the storage access sequence generated in step S72 are contiguous in physical address, the external storage controller can execute each access request in the sequence item by item in batch read mode. In batch read mode, the external storage controller sends the starting address and read length once, and the external storage device (such as DRAM) can automatically complete the data reading and transfer within the contiguous address range without re-initiating a read command for each data block. This significantly reduces command interaction overhead and row activation counts, improving the utilization of actual effective bandwidth.

[0088] The prefetched weight data is loaded into the on-chip cache. In step S10 of the next loop, the system directly retrieves the weight data corresponding to the set of activated neurons in the current layer from the on-chip cache, without needing to access external storage again. Since the access latency of the on-chip cache is much lower than that of external storage (typically a few clock cycles compared to tens to hundreds of clock cycles), the computing unit can retrieve the required weight data with almost no waiting time, further improving inference efficiency.

[0089] For example, suppose there are 1000 predictive activation neurons in the next layer, and each neuron corresponds to 100 weight parameters (i.e., connection weights with the 100 neurons in the next layer). The traditional random access method requires 1000 independent weight read requests, reading 100 parameters each time, for a total of 1000 random accesses. However, after sorting and merging processing in this embodiment, if the distribution of these 1000 neurons in the physical address can be merged into 10 consecutive address segments, only 10 batch read requests are needed, reading 10,000 parameters at a time, reducing the number of random accesses to one percent of the original, and significantly improving the bandwidth utilization of external storage.

[0090] In this embodiment, by converting the activated neuron index into a physical address and generating a continuous storage access sequence through sorting and merging, prefetching is performed. This maps the logically sparse set of activated neurons (with discontinuous indexes) to a physically continuous access sequence, significantly reducing the number of random accesses to external storage and improving the bandwidth utilization efficiency of weight data reading. This allows for efficient prefetching of weight data even under bandwidth-constrained conditions.

[0091] It should be noted that the technical features in the above embodiments can be arbitrarily combined according to actual needs, and as long as the combined technical solutions do not contradict each other, they all fall within the protection scope of this application. For example, the implementation method of correcting the initial set based on historical error compensation can be combined with the implementation method of expanding the set based on correlation degree—first updating the initial set through historical error compensation, and then expanding the correlation degree based on the updated initial set to achieve dual optimization; the implementation method of grouped step-by-step prediction can also be combined with the implementation method of correlation degree expansion—expanding within each group and then merging. Similarly, the implementation method of using computational blocks to enable or skip can be combined with any of the above prediction optimization implementation methods. This application does not specifically limit the combination method between the various implementation methods.

[0092] It should be noted that the above examples are only for understanding this application and do not constitute a limitation on the control method for the execution of the neural network in this application. Any simple transformations based on this technical concept are within the protection scope of this application.

[0093] This application also provides a control device for neural network execution; please refer to... Figure 4 The control device executed by the neural network includes: Data module 10 is used to obtain the set of activated neurons in the current neural network layer and the weight data corresponding to the set of activated neurons in the current neural network layer; The calculation module 20 is used to perform calculations on the current neural network layer based on the set of activated neurons and the weight data corresponding to the set of activated neurons in the current neural network layer. The weight prefetching module 30 is used to determine the set of activated neurons for the next neural network layer based on the intermediate results of the current neural network layer calculation during the execution of the current neural network layer calculation, and to prefetch the weight data corresponding to the set of activated neurons for the next neural network layer based on the set of activated neurons for the next neural network layer.

[0094] The neural network execution control device provided in this application, employing the neural network execution control method in the above embodiments, can solve the technical problem of reducing the amount of data accessed during neural network inference under bandwidth-constrained conditions. Compared with the prior art, the beneficial effects of the neural network execution control device provided in this application are the same as those of the neural network execution control method provided in the above embodiments, and other technical features in the neural network execution control device are the same as those disclosed in the methods of the above embodiments, and will not be repeated here.

[0095] This application provides a control device for neural network execution, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the control method for neural network execution in the above embodiment 1.

[0096] The following is for reference. Figure 5 This document illustrates a schematic diagram of a control device suitable for implementing neural network execution in the embodiments of this application. The control device for neural network execution in the embodiments of this application may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Description), PMPs (Portable Media Players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 5 The control device for neural network execution shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.

[0097] like Figure 5As shown, the control device for neural network execution may include a processing unit 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in read-only memory 1002 or a program loaded from storage device 1003 into random access memory 1004. Random access memory 1004 also stores various programs and data required for the operation of the control device for neural network execution. The processing unit 1001, read-only memory 1002, and random access memory 1004 are interconnected via bus 1005. Input / output interface 1006 is also connected to the bus. Typically, the following systems can be connected to input / output interface 1006: input devices 1007 including, for example, touch screens, touchpads, keyboards, mice, image sensors, microphones, accelerometers, gyroscopes, etc.; output devices 1008 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 1003 including, for example, magnetic tapes, hard disks, etc.; and communication devices 1009. The communication device 1009 allows the control device executing the neural network to communicate wirelessly or wiredly with other devices to exchange data. Although the figure shows control devices executing the neural network with various systems, it should be understood that it is not required to implement or possess all of the systems shown. More or fewer systems may be implemented alternatively.

[0098] Specifically, according to the embodiments disclosed in this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from storage device 1003, or installed from read-only memory 1002. When the computer program is executed by processing device 1001, it performs the functions defined in the methods of the embodiments disclosed in this application.

[0099] The neural network execution control device provided in this application, employing the neural network execution control method in the above embodiments, can solve the technical problem of reducing the amount of data accessed during neural network inference under bandwidth-constrained conditions. Compared with the prior art, the beneficial effects of the neural network execution control device provided in this application are the same as those of the neural network execution control method provided in the above embodiments, and other technical features in this neural network execution control device are the same as those disclosed in the previous embodiment method, and will not be repeated here.

[0100] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.

[0101] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0102] This application provides a computer-readable storage medium having computer-readable program instructions (i.e., a computer program) stored thereon, the computer-readable program instructions being used to execute the control method for neural network execution in the above embodiments.

[0103] The computer-readable storage medium provided in this application may be, for example, a USB flash drive, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems or devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.

[0104] The aforementioned computer-readable storage medium may be included in a control device that performs neural network execution; or it may exist independently and not be assembled into a control device that performs neural network execution.

[0105] The aforementioned computer-readable storage medium carries one or more programs, which, when executed by a control device for executing a neural network, cause the control device for executing the neural network to: Obtain the set of activated neurons in the current neural network layer and the weight data corresponding to the set of activated neurons in the current neural network layer; Based on the set of activated neurons in the current neural network layer and the weight data corresponding to the set of activated neurons in the current neural network layer, the calculation of the current neural network layer is performed. During the calculation of the current neural network layer, based on the intermediate results of the calculation of the current neural network layer, the set of activated neurons for the next neural network layer is determined, and based on the set of activated neurons for the next neural network layer, the weight data corresponding to the set of activated neurons for the next neural network layer is pre-fetched.

[0106] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a Local Area Network (LAN) or a Wide Area Network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0107] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0108] The modules described in the embodiments of this application can be implemented in software or hardware. The names of the modules do not necessarily limit the functionality of the unit itself.

[0109] The readable storage medium provided in this application is a computer-readable storage medium that stores computer-readable program instructions (i.e., a computer program) for executing the control method for the above-described neural network execution. This solves the technical problem of reducing the amount of data accessed during neural network inference under bandwidth-constrained conditions. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as those of the control method for neural network execution provided in the above embodiments, and will not be repeated here.

[0110] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the control method for neural network execution as described above.

[0111] The computer program product provided in this application solves the technical problem of reducing the amount of data accessed during neural network inference under bandwidth-constrained conditions. Compared with the prior art, the beneficial effects of the computer program product provided in this application are the same as those of the neural network execution control method provided in the above embodiments, and will not be repeated here.

[0112] The above description is only a part of the embodiments of this application and does not limit the patent scope of this application. All equivalent structural transformations made under the technical concept of this application and using the contents of the specification and drawings of this application, or direct / indirect applications in other related technical fields, are included in the patent protection scope of this application.

Claims

1. A control method for neural network execution, characterized in that, The method includes: Obtain the set of activated neurons in the current neural network layer and the weight data corresponding to the set of activated neurons in the current neural network layer; Based on the set of activated neurons in the current neural network layer and the weight data corresponding to the set of activated neurons in the current neural network layer, the calculation of the current neural network layer is performed. During the calculation of the current neural network layer, based on the intermediate results of the calculation of the current neural network layer, the set of activated neurons for the next neural network layer is determined, and based on the set of activated neurons for the next neural network layer, the weight data corresponding to the set of activated neurons for the next neural network layer is pre-fetched.

2. The method as described in claim 1, characterized in that, The step of performing calculations for the current neural network layer based on the set of activated neurons and the corresponding weight data of the set of activated neurons in the current neural network layer includes: The computation of the current neural network layer is divided into multiple computation blocks; Based on the set of activated neurons in the current neural network layer, each computation block is enabled or skipped; The computation of the current neural network layer is performed based on the weight data corresponding to the set of activated neurons in the enabled computation block.

3. The method as described in claim 1, characterized in that, The step of determining the set of activated neurons for the next neural network layer based on the intermediate results calculated from the current neural network layer includes: When the computation of the current neural network layer reaches the preset computation progress, the intermediate results that the current neural network layer has calculated are extracted as a partial activation state. Based on the partial activation states, predict the set of activated neurons in the next neural network layer.

4. The method as described in claim 3, characterized in that, The step of predicting the set of activated neurons in the next neural network layer based on the partial activation states includes: Based on the aforementioned partial activation states, an initial set of activated neurons is determined; Based on the correlation between each neuron in the next neural network layer that is not included in the initial set of activated neurons and the initial set of activated neurons, select some neurons whose correlation meets a preset condition and add them to the initial set of activated neurons to obtain the set of activated neurons for the next neural network layer.

5. The method as described in claim 4, characterized in that, Following the step of determining the initial set of activated neurons based on the partial activation states, the method further includes: Obtain historical prediction error compensation information of the neural network; The initial set of activated neurons is updated based on the historical prediction error compensation information.

6. The method as described in claim 3, characterized in that, The step of predicting the set of activated neurons in the next neural network layer based on the partial activation states includes: The neurons of the next neural network layer are grouped to obtain neuron groups; Based on the partial activation states, activation prediction is performed on each neuron group to obtain the set of activated neurons for each neuron group. The sets of activated neurons in each of the aforementioned neuron groups are merged to obtain the set of activated neurons in the next neural network layer.

7. The method as described in claim 1, characterized in that, The step of pre-fetching the weight data corresponding to the set of activated neurons in the next neural network layer includes: The index of each activated neuron in the set of activated neurons in the next neural network layer is converted into the physical address of the corresponding weight data. The physical addresses are sorted and merged to generate a continuous memory access sequence; According to the storage access sequence, the weight data corresponding to the activated neurons of the next neural network layer are prefetched from the external storage.

8. A control device for neural network execution, characterized in that, The control device executed by the neural network includes: The data module is used to obtain the set of activated neurons in the current neural network layer and the weight data corresponding to the set of activated neurons in the current neural network layer. The calculation module is used to perform calculations for the current neural network layer based on the set of activated neurons and the weight data corresponding to the set of activated neurons in the current neural network layer. The weight prefetching module is used to determine the set of activated neurons for the next neural network layer based on the intermediate results of the current neural network layer calculation during the execution of the current neural network layer calculation, and to prefetch the weight data corresponding to the set of activated neurons for the next neural network layer based on the set of activated neurons for the next neural network layer.

9. A control device for neural network execution, characterized in that, The device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the control method for neural network execution as described in any one of claims 1 to 7.

10. A storage medium, characterized in that, The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, it implements the steps of the control method for neural network execution as described in any one of claims 1 to 7.

Images