Network parameter processing method, task processing method and corresponding devices
By dividing the neurons of the large model into multiple packets and filtering and loading according to activation information, the problem of storage and bandwidth pressure of the large model on the end device is solved, and the model's inference efficiency and acceleration performance are improved.
Patent Information
- Application Number
- CN202411201416.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-29
- Publication Date
- 2024-12-03
AI Technical Summary
Due to the high requirements of storage space and computing bandwidth when deploying large models on end devices, existing technologies such as model quantization, model pruning and low-rank decomposition are difficult to effectively reduce storage and transmission pressure, especially under different inputs, the parameters are of different importance. Difficult to adaptive compression processing.
The neurons of the preset model are divided into multiple neuron groups, and the packets with a larger proportion of distribution are filtered as the target group based on the predicted activation information, and their network parameter information is continuously stored and loaded to reduce unnecessary parameter transmission. Improve the efficiency of model inference for dynamic loading.
By reducing storage and bandwidth pressure, the model's inference acceleration performance on the end device is improved, and the activation neuron network parameter information of random single read operations is increased, which alleviates the bandwidth loss of the target device.
Smart Images

Figure CN119067182A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the field of computer technology, and in particular to a network parameter processing method, a task processing method, a network parameter processing device, a task processing device, an electronic device, and a computer-readable storage medium. Background Art
[0002] Large models (LM) may have a huge number of parameters exceeding 100 billion, which brings huge challenges to the deployment and application of large models. In addition, as users' demands for privacy, timeliness, etc. increase, it has become necessary to deploy large models locally on mobile phones, laptops and other end devices, which puts higher requirements on reducing the storage space of large models and the transmission bandwidth during operation. In related technologies, the above problems are mainly solved by weight compression, which mainly involves technical means such as model quantization, model pruning, and low-rank decomposition. Summary of the invention
[0003] The present disclosure provides a network parameter processing method, a task processing method, a network parameter processing device, a task processing device, an electronic device, and a computer-readable storage medium.
[0004] In a first aspect, the present disclosure provides a network parameter processing method that can be applied to a preset model, wherein the preset model includes multiple neuron groups, each of the neuron groups includes multiple neurons, and the network parameter information of the multiple neurons in the same neuron group is continuously stored in a preset storage space, and the multiple neuron groups meet the preset simultaneous release conditions. The network parameter processing method includes: selecting a target neuron group from the multiple neuron groups according to the predicted activation information of each of the neuron groups, the predicted activation information is used to predict the pulse release information of each neuron in the neuron group; loading the network parameter information of the target neuron group from the preset storage space to a target device, so that the target device performs data processing of the preset model based on the network parameter information of the target neuron group.
[0005] In a second aspect, the present disclosure provides a task processing method, which is applied to a target device. The task processing method includes: performing data processing based on network parameter information of a target neuron group loaded from a preset storage space, the correspondence between the network parameter information and neurons in a preset model, and input data of the task to be processed to obtain a task processing result; wherein the network parameter information of the target neuron group is obtained according to the network parameter processing method described in any one of the embodiments of the present disclosure.
[0006] In a third aspect, the present disclosure provides a network parameter processing device that can be applied to a preset model, wherein the preset model includes multiple neuron groups, each of the neuron groups includes multiple neurons, and the network parameter information of the multiple neurons in the same neuron group is continuously stored in a preset storage space, and the multiple neuron groups meet the preset simultaneous release conditions. The network parameter processing device includes: a selection module, which is used to select a target neuron group from the multiple neuron groups according to the predicted activation information of each neuron group, and the predicted activation information is used to predict the pulse release information of each neuron in the neuron group; a loading module, which is used to load the network parameter information of the target neuron group from the preset storage space to the target device, so that the target device can perform data processing of the preset model based on the network parameter information of the target neuron group.
[0007] In a fourth aspect, the present disclosure provides a task processing device, which is applied to a target device, and the task processing device includes: a processing module, which is used to perform data processing based on network parameter information of a target neuron group loaded from a preset storage space, the correspondence between the network parameter information and neurons in a preset model, and input data of the task to be processed to obtain a task processing result; wherein the network parameter information of the target neuron group is obtained according to the network parameter processing method described in any one of the embodiments of the present disclosure.
[0008] In a fifth aspect, the present disclosure provides an electronic device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores one or more computer programs executable by the at least one processor, and one or more of the computer programs are executed by the at least one processor so that the at least one processor can execute the network parameter processing method or task processing method described in any one of the embodiments of the present disclosure.
[0009] In a sixth aspect, the present disclosure provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the network parameter processing method or task processing method described in any one of the embodiments of the present disclosure.
[0010] In the embodiments provided by the present disclosure, a preset model includes multiple neuron groups, each neuron group includes multiple neurons, network parameter information of multiple neurons in the same neuron group is continuously stored in a preset storage space, and multiple neuron groups meet preset simultaneous release conditions, and a network parameter processing method mainly includes: selecting a target neuron group from multiple neuron groups according to predicted activation information of each neuron group, the predicted activation information is used to predict the pulse release information of each neuron in the neuron group; loading the network parameter information of the target neuron group from the preset storage space to the target device, so that the target device performs data processing of the preset model based on the network parameter information of the target neuron group.
[0011] It can be seen that since the neurons of the preset model are divided into multiple neuron groups in advance, and the neuron groups meet the simultaneous firing conditions, it is more likely that multiple neurons in the same neuron group tend to fire simultaneously (or not fire at the same time), and the network parameter information of multiple neurons in the same neuron group is continuously stored in the preset storage space, so that the network parameter information of the preset model is centrally stored at the granularity of neuron groups. Based on this, when it is necessary to load the network parameter information to the target device, the proportion of multiple neurons in each neuron group firing pulse signals simultaneously can be predicted according to the predicted activation information of each neuron group, so as to screen out the neuron group with a larger proportion of firing pulse signals simultaneously as the target neuron group to be loaded, and then the network parameter information corresponding to the target neuron group is loaded from the preset storage space to the target device in units of neuron groups. In this way, on the one hand, since only the network parameter information of the neuron group with a larger simultaneous activation ratio (i.e., the target neuron group) is loaded into the target device, there is no need to load the network parameter information of the neuron group with a smaller simultaneous activation ratio, thereby reducing the amount of data of the network parameter information that needs to be loaded, effectively alleviating the storage pressure and bandwidth transmission pressure on the target device. On the other hand, since the neuron grouping is determined by analyzing the correlation of the pulse signals emitted by the neurons, the neurons with a higher probability of simultaneously emitting pulse signals are divided into the same neuron group, and the network parameter information of these neurons is stored adjacently. Compared with the related technology of storing network parameter information at the granularity of a single neuron, this storage method increases the network parameter information of the activated neurons obtained by a random single read operation, thereby further alleviating the bandwidth loss of the target device and improving the model inference acceleration advantage brought by dynamically loading network parameters.
[0012] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS
[0013] The accompanying drawings are used to provide a further understanding of the present disclosure and constitute a part of the specification. Together with the embodiments of the present disclosure, they are used to explain the present disclosure and do not constitute a limitation of the present disclosure. The above and other features and advantages will become more apparent to those skilled in the art by describing detailed example embodiments with reference to the accompanying drawings, in which:
[0014] Figure 1 A flowchart of a network parameter processing method provided by an embodiment of the present disclosure.
[0015] Figure 2 A flowchart of a network parameter processing method provided in an embodiment of the present disclosure.
[0016] Figure 3 A flowchart of a task processing method provided by an embodiment of the present disclosure.
[0017] Figure 4 A block diagram of a network parameter processing device provided in an embodiment of the present disclosure.
[0018] Figure 5 A block diagram of a task processing device provided in an embodiment of the present disclosure.
[0019] Figure 6 A block diagram of an electronic device provided in accordance with a first embodiment of the present disclosure.
[0020] Figure 7 A block diagram of an electronic device provided in Embodiment 2 of the present disclosure. DETAILED DESCRIPTION
[0021] In order to enable those skilled in the art to better understand the technical solutions of the present disclosure, the exemplary embodiments of the present disclosure are described below in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, those of ordinary skill in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, the description of well-known functions and structures is omitted in the following description.
[0022] In the absence of conflict, the various embodiments of the present disclosure and the various features therein may be combined with each other.
[0023] As used herein, the term "and / or" includes any and all combinations of one or more of the associated listed items.
[0024] The terms used herein are only used to describe specific embodiments and are not intended to limit the present disclosure. As used herein, the singular forms "a" and "the" are also intended to include plural forms, unless the context clearly indicates otherwise. It will also be understood that when the terms "including" and / or "made of" are used in this specification, the presence of the features, wholes, steps, operations, elements and / or components is specified, but the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or groups thereof is not excluded. "Connected" or "connected" and similar words are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect.
[0025] Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by those of ordinary skill in the art. It will also be understood that terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant art and the present disclosure, and will not be interpreted as having an idealized or overly formal meaning unless explicitly defined as such herein.
[0026] Large models (for example, GPT-3 (Generative Pre-trained Transformer-3), GLM (General Language Model) and other large language models (LLM)) have a huge number of network parameters exceeding 100 billion, which brings huge challenges to model deployment and application. In addition, with the growing demand for privacy protection and timeliness, users may need to deploy large models locally on the end device, which requires further reducing the storage space of LM and the transmission bandwidth during operation.
[0027] In related technologies, the above problems are usually solved by parameter compression, which mainly involves technical means such as model quantization, model pruning, and low-rank decomposition. Among them, model quantization refers to reducing the amplitude representation accuracy of model parameters, such as compressing the 16-bit floating point number (fp16) representation of the parameter into an 8-bit integer (int8) or a 4-bit integer (int4) or a lower bit width representation. Compared with the model performance of fp16, int4 precision quantization did not bring about a significant decline. Model pruning refers to discarding unimportant parameters in the model parameters to achieve parameter sparsity. Low-rank decomposition refers to using the low-rank characteristics of the parameter matrix to discard some of the singular values and singular vectors with less influence, decomposing the large parameter matrix into the product of small parameter matrices, thereby reducing the parameter storage amount.
[0028] Although the relevant technology has effectively reduced the total storage volume of the model, it is still difficult to deploy it on the end device. In addition, in the model inference stage, the bandwidth to computation ratio of large models is large, so parameter transmission is one of the main bottlenecks that limit the overall inference speed of the model. However, the above methods mostly use static weight quantization or loading methods, which do not effectively take into account the different importance of different rows (columns) of parameters under different inputs, making it difficult to adaptively compress the model parameters according to the input.
[0029] In view of this, the embodiments of the present disclosure provide a network parameter processing method, a task processing method, a network parameter processing device, a task processing device, an electronic device, and a computer-readable storage medium.
[0030] In the embodiment of the present disclosure, the preset model includes multiple neuron groups, each neuron group includes multiple neurons, the network parameter information of multiple neurons in the same neuron group is continuously stored in the preset storage space, and the multiple neuron groups meet the preset simultaneous release conditions, and the network parameter processing method mainly includes: according to the predicted activation information of each neuron group, a target neuron group is selected from multiple neuron groups, and the predicted activation information is used to predict the pulse release information of each neuron in the neuron group; the network parameter information of the target neuron group is loaded from the preset storage space to the target device, so that the target device can perform data processing of the preset model based on the network parameter information of the target neuron group. Among them, the target device belongs to the end device, which can be a hardware device with data processing capabilities such as a chip, and the embodiment of the present disclosure does not limit this.
[0031] It can be seen that since the neurons of the preset model are divided into multiple neuron groups in advance, and the neuron groups meet the simultaneous firing conditions, it is more likely that multiple neurons in the same neuron group tend to fire simultaneously (or not fire at the same time), and the network parameter information of multiple neurons in the same neuron group is continuously stored in the preset storage space, so that the network parameter information of the preset model is centrally stored at the granularity of neuron groups. Based on this, when it is necessary to load the network parameter information to the target device, the proportion of multiple neurons in each neuron group firing pulse signals simultaneously can be predicted according to the predicted activation information of each neuron group, so as to screen out the neuron group with a larger proportion of firing pulse signals simultaneously as the target neuron group to be loaded, and then the network parameter information corresponding to the target neuron group is loaded from the preset storage space to the target device in units of neuron groups. In this way, on the one hand, since only the network parameter information of the neuron group with a larger simultaneous activation ratio (i.e., the target neuron group) is loaded into the target device, there is no need to load the network parameter information of the neuron group with a smaller simultaneous activation ratio, thereby reducing the amount of data of the network parameter information that needs to be loaded, effectively alleviating the storage pressure and bandwidth transmission pressure on the target device. On the other hand, since the neuron grouping is determined by analyzing the correlation of the pulse signals emitted by the neurons, the neurons with a higher probability of simultaneously emitting pulse signals are divided into the same neuron group, and the network parameter information of these neurons is stored adjacently. Compared with the related technology of storing network parameter information at the granularity of a single neuron, this storage method increases the network parameter information of the activated neurons obtained by a random single read operation, thereby further alleviating the bandwidth loss of the target device and improving the model inference acceleration advantage brought by dynamically loading network parameters.
[0032] A first aspect of an embodiment of the present disclosure provides a network parameter processing method.
[0033] Figure 1 A flowchart of a network parameter processing method provided by an embodiment of the present disclosure, the network parameter processing method can be applied to a preset model, the preset model includes multiple neuron groups, each neuron group includes multiple neurons, the network parameter information of multiple neurons in the same neuron group is continuously stored in a preset storage space, and the multiple neuron groups meet the preset simultaneous release conditions. Figure 1 , the network parameter processing method may include the following steps.
[0034] Step S11, selecting a target neuron group from multiple neuron groups according to the predicted activation information of each neuron group, the predicted activation information is used to predict the pulse emission information of each neuron in the neuron group.
[0035] Step S12, loading the network parameter information of the target neuron group from the preset storage space to the target device, so that the target device can perform data processing of the preset model based on the network parameter information of the target neuron group.
[0036] For example, the preset model may be a model constructed based on a neural network, and the preset model includes a plurality of neurons, and these neurons are connected according to a certain network topology and can perform corresponding data processing functions.
[0037] In some optional embodiments, the target device is an AI acceleration chip for accelerating the processing of a preset model, such as a computing chip based on a multi-core architecture; the preset storage space can be an off-chip storage of the AI acceleration chip, such as an external memory storage of the AI acceleration chip. The network parameter processing method is not only applicable to large models, but also to other models composed of linear operators, including brain simulation networks.
[0038] In related technologies, if you want to deploy or run a preset model on an end device, you need to load all the network parameters of the preset model into the end device. For situations where the preset model is large in scale, such loading of network parameters will inevitably put a lot of pressure on the storage and bandwidth of the target device, and may even make it impossible to deploy or run the preset model on the end device. Based on this, some sparse methods can be used to reduce the amount of data of the network parameter information (such as weight parameters) that needs to be loaded, while taking into account the processing performance of the model.
[0039] For example, when a row of weights is read sparsely based on the sparsity of the input data, subsequent weight rows (or weight columns) are actively cached. If the subsequent weight rows do not belong to activated weight rows (such as weight parameters corresponding to neurons that do not emit pulse signals), there is no need to transmit the weight row to the target device, thus wasting the cache and reducing bandwidth utilization.
[0040] For another example, the weight parameters read in an existing read operation may include both activation weights and inactivation weights, thereby reducing the data transmission bandwidth.
[0041] Based on this, in an embodiment of the present disclosure, a preset model is pre-divided into multiple neuron groups, and multiple neurons in each neuron group tend to fire at the same time or tend not to fire at the same time, and the network parameter information of multiple neurons in the group is continuously stored in units of neuron groups for subsequent continuous reading / caching. After the above processing, when loading the network parameter information to the target device, the firing conditions of multiple neurons in each neuron group are predicted based on the predicted activation information of each neuron group, and the neuron group with a larger firing ratio is selected as the target neuron group, and then the network parameter information of the target neuron group is read out and loaded into the target device, so that the target device can perform corresponding data processing using the loaded network parameter information.
[0042] In summary, in the embodiments provided by the present disclosure, the preset model includes multiple neuron groups, each neuron group includes multiple neurons, the network parameter information of multiple neurons in the same neuron group is continuously stored in a preset storage space, and the multiple neuron groups meet the preset simultaneous release conditions, and the network parameter processing method mainly includes: according to the predicted activation information of each neuron group, a target neuron group is selected from multiple neuron groups, and the predicted activation information is used to predict the pulse release information of each neuron in the neuron group; the network parameter information of the target neuron group is loaded from the preset storage space to the target device, so that the target device performs data processing of the preset model based on the network parameter information of the target neuron group.
[0043] It can be seen that since the neurons of the preset model are divided into multiple neuron groups in advance, and the neuron groups meet the simultaneous firing conditions, it is more likely that multiple neurons in the same neuron group tend to fire simultaneously (or not fire at the same time), and the network parameter information of multiple neurons in the same neuron group is continuously stored in the preset storage space, so that the network parameter information of the preset model is centrally stored at the granularity of neuron groups. Based on this, when it is necessary to load the network parameter information to the target device, the proportion of multiple neurons in each neuron group firing pulse signals simultaneously can be predicted according to the predicted activation information of each neuron group, so as to screen out the neuron group with a larger proportion of firing pulse signals simultaneously as the target neuron group to be loaded, and then the network parameter information corresponding to the target neuron group is loaded from the preset storage space to the target device in units of neuron groups. In this way, on the one hand, since only the network parameter information of the neuron group with a larger simultaneous activation ratio (i.e., the target neuron group) is loaded into the target device, there is no need to load the network parameter information of the neuron group with a smaller simultaneous activation ratio, thereby reducing the amount of data of the network parameter information that needs to be loaded, effectively alleviating the storage pressure and bandwidth transmission pressure on the target device. On the other hand, since the neuron grouping is determined by analyzing the correlation of the pulse signals emitted by the neurons, the neurons with a higher probability of simultaneously emitting pulse signals are divided into the same neuron group, and the network parameter information of these neurons is stored adjacently. Compared with the related technology of storing network parameter information at the granularity of a single neuron, this storage method increases the network parameter information of the activated neurons obtained by a random single read operation, thereby further alleviating the bandwidth loss of the target device and improving the model inference acceleration advantage brought by dynamically loading network parameters.
[0044] The network parameter processing method of the embodiment of the present disclosure is described in detail below.
[0045] In some optional implementations, the preset model is a model of the mapping relationship between the input data and the output data of the task to be processed, which is obtained by modeling the task to be processed. In addition, the preset model may include multiple network layers (such as linear processing layers), each of which may include multiple neurons and have certain data processing functions. The multiple network layers form the network topology structure of the preset model in a certain order, so that the corresponding task to be processed can be executed. Among them, the task to be processed can be a text processing task, or it can be an image processing task, an audio processing task, a video processing task, etc., which is not limited by the embodiments of the present disclosure.
[0046] Exemplarily, the preset model includes any one of a text processing model, an image processing model, an audio processing model, and a video processing task.
[0047] In some optional implementations, the network parameter information refers to information about some network parameters in a preset model, and the network parameters include at least one of weight parameters (such as synaptic connection weights of neurons), bias parameters, and the like.
[0048] As mentioned above, the preset model includes multiple neuron groups, each neuron group includes multiple neurons, and multiple neuron groups meet the preset simultaneous emission conditions. Among them, the simultaneous emission condition is a condition about the correlation degree of multiple neurons in the same neuron group when emitting pulse signals. The correlation degree refers to the degree to which multiple neurons tend to emit pulse signals at the same time or the degree to which multiple neurons tend to be activated at the same time. It can also refer to the degree to which multiple neurons tend not to emit pulse signals at the same time or the degree to which multiple neurons tend not to be activated at the same time. In other words, the simultaneous emission condition can be used to characterize the degree to which multiple neurons in the same neuron group emit pulse signals at the same time or do not emit pulse signals at the same time, wherein neurons that emit pulse signals can be called activated neurons, and neurons that do not emit pulse signals can be called inactivated neurons.
[0049] For example, if multiple neurons tend to emit pulse signals simultaneously (or do not emit pulse signals simultaneously), it is determined that the correlation between the multiple neurons is high; conversely, if multiple neurons do not tend to emit pulse signals simultaneously, it is determined that the correlation between the multiple neurons is low.
[0050] It should be noted that, through the restriction of simultaneous firing conditions, multiple neurons that tend to fire at the same time or not fire at the same time are divided into the same neuron group. In other words, for the divided neuron group, it is expected that under any given input, multiple neurons in the group will either fire at the same time (or the proportion of neurons firing at the same time is relatively large) or not fire at the same time (or the proportion of neurons not firing at the same time is relatively large), that is, multiple neurons in the same neuron group tend to fire at the same time or not fire at the same time.
[0051] In some optional implementations, the method may further include: dividing multiple neurons in a preset model into multiple neuron groups according to a simultaneous firing condition, wherein the simultaneous firing condition is used to characterize the extent to which multiple neurons in the same neuron group simultaneously fire pulse signals or do not fire pulse signals at the same time.
[0052] Exemplarily, the preset model includes multiple network layers (such as linear processing layers, equivalent to a linear operator), and each network layer includes multiple neurons; for at least part of the network layers in the preset model, the multiple neurons corresponding to the network layer can be divided into multiple neuron groups based on the simultaneous release condition.
[0053] That is, neurons can be grouped based on the simultaneous firing condition, and the divided neuron groups can meet the simultaneous firing condition. Moreover, the purpose of performing neuron grouping is to group neurons with similar activation correlation into the same neuron group, so that the network parameter information can be stored based on the neuron grouping, and the proportion of activated network parameters in a single read operation can be increased by changing the storage method, thereby increasing the data transmission bandwidth.
[0054] In some optional implementations, the size of each neuron group may be different, which can better balance sparsity and model prediction performance.
[0055] In some optional implementations, the method may further include: for any neuron group, continuously storing network parameter information of multiple neurons in the neuron group into a preset storage space, wherein continuous storage means that the network parameter information of multiple neurons occupies continuous physical memory.
[0056] It should be noted that, since the multiple neurons in each neuron group have a high degree of correlation when emitting pulse signals (i.e., they either tend to emit at the same time or tend not to emit at the same time), the multiple network parameters corresponding to each neuron group are likely to belong to the same activated network parameters or the same inactivated network parameters. Moreover, this possibility can be quantitatively characterized by the neuron activation ratio. For example, if the values of the neuron activation ratios of multiple network parameters in the same neuron group are relatively high, it means that the multiple neurons in the current neuron group tend to emit pulse signals at the same time, and therefore, the division method of the current neuron group is better; or, if the values of the neuron activation ratios of multiple network parameters in the same neuron group are relatively low, it means that the multiple neurons in the current neuron group tend not to emit pulse signals at the same time, and therefore, the division method of the current neuron group is also better.
[0057] It should also be noted that the reason why the network parameter information of neurons that tend to fire simultaneously is stored continuously is mainly to store the activated network parameters as continuously as possible in physical storage, so that the activated network parameters obtained by a single read operation (or cache operation) account for a larger proportion, which is conducive to improving bandwidth utilization.
[0058] In some optional implementations, the grouping of neurons can be achieved based on clustering. Since clustering mainly divides the set of objects to be classified (i.e., multiple neurons included in the preset model) into multiple classes composed of similar objects, the multiple neurons in the neuron group obtained by clustering have certain similarities. Returning to the embodiment of the present disclosure, the similarity of multiple neurons in the same neuron group is manifested in the pulse emission, that is, multiple neurons in the same neuron group tend to emit pulses together or not emit pulses together.
[0059] In some optional implementations, multiple neurons in a preset model are divided into multiple neuron groups according to a simultaneous firing condition, including: for a current iteration state, clustering multiple neurons in the preset model to obtain multiple first neuron clusters; determining the degree of common firing of each first neuron cluster, the common firing degree is used to measure the degree to which multiple neurons in the first neuron cluster tend to jointly fire pulse signals, or to measure the degree to which multiple neurons in the first neuron cluster tend to not jointly fire pulse signals; for each first neuron cluster, determining the average common firing degree corresponding to the first neuron cluster according to the degree of common firing of the first neuron cluster with respect to multiple first sample data; when it is determined that the multiple first neuron clusters meet the simultaneous firing condition according to the average common firing degree corresponding to the multiple first neuron clusters, determining multiple neuron groups based on the multiple first neuron clusters.
[0060] Among them, clustering methods such as k-means, hierarchical clustering, Gaussian mixture model, spectral clustering, etc. can be used during clustering, and the embodiments of the present disclosure are not limited to this.
[0061] In some optional implementations, some preset sparse enhancement strategies may be used when determining whether a neuron is activated, wherein the sparse enhancement strategy may enhance the sparsity of the network parameters of the neuron.
[0062] Exemplarily, the sparse enhancement strategy includes an adaptive threshold-based sparse enhancement strategy and an activation function-based sparse enhancement strategy.
[0063] For example, after determining the input data of the neuron, the sparse threshold of the input data can be determined by using an adaptive threshold, or by using an activation function (such as a Rectified Linear Unit (ReLU) function, an S-type function, etc.) to determine the sparse threshold of the input data. The sparse threshold is usually a value greater than zero. Compared with directly sparsifying with a zero value, the sparse threshold determined by the activation function can effectively improve the sparsity of the input data. Furthermore, the part of the input data that is less than or equal to the sparse threshold can be sparsed. Therefore, the network parameters corresponding to the part of the input data that is less than or equal to the sparse threshold do not need to participate in subsequent data processing, so there is no need to load this part of the network parameters to the target device, thereby further reducing the amount of network parameter loading and alleviating storage pressure and bandwidth pressure.
[0064] That is, the sparse threshold of the input data is a threshold used to perform sparse processing on the input data, and the sparse threshold is greater than the absolute value of the minimum value in the input data, so that the sparsity of the input data determined based on the sparse threshold is definitely greater than zero (that is, at least one data in the input data will be sparse). In other words, through the sparse threshold, the input data can be screened, important information can be screened out, and irrelevant information can be discarded, thereby achieving sparseness of the input data.
[0065] Furthermore, the sparse threshold of the input data can be determined according to the corresponding sparse requirements. For example, if the sparse requirement is high, a sparse threshold with a larger value can be set to improve the sparse level. If the sparse requirement is low, a sparse threshold with a smaller value can be set to meet various processing requirements.
[0066] It should be noted that the sparse threshold in the embodiment of the present disclosure can be a fixed value, or it can be updated according to actual needs (for example, the requirement for the degree of network parameter compression), or it can change with the change of the input data itself, so as to achieve an adaptive sparse processing effect.
[0067] In some optional implementations, determining the degree of common firing of each first neuron cluster includes: based on a preset sparse enhancement strategy, and determining the pulse firing information of each neuron in a preset model through a preset sample set, the preset sample set includes multiple first sample data, and the pulse firing information includes the pulse firing of neurons with respect to each first sample data; determining the number of neurons in each first neuron cluster that fire pulse signals with respect to each first sample data based on the pulse firing information of each neuron; for any first neuron cluster, determining the degree of common firing of the first neuron cluster with respect to each first sample data based on the number of neurons in the first neuron cluster that fire pulse signals with respect to each first sample data and the total number of neurons included in the first neuron cluster.
[0068] Exemplarily, a preset sample set is determined, which includes multiple first sample data, i.e., the first sample data is data to be input into the preset model, and the first sample data is subjected to sparsity enhancement processing using a preset sparsity enhancement strategy to obtain a corresponding sparse threshold, and the first sample data that is less than or equal to the sparse threshold in the first sample data is set to zero to obtain the sparsed first sample data, and the sparsed first sample data is input into the preset model to obtain the pulse emission of each neuron in the preset model with respect to the first sample data.
[0069] Furthermore, for the i-th iteration (equivalent to the current iteration state), multiple neurons of the preset model are clustered based on a preset clustering method to obtain multiple first neuron clusters C m (1<m≤K, K is the total number of clusters of the first neuron cluster); for any neuron cluster C m According to the pulse emission of each neuron, the first neuron cluster C is determined m The number of neurons that emit pulse signals in n m , and according to the number of neurons that emit pulse signals in the first neuron cluster n m The total number of neurons N included in the first neuron cluster m , determine the neuron activation ratio n of the first neuron cluster m / N m . Further, the calculation method of the common firing degree M is set as M=max(neuron activation ratio, 1-neuron activation ratio), where M>0.5. That is, when the neuron activation ratio is greater than 0.5, it means that multiple neurons in the neuron group tend to be activated at the same time, and when the neuron activation ratio is less than 0.5, it means that multiple neurons in the neuron group tend to be inactivated at the same time. M can unify the two situations of neurons tending to be activated and neurons tending to be inactivated for characterization. Then, the average common firing degree can be calculated according to the common firing degree of multiple first neuron clusters to obtain the average common firing degree, and then the average common firing degree is compared with the preset degree threshold. If the average common firing degree is greater than or equal to the preset degree threshold, it is determined that multiple first neuron clusters meet the simultaneous firing condition. Therefore, based on the current multiple first neuron clusters, the corresponding multiple neuron groups are determined (that is, one first neuron cluster is regarded as a neuron group). On the contrary, if the average common firing degree is less than the preset degree threshold, it is determined that multiple first neuron clusters do not meet the simultaneous firing condition, and the next round of iteration or model fine-tuning is required until the simultaneous firing condition is met.
[0070] In the above implementation, the activation ratio is mainly used to determine the degree of common firing of multiple neurons in a neuron cluster to perform neuron grouping. In some optional implementations, in addition to the activation ratio, multiple neuron groups can also be determined by activation variance. Among them, the activation variance is used to characterize the degree of deviation or activation fluctuation of the activation conditions of multiple neurons; for example, if the activation variance corresponding to the neuron cluster is large, it means that the activation conditions of multiple neurons in the neuron cluster are relatively different (i.e., the activation fluctuation is large), and it is not appropriate to classify the multiple neurons into the same neuron group; on the contrary, if the activation variance corresponding to the neuron cluster is small, it means that the activation conditions of multiple neurons in the neuron cluster are relatively different (i.e., the activation fluctuation is small), tend to be consistent, and the multiple neurons can be classified into the same neuron group. Among them, a large activation variance may be characterized as multiple neurons tending to fire at the same time, or it may be characterized as multiple neurons tending not to fire at the same time.
[0071] In some optional implementations, according to the simultaneous firing condition, multiple neurons in the preset model are divided into multiple neuron groups, including: for the current iteration state, clustering multiple neurons of the preset model to obtain multiple first neuron clusters; determining the neuron activation ratio of each first neuron cluster; determining the average activation variance corresponding to the multiple first neuron clusters according to the neuron activation ratio of the multiple first neuron clusters; in the case where it is determined that the multiple first neuron clusters meet the simultaneous firing condition according to the average activation variance corresponding to the multiple first neuron clusters, multiple neuron groups are determined based on the multiple first neuron clusters; wherein the simultaneous firing condition includes that the average activation variance is less than or equal to a preset variance threshold. Wherein, for any neuron cluster, if the corresponding neuron activation ratio is large or small, the corresponding activation variance value is small, otherwise, if the neuron activation ratio value is in the middle (for example, the value is close to 0.5), it means that the firing conditions of the multiple neurons are uneven and the activation variance value is large.
[0072] It should be noted that in addition to the activation ratio and activation variance, other variables (such as standard deviation, mean square error, dispersion coefficient, etc.) can also be used to characterize the degree of common firing of multiple neurons, thereby performing neuron grouping, and the embodiments of the present disclosure are not limited to this.
[0073] Furthermore, when clustering neurons, the number of clusters may be specified in advance and clustering may be performed based on the specified number of clusters. In addition, the range of the number of neurons included in the neuron cluster may be specified in advance and clustering may be performed based on the range. The embodiments of the present disclosure do not impose any restrictions on this.
[0074] In some optional implementations, clustering multiple neurons of a preset model to obtain multiple first neuron clusters in a current iteration state includes:
[0075] Based on a preset number of clusters and according to a preset clustering method, a plurality of neurons of a preset model are clustered into a plurality of first neuron clusters in a current iteration state; wherein the preset number of clusters is determined according to the data volume and unit cache volume of the network parameters of the neurons, and the difference between the total data volume and the unit cache volume of the network parameters of the plurality of neurons in each first neuron cluster is less than or equal to a preset difference threshold;
[0076] Or, based on preset grouping scale information, multiple neurons of the preset model are divided into multiple first neuron clusters in the current iteration state; wherein the preset grouping scale information is used to characterize the range of the number of neurons in the first neuron cluster, and the preset grouping scale information is determined based on the sparsity of the network parameter information and the model performance data of the preset model.
[0077] It can be seen that a preset number of clusters can be determined in advance according to the data volume of the neuron network parameters and the unit buffer volume, so that the data volume V of the network parameters corresponding to each neuron cluster is C and unit buffer size U C The difference between them is small (such as V C -U C ≤thr, thr is the preset difference threshold), that is, all network parameter information corresponding to a neuron cluster can be read out through one data cache. Among them, the unit cache size is the amount of data that can be cached in a single cache operation.
[0078] In addition, the preset grouping scale information of the first neuron cluster can be determined in advance based on the sparsity of the network parameter information and the model performance data of the preset model, so as to perform clustering processing of multiple neurons of a preset scale according to the preset grouping scale information. For example, if the sparsity is large, in order to ensure the model performance as much as possible, a smaller neuron grouping can be set. If the sparsity is small and the model performance is relatively high, a larger neuron grouping can be set to reduce the amount of data processing. Moreover, the number of neurons included in each neuron grouping can be inconsistent. In this way, the sparsity and model performance can be better balanced.
[0079] In some optional implementations, the clustering process can also be combined with model fine-tuning to achieve grouping of neurons.
[0080] In some optional implementations, after determining the average degree of common firing corresponding to multiple first neuron clusters, the method may also include: when it is determined that the multiple first neuron clusters do not meet the simultaneous firing condition based on the average degree of common firing, fine-tuning the preset model to obtain a fine-tuned preset model; determining the degree of common firing of each second neuron cluster in the fine-tuned preset model, the neurons in the second neuron cluster correspond one-to-one to the neurons in the first neuron cluster, and the network parameter information of at least some of the one-to-one corresponding neurons is different; determining the average degree of common firing corresponding to the multiple second neuron clusters based on the degree of common firing of the multiple second neuron clusters; when it is determined that the multiple second neuron clusters meet the simultaneous firing condition based on the average degree of common firing corresponding to the multiple second neuron clusters, determining multiple neuron groups based on the multiple second neuron clusters.
[0081] It can be seen that if the neuron clusters obtained by clustering do not meet the simultaneous firing condition, in addition to directly entering the next round of iteration to re-execute the clustering process, the model can also be fine-tuned first, and it can be determined whether the corresponding neuron clusters in the preset model after fine-tuning meet the simultaneous firing condition. For example, the average common firing degree of multiple first neuron clusters for input data is low. After fine-tuning the network parameters of some neurons, multiple second neuron clusters are obtained. For the second neuron clusters, the number of neurons and the affiliation of the clusters have not changed, only the network parameter values of some neurons have changed. In this case, the average common firing degree of multiple second neuron clusters is re-determined. If the average common firing degree is effectively improved and meets the simultaneous firing condition, the second neuron cluster at this time can be used as the corresponding neuron grouping.
[0082] It should be noted that, in addition to determining that multiple first neuron clusters do not meet the simultaneous firing condition based on the average common firing degree, it can also be determined based on variables such as activation variance that they do not meet the simultaneous firing condition, and the embodiments of the present disclosure are not limited to this.
[0083] In some optional implementations, the method may also include: when it is determined that multiple first neuron clusters do not meet the simultaneous firing condition, fine-tuning the preset model to obtain a fine-tuned preset model; determining the neuron activation ratio of each second neuron cluster in the fine-tuned preset model, the neurons in the second neuron cluster correspond one-to-one to the neurons in the first neuron cluster, and the network parameter information of at least some of the one-to-one corresponding neurons is different; determining the average activation variance corresponding to the multiple second neuron clusters based on the neuron activation ratios of the multiple second neuron clusters; when it is determined that the multiple second neuron clusters meet the simultaneous firing condition based on the average activation variance corresponding to the multiple second neuron clusters, determining multiple neuron groups based on the multiple second neuron clusters.
[0084] That is to say, after fine-tuning the model, it is also possible to determine whether the current multiple second neuron clusters meet the simultaneous firing condition based on the average activation variance, thereby determining whether to continue a new round of clustering. For example, if the simultaneous firing condition is still not met after fine-tuning the model, clustering is performed again. If the simultaneous firing condition is still not met after re-clustering, fine-tuning is performed again, and so on, until the simultaneous firing condition is met, and the corresponding neuron grouping is determined based on the neuron cluster at this time (which may be the first neuron cluster or the second neuron cluster).
[0085] It can be seen that the embodiment of the present disclosure is equivalent to providing an iterative processing method of clustering-fine-tuning, that is, clustering first (clustering itself can also be iterative, such as kmeans clustering, etc.), then fine-tuning, clustering again, fine-tuning again, and so on, until the stopping condition (such as the simultaneous release condition) is met. This two-layer nested processing method helps to improve the degree of co-release of neuron groupings, thereby improving the accuracy and rationality of neuron groupings, can further alleviate the bandwidth loss of the target device, and improve the model inference acceleration advantage brought by dynamically loading network parameters.
[0086] In some optional implementations, the preset model may be fine-tuned through a preset constraint function.
[0087] In some optional implementations, fine-tuning the preset model to obtain the fine-tuned preset model includes: fine-tuning the preset model based on a preset fine-tuning data set according to a preset constraint function to obtain the fine-tuned preset model;
[0088] The constraint function can be represented by the following formula:
[0089]
[0090] Among them, f(θ,x) represents the preset model, θ represents the network parameters of the preset model, x represents the preset fine-tuning dataset, δθ represents the network parameter adjustment amount, std(g i ) represents the i-th neuron cluster g i The activation standard deviation of multiple neurons in , and λ represents the balance coefficient.
[0091] It can be seen that the constraint function mainly consists of two terms. The first term is ‖f(θ,x)-f(θ+δθ,x)‖2, which is mainly used to constrain the output of the preset model after fine-tuning to not change drastically. The second term is ∑ i std(g i), which is mainly used to make the firing conditions of multiple neurons in the same neuron group converge (that is, multiple neurons in the same neuron group either tend to all fire or tend not to fire); the balance coefficient λ is used to balance the losses of the first and second items, and its value can be adjusted according to the actual situation.
[0092] Furthermore, E represents the statistical mean, min represents the function of finding the minimum value, x obeys the probability distribution D, and solving the constraint function aims to make the statistical mean value E of the constraint function on the input data x x~D The significance of this formula is to make the firing conditions of multiple neurons in the same neuron group converge while minimizing the impact of fine-tuning of network parameters on the output results of the preset model.
[0093] Exemplarily, the constraint function may also be represented by the following formula:
[0094]
[0095] Among them, S() represents sparsity, and the meanings of other parameters refer to the relevant content, which will not be repeated here. The significance of this formula is to make the firing conditions of multiple neurons in the same neuron group converge while maintaining the sparsity corresponding to the preset model as much as possible.
[0096] It should be noted that the above constraint function is only an example, and the embodiments of the present disclosure do not limit this.
[0097] In some optional implementations, in step S11, according to the predicted activation information of each neuron group, a target neuron group is selected from multiple neuron groups, including: determining the predicted activation ratio of each neuron group according to the predicted activation information of each neuron group; and selecting the target neuron group from multiple neuron groups according to the predicted activation ratio of each neuron group. The predicted activation information is information used to predict the pulse emission of multiple neurons in the neuron group.
[0098] Exemplarily, the predicted activation ratio of each neuron group is determined according to the predicted activation information, and the neuron group whose predicted activation ratio is greater than or equal to the predicted ratio threshold is screened out as the target neuron group.
[0099] Exemplarily, the predicted activation ratio of each neuron group is determined according to the predicted activation information, and multiple neuron groups are sorted in order from high to low according to the predicted activation ratio to obtain a group sequence, and the neuron groups ranked in the top r positions in the group sequence are screened out as target neuron groups, where 1≤r<R, and R is the total number of neuron groups.
[0100] In some optional implementations, the predicted activation information may include input data of each neuron in the neuron group, or may include firing information of each neuron in the neuron group. In other words, it is possible to predict whether a neuron fires a pulse signal by using the input data of the neuron, or by using some firing information of the neuron itself.
[0101] In some optional implementations, the predicted activation information includes input data of each neuron in the neuron group; accordingly, based on the predicted activation information of each neuron group, the predicted activation ratio of each neuron group is determined, including: for any neuron group, determining the first input data among the input data of each neuron in the neuron group, the first input data being the non-sparsed input data determined based on the sparsity threshold; based on the first input data, determining the first predicted activation probability of the neuron corresponding to the first input data; based on the first predicted activation probabilities of multiple neurons in each neuron group, determining the predicted activation ratio of each neuron group.
[0102] In some optional implementations, when determining the predicted activation ratio according to the first predicted activation probability, the activation probability threshold may be used for determination. For example, for a certain neuron, if its first predicted activation probability is greater than or equal to the activation probability threshold, the neuron is considered to be activated, and is used as the numerator when calculating the predicted activation ratio (the numerator is the activated neuron part, and the denominator is the total number of neurons in the neuron group to which the neuron belongs).
[0103] In some optional implementations, determining the first input data in the input data of each neuron in each neuron group includes: determining a sparse threshold based on a preset sparse enhancement strategy and input data corresponding to at least part of the neuron group; and determining the first input data in the input data of each neuron in the neuron group according to the sparse threshold. The sparse enhancement strategy is to enhance the sparsity of the input data, thereby reducing the amount of network parameter information that needs to be loaded.
[0104] Exemplarily, based on a preset sparse enhancement strategy and input data corresponding to at least part of the neuron grouping, a sparse threshold is determined, including: determining the sparse threshold according to the quantile value of the input data. Among them, the quantile value (Quantile) can also be called a quantile point, which can be used to divide a set of data according to size and determine the percentage of data contained in each division. The distribution and central tendency of the input data can be analyzed by the quantile value, and the relative value position of the data point in the entire input data can be understood. Common quantile values include median values (50% quantile values), quartile values (such as 25% quantile values, 75% quantile values) and percentile values (quantile values of any percentage), etc., and the embodiments of the present disclosure are not limited to this.
[0105] Exemplarily, based on a preset sparse enhancement strategy and input data corresponding to at least part of the neuron groupings, a sparse threshold is determined, including: activating the input data based on a preset activation function to obtain activated input data in the input data, and then determining the sparse threshold based on the value range of the activated input data.
[0106] In some optional implementations, the predicted activation information includes input data of each neuron in the neuron group; accordingly, according to the predicted activation information of each neuron group, a target neuron group is selected from multiple neuron groups, including: for any neuron group, the input index value of the neuron group is determined according to the absolute value of the input data of multiple neurons in the neuron group; according to the input index values of multiple neuron groups, the target neuron group is selected from multiple neuron groups. The input index value may include the average value, maximum value and minimum value of the input data, etc., which is not limited in the embodiments of the present disclosure.
[0107] Exemplarily, for any neuron grouping, the average value of the input data of multiple neurons in the neuron grouping is determined, and the neuron grouping whose average value is greater than a preset average threshold is selected as the target neuron grouping. Alternatively, L neuron groups whose average values are ranked in the top L positions can be selected as the target neuron grouping, 1≤L<N, where N is the total number of neuron groups.
[0108] In some optional implementations, the predicted activation information includes the firing information of each neuron in the neuron group; accordingly, based on the predicted activation information of each neuron group, a target neuron group is selected from multiple neuron groups, including: for any neuron group, an average firing value of the neuron group is determined based on the firing information of each neuron in the neuron group; based on the average firing value of each neuron group, a target neuron group is selected from multiple neuron groups.
[0109] Exemplarily, the emission information is the historical emission information of each neuron, and the emission frequency and other information of the neuron in the historical time can be determined based on the emission information. Based on this, if the emission frequency of a neuron in the historical time is high, it is considered that the neuron is more likely to emit pulse signals during this signal processing process, and is a neuron that is easy to activate. On the contrary, if the emission frequency of a neuron in the historical time is low, it is considered that the neuron is less likely to emit pulse signals during this signal processing process, and is a neuron that is not easy to activate. Further, the average emission value of a neuron group can refer to its average emission frequency. In other words, for a certain neuron group, it can calculate the corresponding average emission frequency based on the historical emission information of multiple neurons in the neuron group, and then select the target neuron group from multiple neuron groups based on the average emission frequency. For example, a neuron group whose average emission frequency is greater than or equal to a preset frequency threshold can be screened as a target neuron group.
[0110] In summary, when loading the network parameter information of the preset model to the target device, the network parameter information is loaded at the granularity of neuron grouping, and the network parameter information of the neuron groups with higher activation levels is selected and loaded into the target device, while the network parameter information of the remaining neuron groups with lower activation levels does not need to be loaded.
[0111] Furthermore, as mentioned above, after the neuron grouping is determined, the network parameter information of a plurality of neurons in each neuron grouping may be continuously stored in a preset storage space in units of neuron groups.
[0112] In some optional implementations, a hierarchical storage method may be used to store network parameter information corresponding to multiple neuron groups.
[0113] Exemplarily, the storage level of the neuron group is determined according to the average activation ratio and / or the average activation variance of the neuron group, and the network parameter information corresponding to the neuron group is stored according to the storage level of the neuron group and the storage level of the preset storage space. The preset storage space may include a first-level cache (L1 Cache), a second-level cache (L2Cache), a third-level cache (L3 Cache), a memory (such as RAM) and a solid-state drive (such as SSD), and since the read and write speeds decrease in sequence, the corresponding storage levels also decrease in sequence.
[0114] For example, for neuron group G1 and neuron group G2, if the average activation ratio of neuron group G1 is greater than the average activation ratio of neuron group G2 (or the average activation variance of neuron group G1 is less than the average activation variance of neuron group G2), it means that neuron group G1 is relatively easier to be activated. Therefore, neuron group G1 is stored in a preset storage space with a relatively high storage level to reduce its reading and writing time, increase the reading and writing speed, and improve data processing efficiency.
[0115] In some optional implementations, network parameter information of a target neuron group is loaded from a preset storage space to a target device, including: determining a storage address of the network parameter information of the target neuron group in the preset storage space; reading the network parameter information corresponding to the storage address from the preset storage space, and transmitting the read network parameter information to the target device.
[0116] That is to say, since the network parameter information is stored continuously in units of neuron groups, the network parameter information of multiple neurons in the same neuron group is also read continuously when reading.
[0117] In some optional implementations, the network parameter processing method of the embodiment of the present disclosure can be applied to a multi-core system, which includes multiple processing cores; wherein the network parameter information of multiple neurons in the same neuron group is continuously stored in the same processing core. Since the method of continuous storage in the core is adopted, the routing cost and communication cost can be reduced to a certain extent, thereby further improving the data reading efficiency and accelerating the data processing efficiency.
[0118] In some optional implementations, the network parameter information of the neuron grouping can also be stored in a hierarchical cache manner based on the many-core system. Exemplarily, the preset storage space of the many-core system corresponds to multiple storage levels, and the corresponding storage level is determined according to the probability of the neuron grouping to emit pulses, and then the network parameter information of the neuron grouping is stored in the preset storage space corresponding to the storage level. For example, the neuron grouping with a higher proportion of pulses is stored in the preset storage space corresponding to the higher storage level, so as to facilitate the rapid reading of these data, while the possibility of using the neuron grouping with a lower proportion of pulses is relatively low, so the network parameter information of this part of the neuron grouping is stored in the preset storage space corresponding to the lower storage level.
[0119] Figure 2 A flow chart of a network parameter processing method provided by an embodiment of the present disclosure, referring to Figure 2 , the network parameter processing method may include the following steps.
[0120] Step S201, based on a preset sparse enhancement strategy and using preset sample data, the pulse emission information of each neuron in a preset model is determined.
[0121] Step S202: clustering multiple neurons of a preset model according to the current iteration state to obtain multiple first neuron clusters.
[0122] Step S203, determining the neuron activation ratio of each first neuron cluster according to the pulse emission information of each neuron.
[0123] Step S204, determining the common firing degree of each first neuron cluster according to the neuron activation ratio of the multiple first neuron clusters, and determining the average common firing degree corresponding to the multiple first neuron clusters according to the common firing degree of each first neuron cluster.
[0124] Step S205 , determining whether the plurality of first neuron clusters satisfy a simultaneous firing condition according to the average common firing degrees corresponding to the plurality of first neuron clusters.
[0125] Step S206: when the simultaneous firing condition is met, a plurality of neuron groups are determined based on the plurality of first neuron clusters.
[0126] Step S207: When the simultaneous issuance condition is not met, fine-tune the preset model to obtain a fine-tuned preset model.
[0127] Step S208, determining the neuron activation ratio of each second neuron cluster in the fine-tuned preset model.
[0128] Step S209, determining the common firing degree of each second neuron cluster according to the neuron activation ratio of the plurality of second neuron clusters, and determining the average common firing degree corresponding to the plurality of second neuron clusters according to the common firing degree of each second neuron cluster.
[0129] Step S210, determining whether the plurality of second neuron clusters satisfy a simultaneous firing condition according to the average common firing degrees corresponding to the plurality of second neuron clusters.
[0130] Step S211, when the simultaneous firing condition is met, a plurality of neuron groups are determined based on a plurality of second neuron clusters.
[0131] If the simultaneous firing condition is not met, the next round of iteration is entered and the process jumps to step S202 to perform clustering again to determine the neuron grouping.
[0132] Step S212, for any neuron group, continuously storing network parameter information of multiple neurons in the neuron group into a preset storage space.
[0133] Step S213: selecting a target neuron group from multiple neuron groups according to the predicted activation information of each neuron group.
[0134] Step S214, loading the network parameter information of the target neuron group from the preset storage space to the target device.
[0135] Step S215, the target device performs data processing of a preset model based on the network parameter information of the target neuron group.
[0136] A second aspect of an embodiment of the present disclosure provides a task processing method.
[0137] Figure 3 The following is a flowchart of a task processing method provided by an embodiment of the present disclosure, and the task processing method can be applied to a target device. Figure 3 , the task processing method may include the following steps.
[0138] Step S31, performing data processing according to the network parameter information of the target neuron group loaded from the preset storage space, the correspondence between the network parameter information and the neurons in the preset model, and the input data of the task to be processed to obtain the task processing result.
[0139] The network parameter information of the target neuron group is obtained according to the network parameter processing method described in any one of the embodiments of the present disclosure.
[0140] In some optional implementations, the task to be processed includes at least one of a text processing task, an image processing task, an audio processing task, and a video processing task, etc., which is not limited in the embodiments of the present disclosure. Correspondingly, the task processing result corresponds to the task to be processed, which may include at least one of a text processing result, an image processing result, an audio processing result, and a video processing result, etc.
[0141] In some optional implementations, after the input data is input into the preset model, the preset model will perform certain data processing (e.g., multiplication and addition operations) on the input data to obtain corresponding processing results. Among them, the network parameters of the neurons in the preset model (such as the network parameters of multiple neurons in the linear processing layer of the preset model) are actually involved in the data processing of the input data. Therefore, there is a certain correspondence between the input data and the neurons, and based on the correspondence, operations such as multiplication and addition can be performed between the input data and the network parameters of the neurons.
[0142] In step S31, after the network parameter information of the loaded target neuron group is obtained, data processing can be performed according to the corresponding relationship between the network parameter information and the neurons, and the corresponding input data, so as to obtain the corresponding task processing result.
[0143] In the disclosed embodiment, when the model is deployed on the target device or other end device, due to the randomness of loading network parameters based on sparsity, the effective data (i.e., the network parameters of activated neurons) that can be read in a single data reading operation are small, which brings greater storage pressure and bandwidth transmission pressure to the target device. Based on this, the neurons of the preset model are pre-divided into multiple neuron groups that meet the simultaneous release conditions, and the network parameter information of multiple neurons in the same neuron group is continuously stored in the preset storage space. Therefore, more effective data can be obtained through a single reading operation, which alleviates the storage pressure and bandwidth transmission pressure on the target device; in addition, since only the network parameter information of the target neuron group needs to be loaded into the target device, and there is no need to load the full amount of network parameter information, the data loading amount can be reduced, which alleviates the storage pressure and bandwidth pressure of the target device. On the other hand, since the target neuron group is a neuron group with a higher probability of excitation, this part of the network parameter information belongs to the important data of data processing, so the accuracy of the task processing results can be guaranteed as much as possible.
[0144] It can be understood that the above-mentioned embodiments mentioned in the present disclosure can be combined with each other to form a combined embodiment without violating the principle logic. Due to space limitations, the present disclosure will not repeat them. Those skilled in the art can understand that in the above-mentioned method of the specific implementation method, the specific execution order of each step and the setting of the functional module should be determined by its function and possible internal logic.
[0145] A third aspect of an embodiment of the present disclosure provides a network parameter processing device.
[0146] Figure 4 This is a block diagram of a network parameter processing device provided by an embodiment of the present disclosure. The network parameter processing device can be applied to a preset model. The preset model includes multiple neuron groups, each neuron group includes multiple neurons, and the network parameter information of multiple neurons in the same neuron group is continuously stored in a preset storage space, and the multiple neuron groups meet the preset simultaneous release conditions. Figure 4 , the network parameter processing device 400 may include the following modules.
[0147] The selection module 401 is used to select a target neuron group from multiple neuron groups according to the predicted activation information of each neuron group, and the predicted activation information is used to predict the pulse emission information of each neuron in the neuron group.
[0148] The loading module 402 is used to load the network parameter information of the target neuron group from the preset storage space to the target device, so that the target device can perform data processing of the preset model based on the network parameter information of the target neuron group.
[0149] In an embodiment of the present disclosure, a preset model includes multiple neuron groups, each neuron group includes multiple neurons, network parameter information of multiple neurons in the same neuron group is continuously stored in a preset storage space, and multiple neuron groups meet preset simultaneous release conditions; in a network parameter processing device, a target neuron group is selected from multiple neuron groups through a selection module according to predicted activation information of each neuron group, the predicted activation information is used to predict the pulse release information of each neuron in the neuron group, and the network parameter information of the target neuron group is loaded from the preset storage space to the target device through a loading module, so that the target device can perform data processing of the preset model based on the network parameter information of the target neuron group.
[0150] A fourth aspect of the embodiments of the present disclosure provides a task processing device.
[0151] Figure 5 The block diagram of a task processing device provided by an embodiment of the present disclosure is shown in FIG. Figure 5 , the task processing device 500 may include the following modules.
[0152] The processing module 501 is used to perform data processing according to the network parameter information of the target neuron group loaded from the preset storage space, the correspondence between the network parameter information and the neurons in the preset model, and the input data of the task to be processed to obtain the task processing result.
[0153] The network parameter information of the target neuron group is obtained according to the network parameter processing method described in any one of the embodiments of the present disclosure.
[0154] In the disclosed embodiment, when the model is deployed on the target device or other end device, due to the randomness of loading network parameters based on sparsity, the effective data (i.e., the network parameters of activated neurons) that can be read in a single data reading operation are small, which brings greater storage pressure and bandwidth transmission pressure to the target device. Based on this, the neurons of the preset model are pre-divided into multiple neuron groups that meet the simultaneous release conditions, and the network parameter information of multiple neurons in the same neuron group is continuously stored in the preset storage space. Therefore, more effective data can be obtained through a single reading operation, which alleviates the storage pressure and bandwidth transmission pressure on the target device; in addition, since only the network parameter information of the target neuron group needs to be loaded into the target device, and there is no need to load the full amount of network parameter information, the data loading amount can be reduced, which alleviates the storage pressure and bandwidth pressure of the target device. On the other hand, since the target neuron group is a neuron group with a higher probability of excitation, this part of the network parameter information belongs to the important data of data processing, so the accuracy of the task processing results can be guaranteed as much as possible.
[0155] In addition, the present disclosure also provides an electronic device and a computer-readable storage medium.
[0156] Figure 6 A block diagram of an electronic device provided in accordance with a first embodiment of the present disclosure.
[0157] Reference Figure 6 An embodiment of the present disclosure provides an electronic device, which includes: at least one processor 601; at least one memory 602, and one or more I / O interfaces 603, connected between the processor 601 and the memory 602; wherein the memory 602 stores one or more computer programs that can be executed by at least one processor 601, and the one or more computer programs are executed by at least one processor 601 so that the at least one processor 601 can execute the network parameter processing method or task processing method described in any one of the embodiments of the present disclosure.
[0158] Figure 7 A block diagram of an electronic device provided in Embodiment 2 of the present disclosure.
[0159] Reference Figure 7 An embodiment of the present disclosure provides an electronic device, which includes multiple processing cores 701 and an on-chip network 702, wherein the multiple processing cores 701 are connected to the on-chip network 702, and the on-chip network 702 is used to exchange data between the multiple processing cores and external data.
[0160] Among them, one or more instructions are stored in one or more processing cores 701, and one or more instructions are executed by one or more processing cores 701, so that one or more processing cores 701 can execute the network parameter processing method or task processing method described in any one of the embodiments of the present disclosure.
[0161] In some embodiments, the electronic device may be a brain-like chip, which can use vectorized computing and needs to load the weight information and other parameters of the neural network model through an external memory such as a double data rate (DDR) synchronous dynamic random access memory. Therefore, the batch processing used in the embodiments of the present disclosure has a higher computing efficiency.
[0162] The embodiments of the present disclosure also provide a computer-readable storage medium on which a computer program is stored, wherein the computer program, when executed by a processor / processing core, implements the network parameter processing method or task processing method described in any one of the embodiments of the present disclosure. The computer-readable storage medium may be a volatile or non-volatile computer-readable storage medium.
[0163] The embodiments of the present disclosure also provide a computer program product, including a computer-readable code, or a non-volatile computer-readable storage medium carrying the computer-readable code. When the computer-readable code runs in a processor of an electronic device, the processor in the electronic device executes the network parameter processing method or task processing method described in any one of the embodiments of the present disclosure.
[0164] It will be appreciated by those skilled in the art that all or some of the steps, systems, and functional modules / units in the methods disclosed above may be implemented as software, firmware, hardware, and appropriate combinations thereof. In a hardware implementation, the division between the functional modules / units mentioned in the above description does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed by several physical components in cooperation. Some or all physical components may be implemented as software executed by a processor, such as a central processing unit, a digital signal processor, or a microprocessor, or may be implemented as hardware, or may be implemented as an integrated circuit, such as an application-specific integrated circuit. Such software may be distributed on a computer-readable storage medium, which may include a computer storage medium (or a non-transitory medium) and a communication medium (or a temporary medium).
[0165] As is known to those of ordinary skill in the art, the term computer storage media includes volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable program instructions, data structures, program modules or other data). Computer storage media include, but are not limited to, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), static random access memory (SRAM), flash memory or other memory technology, portable compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical disk storage, magnetic cassettes, magnetic tapes, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and can be accessed by a computer. In addition, it is known to those of ordinary skill in the art that communication media typically contain computer-readable program instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transmission mechanism, and may include any information delivery medium.
[0166] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, optical fiber transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in the computer-readable storage medium in each computing / processing device.
[0167] The computer program instructions for performing the operation of the present disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-related instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages, such as Smalltalk, C++, etc., and conventional procedural programming languages, such as "C" language or similar programming languages. Computer-readable program instructions may be executed completely on a user's computer, partially on a user's computer, as an independent software package, partially on a user's computer, partially on a remote computer, or completely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., using an Internet service provider to connect via the Internet). In some embodiments, an electronic circuit, such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), may be customized by utilizing the state information of the computer-readable program instructions, and the electronic circuit may execute the computer-readable program instructions, thereby realizing various aspects of the present disclosure.
[0168] The computer program product described herein may be implemented in hardware, software, or a combination thereof. In one optional embodiment, the computer program product is embodied as a computer storage medium, and in another optional embodiment, the computer program product is embodied as a software product, such as a software development kit (SDK), etc.
[0169] Various aspects of the present disclosure are described herein with reference to the flowcharts and / or block diagrams of the methods, devices (systems) and computer program products according to the embodiments of the present disclosure. It should be understood that each box in the flowchart and / or block diagram and the combination of each box in the flowchart and / or block diagram can be implemented by computer-readable program instructions.
[0170] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processor of the computer or other programmable data processing device, a device that implements the functions / actions specified in one or more boxes in the flowchart and / or block diagram is generated. These computer-readable program instructions can also be stored in a computer-readable storage medium, and these instructions cause the computer, programmable data processing device, and / or other equipment to work in a specific manner, so that the computer-readable medium storing the instructions includes a manufactured product, which includes instructions for implementing various aspects of the functions / actions specified in one or more boxes in the flowchart and / or block diagram.
[0171] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operating steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions / actions specified in one or more boxes in the flowchart and / or block diagram.
[0172] The flow chart and block diagram in the accompanying drawings show the possible architecture, function and operation of the system, method and computer program product according to multiple embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a part of a module, program segment or instruction, and a part of the module, program segment or instruction includes one or more executable instructions for realizing the specified logical function. In some alternative implementations, the function marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two continuous square boxes can actually be executed substantially in parallel, and they can sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs the specified function or action, or can be implemented with a combination of special hardware and computer instructions.
[0173] Example embodiments have been disclosed herein, and although specific terms are employed, they are used and should be interpreted only in a general illustrative sense and not for limiting purposes. In some instances, it will be apparent to those skilled in the art that, unless otherwise expressly noted, features, characteristics, and / or elements described in conjunction with a particular embodiment may be used alone or in combination with features, characteristics, and / or elements described in conjunction with other embodiments. Therefore, those skilled in the art will appreciate that various changes in form and detail may be made without departing from the scope of the present disclosure as set forth in the appended claims.
Claims
1. A network parameter processing method, characterized in that: Applied to a preset model, the preset model includes a plurality of neuron groups, each of the neuron groups includes a plurality of neurons, network parameter information of a plurality of neurons in the same neuron group is continuously stored in a preset storage space, and the plurality of neuron groups meet a preset simultaneous release condition, the method includes: Selecting a target neuron group from the plurality of neuron groups according to the predicted activation information of each neuron group, wherein the predicted activation information is used to predict the pulse emission information of each neuron in the neuron group; The network parameter information of the target neuron group is loaded from the preset storage space to the target device, so that the target device performs data processing of the preset model based on the network parameter information of the target neuron group.
2. The method according to claim 1, characterized in that The method further comprises: According to the simultaneous firing condition, the plurality of neurons in the preset model are divided into a plurality of the neuron groups; The simultaneous firing condition is used to characterize the degree to which multiple neurons in the same neuron group simultaneously fire pulse signals or do not fire pulse signals at the same time.
3. The method according to claim 2, characterized in that The dividing the plurality of neurons in the preset model into a plurality of the neuron groups according to the simultaneous emission condition comprises: According to the current iteration state, clustering multiple neurons of the preset model to obtain multiple first neuron clusters; Determining the degree of common firing of each first neuron cluster, wherein the degree of common firing is used to measure the degree to which multiple neurons in the first neuron cluster tend to jointly fire pulse signals, or to measure the degree to which multiple neurons in the first neuron cluster tend to not jointly fire pulse signals; For each first neuron cluster, determining an average common firing degree corresponding to the first neuron cluster according to the common firing degree of the first neuron cluster with respect to a plurality of first sample data; When it is determined that the plurality of first neuron clusters satisfy the simultaneous firing condition according to the average common firing degrees corresponding to the plurality of first neuron clusters, the plurality of neuron groups are determined based on the plurality of first neuron clusters.
4. The method according to claim 3, characterized in that The determining the degree of co-sending of each first neuron cluster comprises: Based on a preset sparse enhancement strategy, the pulse emission information of each neuron in the preset model is determined through a preset sample set, wherein the preset sample set includes a plurality of first sample data, and the pulse emission information includes the pulse emission of the neuron with respect to each first sample data; Determine the number of neurons in each first neuron cluster that emit pulse signals for each first sample data according to the pulse emission information of each neuron; For any first neuron cluster, the degree of common emission of the first neuron cluster with respect to each first sample data is determined according to the number of neurons of the first neuron cluster that emit pulse signals with respect to each first sample data and the total number of neurons included in the first neuron cluster.
5. The method according to claim 3, characterized in that: After determining the average co-firing degree corresponding to the first neuron cluster, the method further includes: When it is determined according to the average co-firing degree that the plurality of first neuron clusters do not satisfy the co-firing condition, fine-tuning the preset model to obtain a fine-tuned preset model; Determining the degree of common firing of each second neuron cluster in the fine-tuned preset model, the neurons in the second neuron cluster correspond one-to-one with the neurons in the first neuron cluster, and network parameter information of at least some of the one-to-one corresponding neurons is different; For each second neuron cluster, determining an average common firing degree corresponding to the second neuron cluster according to the common firing degree of the second neuron cluster with respect to a plurality of second sample data; When it is determined that the plurality of second neuron clusters satisfy the simultaneous firing condition according to the average common firing degrees corresponding to the plurality of second neuron clusters, the plurality of neuron groups are determined based on the plurality of second neuron clusters.
6. The method according to claim 5, characterized in that The step of fine-tuning the preset model to obtain a fine-tuned preset model includes: According to a preset constraint function, fine-tuning the preset model based on a preset fine-tuning data set to obtain the fine-tuned preset model; The constraint function is represented by the following formula: Among them, f(θ,x) represents the preset model, θ represents the network parameters of the preset model, x represents the preset fine-tuning dataset, δθ represents the network parameter adjustment amount, std(g i ) represents the i-th neuron cluster g i The activation standard deviation of multiple neurons in , and λ represents the balance coefficient.
7. The method according to claim 3, characterized in that The clustering of the plurality of neurons of the preset model to obtain a plurality of first neuron clusters in the current iteration state includes: Based on a preset number of clusters and according to a preset clustering method, clustering the multiple neurons of the preset model into multiple first neuron clusters in a current iteration state; The preset number of clusters is determined according to the data volume of the network parameters of the neurons and the unit buffer volume, and the difference between the total data volume of the network parameters of the plurality of neurons in each of the first neuron clusters and the unit buffer volume is less than or equal to a preset difference threshold; or, Based on preset grouping scale information, multiple neurons of the preset model are divided into multiple first neuron clusters in a current iteration state; The preset grouping scale information is used to characterize the range of the number of neurons in the first neuron cluster, and the preset grouping scale information is determined according to the sparsity of the network parameter information and the model performance data of the preset model.
8. The method according to any one of claims 2 to 7, characterized in that: The method further comprises: For any of the neuron groups, network parameter information of a plurality of neurons in the neuron group is continuously stored in the preset storage space.
9. The method according to claim 1, characterized in that: The step of selecting a target neuron group from the plurality of neuron groups according to the predicted activation information of each neuron group comprises: Determining the predicted activation ratio of each of the neuron groups according to the predicted activation information of each of the neuron groups; The target neuron group is selected from the plurality of neuron groups according to the predicted activation ratios of the respective neuron groups.
10. The method according to claim 9, characterized in that The predicted activation information includes input data of each neuron in the neuron group; Determining the predicted activation ratio of each neuron group according to the predicted activation information of each neuron group includes: For any of the neuron groups, determine first input data among input data of each neuron in the neuron group, wherein the first input data is input data that is not sparsely distributed and is determined based on a sparse threshold; Determining, based on the first input data, a first predicted activation probability of a neuron corresponding to the first input data; The predicted activation ratio of each neuron group is determined according to the first predicted activation probability of multiple neurons in each neuron group.
11. The method according to claim 10, characterized in that The determining of first input data among the input data of each neuron in each neuron group comprises: Determining the sparse threshold based on a preset sparse enhancement strategy and input data corresponding to at least part of the neuron grouping; According to the sparse threshold, first input data among the input data of each neuron in the neuron group is determined.
12. The method according to claim 1, characterized in that The predicted activation information includes input data of each neuron in the neuron group; The step of selecting a target neuron group from the plurality of neuron groups according to the predicted activation information of each neuron group comprises: For any of the neuron groups, determining an input index value of the neuron group according to absolute values of input data of a plurality of neurons in the neuron group; The target neuron group is selected from the plurality of neuron groups according to the input index values of the plurality of neuron groups.
13. The method according to claim 1, characterized in that The predicted activation information includes firing information of each neuron in the neuron group; The step of selecting a target neuron group from the plurality of neuron groups according to the predicted activation information of each neuron group comprises: For any of the neuron groups, determining an average firing value of the neuron group according to firing information of each neuron in the neuron group; The target neuron group is selected from the plurality of neuron groups according to the average firing values of the respective neuron groups.
14. The method according to claim 1, characterized in that The step of loading the network parameter information of the target neuron group from the preset storage space to the target device includes: Determine the storage address of the network parameter information of the target neuron group in the preset storage space; The network parameter information corresponding to the storage address is read from the preset storage space, and the read network parameter information is transmitted to the target device.
15. A task processing method, characterized in that: Applied to a target device, the method comprises: Performing data processing according to the network parameter information of the target neuron group loaded from the preset storage space, the correspondence between the network parameter information and the neurons in the preset model, and the input data of the task to be processed to obtain the task processing result; The network parameter information of the target neuron group is obtained according to the network parameter processing method as described in any one of claims 1 to 14.
16. A network parameter processing device, characterized in that: Applied to a preset model, the preset model includes a plurality of neuron groups, each of the neuron groups includes a plurality of neurons, network parameter information of a plurality of neurons in the same neuron group is continuously stored in a preset storage space, and the plurality of neuron groups meet a preset simultaneous release condition, the device includes: A selection module, used to select a target neuron group from the plurality of neuron groups according to the predicted activation information of each neuron group, wherein the predicted activation information is used to predict the pulse emission information of each neuron in the neuron group; A loading module is used to load the network parameter information of the target neuron group from the preset storage space to the target device, so that the target device can perform data processing of the preset model based on the network parameter information of the target neuron group.
17. A task processing device, characterized in that: Applied to a target device, the apparatus comprises: A processing module, used to perform data processing according to the network parameter information of the target neuron group loaded from the preset storage space, the correspondence between the network parameter information and the neurons in the preset model, and the input data of the task to be processed, to obtain a task processing result; The network parameter information of the target neuron group is obtained according to the network parameter processing method as described in any one of claims 1 to 14.
18. An electronic device, characterized in that: include: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores one or more computer programs that can be executed by the at least one processor, and the one or more computer programs are executed by the at least one processor so that the at least one processor can execute the network parameter processing method as described in any one of claims 1-14, or the task processing method as described in claim 15.
19. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, it implements the network parameter processing method according to any one of claims 1 to 14, or the task processing method according to claim 15.
Citation Information
Cited By
Question and answer method and product of question and answer large language model based on data-driven regularization
CN120336497A
