Data intelligence model training and hardware acceleration method and system based on artificial intelligence

Abstract

The present invention relates to the field of artificial intelligence and specifically discloses a data intelligence model training and hardware acceleration method based on artificial intelligence, including: setting the network structure of an original deep convolutional neural network model (DNN); obtaining the convolutional neural network model through initial training; then dividing the model into stages and categories to obtain blocks; calculating the filter channel size and pruning weight of each category convolution layer in each block; using the pruned intelligent model to perform inference tasks; selecting the corresponding activation function according to the task type; and performing quantization and knowledge distillation; designing a universal hardware accelerator to deploy the optimized deep convolutional neural network model (DNN) in an FPGA. The present invention constructs the network structure of the deep convolutional neural network model (DNN) and designs a universal hardware architecture for multiple activation functions, making it more lightweight while improving the expressiveness of the convolutional neural network and accurately improving the intelligence level.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence, and in particular, to a data intelligence model training and hardware acceleration method based on artificial intelligence. Background Art

[0002] The rapid development of deep neural networks (DNNs) has led to a growing number of researchers engaging in related research, resulting in significant progress in areas such as object detection, natural language processing, and computer vision. With the advancement of integrated circuits and the continuous improvement of computer processing power, neural networks have once again become a hot topic in artificial intelligence research and development.

[0003] However, neural networks consist of a series of hidden layers operating in parallel, each containing a large number of neurons. Each neuron draws input from other neurons and computes its output. Therefore, activation functions are used to increase their expressive power and make them truly meaningful. Activation functions have their own advantages and disadvantages, ranging from linear to nonlinear, and each has varying hardware requirements and implementation difficulty. Furthermore, to further improve neural network performance, the structural design of neural network models has become increasingly complex, with an increasing number of network layers. This makes neural network algorithms computationally more complex, making them difficult to deploy in scenarios with stringent power requirements.

[0004] Therefore, a general activation function hardware accelerator is needed that is suitable for DNN and can improve computing efficiency, reduce power consumption, and minimize storage resources. Summary of the Invention

[0005] In view of this, the present invention provides a data intelligence model training and hardware acceleration method based on artificial intelligence, the purposes of which are: (1) to propose a data intelligence model training and hardware acceleration method based on artificial intelligence, which can accurately improve the intelligence level while making it more lightweight by constructing the network structure of a deep convolutional neural network model DNN; (2) to design a universal hardware architecture for multiple activation functions, which can improve the expressive power of convolutional neural networks while giving them the true meaning of artificial intelligence.

[0006] To achieve the above objectives, the present invention provides an artificial intelligence-based data intelligence model training and hardware acceleration method, comprising the following steps:

[0007] Step S1, setting the network structure of the original deep convolutional neural network model DNN, and initializing the original deep convolutional neural network model DNN by randomly taking weights;

[0008] Step S2: Collect control instructions and network parameters of historical task data, input them into the task queue, and obtain network weights through initial training to obtain a convolutional neural network model after initial training.

[0009] Step S3, dividing the convolutional layers of the first trained convolutional neural network model DNN into stages and categories according to the commonality of the attributes of each convolutional layer, thereby obtaining blocks;

[0010] Step S4, calculating the filter channel size and pruning weight of each category convolution layer in each block, and screening the importance of the weights by using a filter that can adjust the filter channel size;

[0011] Step S5: Use the pruned intelligent model to perform the inference task. If the inference task is not satisfied, adjust the filter channel size.

[0012] Step S6: Select the corresponding activation function according to the task type and retrain the optimized neural network model;

[0013] Step S7: quantizing the pruned deep convolutional neural network model DNN;

[0014] Step S8, performing knowledge distillation and optimization on the network structure of the quantized deep convolutional neural network model DNN;

[0015] Step S9: designing a general-purpose hardware accelerator, wherein the integrated circuit of the general-purpose hardware accelerator is composed of a CPU, a GPU, and a field-programmable gate array (FPGA), the linear training task is completed by the CPU, and the nonlinear training task is completed by the GPU;

[0016] Step S10, deploying the optimized deep convolutional neural network model DNN in the FPGA; according to the resource constraints of the optimized deep convolutional neural network model DNN and the FPGA, selecting the corresponding hardware parallelization structure, including computing units, control units and storage units, deploying the optimized deep convolutional neural network model DNN in the FPGA to achieve hardware acceleration.

[0017] As a further improvement method of the present invention:

[0018] Optionally, step S3, dividing the convolutional layers of the first-trained convolutional neural network model DNN into stages and categories according to the commonality of properties of each convolutional layer, includes:

[0019] S31, grouping according to the size of the input feature map, and grouping the convolutional layers with the same input feature map size into one stage;

[0020] S32: For the convolution layers after the division stage, group and adjust them according to the size of the convolution kernel and the size of the adjacent pooling layer, and group the convolution layers with the same convolution kernel size and the pooling layers with the same pooling layer size into one large category;

[0021] S33, within each major category, according to the residual connection characteristics, all convolutional layers within a residual connection span are grouped into a block; finally, within each block, each convolutional layer is considered separately.

[0022] As a further improvement method of the present invention:

[0023] Optionally, in step S4, the filter channel size and pruning weight of each category convolution layer in each block are calculated, and the importance of the weight is screened by using a filter capable of adjusting the filter channel size, including:

[0024] S41, set the initial size of the filter channel to ,in, is the convolution kernel size, k is the preset coefficient;

[0025] S42: Training the obtained deep convolutional neural network model DNN network architecture until convergence;

[0026] S43: According to the formula Calculate the contribution of each parameter, where the error ; is the weight from neuron i to neuron j; , , is a set of index pairs; The contribution of each parameter; , represents the connection of one or more neurons that are selected for control; is the predetermined disturbance parameter vector Elements in It is the second-order derivative on the diagonal of the Hessian matrix H in the deep convolutional neural network model DNN;

[0027] S44: Sort the parameters by contribution and delete some parameters with low contribution to screen the importance of weights, cut off unimportant neurons, establish a local model of the error function to predict the impact of the perturbation parameters on the optimization target, and find the neurons with the least impact on the loss from the loss;

[0028] S45: Iterate to step S41, re-participate in the operation, and update the deep convolutional neural network model DNN again.

[0029] As a further improvement method of the present invention:

[0030] Optionally, in step S5, the pruned intelligent model is used to perform the reasoning task, and when the reasoning task is not satisfied, the filter channel size is adjusted, including:

[0031] S51, according to the formula Calculate the preset error of the reasoning task at each stage , preset threshold , then the cumulative error ; s is the number of stages;

[0032] S52: When When , adjust the parameters in the filter channel size so that the preset coefficient k=k+0.1;

[0033] S53: Iterate step S52 until Until then, re-participate in the calculation and update the deep convolutional neural network model DNN again.

[0034] As a further improvement method of the present invention:

[0035] Optionally, the step S6 selects a corresponding activation function according to the task type and retrains the optimized neural network model, including: selecting a corresponding activation function according to the selection signal, the activation function including Sgn function, Sigmoid function, and Tanh function,

[0036] in,

[0037] The Sgn function expression is: ;

[0038] The Sigmoid function expression is: ;

[0039] The Tanh function expression is: ;

[0040] Where x is the input of the neuron, w is the neuron weight, p is the bias coefficient; e represents the exponential function;

[0041] The selection signal is set to be two bits wide.

[0042] As a further improvement method of the present invention:

[0043] Optionally, step S7 quantizes the pruned deep convolutional neural network model DNN, including: converting floating-point parameters in the network into int integers that are more friendly to hardware parallel computing for calculation, extracting parameters, biases and feature map data from the lightweight model, dynamically solving the quantization factor of each set of data according to the established quantization bit width, and reloading all data into the deep convolutional neural network model DNN after applying the quantization method; then retraining and fine-tuning the quantized model to restore the original performance. The quantization formula is:

[0044] ;

[0045] W i = ∑ n = 1 N R n [ idx i ( n )] ;

[0046] ;

[0047] Among them, r i and r i-1 The value range R of the quantized weight n adjacent values within; n is an integer, M is the quantization bit width of the weight value, and N is the number of segments after weight quantization, which is used to determine the quantization value range; Indicates indexing function through NB bit; represents the absolute value function; is the original weight of the neural network model; After quantizing the weights of the neural network model, R n A value range collection.

[0048] As a further improvement method of the present invention:

[0049] Optionally, the step S8, performing knowledge distillation and optimization on the network structure of the quantized deep convolutional neural network model DNN, includes:

[0050] Distilled in a teacher-student integration manner, specifically:

[0051] The teacher model is trained on dataset D. For the classifier, the neural network output before softmax is taken as the soft target to guide the student model training. Through the attention mechanism, the teacher's attention map information is used to supervise the training of the student model to achieve spatial information transfer. In addition, the temperature scaling temptature scaler is used to scale the soft targets to obtain a flatter soft target distribution, which can further improve the performance of the student model, as shown in the following formula:

[0052] The loss function is:

[0053] ;

[0054] is the attention map of the teacher network, is the student network's attention map, h and f are the width and height of the attention map, respectively. The student model's overall loss function, Loss, consists of two parts: the cross entropy with the soft target and the cross entropy with the ground truth. After training, a smaller student model can achieve performance equivalent to or close to that of a more complex teacher model.

[0055] The network structure of the deep convolutional neural network model DNN includes an input layer, an output layer, 3N convolutional layers, 2N pooling layers, 2N fully connected layers and a Gaussian connection layer, wherein a convolutional layer is connected to a pooling layer, and then to a convolutional layer;

[0056] In step S9, a universal hardware accelerator is designed, wherein the universal hardware accelerator includes an input signal part, a bias unit register, an effective weight selection unit, a SUM unit, an output data processing unit, and an output unit; wherein the bias unit register is connected in parallel with the effective weight selection unit.

[0057] As a further improvement method of the present invention:

[0058] Optionally, the present invention further provides an artificial intelligence-based data intelligence model training and hardware acceleration system, comprising:

[0059] The original network structure module sets the network structure of the original deep convolutional neural network model DNN, and initializes the original deep convolutional neural network model DNN by randomly taking values for the weights;

[0060] The intermediate network structure module collects control instructions and network parameters from historical task data, inputs them into the task queue, and obtains network weights after the initial training, which are used to obtain the convolutional neural network model after the initial training;

[0061] The block division unit divides the convolutional layers of the first trained convolutional neural network model DNN into stages and categories according to the common properties of each convolutional layer, thereby obtaining blocks;

[0062] The screening unit calculates the filter channel size and pruning weight of each category convolution layer in each block, and uses the filter that can adjust the filter channel size to screen the importance of the weight;

[0063] Adjustment unit: Use the pruned intelligent model to perform inference tasks. If the inference task is not satisfied, adjust the filter channel size.

[0064] Retraining unit: selects the corresponding activation function according to the task type and retrains the optimized neural network model;

[0065] Quantization unit: quantizes the deep convolutional neural network model DNN after pruning;

[0066] The optimization unit performs knowledge distillation and optimization on the network structure of the quantized deep convolutional neural network model DNN;

[0067] Integrated circuit module, designing a universal hardware accelerator, wherein the integrated circuit of the universal hardware accelerator is composed of a CPU, a GPU and a field programmable gate array FPGA, linear training tasks are completed by the CPU, and nonlinear training tasks are completed by the GPU;

[0068] The accelerator module deploys the optimized deep convolutional neural network model DNN in the FPGA; according to the resource limitations of the optimized deep convolutional neural network model DNN and FPGA, the corresponding hardware parallelization structure is selected, including computing units, control units and storage units, and the optimized deep convolutional neural network model DNN is deployed in the FPGA to achieve hardware acceleration.

[0069] In order to solve the above problem, the present invention provides an electronic device, comprising:

[0070] A memory, a processor, and a computer program stored in the memory and runnable on the processor, characterized in that when the processor executes the computer program, the steps of the artificial intelligence-based data intelligence model training and hardware acceleration method are implemented.

[0071] In order to solve the above problems, the present invention also provides a computer-readable storage medium, which stores at least one instruction, and the at least one instruction is executed by a processor in an electronic device to implement the steps of the artificial intelligence-based data intelligence model training and hardware acceleration method.

[0072] This solution proposes an AI-based data intelligence model training and hardware acceleration method, which can achieve the following beneficial technical effects:

[0073] (1) The data intelligence model training and hardware acceleration method based on artificial intelligence of the present invention significantly improves the computational efficiency and hardware resource utilization of deep convolutional neural network models in reasoning tasks by innovatively combining model pruning, quantization, knowledge distillation and hardware acceleration technologies. Compared with traditional methods, the present invention can significantly reduce the number of model parameters and computational overhead while maintaining model accuracy, especially on resource-constrained embedded systems or FPGAs, achieving more efficient reasoning capabilities and lower energy consumption. In addition, by dynamically adjusting the filter channel and adaptively selecting the activation function, the method can optimize the model structure according to specific task requirements, further improving the response speed and adaptability of the system. Therefore, the present invention has broad application prospects and significant technical advantages in smart devices, edge computing and other applications with high requirements for real-time performance and resource efficiency.

[0074] (2) The present invention constructs a network structure of a deep convolutional neural network model DNN, making it more lightweight and accurately improving the level of intelligence. At the same time, this solution improves the expressive power of the convolutional neural network by designing a universal hardware architecture for multiple activation functions, making it truly meaningful as artificial intelligence. By introducing the stage and category division of the convolutional layer and the pruning technology of the filter channel: The present invention divides the convolutional layer of the deep convolutional neural network into stages and categories, combines the calculation of the filter channel size of each convolutional layer and performs weight pruning, greatly reducing the redundant parameters of the model, significantly reducing the computational complexity of the model, and maintaining the prediction accuracy of the model.

[0075] (3) The present invention combines quantization and knowledge distillation to optimize the network structure. Based on the quantization of the pruned network model, the present invention optimizes the quantized network structure through knowledge distillation technology, thereby further improving the accuracy and reasoning performance of the model while maintaining the lightweight of the model, especially realizing efficient reasoning calculation on resource-constrained hardware platforms.

[0076] (4) The present invention designs a general hardware accelerator and performs dynamic adaptive optimization. By designing a general hardware accelerator that integrates CPU, GPU and FPGA, combined with an adaptive optimization strategy that dynamically adjusts the filter channel and selects the activation function, the model can dynamically adapt to hardware resource limitations according to task requirements, achieving more efficient computing parallelism and lower energy consumption, and showing great advantages in embedded systems and edge computing scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0077] Figure 1 A flowchart of an artificial intelligence-based data intelligence model training and hardware acceleration method provided by one embodiment of the present invention;

[0078] Figure 2 A schematic diagram of the functional modules of a data intelligence model training and hardware acceleration method based on artificial intelligence provided by one embodiment of the present invention;

[0079] Figure 3 A schematic diagram of the structure of an electronic device for implementing an artificial intelligence-based data intelligence model training and hardware acceleration method provided by one embodiment of the present invention.

[0080] In the figure: 1 original network structure module, 2 intermediate network structure module, 3 blocking unit, 4 screening unit, 5 adjustment unit, 6 retraining unit, 7 quantization unit, 8 optimization unit, 9 integrated circuit module, 10 accelerator module;

[0081] 100 electronic device, 101 processor, 11 memory, 12 program, 13 communication interface.

[0082] The purpose, features and advantages of the present invention will be further described with reference to the accompanying drawings and in conjunction with the embodiments. DETAILED DESCRIPTION

[0083] It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0084] The present invention provides an artificial intelligence-based data intelligence model training and hardware acceleration method. The method may be performed by at least one electronic device, such as a server or terminal, that can be configured to perform the method provided by the present invention. In other words, the method may be performed by software or hardware installed on a terminal or server device, where the software may be a blockchain platform. The server may include, but is not limited to, a single server, a server cluster, a cloud server, or a cloud server cluster.

[0085] Example 1:

[0086] Step S1, setting the network structure of the original deep convolutional neural network model DNN, and initializing the original deep convolutional neural network model DNN by randomly taking weights.

[0087] Specifically, artificial neural networks perform computations based on the weights of neurons within the network. These computations fulfill specific functions, a process known as prediction. Prior to prediction, the neural network must calculate the network weights using a specific algorithm, a process known as training. The weights of the original deep convolutional neural network (DNN), or the initial deep convolutional neural network model, can be assigned arbitrary functions, and then trained to accurately determine these weights.

[0088] Step S2: collect control instructions and network parameters of historical task data, input them into the task queue, and obtain network weights after the first training to obtain the convolutional neural network model after the first training.

[0089] A deep neural network is an artificial neural network with multiple layers between the input and output layers. Whether the relationship is linear or nonlinear, a DNN finds the correct mathematical operation to transform the input into the output. As it iterates through each layer, it calculates the probability of each output. Specifically, a DNN trained to identify dog breeds would iterate over a given image and calculate the probability that the dog in the image is of a specific breed. Users can review the results and select the probability the network should display (e.g., above a certain threshold) and return the label suggested by the experiment.

[0090] Each mathematical operation is considered a layer, and complex DNNs have many layers, hence the name "deep" network. DNNs synthesize models by learning complex nonlinear relationships. These models are typically composed of multiple specific layers, each with corresponding features. It is known that as the number of layers increases, the model performs better, but some shallow networks can model with fewer units to process complex data. DNNs are typically feedforward neural networks, where data is transmitted from front to back without looping. Neurons in a DNN are mapped to each other, and this mapping connection initially has a random value, namely the weight of the neural network. The weight is multiplied by the output of the previous neuron to get the input of the next neuron, where the product is usually between 0 and 1. If the current network cannot achieve the desired effect, the algorithm will adjust the weights between neurons until the training achieves the desired effect.

[0091] In step S3, the convolutional layers of the first-trained convolutional neural network model DNN are divided into stages and categories according to the common properties of each convolutional layer, thereby obtaining blocks.

[0092] Specifically, in step S3, the convolutional layers of the first-trained convolutional neural network model DNN are divided into stages and categories according to the common properties of each convolutional layer, including:

[0093] S31, grouping according to the size of the input feature map, and grouping the convolutional layers with the same input feature map size into one stage;

[0094] S32: For the convolution layers after the division stage, group and adjust them according to the size of the convolution kernel and the size of the adjacent pooling layer, and group the convolution layers with the same convolution kernel size and the pooling layers with the same pooling layer size into one large category;

[0095] In S33, within each major category, all convolutional layers within a residual connection span are grouped into one block based on the residual connection characteristics; finally, within each block, each convolutional layer is calculated separately.

[0096] In one embodiment, a deep convolutional neural network model (DNN) is used for image classification tasks. The model consists of multiple convolutional layers, pooling layers, and fully connected layers. According to step S3 of the present invention, these convolutional layers are divided into stages and categories in the following manner:

[0097] Step S31 - Grouping Input Feature Map Sizes: There are three convolutional layers in the model, with input feature map sizes of 32x32, 16x16, and 8x8, respectively. In this step, convolutional layers with the same input feature map size are first grouped into one stage. For example, all convolutional layers with a feature map size of 32x32 are grouped into stage 1, convolutional layers with a feature map size of 16x16 are grouped into stage 2, and convolutional layers with a feature map size of 8x8 are grouped into stage 3.

[0098] Step S32 - Grouping Convolution Kernel Size and Pooling Layer Size Adjustment: Within each stage, further grouping adjustments are made based on the convolution kernel size and the size of adjacent pooling layers. For example, in stage 1, there are two convolution layers with kernel sizes of 3x3 and 5x5, respectively. The adjacent pooling layers have sizes of 2x2 and 3x3, respectively. Convolution layers with the same kernel size are grouped together, and pooling layers with the same pooling layer size are grouped together. Suppose that in stage 1, all convolution layers with a pooling layer size of 2x2 are grouped together as class A, and all convolution layers with a pooling layer size of 3x3 are grouped together as class B.

[0099] Step S33 - Blocking by Residual Connections: Within each cluster, convolutional layers are further divided into blocks based on residual connections. Residual connections are a connection method in deep neural networks where the output of certain layers is passed directly to subsequent layers, skipping intermediate layers. In cluster A, three convolutional layers use residual connections, and their residual connections have the same span. These convolutional layers are grouped into the same block. Similar block division is performed for the remaining convolutional layers in cluster B based on their residual connection characteristics.

[0100] In one embodiment, a convolutional neural network (DNN) model is used to recognize handwritten digits. The model's input is a 28x28 grayscale image. The model contains 10 convolutional layers, distributed across different stages and categories. By dividing the data in step S3 above, the following classification results are obtained:

[0101] Stage 1 (28x28 feature map size): Category A: Contains two convolutional layers with a kernel size of 3x3, adjacent pooling layers of 2x2, and residual connections. Category B: Contains one convolutional layer with a kernel size of 5x5, and adjacent pooling layers of 3x3.

[0102] Stage 2 (14x14 feature map size): Category C: Contains three convolutional layers with a kernel size of 3x3, adjacent pooling layers of size 2x2, and no residual connections.

[0103] Stage 3 (7x7 feature map size): Large Class D: Contains four convolutional layers with a 3x3 kernel size, adjacent pooling layers with a 2x2 size, and some convolutional layers use residual connections. Within each block, the weights of each convolutional layer are calculated independently, and the model structure is further optimized to improve overall inference speed and accuracy.

[0104] Step S4: Calculate the filter channel size and pruning weight of each category convolution layer in each block, and use the filter that can adjust the filter channel size to screen the importance of the weight.

[0105] For the calculation of the filter channel size and pruning weight of each category convolution layer in each block in step S4, the importance of weights is screened by using filters that can adjust the filter channel size, including:

[0106] S41, set the initial size of the filter channel to ,in, is the convolution kernel size, k is the preset coefficient;

[0107] S42: Training the obtained deep convolutional neural network model DNN network architecture until convergence;

[0108] S43: According to the formula Calculate the contribution of each parameter, where the error ; is the weight from neuron i to neuron j; , , is a set of index pairs; The contribution of each parameter; , represents the connection of one or more neurons that are selected for control; is the predetermined disturbance parameter vector Elements in It is the second-order derivative on the diagonal of the Hessian matrix H in the deep convolutional neural network model DNN;

[0109] S44: Sort the parameters by contribution and delete some parameters with low contribution to screen the importance of weights, cut off unimportant neurons, establish a local model of the error function to predict the impact of the perturbation parameters on the optimization target, and find the neurons with the least impact on the loss from the loss;

[0110] S45: Iterate to step S41, re-participate in the operation, and update the deep convolutional neural network model DNN again.

[0111] In one embodiment, a convolutional neural network model (DNN) is used for road sign recognition in an autonomous vehicle. A block of the model contains five convolutional layers, and the filter channel size of each convolutional layer is 64. The initial channel size is set in step S41, and the model is trained in step S42 until the model performs stably on the validation set. In step S43, after calculating the contribution of each filter channel, it is found that 15 channels have a negligible effect on the recognition accuracy, and may even reduce the overall performance due to overfitting. Therefore, in step S44, these 15 low-contribution channels are cut off, and 49 high-contribution channels are retained. After several such iterations, the complexity of the model is significantly reduced, while the performance in real driving scenarios remains excellent.

[0112] Step S5: Use the pruned intelligent model to perform the inference task. If the inference task is not satisfied, adjust the filter channel size.

[0113] In step S5, when the pruned intelligent model is used to perform reasoning tasks, if the reasoning tasks are not satisfied, the filter channel size is adjusted, including:

[0114] S51, according to the formula Calculate the preset error of the reasoning task at each stage , preset threshold , then the cumulative error ; s is the number of stages;

[0115] S52: When When , adjust the parameters in the filter channel size so that the preset coefficient k=k+0.1;

[0116] S53: Iterate step S52 until Until then, re-participate in the calculation and update the deep convolutional neural network model DNN again.

[0117] Step S6: Select the corresponding activation function according to the task type and retrain the optimized neural network model.

[0118] Step S6 selects the corresponding activation function according to the task type and retrains the optimized neural network model, including: selecting the corresponding activation function according to the selection signal, the activation function includes Sgn function, Sigmoid function, Tanh function,

[0119] in,

[0120] The Sgn function expression is: ;

[0121] The Sigmoid function expression is: ;

[0122] The Tanh function expression is: ;

[0123] Wherein, x is the input of the neuron, w is the neuron weight, p is the bias coefficient; e represents the exponential function; and the selection signal is set to be two bits wide.

[0124] In deep learning models, the activation function is a key component that determines how the output of a neuron affects the next level of the network. In different tasks, choosing the right activation function can significantly affect the performance and convergence speed of the model. Step S6 describes how to select the appropriate activation function based on the task type and retrain the optimized neural network model. The following is a specific operation description and example: Task type and activation function selection:

[0125] Sgn function: Suitable for binary classification tasks. The Sgn function limits the output value to -1 or 1, and is suitable for tasks that output clear positive and negative categories, such as black and white binarization of images and binary classification.

[0126] Sigmoid function: Suitable for classification tasks that require probabilistic output. The Sigmoid function limits the output value to between 0 and 1. It is often used in binary classification problems or scenarios where probabilistic output is required, such as determining whether an image contains a specific object.

[0127] Tanh function: Suitable for tasks that require symmetrical processing of positive and negative values. The Tanh function limits the output value to between -1 and 1. It is suitable for tasks that require normalizing the input data and maintaining data symmetry, such as calculating sentiment scores in text sentiment analysis.

[0128] A selection signal is used to automatically select an appropriate activation function based on task requirements. For example, in a comprehensive neural network system, the model generates a selection signal based on the input task type. This signal can be a flag or a coded value. Based on this signal, the model automatically selects the appropriate activation function and applies it to the output layer of neurons. Once the appropriate activation function is selected, the model readjusts the network weights based on the new activation function and retrains the network to ensure optimal performance for the specific task.

[0129] In one embodiment, determining whether an image contains a cat begins by selecting an activation function. For a binary classification task like this, the system generates a selection signal and selects the Sgn function as the activation function because it directly outputs a clear classification result of -1 or 1. After selecting the Sgn function, the model is retrained on a dataset containing a large number of cat and non-cat images. The model is ultimately able to quickly and accurately determine whether an input image contains a cat.

[0130] Step S7: quantize the pruned deep convolutional neural network model DNN.

[0131] In step S7, the pruned deep convolutional neural network model DNN is quantized, including: converting the floating-point parameters in the network into int integers that are more friendly to hardware parallel computing for calculation, extracting parameters, biases, and feature map data from the lightweight model, dynamically solving the quantization factor of each set of data according to the established quantization bit width, and reloading all the data into the deep convolutional neural network model DNN after applying the quantization method; then retraining and fine-tuning the quantized model to restore the original performance. The quantization formula is:

[0132] ;

[0133] W i = ∑ n = 1 N R n [ idx i ( n )] ;

[0134] ;

[0135] Among them, r i and r i-1 The value range R of the quantized weight n adjacent values within; n is an integer, M is the quantization bit width of the weight value, and N is the number of segments after weight quantization, which is used to determine the quantization value range; Indicates indexing function through NB bit; represents the absolute value function; is the original weight of the neural network model; After quantizing the weights of the neural network model, R n A value range collection.

[0136] Step S8: perform knowledge distillation and optimization on the network structure of the quantized deep convolutional neural network model DNN.

[0137] Due to differences in the depth or structure of the teacher and student networks, the number of intermediate feature channels extracted by the teacher and student networks may be different. In order to learn rich hierarchical information, the spatial transfer module also extracts spatial information from high-level features. The spatial transfer module converts the three-dimensional feature maps obtained from the teacher network and the student network into two-dimensional attention maps by superimposing them in the channel dimension, and then uses the corresponding loss function to make the attention maps of the teacher and student networks as similar as possible. The spatial feature distillation loss is used to measure the similarity between the student's attention map and the teacher's attention map. By using the teacher's attention map information to supervise the student's training, the transfer of spatial information can be achieved, improving the student's learning effect.

[0138] In step S8, knowledge distillation is performed on the network structure of the quantized deep convolutional neural network model DNN, and optimization is performed, including:

[0139] Distilled in a teacher-student integration manner, specifically:

[0140] The teacher model is trained on dataset D. For the classifier, the neural network output before softmax is taken as the soft target to guide the student model training. Through the attention mechanism, the teacher's attention map information is used to supervise the training of the student model to achieve spatial information transfer. In addition, the temperature scaling temptature scaler is used to scale the soft targets to obtain a flatter soft target distribution, which can further improve the performance of the student model, as shown in the following formula:

[0141] The loss function is:

[0142] ;

[0143] is the attention map of the teacher network, is the student network's attention map, h and f are the width and height of the attention map, respectively. The student model's overall loss function, Loss, consists of two parts: the cross entropy with the soft target and the cross entropy with the ground truth. After training, a smaller student model can achieve performance equivalent to or close to that of a more complex teacher model.

[0144] Among them, the network structure of the deep convolutional neural network model DNN includes an input layer, an output layer, 3N convolutional layers, 2N pooling layers, 2N fully connected layers and a Gaussian connection layer, wherein a convolutional layer is connected to an adjacent pooling layer, and then to an adjacent convolutional layer.

[0145] Step S9: designing a general-purpose hardware accelerator, wherein the integrated circuit of the general-purpose hardware accelerator is composed of a CPU, a GPU, and a field-programmable gate array (FPGA), the linear training task is completed by the CPU, and the nonlinear training task is completed by the GPU;

[0146] For step S9, a general hardware accelerator is designed, wherein the general hardware accelerator includes an input signal part, a bias unit register, an effective weight selection unit, a SUM unit, an output data processing unit, and an output unit; wherein the bias unit register is connected in parallel with the effective weight selection unit.

[0147] Step S10, deploying the optimized deep convolutional neural network model DNN in the FPGA; according to the resource constraints of the optimized deep convolutional neural network model DNN and the FPGA, selecting the corresponding hardware parallelization structure, including computing units, control units and storage units, deploying the optimized deep convolutional neural network model DNN in the FPGA to achieve hardware acceleration.

[0148] Example 2:

[0149] like Figure 2FIG. 1 is a schematic diagram of the structure of the functional modules of the method provided by one embodiment of the present invention.

[0150] This paper proposes an artificial intelligence-based data intelligence model training and hardware acceleration system, including:

[0151] Original network structure module 1, sets the network structure of the original deep convolutional neural network model DNN, and initializes the original deep convolutional neural network model DNN by randomly taking values for the weights;

[0152] The intermediate network structure module 2 collects the control instructions and network parameters of the historical task data, inputs them into the task queue, and obtains the network weights after the first training, which are used to obtain the convolutional neural network model after the first training;

[0153] The block division unit 3 divides the convolutional layers of the first trained convolutional neural network model DNN into stages and categories according to the common properties of each convolutional layer, thereby obtaining blocks;

[0154] Screening unit 4 calculates the filter channel size and pruning weight of each category convolution layer in each block, and screens the importance of the weights through a filter that can adjust the filter channel size;

[0155] Adjustment unit 5: Use the pruned intelligent model to perform reasoning tasks. If the reasoning task is not satisfied, adjust the filter channel size.

[0156] Retraining unit 6: Select the corresponding activation function according to the task type and retrain the optimized neural network model;

[0157] Quantization Unit 7: Quantize the pruned deep convolutional neural network model DNN;

[0158] Optimization unit 8 performs knowledge distillation and optimization on the network structure of the quantized deep convolutional neural network model DNN;

[0159] Integrated circuit module 9 designs a general hardware accelerator, wherein the integrated circuit of the general hardware accelerator is composed of a CPU, a GPU, and a field programmable gate array (FPGA). The linear training task is completed by the CPU, and the nonlinear training task is completed by the GPU.

[0160] The accelerator module 10 deploys the optimized deep convolutional neural network model DNN in the FPGA; according to the resource limitations of the optimized deep convolutional neural network model DNN and the FPGA, the corresponding hardware parallelization structure is selected, including a computing unit, a control unit and a storage unit, and the optimized deep convolutional neural network model DNN is deployed in the FPGA to achieve hardware acceleration.

[0161] Example 3:

[0162] like Figure 3 FIG. 1 is a schematic diagram of the structure of an electronic device according to a method provided by an embodiment of the present invention.

[0163] The electronic device 100 may include a processor 101 , a memory 11 , a communication interface 13 , and a bus, and may further include a computer program stored in the memory 11 and executable on the processor 101 , such as program 12 .

[0164] The memory 11 includes at least one type of readable storage medium, including flash memory, a removable hard disk, a multimedia card, a card-type memory (e.g., SD or DX memory), a magnetic memory, a magnetic disk, an optical disk, etc. In some embodiments, the memory 11 may be an internal storage unit of the electronic device 100, such as a removable hard disk of the electronic device 100. In other embodiments, the memory 11 may also be an external storage device of the electronic device 100, such as a plug-in removable hard disk, a Smart Media Card (SMC), a Secure Digital (SD) card, a flash memory card, etc. equipped on the electronic device 100. Furthermore, the memory 11 may include both an internal storage unit of the electronic device 100 and an external storage device. The memory 11 can be used not only to store application software installed in the electronic device 100 and various types of data, such as the code of the program 12, but also to temporarily store data that has been output or is about to be output.

[0165] In some embodiments, the processor 101 may be composed of an integrated circuit, such as a single packaged integrated circuit or a plurality of packaged integrated circuits with the same or different functions, including one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and a combination of various control chips. The processor 101 is the control core (Control Unit) of the electronic device, connecting the various components of the entire electronic device using various interfaces and circuits. It executes or runs programs or modules stored in the memory 11 (such as the program 12 for implementing artificial intelligence-based data intelligence model training and hardware acceleration methods) and calls data stored in the memory 11 to perform various functions of the electronic device 100 and process data.

[0166] The communication interface 13 may include a wired interface and / or a wireless interface (such as a WI-FI interface, a Bluetooth interface, etc.), which is generally used to establish a communication connection between the electronic device 100 and other electronic devices, and to achieve connection and communication between internal components of the electronic devices.

[0167] The bus may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus. The bus may be divided into an address bus, a data bus, a control bus, etc. The bus is configured to enable connection and communication between the memory 11 and at least one processor 101, etc.

[0168] Figure 2 Only the electronic device with components is shown, and it can be understood by those skilled in the art that Figure 2 The structure shown does not constitute a limitation on the electronic device 100 , and the electronic device 100 may include fewer or more components than shown in the figure, or combine certain components, or arrange the components differently.

[0169] For example, although not shown, the electronic device 100 may further include a power supply (e.g., a battery) to power various components. Preferably, the power supply may be logically connected to the at least one processor 101 via a power management device, thereby enabling functions such as charge management, discharge management, and power consumption management via the power management device. The power supply may further include any components such as one or more DC or AC power supplies, a recharging device, a power failure detection circuit, a power converter or inverter, and a power status indicator. The electronic device 100 may also include various sensors, Bluetooth modules, Wi-Fi modules, etc., which are not further described here.

[0170] Optionally, the electronic device 100 may further include a user interface, which may be a display or an input unit (such as a keyboard). Optionally, the user interface may also be a standard wired interface or a wireless interface. Optionally, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, or an OLED (Organic Light-Emitting Diode) touchscreen. The display may also be appropriately referred to as a display screen or a display unit, and is used to display information processed in the electronic device 100 and to display a visual user interface.

[0171] It should be understood that the embodiment is for illustration only and the scope of the patent application is not limited to this structure.

[0172] The program 12 stored in the memory 11 of the electronic device 100 is a combination of multiple instructions. When executed in the processor 101, it can achieve the following:

[0173] Set the network structure of the original deep convolutional neural network model DNN, and initialize the original deep convolutional neural network model DNN by randomly taking values for the weights;

[0174] Collect control instructions and network parameters from historical task data, input them into the task queue, and obtain network weights after the first training to obtain the convolutional neural network model after the first training;

[0175] The convolutional layers of the first trained convolutional neural network model DNN are divided into stages and categories according to the common properties of each convolutional layer, thereby obtaining blocks;

[0176] Calculate the filter channel size and pruning weight of each category convolution layer in each block, and use filters that can adjust the filter channel size to screen the importance of the weights;

[0177] Use the pruned intelligent model to perform inference tasks. If the inference task is not satisfied, adjust the filter channel size.

[0178] Select the corresponding activation function according to the task type and retrain the optimized neural network model;

[0179] Quantify the deep convolutional neural network model DNN after pruning;

[0180] Perform knowledge distillation and optimization on the network structure of the quantized deep convolutional neural network model DNN;

[0181] Design a general-purpose hardware accelerator, wherein the integrated circuit of the general-purpose hardware accelerator is composed of a CPU, a GPU, and a field-programmable gate array (FPGA), linear training tasks are completed by the CPU, and nonlinear training tasks are completed by the GPU;

[0182] The optimized deep convolutional neural network model DNN is deployed in the FPGA; according to the resource limitations of the optimized deep convolutional neural network model DNN and FPGA, the corresponding hardware parallelization structure is selected, including computing units, control units and storage units, and the optimized deep convolutional neural network model DNN is deployed in the FPGA to achieve hardware acceleration.

[0183] Specifically, the specific implementation method of the processor 101 for the above instructions can refer to Figures 1 to 2 The description of the relevant steps in the corresponding embodiments will not be repeated here.

[0184] It should be noted that the serial numbers of the above-mentioned embodiments of the present invention are for descriptive purposes only and do not represent the advantages or disadvantages of the embodiments. In addition, the terms "including", "comprising" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, device, article or method comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, device, article or method. In the absence of further restrictions, an element defined by the sentence "including a ..." does not exclude the presence of other identical elements in the process, device, article or method comprising the element.

[0185] Through the description of the above embodiments, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus the necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better embodiment. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes a number of instructions for enabling a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods described in each embodiment of the present invention.

[0186] The above are only preferred embodiments of the present invention and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention description and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A data intelligence model training and hardware acceleration method based on artificial intelligence, characterized in that: The method comprises: Step S1, setting the network structure of the original deep convolutional neural network model DNN, and initializing the original deep convolutional neural network model DNN by randomly taking weights; Step S2: Collect control instructions and network parameters of historical task data, input them into the task queue, and obtain network weights through initial training to obtain a convolutional neural network model after initial training. Step S3, dividing the convolutional layers of the first trained convolutional neural network model DNN into stages and categories according to the commonality of the attributes of each convolutional layer, thereby obtaining blocks; Step S4, calculating the filter channel size and pruning weight of each category convolution layer in each block, and screening the importance of the weights by using a filter that can adjust the filter channel size; Step S5: Use the pruned intelligent model to perform the inference task. If the inference task is not satisfied, adjust the filter channel size, including: S51: According to the error calculation formula Calculate the preset error E' of the reasoning task at each stage s , then the cumulative error s is the number of stages; S52: When E'>ξ, adjust the parameters in the filter channel size so that the preset coefficient k=k+0.1; ξ is the preset error threshold; S53: Return to step S52 until E'≤ξ, re-participate in the calculation, and update the deep convolutional neural network model DNN again; Step S6: selecting the corresponding activation function according to the task type and retraining the optimized neural network model, including: the neural network model generates a selection signal according to the input task type, the selection signal is a flag bit or a coding value, and the neural network model automatically selects the corresponding activation function according to the selection signal and applies it to the output layer of the neuron. After the activation function is selected, the neural network model readjusts the network weight parameters according to the selected activation function and retrains; Step S7: quantizing the pruned deep convolutional neural network model DNN; Step S8, performing knowledge distillation and optimization on the network structure of the quantized deep convolutional neural network model DNN; Step S9: designing a general-purpose hardware accelerator, wherein the integrated circuit of the general-purpose hardware accelerator is composed of a CPU, a GPU, and a field-programmable gate array (FPGA), the linear training task is completed by the CPU, and the nonlinear training task is completed by the GPU; Step S10, deploying the optimized deep convolutional neural network model DNN in the FPGA; according to the resource constraints of the optimized deep convolutional neural network model DNN and the FPGA, selecting the corresponding hardware parallelization structure, including computing units, control units and storage units, deploying the optimized deep convolutional neural network model DNN in the FPGA to achieve hardware acceleration.

2. The data intelligence model training and hardware acceleration method based on artificial intelligence according to claim 1, characterized in that: The step S3 divides the convolutional layers of the first trained convolutional neural network model DNN into stages and categories according to the common attributes of each convolutional layer, including: S31, grouping according to the size of the input feature map, and grouping the convolutional layers with the same input feature map size into one stage; S32: For the convolution layers after the division stage, group and adjust them according to the size of the convolution kernel and the size of the adjacent pooling layer, and group the convolution layers with the same convolution kernel size and the pooling layers with the same pooling layer size into one large category; S33, within each major category, according to the residual connection characteristics, all convolutional layers within a residual connection span are grouped into one block; within each block, each convolutional layer is processed independently.

3. The data intelligence model training and hardware acceleration method based on artificial intelligence according to claim 1, characterized in that: In the step S4, the filter channel size and pruning weight of each category convolution layer in each block are calculated, and the importance of the weight is screened by using a filter that can adjust the filter channel size, including: S41, set the initial size of the filter channel to Where b' is the convolution kernel size and k is the preset coefficient; S42: Training the obtained deep convolutional neural network model DNN network architecture until convergence; S43: According to the formula Calculate the contribution of each parameter, where the error w ij is the weight from neuron i to neuron j; V k is a set of index pairs; C d The contribution of each parameter; u k =w ij , represents the connection of one or more neurons to be controlled; δu i is an element in the predetermined disturbance parameter vector δU; h ii It is the second-order derivative on the diagonal of the Hessian matrix H in the deep convolutional neural network model DNN; S44: Sort the parameters by contribution and delete some parameters whose contribution is less than the set threshold to screen the importance of the weights, prune unimportant neurons, and establish a local model of the error function to predict the impact of the perturbation parameters on the optimization target, and find the neurons with the least impact on the loss; S45: Return to step S41, re-participate in the calculation, and update the deep convolutional neural network model DNN again.

4. The method for data intelligent model training and hardware acceleration based on artificial intelligence according to claim 1, wherein: The step S6, selecting the corresponding activation function according to the task type and retraining the optimized neural network model, includes: selecting the corresponding activation function according to the selection signal, the activation function includes Sgn function, Sigmoid function, Tanh function, in, The Sgn function expression is: The Sigmoid function expression is: The Tanh function expression is: Where x is the input of the neuron, w is the neuron weight, p is the bias coefficient; e represents the base of the natural logarithm, and T is the transpose; The selection signal is set to be two bits wide.

5. The method for data intelligent model training and hardware acceleration based on artificial intelligence according to claim 4, characterized in that: The step S7 quantizes the pruned deep convolutional neural network model DNN, including converting floating-point parameters in the network into int integers that are more friendly to hardware parallel computing for calculation, extracting parameters, biases, and feature map data from the lightweight model, dynamically solving the quantization factor of each set of data according to the established quantization bit width, and reloading all data into the deep convolutional neural network model DNN after applying the quantization method; then retraining and fine-tuning the quantized model to restore the original performance. The quantization formula is: Among them, r i and r i-1 The value range R of the quantized weight n adjacent values within; n is an integer, M is the quantization bit width of the weight value, and N is the number of segments after weight quantization, which is used to determine the quantization value range; idx i () indicates the index function through the NB bit; abs() indicates the absolute value function; W i is the original weight of the neural network model; After quantizing the weights of the neural network model, R n A value range collection.

6. The method for data intelligent model training and hardware acceleration based on artificial intelligence according to claim 1, characterized in that: The step S8 performs knowledge distillation and optimization on the network structure of the quantized deep convolutional neural network model DNN, including: Distilled in a teacher-student integration manner, specifically: The teacher model is trained on dataset D. For the classifier, the neural network output before softmax is taken as the soft target to guide the student model training. Through the attention mechanism, the teacher's attention map information is used to supervise the training of the student model to achieve spatial information transfer. In addition, the temperature scaling Temptature scaler is used to scale the soft targets to obtain a flatter soft target distribution, which can further improve the performance of the student model, as shown in the following formula: The loss function is: a T is the attention map of the teacher network, a s is the attention map of the student network, h and f are the width and height of the attention map respectively; the overall loss function of the student model Loss consists of two parts: the cross entropy with the soft target and the cross entropy with the ground truth. After training, the smaller student model can achieve performance equivalent to or close to that of the complex teacher model; The network structure of the deep convolutional neural network model DNN includes an input layer, an output layer, 3N convolutional layers, 2N pooling layers, 2N fully connected layers and a Gaussian connection layer, wherein a convolutional layer is connected to a pooling layer, and then to a convolutional layer; In step S9, a universal hardware accelerator is designed, wherein the universal hardware accelerator includes an input signal part, a bias unit register, an effective weight selection unit, a SUM unit, an output data processing unit, and an output unit; wherein the bias unit register is connected in parallel with the effective weight selection unit.

7. An artificial intelligence-based data intelligence model training and hardware acceleration system, characterized in that: include: The original network structure module sets the network structure of the original deep convolutional neural network model DNN, and initializes the original deep convolutional neural network model DNN by randomly taking values for the weights; The intermediate network structure module collects control instructions and network parameters from historical task data, inputs them into the task queue, and obtains network weights after the initial training, which are used to obtain the convolutional neural network model after the initial training; The block division unit divides the convolutional layers of the first trained convolutional neural network model DNN into stages and categories according to the common properties of each convolutional layer, thereby obtaining blocks; The screening unit calculates the filter channel size and pruning weight of each category convolution layer in each block, and uses the filter that can adjust the filter channel size to screen the importance of the weight; Adjustment unit: Use the pruned intelligent model to perform inference tasks. If the inference task is not satisfied, adjust the filter channel size, including: According to the error calculation formula Calculate the preset error E' of the reasoning task at each stage s , then the cumulative error s is the number of stages; When E'>ξ, adjust the parameters in the filter channel size so that the preset coefficient k=k+0.1; ξ is the preset error threshold; Until E'≤ξ, re-participate in the calculation and update the deep convolutional neural network model DNN again; Retraining unit: Selects the corresponding activation function according to the task type and retrains the optimized neural network model. This includes: the neural network model generates a selection signal based on the input task type. The selection signal is a flag or a coded value. Based on the selection signal, the neural network model automatically selects the corresponding activation function and applies it to the output layer of the neuron. After the activation function is selected, the neural network model readjusts the network weight parameters according to the selected activation function and retrains. Quantization unit: quantizes the deep convolutional neural network model DNN after pruning; The optimization unit performs knowledge distillation and optimization on the network structure of the quantized deep convolutional neural network model DNN; Integrated circuit module, designing a universal hardware accelerator, wherein the integrated circuit of the universal hardware accelerator is composed of a CPU, a GPU and a field programmable gate array FPGA, linear training tasks are completed by the CPU, and nonlinear training tasks are completed by the GPU; The accelerator module deploys the optimized deep convolutional neural network model DNN in the FPGA; according to the resource limitations of the optimized deep convolutional neural network model DNN and FPGA, the corresponding hardware parallelization structure is selected, including computing units, control units and storage units, and the optimized deep convolutional neural network model DNN is deployed in the FPGA to achieve hardware acceleration.

8. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the steps of the artificial intelligence-based data intelligence model training and hardware acceleration method as described in any one of claims 1 to 6 are implemented.

9. A computer storage medium storing a computer program, wherein: When the computer program is executed by a processor, the steps of the artificial intelligence-based data intelligence model training and hardware acceleration method as described in any one of claims 1 to 6 are implemented.

Images