A data processing method based on an artificial intelligence chip

Abstract

The application discloses a data processing method based on an artificial intelligence chip, comprising the following steps: S1, monitoring the complexity parameter of the input data stream in real time; S2, dynamically adjusting the computing power distribution ratio of each computing unit in the chip by using a reinforcement learning algorithm; S3, adopting an attention mechanism to perform space-time alignment on multi-modal data such as vision, voice and text; S4, predicting the task execution time based on a long short-term memory network; S5, automatically switching the computing precision mode; and S6, embedding a federated learning framework in the data preprocessing stage.The application dynamically adjusts the computing power distribution ratio by using the reinforcement learning algorithm, real-time adapts the data stream complexity, improves the chip resource utilization rate, and combines the long short-term memory network to predict the task execution time, so that the average waiting time is optimized by the scheduling strategy, the delay is greatly reduced compared with a traditional FCFS algorithm, and the interference of redundant information is effectively filtered.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, and in particular to a data processing method based on an artificial intelligence chip. BACKGROUND

[0002] With the rapid development of artificial intelligence technology, artificial intelligence chips are increasingly widely used in intelligent terminals, edge computing and other scenarios. The design of traditional artificial intelligence chips usually adopts a fixed computing power allocation mode, relying on manually preset computing resource division strategies.

[0003] However, the existing artificial intelligence chips have the following technical bottlenecks in actual application: low computing power allocation efficiency, the existing chips cannot dynamically adjust the computing power allocation according to the complexity of real-time data flow, resulting in delay of high complexity tasks due to insufficient resources, and waste of computing resources for low complexity tasks; insufficient multi-modal data fusion accuracy, the temporal and spatial alignment of visual, speech, text and other multi-modal data relies on manually designed fusion rules, lacking adaptability; single task scheduling strategy, the existing task scheduling is based on static priority or first-come-first-served algorithm, which cannot predict task execution time, easily leading to queue congestion or resource idleness; difficulty in balancing energy consumption and performance, to reduce energy consumption, the existing chips often adopt fixed frequency reduction or precision reduction mode, but lack real-time perception of temperature and task type; prominent data privacy risk, traditional centralized training requires uploading original data to the cloud, which has the risk of privacy leakage. SUMMARY

[0004] The purpose of the present application is to solve the problems in the prior art and provide a data processing method based on an artificial intelligence chip.

[0005] To achieve the above purpose, the present application adopts the following technical solutions:

[0006] A data processing method based on an artificial intelligence chip, comprising the following steps:

[0007] S1, real-time monitoring of the complexity parameters of the input data flow, including data dimension D, feature sparsity S and computation intensity C;

[0008] S2, dynamically adjusting the computing power allocation ratio of each computing unit in the chip using a reinforcement learning algorithm;

[0009] S3, using an attention mechanism to perform temporal and spatial alignment on visual, speech, text and other multi-modal data, the attention mechanism including a multi-head self-attention layer and a cross-modal attention layer, the number of heads h of the multi-head self-attention layer being dynamically adjustable, ranging from 4 to 16 heads;

[0010] S4, predicting the task execution time based on a long short-term memory network, the input of the long short-term memory network unit being the historical execution time sequence T of the taskhistory = [t1, t2, …, t n ], output is the predicted task execution time

[0011] S5, when the chip temperature T chip exceeds 75℃, automatically switch the calculation precision mode FP32→FP16→INT8, predict the precision loss rate in different modes through on-chip neural network, the input of the on-chip neural network is the current running parameter of the chip, the output is the precision loss rate in different calculation precision modes, the training data of the on-chip neural network comes from the chip historical running log, containing 10 6 pairs of energy consumption-precision data under different working conditions;

[0012] S6, embed the federated learning framework in the data preprocessing stage, adopt the asynchronous update strategy, the local device trains the model on the local data set to obtain the local model parameter θ local , the local model parameter update period is 50-200 training batches, and the global server aggregates the local model parameters of multiple local devices.

[0013] Preferably, in the step S1, the calculation formula of the feature sparsity S is wherein N zeros represents the number of zero elements in the feature vector, and N total represents the total number of elements in the feature vector; and the calculation formula of the calculation intensity C is wherein N ops represents the number of operations required to complete the calculation, and N data represents the amount of data participating in the calculation.

[0014] Preferably, in the step S2, the reinforcement learning algorithm adopts the proximal policy optimization, and the proximal policy optimization comprises the following steps: defining a state space S state ={D, S, C}, defining an action space A action is a set of computing unit computing power allocation ratios, the ratio range is 1:3 to 3:1, and the reward function R is defined as R=α×E-β×T, wherein E is the computing power allocation efficiency, calculated by the formula , N completed is the number of completed computing tasks within a certain time, N total is the total number of computing tasks; T is the task completion time; and α and β are weighting coefficients, and α+β=1.

[0015] Preferably, in the step S3, the calculation process of the multi-head self-attention layer is as follows:

[0016] a. For the input modal data X, the query matrix Q, the key matrix K and the value matrix V are calculated respectively, as follows: Q = XW Q , K = XW K , V = XW V , where W Q , W K and W V are learnable weight matrices;

[0017] b. The attention score is calculated as follows: where d k is the dimension of the query matrix Q;

[0018] C. The output of the multi-head self-attention layer is:

[0019] MultiHead(Q, K, V) = Concat(head1, …, head h )W O ,

[0020] where head i = Attention(QW Qi , KW Ki , VW Vi ), W Qi , W Ki , W Vi and W O are learnable weight matrices;

[0021] d. The cross-modal attention layer is used to fuse the features of different modalities, by taking the features of one modality as the query, the features of another modality as the key and the value, calculating the attention score and updating the feature representation;

[0022] e. Finally, a fusion feature vector with a dimension of 2048 is generated.

[0023] Preferably, in the step S4, the calculation formula of the long short-term memory network unit is as follows:

[0024] Input gate i t = σ(W ii x t +W hi h t-1 +b i )

[0025] Forget gate f t = σ(W if x t +W hf h t-1 +b f )

[0026] Cell state update

[0027] Cell state

[0028] Output gate o t = sigma(W io x t + W ho h t-1 + b o )

[0029] Hidden state h t = o t tanh(C t )

[0030] Predicted task execution time

[0031] According to the predicted task execution time, the waiting time of the task queue is optimized to ≤15ms, while the chip utilization rate is ≥85%, the task scheduling strategy adopts the priority scheduling algorithm, and the priority of the task is determined according to the predicted execution time and the urgency of the task.

[0032] Preferably, in the step S6, the local model parameter aggregation formula is Wherein N is the number of local devices participating in federated learning, and the local device updates the local model according to the global model parameter.

[0033] Preferably, in the step S3, the learnable weight matrix of the multi-head self-attention layer and the cross-modal attention layer is trained by a back propagation algorithm, and the optimization target is to minimize the loss function of multi-modal data fusion, and the loss function adopts a cross-entropy loss function.

[0034] Preferably, in the step S4, the training of the long short-term memory network adopts a stochastic gradient descent algorithm, the learning rate is 0.001-0.01, and the training round number is 100-500 rounds.

[0035] Preferably, in the step S5, the on-chip neural network adopts a convolutional neural network structure, and performs feature extraction and prediction on the input chip operation parameters through a convolutional layer, a pooling layer and a fully connected layer.

[0036] Preferably, in the step S6, in the federated learning framework, the local device performs differential privacy protection processing on the parameters before uploading the local model parameters, and adds Laplace noise.

[0037] The present application has the following beneficial effects:

[0038] 1、The application dynamically adjusts the computing power allocation ratio through the reinforcement learning algorithm, real-time adapts the data stream complexity, improves the chip resource utilization rate, and combines the long short-term memory network to predict the task execution time, the scheduling strategy optimizes the average waiting time, and reduces the delay by 40% compared with the traditional FCFS algorithm;

[0039] 2、The multi-head self-attention layer can realize the spatio-temporal alignment of multi-modal data with the cross-modal attention mechanism, generate a 2048-dimensional fusion feature vector, improve the classification accuracy in the multi-modal sentiment analysis task, and the attention mechanism automatically focuses on the key features, effectively filters the environment and reduces the interference of redundant information;

[0040] 3、When the temperature exceeds 75 DEG C, the calculation precision mode is automatically switched to FP32→FP16→INT8, the precision loss rate is combined with the on-chip neural network, in the ResNet50 model inference, the energy consumption is reduced by 45%, while the precision loss is controlled within 1.5%, and through dynamic frequency reduction and precision adjustment, the long-term running temperature of the chip is stabilized within 80 DEG C, and the thermal aging effect is reduced;

[0041] 4、The local device only uploads the model parameters, combined with differential privacy protection, in the MNIST data set training, the privacy leakage risk is greatly reduced, and the step update strategy makes the global model convergence speed faster than the synchronous federated learning by 30%, which is suitable for real-time edge computing scenarios with high requirements;

[0042] 5、The on-chip neural network and the reinforcement learning framework realize local data processing and strategy optimization, reduce the demand for cloud interaction, and through the training of 10 6 + historical working condition data, the system can quickly adapt to new task types without re-deploying the model. BRIEF DESCRIPTION OF DRAWINGS

[0043] Figure 1 A flowchart of a data processing method based on an artificial intelligence chip is proposed. DETAILED DESCRIPTION

[0044] The technical solutions in the embodiments of the application will be described clearly and completely below with reference to the drawings in the embodiments of the application. Obviously, the described embodiments are only part of the embodiments of the application, not all.

[0045] Referring to Figure 1 A data processing method based on an artificial intelligence chip, comprising the following steps:

[0046] S1, real-time monitoring of the complexity parameters of the input data stream, including data dimension D, feature sparsity S and calculation intensity C, the calculation formula of the feature sparsity S is Wherein, N zerosN represents the number of zero elements in the feature vector total N represents the total number of elements in the feature vector; the calculation formula of the calculation intensity C is N represents the number of zero elements in the feature vector ops N represents the number of operations required to complete the calculation data N represents the amount of data involved in the calculation

[0047] S2, dynamically adjusts the computing power allocation ratio of each computing unit in the chip by using a reinforcement learning algorithm, the training process of the reinforcement learning algorithm adopts an experience replay mechanism, a batch of data is randomly sampled from the experience replay buffer for training, and the reinforcement learning algorithm adopts a proximal policy optimization, which includes the following steps: defining a state space S state ={D, S, C}, defining an action space A action is a set of computing power allocation ratios of each computing unit, the ratio range is 1:3 to 3:1, and a reward function R is defined as R=α×E-β×T, wherein E is the computing power allocation efficiency, which is calculated by the formula N represents the number of zero elements in the feature vector completed N represents the number of computing tasks completed within a certain time total N represents the total number of computing tasks; T represents the task completion time; and α and β are weighting coefficients, and α+β=1

[0048] S3, a spatio-temporal alignment is performed on multi-modal data such as vision, voice and text by using an attention mechanism, the attention mechanism includes a multi-head self-attention layer and a cross-modal attention layer, and the learnable weight matrix of the multi-head self-attention layer and the cross-modal attention layer is trained by a back propagation algorithm, and the optimization target is to minimize the loss function of multi-modal data fusion, and the loss function adopts a cross-entropy loss function, the number of heads h of the multi-head self-attention layer can be dynamically adjusted, and the range is 4-16 heads, and the calculation process of the multi-head self-attention layer is as follows:

[0049] a. For the input modal data X, the query matrix Q, the key matrix K and the value matrix V are calculated respectively as follows: Q=XW Q , K=XW K , V=XW V , wherein W Q , W K and W V are learnable weight matrices;

[0050] b. The attention score is calculated as where d k is the dimension of the query matrix Q

[0051] C. The output of the multi-head self-attention layer is:

[0052] MultiHead(Q, K, V) = Concat(head1,..., headH) WO h )W O ,

[0053] where head i = Attention(QW Qi , KW Ki , VW Vi ), W Qi , W Ki , W Vi and W O are learnable weight matrices;

[0054] d. Cross-modal attention layer is used to fuse features of different modalities, by taking the features of one modality as query, the features of another modality as key and value, calculating attention score and updating feature representation;

[0055] e. Finally, a fusion feature vector with a dimension of 2048 is generated;

[0056] S4, based on long short-term memory network to predict task execution time, the input of long short-term memory network unit is the historical execution time sequence T history = [t1, t2,..., t n ] of task, and the output is the predicted task execution time The calculation formula of long short-term memory network unit is as follows:

[0057] Input gate i t = σ(W ii x t +W hi h t-1 +b i )

[0058] Forget gate f t = σ(W if x t +W hf h t-1 +b f )

[0059] Cell state update

[0060] Cell state

[0061] Output gate o t = σ(W io x t +W ho h t-1 +b o )

[0062] Hidden state h t = o t ⊙ tanh(C t )

[0063] Predicted task execution time

[0064] According to the predicted task execution time, the waiting time of the task queue is optimized to ≤15ms while maintaining the chip utilization rate ≥85%, the task scheduling strategy adopts the priority scheduling algorithm, the priority of the task is determined according to the predicted execution time and the urgency of the task, and the training of the long short-term memory network adopts the stochastic gradient descent algorithm, the learning rate is 0.001-0.01, and the training rounds are 100-500 rounds.

[0065] S5, when the chip temperature T chip exceeds 75℃, automatically switch the calculation precision mode FP32→FP16→INT8, predict the precision loss rate under different modes through the on-chip neural network, the input of the on-chip neural network is the current running parameter of the chip, and the output is the precision loss rate under different calculation precision modes, the training data of the on-chip neural network comes from the chip historical running log, containing 10 6 + pairs of energy-precision data under different working conditions, the on-chip neural network adopts a convolutional neural network structure, and the input chip running parameters are feature extracted and predicted through convolutional layers, pooling layers and fully connected layers;

[0066] S6, embed the federated learning framework in the data preprocessing stage, adopt the asynchronous update strategy, and train the model on the local data set to obtain the local model parameter θ local Before uploading the local model parameters, the local device performs differential privacy protection processing on the parameters and adds Laplace noise, the local model parameter update period is 50-200 training batches, the global server aggregates the local model parameters of multiple local devices, and the local model parameter aggregation formula is Where N is the number of local devices participating in federated learning, and the local device updates the local model according to the global model parameter.

[0067] Example 1

[0068] Image classification optimization based on small computer

[0069] Scene description: deploy image classification tasks on an edge computing platform, and compare the performance of the method of the application and the fixed computing power allocation method.

[0070] Implementation steps:

[0071] 1. Hardware configuration: processor, GPU, memory, data set and framework are all common and purchasable configurations on the market.

[0072] 2, The method of the application (experimental group): dynamic resource allocation, the reinforcement learning algorithm dynamically adjusts the GPU and CPU computing power allocation (default 3:1) according to the image data dimension (32x32→224x224), and when the precision switching temperature is greater than or equal to 75℃, it automatically switches from FP32→FP16→INT8, and the on-chip neural network prediction accuracy loss (based on 100,000 historical data training).

[0073] 3, The comparison method: fixed allocation (control group 1), GPU accounting for 70%, CPU accounting for 30%; TensorRT optimization (control group 2), using only FP16 inference (prior art).

[0074] 4, Record the computing power utilization rate, single image inference time, energy consumption, classification accuracy and temperature peak data of the experimental group, control group 1 and control group 2 during the execution of the image classification task, and arrange the recorded data into a table, the specific results are shown in the following table.

[0075] Table 1: Comparison of image classification optimization task data

[0076] Indicators / Items Experimental Group Control Group 1 Control Group 2 Computing power utilization rate 86.2％ 62.5％ 78.3％ Single image inference time 12.3ms 18.7ms 15.1ms Energy consumption (average power) 4.2W 5.8W 5.1W Classification accuracy rate (INT8 mode) 92.1％ 91.5％ 91.8％ Temperature peak value 79℃ 88℃ 85℃

[0077] From the above table, it can be seen that when the data processing method of the application is used for image classification optimization, the computing power utilization rate is improved by 38%, the inference time is shortened by 34%, the energy consumption is reduced by 27%, and the accuracy loss under INT8 mode is only 0.6%, which is better than TensorRT optimization.

[0078] Example 2

[0079] USB acceleration device real-time target detection

[0080] Scene description

[0081] Deploy YOLOv5 target detection task on x86 architecture host with USB acceleration device (2TOPS computing power), compare the method of the application with traditional task scheduling algorithm.

[0082] Implementation steps:

[0083] 1, Hardware configuration: host, USB acceleration device, data set and framework are common commercially available configurations.

[0084] 2, The method of the application (experimental group): adaptive task scheduling, LSTM predicts task execution time (based on 1000 historical inference data training), priority scheduling strategy optimizes the queue, multi-modal fusion, fuses camera RGB image and IMU sensor data (requires external IMU module).

[0085] 3. Comparison method: FCFS scheduling (control group 1), first come first serve; static priority (control group 2), manually set the priority of the target detection task to be the highest.

[0086] 4. Record the average queue waiting time, detection frame rate per second, missed detection rate and device utilization rate data of the experimental group, control group 1 and control group 2 during the execution of the YOLOv5 target detection task, and organize the recorded data into a table, the specific results are shown in the following table.

[0087] Table 2: YOLOv5 target detection task data comparison record table

[0088] Indicators / Items Experimental Group Control Group 1 Control Group 2 Average queue waiting time 11.8ms 24.1ms 18.5ms Detection frame rate per second (FPS) 31.5 25.1 28.3 Missed detection rate (small target) 4.5％ 7.3％ 5.9％ Device utilization rate 88.9％ 70.8％ 78.1％

[0089] From the above table, it can be seen that when executing the OLOv5 target detection task, the data processing method proposed in the application reduces the queue waiting time by 51%, increases the detection frame rate by 25%, reduces the small target missed detection rate by 38%, and the device utilization rate is close to the theoretical upper limit.

[0090] Example 3

[0091] Low-power microcontroller speech recognition

[0092] Scene description

[0093] Deploy a lightweight speech recognition model on a dual-core microcontroller (168MHz), and compare the method of the application with traditional energy optimization strategies.

[0094] Implementation steps

[0095] 1. Hardware configuration: microcontroller, memory, data set and framework are common configurations available on the market.

[0096] 2. Method of the application (experimental group): dynamic precision switching, when the temperature is greater than or equal to 65℃, automatically switch from FP32→INT8, on-chip neural network prediction precision loss (based on 5000 speech data training), federated learning, local device training model (50 batch update period), global server parameter aggregation.

[0097] 3. Comparison method: fixed INT8 (control group 1), forced to use INT8 inference; no federated learning (control group 2), centralized training (need to upload original speech data).

[0098] 4. Record the recognition accuracy, single instruction inference energy consumption, privacy leakage risk and model convergence time data of the experimental group, control group 1 and control group 2 during the execution of the lightweight speech recognition model task, and organize the recorded data into a table, the specific results are shown in the following table.

[0099] Table 3: Lightweight speech recognition model task data comparison record table

[0100] Indicators / Items Experimental Group Control Group 1 Control Group 2 Recognition accuracy rate 90.1％ 85.5％ 91.2％ Single instruction inference energy consumption 12.5μJ 10.3μJ 15.6μJ Privacy leakage risk (differential privacy) ≤0.1％ - 100％ Model convergence time 15 minutes - 25 minutes

[0101] From the above table, it can be seen that when the data processing method proposed in the application is used to execute a lightweight speech recognition model task, the energy consumption is only increased by 21% compared with the fixed INT8 mode, but the accuracy is increased by 5.4%, the federal learning makes the privacy disclosure risk close to zero, and the training time is reduced by 40%.

[0102] The above describes only the preferred specific embodiments of the application, but the protection scope of the application is not limited thereto, any person skilled in the art can make equivalent replacements or changes within the technical range disclosed by the application according to the technical solution and the inventive concept of the application, which should be covered within the protection scope of the application.

Claims

1. A data processing method based on an artificial intelligence chip, characterized in that, Comprising the following steps: S1, real-time monitoring of the complexity parameters of the input data stream, including data dimension D, feature sparsity S and calculation intensity C, the calculation formula of the feature sparsity S is Wherein, N zeros represents the number of zero elements in the feature vector, N total represents the total number of elements of the feature vector; the calculation formula of the calculation intensity C is Wherein, N ops represents the number of operations required to complete the calculation, N data represents the amount of data participating in the calculation; S2, dynamically adjusting the computing power allocation ratio of each computing unit in the chip by using a reinforcement learning algorithm, the reinforcement learning algorithm adopts a proximal policy optimization, and the proximal policy optimization comprises the following steps: defining a state space S state ={D, S, C}, defining an action space A action is a set of computing power allocation ratios of each computing unit, the ratio range is 1:3 to 3:1, and a reward function R is defined as R = a x E - b x T, wherein E is an efficiency of computing power allocation, N is calculated by a formula completed is the number of computing tasks completed within a certain time, N total is the total number of computing tasks; T is the task completion time; a and b are weighting coefficients, and a + b = 1. S3, using attention mechanism for spatio-temporal alignment of visual, speech and text multi-modal data, the attention mechanism includes multi-head self-attention layer and cross-modal attention layer, the number of heads h of the multi-head self-attention layer can be dynamically adjusted, ranging from 4 to 16 heads; S4, predicting the task execution time based on a long short-term memory network, the input of the long short-term memory network unit is the historical execution time sequence T of the task history = [t1, t2, …, t n ], and the output is the predicted task execution time S5、When the chip temperature T chip When the chip temperature T exceeds 75℃, automatically switch the calculation precision mode FP32→FP16→INT8, predict the precision loss rate in different modes through the on-chip neural network, the input of the on-chip neural network is the current running parameter of the chip, the output is the precision loss rate ∈ in different calculation precision modes, the training data of the on-chip neural network comes from the chip historical running log, containing 10 6 pairs of energy-precision data under different working conditions; S6, embed the federated learning framework in the data preprocessing stage, adopt the asynchronous update strategy, and train the model on the local data set to obtain the local model parameter θ local The local model parameter update period is 50-200 training batches, and the global server aggregates the local model parameters of multiple local devices.

2. The data processing method based on the artificial intelligence chip according to claim 1, characterized in that, In the step S3, the calculation process of the multi-head self-attention layer is as follows: a. For the input modal data X, calculate the query matrix Q, key matrix K, and value matrix V, respectively: Q = XW Q K = XW K V = XW V W Q W K and W V The weight matrix is ​​a learnable matrix; b. Calculating attention scores where d k is the dimension of the query matrix Q; C. The output of the multi-head self-attention layer is: MultiHead(Q, K, V) = Concat(head1,..., head h )W O , where head i = Attention(QW Qi ,KW Ki ,VW Vi ), W Qi , W Ki , W Vi and W O are learnable weight matrices; d. The cross-modal attention layer is used to fuse the features of different modalities, by taking the features of one modality as the query, the features of another modality as the key and value, calculating the attention score and updating the feature representation; e. Finally, a fusion feature vector with a dimension of 2048 is generated. 3.The data processing method based on the artificial intelligence chip according to claim 1, characterized in that, In the step S4, the calculation formula of the long short-term memory network unit is as follows: input gate i t = σ(W ii x t + W hi h t-1 + b i ) forget gate f t = σ(w if x t + w hf h t-1 + b f ) Cell status update Cell state Output gate o t = σ(W io x t + W ho h t-1 + b o ) Hidden state h t = o t ⊙ tanh(C t ) Predicted task execution time According to the predicted task execution time, the waiting time of the task queue is optimized to ≤15ms, while keeping the chip utilization rate ≥85%, the task scheduling strategy adopts priority scheduling algorithm, and the priority of the task is determined according to the predicted execution time and the urgency of the task.

4. The data processing method based on the artificial intelligence chip according to claim 1, characterized in that, In the step S6, the local model parameter aggregation formula is where N is the number of local devices participating in federated learning, and the local device updates the local model according to the global model parameters.

5. The data processing method based on the artificial intelligence chip according to claim 1, characterized in that, In the step S3, the learnable weight matrix of the multi-head self-attention layer and the cross-modal attention layer is trained by the back propagation algorithm, and the optimization target is to minimize the loss function of multi-modal data fusion, and the loss function adopts cross entropy loss function.

6. The data processing method based on the artificial intelligence chip according to claim 1, characterized in that, In the step S4, the training of the long short-term memory network adopts the stochastic gradient descent algorithm, the learning rate is 0.001-0.01, and the training rounds are 100-500 rounds.

7. The data processing method based on the artificial intelligence chip according to claim 1, characterized in that, In the step S5, the on-chip neural network adopts convolutional neural network structure, which extracts features and predicts the input chip running parameters through convolutional layer, pooling layer and fully connected layer.

8. The data processing method based on the artificial intelligence chip according to claim 1, characterized in that, In the step S6, in the federated learning framework, the local device performs differential privacy protection processing on the parameters before uploading the local model parameters, and adds Laplace noise.

Images