Mobile terminal model lightweight method based on artificial intelligence practical training platform

Abstract

The invention provides a mobile terminal model lightweight method based on an artificial intelligence practical training platform, and belongs to the technical field of artificial intelligence model optimization and mobile terminal deployment, and the method comprises the steps: 1, carrying out the weight parameter monitoring of each convolution layer of a target AI voice model, calculating the importance score and parameter sparseness of each layer when a redundant parameter is detected, and carrying out the weight parameter monitoring of each convolution layer; performing pruning feasibility analysis and quantitative feasibility analysis to obtain a target pruning coefficient and a target quantitative coefficient; 2, calculating the dependency weight of the association reasoning task on each layer of the target AI voice model, and obtaining a dependency relationship network; 3, obtaining a final pruning threshold value and a pruning proportion; 4, hierarchical pruning operation is carried out by adopting the final pruning threshold value and the pruning proportion, and a model after pruning is obtained; and 5, generating a lightweight model, and outputting a mobile terminal deployment performance report. And the deployment efficiency and response performance of the AI voice model at the mobile terminal are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of artificial intelligence model optimization and mobile terminal deployment, and in particular to a lightweight method for mobile terminal models based on an artificial intelligence training platform. Background Art

[0002] With the rapid development of artificial intelligence (AI) technology, AI models for speech recognition and semantic understanding are widely used in education, security, finance, and other fields. Demand for deployment on smart mobile devices is particularly growing. To meet the stringent requirements of mobile devices for computing efficiency, memory usage, and power consumption, model lightweighting has become a key research direction in current AI engineering practice.

[0003] In the existing technology, some lightweight methods mainly rely on static structure pruning or unified precision quantization. These methods have achieved certain results in server-side reasoning, but still have obvious limitations in mobile training scenarios. On the one hand, most existing methods fail to combine the hierarchical dependency characteristics of specific AI reasoning tasks for dynamic structural optimization, and the pruning operation is blind, which can easily affect model performance; on the other hand, when faced with AI voice models with complex model layers and dense parameter structures, traditional methods lack task correlation analysis and topology security verification mechanisms, which can easily destroy the integrity of the network structure, leading to reasoning failure or a sharp drop in accuracy. In addition, the current mainstream solutions fail to provide an integrated process from pruning prediction to quantization implementation, and lack the ability to automatically optimize the entire link for the deployment goals of the training platform.

[0004] Therefore, the present invention provides a lightweight method for mobile terminal models based on an artificial intelligence training platform. Summary of the Invention

[0005] This paper provides a lightweight model quantization method for mobile terminals based on an artificial intelligence training platform. This method uses dynamic pruning and quantization to precisely control the model structure and computational accuracy of AI speech models, reduce redundant parameters, and improve model efficiency. By integrating a dependency network, it ensures that pruning operations do not affect model performance. Simultaneously, mixed-precision quantization is used to achieve model lightweighting, effectively improving mobile terminal deployment performance, reducing computing resource consumption, and optimizing the execution efficiency of real-time inference tasks.

[0006] The present invention provides a lightweight method for a mobile terminal model based on an artificial intelligence training platform, comprising:

[0007] Step 1: Monitor the weight parameters of each convolutional layer of the target AI speech model. When redundant parameters are detected, calculate the layer importance score and parameter sparsity, and perform pruning feasibility analysis and quantization feasibility analysis to obtain the target pruning coefficient and target quantization coefficient.

[0008] Step 2: Obtain the associated reasoning task of the target AI speech model, calculate the dependency weight of the associated reasoning task on each layer of the target AI speech model, and obtain a dependency network;

[0009] Step 3: Dynamically adjust the pruning threshold and predict the pruning ratio according to the layer importance score to obtain the final pruning threshold and pruning ratio;

[0010] Step 4: According to the dependency network, a hierarchical pruning operation is performed using the final pruning threshold and pruning ratio to obtain a pruned model;

[0011] Step 5: Combine the pruned model with the target quantization coefficient for mixed precision quantization to generate a lightweight model and output a mobile terminal deployment performance report.

[0012] Preferably, the weight parameters of each convolutional layer of the target AI speech model are monitored. When redundant parameters are detected, the layer importance score and parameter sparsity are calculated, and pruning feasibility analysis and quantization feasibility analysis are performed to obtain the target pruning coefficient and target quantization coefficient, including:

[0013] Monitor the weight parameters of each convolutional layer of the target AI speech model to obtain real-time weight distribution;

[0014] Determining whether the real-time weight distribution meets a preset redundancy determination condition, if so, detecting a redundant parameter, and if not, not detecting;

[0015] When redundant parameters are detected, the L1 norm and zero value ratio of the weight parameters of the convolutional layer within a preset time range are calculated to obtain the layer importance score and parameter sparsity;

[0016] A pruning feasibility analysis and a quantization feasibility analysis are performed according to the layer importance score and the parameter sparsity to obtain a target pruning coefficient and a target quantization coefficient.

[0017] Preferably, when redundant parameters are detected, the L1 norm and zero value ratio of the weight parameters of the convolutional layer within a preset time range are calculated to obtain the layer importance score and parameter sparsity, including:

[0018] For the target convolutional layer, collect the historical parameter values ​​of the corresponding weight parameters in several consecutive training iterations;

[0019] The L1 norm of each channel is calculated separately for the corresponding weight parameters of each target convolution layer. The specific calculation formula is:

[0020]

[0021] Among them, w c,irepresents the i-th weight value of the c-th channel, and K is the total weight of a single channel;

[0022] Count the number of zero-value parameters whose absolute values ​​in the corresponding weight parameters of each target convolutional layer are less than the preset threshold, and calculate the zero-value ratio. The formula is:

[0023]

[0024] The mean of the L1 norm of each channel is used as the importance score of the layer, and the calculation formula is:

[0025]

[0026] Where C is the total number of channels;

[0027] With zero value S zero As parameter sparsity output, combined with the layer importance score I layer Generate a two-dimensional evaluation vector (I layer ,S zero ).

[0028] Preferably, performing pruning feasibility analysis and quantization feasibility analysis based on the layer importance score and parameter sparsity to obtain a target pruning coefficient and a target quantization coefficient includes:

[0029] Calculate the deviation between the layer importance score and a preset pruning baseline as the target pruning coefficient;

[0030] The degree of fit between the parameter sparsity and a preset quantization baseline is calculated as a target quantization coefficient.

[0031] Preferably, obtaining the associated reasoning task of the target AI speech model, calculating the dependency weight of the associated reasoning task on each layer of the target AI speech model, and obtaining a dependency network includes:

[0032] Extract several types of speech inference tasks supported by the target AI speech model and count the call frequencies of each convolutional layer for each type of speech inference task.

[0033] Calculating the inter-layer dependency strength based on the call frequency, and then constructing a directed graph structure with the convolutional layer as the node and the dependency strength as the edge;

[0034] Dependency weights are assigned to the directed graph structure based on a graph propagation algorithm to generate a dependency network.

[0035] Preferably, dynamically adjusting the pruning threshold and predicting the pruning ratio according to the layer importance score to obtain the final pruning threshold and pruning ratio includes:

[0036] Obtain historical pruning effect data and establish an LSTM prediction model with historical pruning effect data as input;

[0037] Input the layer importance score of the current layer into the LSTM prediction model and output the recommended pruning threshold range;

[0038] Constraining and modifying the recommended pruning threshold interval according to the dependency network to obtain a final pruning threshold;

[0039] Based on the distribution of the final pruning thresholds of each layer, the global pruning ratio is calculated.

[0040] Preferably, according to the dependency network, a hierarchical pruning operation is performed using the final pruning threshold and pruning ratio to obtain a pruned model, including:

[0041] Based on the topological sorting result of the dependency network, identifying the terminal convolutional layer in the network that has no downstream dependencies and marking it as a high-redundancy candidate layer, wherein the determination condition of the terminal convolutional layer is: the output tensor of the layer is not used as the input of any other convolutional layer;

[0042] Compare the target pruning coefficient of each high-redundancy candidate layer with the final pruning threshold, and select the candidate layers whose target pruning coefficient is greater than the final pruning threshold to form the first round of pruned layer set;

[0043] Starting from the first round of pruned layers, the information is propagated backwards along the edges of the dependency network to the upstream layers, and pruning feasibility verification is performed on each upstream layer.

[0044] Perform structured pruning on all layers that pass validation to obtain the pruned model.

[0045] Preferably, pruning feasibility verification is performed for each upstream layer, including:

[0046] Calculate the dimensionality matching between the pruned output tensor of each upstream layer and the input tensor of the downstream unpruned layer;

[0047] Verify whether the pruning operation of each upstream layer triggers a break in the network connection topology, and allow pruning if the topology is maintained.

[0048] Preferably, verifying whether the pruning operation of each upstream layer triggers a break in the network connection topology includes:

[0049] Construct the adjacency matrix A∈{0,1} of the network connection topology L×L , where L is the total number of layers, and the matrix element A i,j =1 means the output of layer i is used as the input of layer j;

[0050] Adjacency matrix A after simulated pruning operation' , set the corresponding rows and columns of the pruned layer to zero;

[0051] Based on the matrix A ' Perform reachability analysis to verify whether the following preset conditions exist, where the preset conditions include: the input degree of any unpruned layer becomes zero and there are isolated subgraphs in the network that are not connected to the input and output layers;

[0052] If A ' If the reachability verification is passed and none of the preset conditions are met, the topology is determined to be intact.

[0053] Output the topology integrity verification result. If A ' If the reachability verification fails, pruning the layer to be pruned is prohibited.

[0054] Compared with the prior art, the present invention has the following advantages:

[0055] Through dynamic pruning and quantization methods, the model structure and computational accuracy of the AI ​​voice model are precisely controlled, redundant parameters are reduced, and model efficiency is improved. Combined with the dependency network, pruning operations are ensured to not affect model performance. At the same time, mixed-precision quantization is used to achieve model lightweighting, effectively improving mobile deployment performance, reducing computing resource consumption, and optimizing the execution efficiency of real-time inference tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] In order to more clearly illustrate the technical solutions in the present invention or the prior art, a brief introduction is given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0057] Figure 1 It is a flow chart of a lightweight method for a mobile terminal model based on an artificial intelligence training platform provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0058] To make the objectives, technical solutions, and advantages of the present invention more clear, the technical solutions of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the embodiments described are only some of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0059] Example 1:

[0060] The embodiment of the present invention provides a lightweight method for mobile terminal models based on artificial intelligence training platform, such as Figure 1 Shown, including:

[0061] Step 1: Monitor the weight parameters of each convolutional layer of the target AI speech model. When redundant parameters are detected, calculate the layer importance score and parameter sparsity, and perform pruning feasibility analysis and quantization feasibility analysis to obtain the target pruning coefficient and target quantization coefficient.

[0062] Step 2: Obtain the associated reasoning task of the target AI speech model, calculate the dependency weight of the associated reasoning task on each layer of the target AI speech model, and obtain a dependency network;

[0063] Step 3: Dynamically adjust the pruning threshold and predict the pruning ratio according to the layer importance score to obtain the final pruning threshold and pruning ratio;

[0064] Step 4: According to the dependency network, a hierarchical pruning operation is performed using the final pruning threshold and pruning ratio to obtain a pruned model;

[0065] Step 5: Combine the pruned model with the target quantization coefficient for mixed precision quantization to generate a lightweight model and output a mobile terminal deployment performance report.

[0066] In this embodiment, the convolutional layers of the target AI speech model refer to the convolutional neural network structural units within the target AI speech model used to extract and process features of speech data. For example, in a speech recognition model, the convolutional layer slides the convolution kernel across the spectrogram of the speech signal to extract speech features at different frequency and time scales, such as pitch, timbre, and other characteristic information.

[0067] In this embodiment, redundant parameters refer to weight parameters in the convolutional layers of the target AI speech model that have little impact on the model's final output and whose removal will not significantly reduce model performance. For example, in a trained speech emotion classification model, the connection relationships corresponding to some parameters in certain convolutional layers contribute very little to the emotion classification results, and these parameters can be considered redundant.

[0068] In this embodiment, the layer importance score and parameter sparsity are used. The layer importance score is a numerical indicator that uses a specific algorithm to evaluate the impact of each convolutional layer on the overall performance of the model. The parameter sparsity indicates the proportion of zero-valued parameters in the convolutional layer, reflecting the redundancy of the layer's parameters. For example, in a voice wake-up model, after calculation, a convolutional layer may have a low layer importance score and a high parameter sparsity, indicating that the layer has a lot of redundant components that can be optimized.

[0069] In this embodiment, the mobile deployment performance report summarizes various performance indicators of the lightweight model when running on a mobile device, including but not limited to model inference speed (such as the number of speech frames processed per second), memory usage, energy consumption, accuracy, and recall. For example, after deploying the lightweight speech translation model on a mobile phone, the report will show that the model uses 50MB of memory at runtime, takes 0.3 seconds to infer each 10-second speech segment, and has a translation accuracy of 92%.

[0070] In this embodiment, the pruned model obtained is subjected to mixed-precision quantization processing using quantization strategies of different precisions for parameters of different layers in the model according to the determined target quantization coefficient, so as to reduce model storage and computing resource consumption and generate a lightweight model that meets the requirements of mobile terminal deployment; and the performance of the lightweight model running on the mobile terminal is evaluated to form and output a mobile terminal deployment performance report including indicators such as inference speed, memory usage, energy consumption and accuracy.

[0071] The beneficial effects of this technical solution include: precise control of the AI ​​voice model's structure and computational accuracy through dynamic pruning and quantization, reducing redundant parameters and improving model efficiency. This, combined with a dependency network, ensures that pruning does not impact model performance. Furthermore, mixed-precision quantization enables model lightweighting, effectively improving mobile deployment performance, reducing computing resource consumption, and optimizing the execution efficiency of real-time inference tasks.

[0072] Example 2:

[0073] The embodiment of the present invention provides a lightweight model method for a mobile terminal based on an artificial intelligence training platform. The method monitors the weight parameters of each convolutional layer of the target AI speech model. When redundant parameters are detected, the layer importance score and parameter sparsity are calculated, and pruning feasibility analysis and quantization feasibility analysis are performed to obtain the target pruning coefficient and target quantization coefficient, including:

[0074] Monitor the weight parameters of each convolutional layer of the target AI speech model to obtain real-time weight distribution;

[0075] Determining whether the real-time weight distribution meets a preset redundancy determination condition, if so, detecting a redundant parameter, and if not, not detecting;

[0076] When redundant parameters are detected, the L1 norm and zero value ratio of the weight parameters of the convolutional layer within a preset time range are calculated to obtain the layer importance score and parameter sparsity;

[0077] A pruning feasibility analysis and a quantization feasibility analysis are performed according to the layer importance score and the parameter sparsity to obtain a target pruning coefficient and a target quantization coefficient.

[0078] In this embodiment, weight parameter monitoring refers to the process of continuously collecting and analyzing data on the weight parameters in each convolutional layer of the target AI speech model. For example, during the training of a deep learning-based speech recognition model, a specific algorithm is used to dynamically track the weight parameters of each neuron in the convolutional layer, capturing in real time how these parameters change during training iterations.

[0079] In this embodiment, real-time weight distribution refers to the numerical distribution of the weight parameters of each convolutional layer of the target AI speech model at a specific moment, as obtained during the weight parameter monitoring process. Taking a voice wake-up model as an example, after completing 1000 training iterations, the weight parameters of a certain convolutional layer exhibit a Gaussian distribution centered on 0, with most weight values ​​concentrated in the range of -0.5 to 0.5. This is the real-time weight distribution of the convolutional layer at that moment.

[0080] The beneficial effects of this technical solution include: real-time monitoring of the weight distribution of each convolutional layer in the AI ​​speech model, timely identification of redundant parameters, and accurate assessment of layer importance and parameter sparsity by calculating the L1 norm and zero-value ratio. This method combines pruning and quantization feasibility analysis to obtain appropriate pruning and quantization coefficients, effectively reducing redundant parameters, improving model efficiency, optimizing model structure, adapting to mobile deployment requirements, reducing computing resource consumption, and improving inference speed.

[0081] Example 3:

[0082] The embodiment of the present invention provides a lightweight method for a mobile terminal model based on an artificial intelligence training platform. When redundant parameters are detected, the L1 norm and zero value ratio of the weight parameters of the convolutional layer within a preset time range are calculated to obtain the layer importance score and parameter sparsity, including:

[0083] For the target convolutional layer, collect the historical parameter values ​​of the corresponding weight parameters in several consecutive training iterations;

[0084] The L1 norm of each channel is calculated separately for the corresponding weight parameters of each target convolution layer. The specific calculation formula is:

[0085]

[0086] Among them, w c,i represents the i-th weight value of the c-th channel, and K is the total weight of a single channel;

[0087] Count the number of zero-value parameters whose absolute values ​​in the corresponding weight parameters of each target convolutional layer are less than the preset threshold, and calculate the zero-value ratio. The formula is:

[0088]

[0089] The mean of the L1 norm of each channel is used as the importance score of the layer, and the calculation formula is:

[0090]

[0091] Where C is the total number of channels;

[0092] With zero value S zero As parameter sparsity output, combined with the layer importance score I layer Generate a two-dimensional evaluation vector (I layer ,S zero ).

[0093] In this embodiment, for a specific convolutional layer in the AI ​​speech model, the numerical changes of the weight parameters of this layer in multiple consecutive training iteration steps are continuously tracked during the model training process to obtain its historical parameter values. Specifically, the weight parameters of the target convolutional layer are divided according to channels, and the weight parameters in each channel are analyzed separately.

[0094] In this embodiment, the preset threshold is a preset threshold set by a person to determine whether a weight parameter can be regarded as "approximately zero". When the absolute value of a weight value is less than the threshold, it is regarded as a "zero value parameter".

[0095] The beneficial effect of this technical solution is that by calculating the L1 norm and zero-value ratio of the convolutional layer weight parameters, the layer importance score and parameter sparsity of each convolutional layer can be accurately assessed, effectively identifying redundant parameters. This method can accurately guide pruning operations, optimize model structure, reduce unnecessary computation, while retaining parameters that are critical to model performance, and improve the model's lightweight efficiency, making it particularly suitable for mobile deployment.

[0096] Example 4:

[0097] The embodiment of the present invention provides a lightweight method for a mobile terminal model based on an artificial intelligence training platform, which performs pruning feasibility analysis and quantization feasibility analysis based on the layer importance score and parameter sparsity to obtain a target pruning coefficient and a target quantization coefficient, including:

[0098] Calculate the deviation between the layer importance score and a preset pruning baseline as the target pruning coefficient;

[0099] The degree of fit between the parameter sparsity and a preset quantization baseline is calculated as a target quantization coefficient.

[0100] In this embodiment, the deviation from the preset pruning baseline refers to comparing the layer importance score of a convolutional layer with a preset pruning reference value, and calculating the difference between the two to quantitatively measure whether the layer is suitable for pruning and the degree of pruning. For example, in a speech synthesis model, the preset pruning baseline is set to 0.5, and the layer importance score of a convolutional layer is 0.3. The deviation of the layer importance score from the preset pruning baseline reflects the difference in importance compared to the baseline. The greater the deviation, the less important the layer is in the model and may be more suitable for a larger degree of pruning.

[0101] In this embodiment, the adaptability of the preset quantization baseline is determined by comparing the parameter sparsity of the convolutional layer with a pre-set quantization reference value. By evaluating the degree of fit between the two, the quantization indicator of the quantization strategy to be adopted by the layer is determined. For example, in a speech noise reduction model, the preset quantization baseline is 30%, and the parameter sparsity of a convolutional layer is 40%. This indicates that the parameter sparsity of the layer has a good adaptability to the preset quantization baseline. The higher the adaptability, the more likely it is that the layer is more suitable for low-precision quantization during the quantization process to further compress the model size.

[0102] The beneficial effect of this technical solution is that by calculating the deviation of layer importance scores from the pruning baseline and the fit of parameter sparsity to the quantization baseline, pruning and quantization coefficients can be accurately determined. This method optimizes pruning and quantization strategies while ensuring model performance, reduces redundant parameters, improves model lightweighting, enhances mobile deployment capabilities, reduces computing resource requirements, and improves inference efficiency.

[0103] Example 5:

[0104] An embodiment of the present invention provides a lightweight model method for a mobile terminal based on an artificial intelligence training platform, which obtains an associated reasoning task of the target AI voice model, calculates the dependency weight of the associated reasoning task on each layer of the target AI voice model, and obtains a dependency network, including:

[0105] Extract several types of speech inference tasks supported by the target AI speech model and count the call frequencies of each convolutional layer for each type of speech inference task.

[0106] Calculating the inter-layer dependency strength based on the call frequency, and then constructing a directed graph structure with the convolutional layer as the node and the dependency strength as the edge;

[0107] Dependency weights are assigned to the directed graph structure based on a graph propagation algorithm to generate a dependency network.

[0108] In this embodiment, several types of speech reasoning tasks refer to various different types of speech processing tasks that can be performed by the target AI speech model. In actual applications, these tasks have different functions and purposes. For example, in an intelligent voice assistant system, the target AI speech model may support speech recognition tasks, that is, converting the user's voice into text; speech synthesis tasks, that is, converting text information into natural and fluent speech; and speech sentiment analysis tasks, which are used to judge the emotional state contained in the speech, such as happiness, anger, sadness, etc. In addition, tasks such as voice wake-up tasks (recognizing specific wake-up words to activate the device) and speech translation tasks (translating speech in one language into another language) are also common types of speech reasoning tasks. These different tasks have different degrees of dependence on each layer of the model when processing speech data.

[0109] In this embodiment, the calling frequency refers to the frequency with which each convolutional layer of the target AI speech model is used in the process of performing various speech reasoning tasks. Taking the speech recognition task as an example, in the process of converting speech into text, the first few convolutional layers of the model are mainly responsible for extracting the basic features of the speech signal, such as spectral features, short-time energy, etc. The extraction of these basic features is crucial for the subsequent recognition process, so the first few convolutional layers will be called frequently. Assuming that in a speech recognition task that processes 100 pieces of speech data, a certain convolutional layer is used 80 times, then the calling frequency of this convolutional layer for this task is 80%. Different speech reasoning tasks have different calling frequencies for the same convolutional layer due to their different processing logic and requirements. For example, speech synthesis tasks focus more on the processing of semantics and rhythm, and may call the convolutional layers later in the model more frequently.

[0110] In this embodiment, the inter-layer dependency strength is calculated based on the frequency of calling the convolutional layer by various speech reasoning tasks, and is used as an indicator to measure the closeness of the mutual dependence between different convolutional layers in the model. If two convolutional layers are often called continuously in multiple speech reasoning tasks, that is, the output of the previous convolutional layer is frequently used as the input of the next convolutional layer, then the dependency strength between the two convolutional layers is high. For example, in the two tasks of voice wake-up and speech recognition, convolutional layer A is responsible for preliminarily extracting the basic features of the speech, and convolutional layer B further extracts features related to the wake-up word or recognition keyword based on these basic features. During the execution of these two tasks, the output of convolutional layer A is always used as the input of convolutional layer B. After statistical calculation, the inter-layer dependency strength between them is 0.9 (assuming a 0-1 quantization standard, 1 indicates the highest dependency strength); conversely, if two convolutional layers are rarely called continuously by the same task or multiple tasks, the dependency strength between them is low.

[0111] In this embodiment, constructing a directed graph structure with convolutional layers as nodes and dependency strengths as edges is to abstract the convolutional layers in the target AI speech model as nodes in the graph structure, use the inter-layer dependency strengths as edges connecting the nodes, and determine the direction of the edges according to the direction of data flow between layers, thereby forming a directed graph. For example, for a speech model containing three convolutional layers A, B, and C, the inter-layer dependency strengths are known to be: A to B is 0.8, B to C is 0.7, and A to C is 0.3. When constructing a directed graph, the three convolutional layers A, B, and C are respectively regarded as three nodes, and then an edge with an arrow is drawn from node A to node B, and the weight 0.8 is marked on the edge, indicating that the data flows from layer A to layer B and the dependency strength is 0.8; similarly, an edge is drawn from node B to node C and marked with 0.7, and an edge is drawn from node A to node C and marked with 0.3. In this way, the dependency relationship between the convolutional layers of the model is presented in the form of an intuitive directed graph, which is convenient for subsequent analysis and processing.

[0112] In this embodiment, the dependency weights of the directed graph structure are assigned based on the graph propagation algorithm, and the dependency network is generated by using a graph propagation algorithm (such as the PageRank algorithm, the random walk algorithm, etc.). On the directed graph structure that has been constructed, the dependency weights between each node (convolutional layer) are recalculated and assigned according to the connection relationship between the nodes and the initial weight of the edge (i.e., the inter-layer dependency strength), so that the representation of the dependency relationship is more accurate and comprehensive. For example, using the PageRank algorithm, the dependency weights between each node are iteratively updated considering factors such as the in-degree, out-degree, and edge weights of the nodes in the directed graph. After multiple iterative calculations, a new weight assignment result is obtained. These new weights can more reasonably reflect the actual dependency relationships of each convolutional layer in various speech reasoning tasks, and ultimately form a complete dependency network. This network can provide an important basis for subsequent model pruning, optimization, and other operations. For example, when pruning, the dependency network can be referred to avoid pruning layers that are highly dependent on key tasks.

[0113] The beneficial effects of this technical solution include extracting and counting the frequency of convolutional layer calls in speech inference tasks, accurately calculating the strength of inter-layer dependencies, and constructing a dependency network. Dependency weights are assigned using a graph propagation algorithm, effectively identifying the criticality and task dependencies of each convolutional layer. This ensures that layers critical to inference tasks are retained during model lightweighting, thereby optimizing model performance and efficiency, adapting to mobile deployment requirements, and improving inference speed and computing resource utilization.

[0114] Example 6:

[0115] The embodiment of the present invention provides a lightweight method for a mobile terminal model based on an artificial intelligence training platform, which dynamically adjusts the pruning threshold and predicts the pruning ratio according to the layer importance score to obtain the final pruning threshold and pruning ratio, including:

[0116] Obtain historical pruning effect data and establish an LSTM prediction model with historical pruning effect data as input;

[0117] Input the layer importance score of the current layer into the LSTM prediction model and output the recommended pruning threshold range;

[0118] Constraining and modifying the recommended pruning threshold interval according to the dependency network to obtain a final pruning threshold;

[0119] Based on the distribution of the final pruning thresholds of each layer, the global pruning ratio is calculated.

[0120] In this embodiment, establishing an LSTM prediction model with historical pruning effect data as input refers to using a deep learning model called a long short-term memory network (LSTM) to use various types of data accumulated in the past during the model pruning process, such as the change in model accuracy, the degree of improvement in inference speed, and the amount of parameter reduction after specific pruning operations on different layers, as training data to train a model that can predict the appropriate pruning threshold based on the input information. For example, when pruning multiple speech recognition models in the past, the layer importance score of each layer, the pruning threshold used, and the accuracy and running time of the pruned model on the test set were recorded during each pruning. This data was used as training samples, and the LSTM network was used to learn the patterns therein, thereby constructing the prediction model.

[0121] In this embodiment, the layer importance score of the current layer is input into the LSTM prediction model, and outputting the recommended pruning threshold interval means that before performing a pruning operation on a layer of the target AI speech model, the layer importance score currently calculated for the layer is used as input data and passed into the trained LSTM prediction model. After calculation and reasoning, the model outputs a reasonable pruning threshold range. For example, for a certain convolutional layer in the speech synthesis model, its layer importance score is 0.4. After inputting this value into the LSTM prediction model, the model outputs a recommended pruning threshold interval of [0.2, 0.6]. This means that selecting a pruning threshold within this interval may achieve better parameter compression while ensuring model performance.

[0122] In this embodiment, the recommended pruning threshold interval is constrained and modified based on the dependency network. The final pruning threshold represents the dependency network constructed in the previous steps. The interdependencies between layers are taken into account to adjust and optimize the recommended pruning threshold interval output by the LSTM prediction model. For example, if the dependency network shows that a layer has a strong dependency with other key layers, even if the recommended pruning threshold interval given by the LSTM prediction model is large, in order to not affect the performance of the overall model, this interval needs to be narrowed, and a more conservative pruning threshold is selected as the final pruning threshold to ensure that the dependencies between model layers are not destroyed during the pruning process.

[0123] In this embodiment, based on the distribution of the final pruning thresholds of each layer, calculating the global pruning ratio means that after determining the final pruning thresholds of each layer of the target AI speech model, the proportion of parameters expected to be pruned from the entire model is calculated according to the number of parameters in each layer and the corresponding final pruning threshold. For example, the model has 10 convolutional layers, and the number of parameters and the final pruning threshold of each layer are known. By calculating the number of parameters expected to be pruned from each layer, accumulating and dividing by the total number of parameters in the model, a percentage value is obtained. This value is the global pruning ratio, which is used to measure the overall compression degree of the model.

[0124] The beneficial effect of this technical solution is that it dynamically adjusts the pruning threshold by combining historical pruning performance data with the LSTM prediction model, and corrects the recommended threshold based on the dependency network, ensuring accurate and effective pruning operations. This method automatically predicts the pruning threshold and calculates the global pruning ratio, improving the intelligence of the pruning strategy. While ensuring model performance, it also optimizes the model's lightweightness, improving the efficiency of mobile deployment and inference speed.

[0125] Example 7:

[0126] An embodiment of the present invention provides a lightweight method for a mobile terminal model based on an artificial intelligence training platform, which performs a hierarchical pruning operation based on the dependency network using the final pruning threshold and pruning ratio to obtain a pruned model, including:

[0127] Based on the topological sorting result of the dependency network, identifying the terminal convolutional layer in the network that has no downstream dependencies and marking it as a high-redundancy candidate layer, wherein the determination condition of the terminal convolutional layer is: the output tensor of the layer is not used as the input of any other convolutional layer;

[0128] Compare the target pruning coefficient of each high-redundancy candidate layer with the final pruning threshold, and select the candidate layers whose target pruning coefficient is greater than the final pruning threshold to form the first round of pruned layer set;

[0129] Starting from the first round of pruned layers, the information is propagated backwards along the edges of the dependency network to the upstream layers, and pruning feasibility verification is performed on each upstream layer.

[0130] Perform structured pruning on all layers that pass validation to obtain the pruned model.

[0131] In this embodiment, before obtaining the pruned model, the method further includes: calculating the ratio of the number of zeroed parameters in the pruned model to the total number of original parameters, and outputting the ratio as the actual ratio of removed parameters to the performance report.

[0132] The beneficial effects of this technical solution include: by combining topological sorting of the dependency network, accurately identifying highly redundant candidate layers and performing layer-by-layer pruning based on the target pruning coefficient and final pruning threshold. Backpropagation pruning ensures the effectiveness of the pruning strategy and minimizes redundant parameters while retaining key layers, optimizing the model structure, improving lightweightness, and enhancing the performance and computational efficiency of mobile deployment.

[0133] Example 8:

[0134] The embodiment of the present invention provides a lightweight method for mobile terminal models based on an artificial intelligence training platform, which performs pruning feasibility verification for each upstream layer, including:

[0135] Calculate the dimensionality matching between the pruned output tensor of each upstream layer and the input tensor of the downstream unpruned layer;

[0136] Verify whether the pruning operation of each upstream layer triggers a break in the network connection topology, and allow pruning if the topology is maintained.

[0137] In this embodiment, the dimensional matching degree between the output tensor of each upstream layer after pruning and the input tensor of the downstream unpruned layer is calculated: in the deep learning model, tensors are the carriers of data, and information is transmitted between layers through tensors. After the upstream layer is pruned, the shape (dimensional size) of its output tensor may change. This operation is to calculate the degree of fit between the dimension of the output tensor of the upstream layer after pruning and the input tensor dimension required by the downstream unpruned layer. For example, in a speech recognition model, the original output tensor dimension of the upstream convolutional layer is [batch_size, 16, 32, 64], which becomes [batch_size, 8, 32, 64] after pruning, and the downstream fully connected layer requires the input tensor dimension to be [batch_size, 8, 32, 64]. At this time, the dimensions of the two are exactly the same, and the dimensional matching degree is 100%; if the downstream fully connected layer requires the input dimension to be [batch_size, 16, 32, 64], the matching degree is low, which may cause the data to not be transmitted correctly.

[0138] In this embodiment, it is verified whether the pruning operation of each upstream layer triggers a break in the network connection topology. If the topology structure is maintained, pruning is allowed: the network connection topology describes the connection relationship between the layers in the model. This operation is to check whether the pruning of the upstream layer will destroy the original network connection structure, resulting in the inability of data to flow between the layers according to the established path. For example, in a speech synthesis model, an upstream layer passes data to multiple downstream layers through a specific connection method. If the upstream layer is pruned and the connection with one of the downstream layers is cut off, a break in the network connection topology is triggered. Pruning of the upstream layer is allowed only when the pruning operation does not destroy the connection relationship between the layers and maintains the integrity of the network topology structure.

[0139] In this embodiment, for each upstream layer of the target AI speech model, after performing the pruning operation, the degree of fit between the dimension of the output tensor of the upstream layer and the dimension of the input tensor of the downstream unpruned layer is calculated to obtain the dimension matching degree; at the same time, it is checked whether the pruning operation of the upstream layer will destroy the original network connection topology of the model, that is, whether the data transmission path between the layers is cut off. If the network connection topology is not broken after pruning and the data can be maintained to flow between the layers according to the established path, the pruning operation of the upstream layer is determined to be feasible and pruning is allowed; otherwise, pruning is not allowed.

[0140] The beneficial effect of this technical solution is that by constructing an adjacency matrix of the network connection topology and simulating the impact of pruning, it can accurately verify whether pruning operations cause network topology disruptions. This method uses reachability analysis to ensure that pruning operations do not disrupt the connectivity of the network structure, thereby avoiding computational errors or performance degradation caused by pruning. This ensures the reliability and integrity of the model pruning process, optimizes lightweighting effects, and improves the stability and efficiency of mobile models.

[0141] Example 9:

[0142] An embodiment of the present invention provides a lightweight method for a mobile terminal model based on an artificial intelligence training platform, which verifies whether the pruning operation of each upstream layer triggers a break in the network connection topology, including:

[0143] Construct the adjacency matrix A∈{0,1} of the network connection topology L×L , where L is the total number of layers, and the matrix element A i,j =1 means the output of layer i is used as the input of layer j;

[0144] Adjacency matrix A after simulated pruning operation ' , set the corresponding rows and columns of the pruned layer to zero;

[0145] Based on the matrix A 'Perform reachability analysis to verify whether the following preset conditions exist, where the preset conditions include: the input degree of any unpruned layer becomes zero and there are isolated subgraphs in the network that are not connected to the input and output layers;

[0146] If A ' If the reachability verification is passed and none of the preset conditions are met, the topology is determined to be intact.

[0147] Output the topology integrity verification result. If A ' If the reachability verification fails, pruning the layer to be pruned is prohibited.

[0148] In this embodiment, the adjacency matrix of the network connection topology is constructed (with layers as nodes): the topology of the network can be represented by the adjacency matrix. Assuming that the model has L layers, the adjacency matrix is ​​an L×L square matrix, and the element A in the i-th row and j-th column of the matrix is ij Indicates whether the output of layer i is used as the input of layer j: If so, then A ij =1; otherwise 0. For example, in a speech model with 3 layers, if layer 1 is connected to layers 2 and 3, and layer 2 is connected to layer 3, then the adjacency matrix is:

[0149]

[0150] This clearly expresses the connection relationship between the layers.

[0151] In this embodiment, the adjacency matrix after pruning is simulated: when a layer is pruned (i.e., the layer and its connections are removed), the rows and columns corresponding to the layer in the adjacency matrix can be set to zero to simulate the network structure after it is "removed". For example, if the second layer in the above matrix is ​​pruned, that is, the second row and the second column are set to zero, the result is:

[0152]

[0153] This means that the second layer is no longer involved in any input and output connections, simulating the changes in network connections after pruning.

[0154] The beneficial effect of this technical solution is that by constructing an adjacency matrix of the network connection topology and simulating the impact of pruning, it can accurately verify whether pruning operations cause network topology disruptions. This method uses reachability analysis to ensure that pruning operations do not disrupt the connectivity of the network structure, thereby avoiding computational errors or performance degradation caused by pruning. This ensures the reliability and integrity of the model pruning process, optimizes lightweighting effects, and improves the stability and efficiency of mobile models.

[0155] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A lightweight method for mobile terminal models based on artificial intelligence training platform, characterized by: include: Step 1: Monitor the weight parameters of each convolutional layer of the target AI speech model. When redundant parameters are detected, calculate the layer importance score and parameter sparsity, and perform pruning feasibility analysis and quantization feasibility analysis to obtain the target pruning coefficient and target quantization coefficient. Step 2: Obtain the associated reasoning task of the target AI speech model, calculate the dependency weight of the associated reasoning task on each layer of the target AI speech model, and obtain a dependency network; Step 3: Dynamically adjust the pruning threshold and predict the pruning ratio according to the layer importance score to obtain the final pruning threshold and pruning ratio; Step 4: According to the dependency network, a hierarchical pruning operation is performed using the final pruning threshold and pruning ratio to obtain a pruned model; Step 5: Combine the pruned model with the target quantization coefficient for mixed precision quantization to generate a lightweight model and output a mobile terminal deployment performance report.

2. The lightweight method for mobile terminal model based on artificial intelligence training platform according to claim 1 is characterized in that: Monitor the weight parameters of each convolutional layer of the target AI speech model. When redundant parameters are detected, calculate the layer importance score and parameter sparsity, and perform pruning feasibility analysis and quantization feasibility analysis to obtain the target pruning coefficient and target quantization coefficient, including: Monitor the weight parameters of each convolutional layer of the target AI speech model to obtain real-time weight distribution; Determining whether the real-time weight distribution meets a preset redundancy determination condition, if so, detecting a redundant parameter, and if not, not detecting; When redundant parameters are detected, the L1 norm and zero value ratio of the weight parameters of the convolutional layer within a preset time range are calculated to obtain the layer importance score and parameter sparsity; A pruning feasibility analysis and a quantization feasibility analysis are performed according to the layer importance score and the parameter sparsity to obtain a target pruning coefficient and a target quantization coefficient.

3. The lightweight method for mobile terminal model based on artificial intelligence training platform according to claim 2 is characterized in that: When redundant parameters are detected, the L1 norm and zero value ratio of the weight parameters of the convolution layer within a preset time range are calculated to obtain the layer importance score and parameter sparsity, including: For the target convolutional layer, collect the historical parameter values ​​of the corresponding weight parameters in several consecutive training iterations; The L1 norm of each channel is calculated separately for the corresponding weight parameters of each target convolution layer. The specific calculation formula is: Among them, w c,i represents the i-th weight value of the c-th channel, and K is the total weight of a single channel; Count the number of zero-value parameters whose absolute values ​​in the corresponding weight parameters of each target convolutional layer are less than the preset threshold, and calculate the zero-value ratio. The formula is: The mean of the L1 norm of each channel is used as the importance score of the layer, and the calculation formula is: Where C is the total number of channels; With zero value S zero As parameter sparsity output, combined with the layer importance score I layer Generate a two-dimensional evaluation vector (I layer ,S zero ).

4. The lightweight method for mobile terminal model based on artificial intelligence training platform according to claim 2 is characterized in that: Performing pruning feasibility analysis and quantization feasibility analysis based on the layer importance score and parameter sparsity to obtain target pruning coefficients and target quantization coefficients, including: Calculate the deviation between the layer importance score and a preset pruning baseline as the target pruning coefficient; The degree of fit between the parameter sparsity and a preset quantization baseline is calculated as a target quantization coefficient.

5. The lightweight method for mobile terminal model based on artificial intelligence training platform according to claim 1 is characterized in that: Obtaining an associated reasoning task for the target AI speech model, calculating the dependency weights of the associated reasoning task on each layer of the target AI speech model, and obtaining a dependency network, including: Extract several types of speech inference tasks supported by the target AI speech model and count the call frequencies of each convolutional layer for each type of speech inference task. Calculating the inter-layer dependency strength based on the call frequency, and then constructing a directed graph structure with the convolutional layer as the node and the dependency strength as the edge; Dependency weights are assigned to the directed graph structure based on a graph propagation algorithm to generate a dependency network.

6. The lightweight method for mobile terminal model based on artificial intelligence training platform according to claim 1 is characterized in that: Dynamically adjust the pruning threshold and predict the pruning ratio based on the layer importance score to obtain the final pruning threshold and pruning ratio, including: Obtain historical pruning effect data and establish an LSTM prediction model with historical pruning effect data as input; Input the layer importance score of the current layer into the LSTM prediction model and output the recommended pruning threshold range; Constraining and modifying the recommended pruning threshold interval according to the dependency network to obtain a final pruning threshold; Based on the distribution of the final pruning thresholds of each layer, the global pruning ratio is calculated.

7. The lightweight method for mobile terminal model based on artificial intelligence training platform according to claim 1 is characterized in that: According to the dependency network, a hierarchical pruning operation is performed using the final pruning threshold and pruning ratio to obtain a pruned model, including: Based on the topological sorting result of the dependency network, identifying the terminal convolutional layer in the network that has no downstream dependencies and marking it as a high-redundancy candidate layer, wherein the determination condition of the terminal convolutional layer is: the output tensor of the layer is not used as the input of any other convolutional layer; Compare the target pruning coefficient of each high-redundancy candidate layer with the final pruning threshold, and select the candidate layers whose target pruning coefficient is greater than the final pruning threshold to form the first round of pruned layer set; Starting from the first round of pruned layers, the information is propagated backwards along the edges of the dependency network to the upstream layers, and pruning feasibility verification is performed on each upstream layer. Perform structured pruning on all layers that pass validation to obtain the pruned model.

8. The lightweight method for mobile terminal model based on artificial intelligence training platform according to claim 7 is characterized in that: Perform pruning feasibility verification for each upstream layer, including: Calculate the dimensionality matching between the pruned output tensor of each upstream layer and the input tensor of the downstream unpruned layer; Verify whether the pruning operation of each upstream layer triggers a break in the network connection topology, and allow pruning if the topology is maintained.

9. The lightweight method for mobile terminal model based on artificial intelligence training platform according to claim 8 is characterized in that: Verify whether the pruning operation of each upstream layer triggers a break in the network connection topology, including: Construct the adjacency matrix A∈{0,1} of the network connection topology L×L , where L is the total number of layers, and the matrix element A i,j =1 means the output of layer i is used as the input of layer j; Adjacency matrix A after simulated pruning operation ' , set the corresponding rows and columns of the pruned layer to zero; Based on the matrix A ' Perform reachability analysis to verify whether the following preset conditions exist, where the preset conditions include: the input degree of any unpruned layer becomes zero and there are isolated subgraphs in the network that are not connected to the input and output layers; If A ' If the reachability verification is passed and none of the preset conditions are met, the topology is determined to be intact. Output the topology integrity verification result. If A ' If the reachability verification fails, pruning the layer to be pruned is prohibited.

Images