Vehicle-mounted large model deployment method based on mixed granularity knowledge distillation

Abstract

The invention discloses a vehicle-mounted large model deployment method based on mixed granularity knowledge distillation. The vehicle-mounted large model deployment method comprises the following steps: performing mask initialization on coarse particle and fine particle structures in a pre-trained large model; initializing low-rank matrixes, wherein the matrixes are used for replacing original weight parameters of the model for updating in the distillation training process; setting a model loss function including knowledge distillation loss and regularization loss; in the model training process, the parameters of the pre-training model are kept unchanged, and only the low-rank matrix and the mask parameters are updated; performing model evaluation after training is completed, and checking whether the model after knowledge distillation meets performance requirements or not; if the evaluation result is not satisfied, returning to the training step for further optimization; a low-rank matrix and mask parameters are fixed, and the parameters are fused into an original model according to masks. The model after knowledge distillation can obtain the capacity of a large model in a small size, and is more suitable for being deployed in a vehicle-mounted system with limited resources so as to provide efficient calculation performance.

Description

Technical Field

[0001] The present invention relates to the field of large language models and vehicle-side intelligent knowledge response models, and in particular to a vehicle-mounted large model deployment method based on mixed granularity knowledge distillation. Background Art

[0002] With the continuous development and iteration of large language model technology, more and more fields have begun to embrace and apply this technology to achieve the goal of adding artificial intelligence to production. In the field of vehicle-side, engineers are constantly conducting experiments to try to add large language models or derived multimodal large models to the vehicle-side. For example, the answering AI smart assistants developed by various manufacturers are all based on multimodal models. Moreover, the on-board computer environment is easier to deal with problems such as heat dissipation and power consumption than mobile devices such as mobile phones and computers. Especially in new energy vehicles. However, the resources required for inference of existing large language models are still considerable. Based on bf16, the commonly used 7B parameter large model requires 14GB of GPU memory for inference, which is too expensive for existing car machines. Deploying smaller large models on the vehicle side has become a common solution. Due to the limitations of its own parameters, the basic knowledge and specific domain knowledge learned during pre-training are relatively poor. Knowledge transfer and cross-domain capabilities cannot be compared with large models. Although the vehicle side can accept higher power consumption when using chips with lower process, the current running memory size of the on-board computer and the CPU and GPU are still unable to withstand large models with billions of parameters. Some people have tried to deploy large models in the cloud. However, the processing speed of the cloud model, the transmission time of the communication module, and the response time of local decisions objectively affect the response speed and user experience of the model on the vehicle. The local intelligent assistants and master control models currently used are often small models with smaller parameters and less resources. Knowledge injection for small models came into being.

[0003] If you want to inject knowledge into a large vehicle-side model, you need to first select a suitable vehicle-side knowledge data set and secondly select a suitable knowledge injection method. The knowledge injection method is the most important one, which determines the effect of the model's operation and the user's intuitive experience.

[0004] In this regard, the current common method in the field of large models is to use data to fine-tune all parameters of the model, or to fine-tune the lora matrix. However, due to the limitations of the parameters of the small model, even after fine-tuning all parameters or lora fine-tuning, its performance still cannot meet the needs after deployment on the vehicle side. Another way is to INT4 quantize the original large model. Although this quantization method can greatly reduce the video memory resources consumed during large model inference, it still takes up a lot of resources and is not suitable for existing car platforms. In addition, the corresponding confusion and hallucinations will increase significantly, which will have a great impact on the final experience effect. Therefore, the existing methods of knowledge injection are not suitable for most of the existing car computers equipped with corresponding models.

[0005] However, the knowledge distillation method is very suitable for the existing environment. Distillation uses the answers generated by a large model with many parameters to guide the small model with fewer parameters to perform full-parameter training. This method ultimately results in a small model that is specialized in certain knowledge areas. And answering knowledge-based questions in this field can partially achieve the effect of a large model. This method greatly saves resources and time compared to retraining the existing small model from scratch. And because the final result is a small model with few parameters, it can also be deployed directly on the existing car computer. At the same time, because it is in the bf16 format, it will not significantly increase the confusion and hallucination. It can be seen that the small model obtained based on the distillation method can indeed adapt to existing needs.

[0006] However, traditional distillation methods also have shortcomings when applied to this field. Existing mainstream models often cause a serious decline in general capabilities after injecting knowledge. Summary of the invention

[0007] The present invention provides a method for deploying a large vehicle model based on mixed-granularity knowledge distillation. By coordinating the structural design of the large model and taking the model performance as the final measurement standard, different fusion strategies are performed on the coarse-grained and fine-grained parts in the structure to ensure that the large vehicle model can fuse new knowledge while retaining the originally acquired knowledge.

[0008] A method for deploying a large vehicle model based on mixed granularity knowledge distillation includes the following steps:

[0009] (1) Prepare a pre-trained large model and initialize the mask parameters of its coarse-grained and fine-grained structures. The mask is represented by Z∈{0,1};

[0010] (2) For matrices that need parameter update, two low-rank matrices A and B are initialized respectively, and only the two low-rank matrices are updated during the process of updating the original parameters of the large model;

[0011] (3) Prepare a teacher model and a student model, and generate soft labels by the teacher model;

[0012] (4) Define the comprehensive distillation loss function L cd , including the traditional loss L of the student model and hard labels CE , distillation loss L of soft labels of student model and teacher model KD , and the loss of the intermediate layer features L IF ;

[0013] (5) Set the mixed particle size loss, which includes the original comprehensive distillation loss L cd With regularization loss

[0014] (6) Perform mixed granularity training on the large model. During the training process, the parameters of the pre-trained large model are not updated, and only the parameters of the low-rank matrices A and B and the mask Z are updated;

[0015] (7) Fix the values ​​of the low-rank matrix and the structure mask Z, and determine the fine-grained parameters of the frozen student model based on the learned mask Z;

[0016] (8) After the knowledge distillation training is completed, the model is evaluated. If the model evaluation result does not meet expectations, return to step (6) to continue training;

[0017] (9) Perform model fusion on the trained model, fuse the obtained low-rank matrices A and B and the mask Z into the pre-trained large model to obtain the final compressed large model, and deploy the obtained large model in the vehicle system to achieve efficient human-vehicle interaction.

[0018] The present invention subdivides the model into different granularities and performs mixed-type knowledge distillation, thereby achieving the purpose of retaining the general capabilities of the model while allowing the model to acquire new domain knowledge.

[0019] Before carrying out this process, you need to prepare a pre-trained large model. The performance of this model can meet the final requirements, but due to the limitations of edge device memory and computing power, further compression is required.

[0020] In step (1), each structure of the large model is covered with a mask Z∈{0,1} representing whether the structure is to be knowledge injected, where Z=1 represents that the structure is frozen, and Z=0 represents that the structure is to be knowledge injected;

[0021] The large model is based on the Transformer model. After the embedding layer, it is composed of multiple layers with the same structure stacked together. Each stacked layer contains an attention layer and a feedforward layer. Each attention layer has multiple attention heads, and each feedforward layer has multiple fully connected linear layers. The input dimension is first projected up and then down. The structure that needs to cover the mask includes the hidden dimension mask Z. hidden , attention head mask Zhead , the intermediate dimension mask Z of the feedforward layer inter and a coarser layer mask Z layer , attention layer mask Z att and the feed-forward layer mask Z ffn .

[0022] In addition to the hidden dimension mask Z hidden In addition to being shared by all layers, each structure has its own separate mask representation; masks at different levels overlap to control whether a certain structure is frozen or not.

[0023] In step (2), the original matrix is ​​defined as The corresponding two low-rank matrices are and Then the matrix update is expressed as:

[0024] Wx+VWx=Wx+BAx

[0025] Among them, x is the input value; after adding two low-rank matrices A and B, the updated parameter amount changes from the original d×k to r×d+r×k. Since the rank r of the matrix is ​​much smaller than min(d, k), the number of parameter updates is greatly reduced.

[0026] In step (3), the teacher model selects a large-scale pre-trained language model. The present invention selects the currently open source Llama2 13B model; the student model selects a relatively small language model, which has a more compact structure than the teacher model and has lower computing and storage requirements than the teacher model. The present invention selects the currently open source bloom 310m model.

[0027] In step (4), the comprehensive distillation loss function L cd The formula is:

[0028] L cd =αL CE +βL KD +γL IF

[0029] Among them, α, β, and γ are weight coefficients used to balance the contribution of different losses;

[0030] Traditional loss L for student model and hard labels CE for:

[0031]

[0032] In the formula, y i is the true label, i.e., hard label; p i is the predicted probability of the student model;

[0033] Distillation loss L of soft labels of student model and teacher model KD for:

[0034]

[0035] Where P i (T) and P i (S) are the output probability distributions of the teacher model and the student model respectively;

[0036] The loss of the intermediate layer features L IF for:

[0037]

[0038] In the formula, f l (T) and f l (S) are the feature representations of the lth layer of the teacher model and the student model respectively, and L is the total number of layers.

[0039] In step (5), the overall expression of mixed particle size loss is:

[0040]

[0041] Regularization Loss The formula is:

[0042]

[0043] in, is the current sparsity of the model, calculated based on the current mask Z; S is the sparsity of the model target, and λ1 and λ2 are coefficients.

[0044] In step (6), when the large model is trained with mixed granularity, during the back propagation process, the parameters in the pre-trained model are frozen during the training process and are not updated. Only the low-rank matrices A and B and the parameter α of the mask Z are updated.

[0045] And during the training process, the coefficients λ1 and λ2 will continue to increase, forcing the sparsity of the model to be equal to the target sparsity; while the target sparsity S is gradually reduced until the actual target sparsity is reached, so that the model gradually adapts to the reduction in sparsity during training.

[0046] In step (7), the value of the structure mask Z is determined as follows:

[0047] After the model training is completed, the Z value is calculated in one sampling, and the size is sorted according to the obtained Z value, and the threshold is calculated according to the target sparsity; Z below the threshold is truncated to 0, indicating that the structure will be injected with knowledge, and Z above the threshold is truncated to 1, indicating that the structure is frozen.

[0048] In step (9), model fusion is represented by updating the weights of the pre-trained model according to the low-rank matrix. The new weight W′ is obtained by directly adding the two low-rank matrices at different granularities for determining mask distinction, and is expressed as:

[0049] W′=W+(1-Z)·BA

[0050] Where W represents the original matrix that needs parameter update in the large model.

[0051] Compared with the prior art, the present invention has the following beneficial effects:

[0052] 1. The present invention is a solution strategy for the knowledge distillation of large vehicle-mounted models, which actually reduces the number of model parameters and the amount of calculation, making the model easier to deploy in the edge device on the vehicle side. At the same time, it accelerates the model reasoning speed and reduces the risks caused by model reasoning delays during driving.

[0053] 2. The mixed-granularity knowledge distillation strategy proposed in the present invention can inject knowledge more accurately, avoid knowledge conflicts, and enable small models to acquire the capabilities of general large models while retaining their original capabilities.

[0054] 3. The present invention utilizes a low-rank matrix to replace the parameter update in the pre-trained model, which greatly reduces the amount of parameters that need to be updated during the model distillation process.

[0055] 4. The present invention uses a regularization method to train the model, which can achieve a balance between the model's distillation ability and general ability. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 This is a flow chart of a method for deploying a large vehicle model based on mixed granularity knowledge distillation according to the present invention;

[0057] Figure 2 Schematic diagram of the structure that a large model needs to cover a mask in an embodiment of the present invention. DETAILED DESCRIPTION

[0058] The present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be pointed out that the embodiments described below are intended to facilitate the understanding of the present invention and do not have any limiting effect on the present invention.

[0059] like Figure 1As shown, a method for deploying a large vehicle model based on mixed granularity knowledge distillation includes the following steps:

[0060] (1) Prepare a pre-trained large model and initialize the mask parameters of its coarse-grained and fine-grained structures. The mask is represented by Z∈{0,1}.

[0061] Before carrying out this process, you need to prepare a pre-trained large model. The performance of this model can meet the final requirements, but due to the limitations of edge device memory and computing power, further compression is required.

[0062] Each structure of the large model will be covered with a mask Z∈{0,1}, which indicates that the structure is for knowledge injection. When Z is 1, it means that the structure is frozen, and when Z is 0, it means that the structure is for knowledge injection. Considering the structure of the large language model constructed based on the Transformer model, the model is composed of multiple layers of the same structure after the embedding layer. Among these layers, each layer contains an attention layer and a feedforward layer. One of the attention layers has multiple attention heads, and the feedforward layer is composed of multiple fully connected linear layers, which will first project the input dimension up and then down. For example Figure 2 As shown, the specific structure of the mask that needs to be covered includes the hidden dimension mask Z hidden , attention head mask Z head , the intermediate dimension mask Z of the feedforward layer inter , and a coarser-grained layer mask Z layer , attention layer mask Z att and the feed-forward layer mask Z ffn These masks, in addition to the hidden dimension mask Z hidden Each structure has its own mask representation, except for the shared mask of each layer. However, masks of different layers overlap to control whether a certain structure is frozen or not. For example, if the Z layer is 0, indicating that knowledge is injected into the layer. Therefore, Z hidden and Z inter The value of has no effect.

[0063] Although such a representation is convenient for knowledge distillation, back propagation optimization is required during model training. Discrete values ​​such as Z cannot be gradient optimized. Therefore, the L0 reparameterization method is used to represent Z, and the following is obtained:

[0064] u~U(0,1)

[0065]

[0066]

[0067]

[0068] Among them, U(0, 1) is a uniform distribution between the interval (0, 1), r, l and β are fixed hyperparameters, and α is the main parameter that needs to be learned during training. During the training process, α and u jointly determine the value of the mask Z. By training the value of α, the value of the mask Z can be indirectly trained.

[0069] (2) For matrices that require parameter updating, two low-rank matrices A and B are initialized respectively. In the process of updating the original parameters of the large model, only the two low-rank matrices are updated.

[0070] For each matrix that needs parameter update, two low-rank matrices A and B are initialized respectively, and A and B are used to update the original matrix weights. That is, the original matrix is The corresponding two low-rank matrices are and Then the matrix update can be expressed as:

[0071]

[0072] Where x is the input value.

[0073] After adding two low-rank matrices A and B, the amount of updated parameters changes from the original d×k to r×d+r×k. Since the rank r of the matrix is ​​much smaller than min(d, k), the number of parameter updates can be greatly reduced.

[0074] (3) Prepare the teacher model and student model, and generate soft labels by the teacher model.

[0075] The present invention selects a large-scale pre-trained language model with high precision and rich knowledge representation capabilities as the teacher model. The model is pre-trained on a large data set and performs well in multiple downstream tasks. The student model selects a relatively small language model with a more compact structure and lower computing and storage requirements. In this step, the soft labels generated by the teacher model are collected in the training set through the teacher model.

[0076] (4) Define the comprehensive distillation loss function L cd , including the traditional loss L of the student model and hard labels CE , distillation loss L of soft labels of student model and teacher model KD , and the loss of the intermediate layer features L IF .

[0077] Traditional loss L for student model and hard labels CE It can be expressed as:

[0078]

[0079] In the formula, y iis the true label, i.e., hard label; p i is the predicted probability of the student model;

[0080] In order to learn the internal knowledge of the teacher model, the distillation loss L between the student model and the soft label of the teacher model is KD for:

[0081]

[0082] Where P i (T) and P i (S) are the output probability distributions of the teacher model and the student model respectively;

[0083] In order to learn the intermediate layer features of the teacher model, so that the student model not only focuses on the final output, but also captures the knowledge representation of the intermediate layer, the loss L of the intermediate layer features IF for:

[0084]

[0085] In the formula, f l (T) and f l (S) are the feature representations of the lth layer of the teacher model and the student model respectively, and L is the total number of layers.

[0086] Finally, in order to comprehensively consider the task loss and the loss in distillation learning, the comprehensive distillation loss function is defined as:

[0087] L cd =αL CE +βL KD +γL IF

[0088] Among them, α, β, and γ are weight coefficients used to balance the contribution of different losses.

[0089] (5) Set the mixed granularity loss, which includes the original comprehensive distillation loss and regularization loss

[0090] The design focus of the regularization loss is to optimize each structural mask. When the mask is not 0, it means that the structure corresponding to the current mask is frozen, which is contrary to the idea of ​​knowledge distillation and will be punished in the loss accordingly. Placing this regularization loss in the overall loss calculation can promote the balance between general ability retention and knowledge distillation effect during model training. However, in order to better control the degree of knowledge distillation of the model, it is not chosen to directly add the mask Z to the loss for calculation, but some conversions are performed.

[0091] Specifically, the regularization loss can be expressed as:

[0092]

[0093] in, is the current sparsity of the model, S is the sparsity of the model target, and λ1 and λ2 are coefficients.

[0094] and It can be calculated based on the current mask Z. After the model sets the mask parameters, the current sparsity of the model can be determined during the forward propagation of the model. It can be expressed as:

[0095]

[0096] Where L is the number of layers, d is the size of the model hidden dimension, and N h is the number of attention heads in each layer, d h is the output dimension size of the attention head, d f is the intermediate dimension size of the feedforward layer, and M is the size of the original model.

[0097] The overall expression of the final loss is:

[0098]

[0099] For the final loss target, find the maximum minimization That is, the goal of the training process, so it can ensure that the sparsity of the final distillation of the model is equal to the target distillation degree.

[0100] (6) The large model is trained with mixed granularity. During the training process, the parameters of the pre-trained large model are not updated, only the parameters of the low-rank matrices A and B and the mask Z are updated.

[0101] In each forward propagation process, although the value of Z will be updated according to the results of random distribution sampling, the parameter that mainly controls the value of Z is α, so α is also the main parameter that needs to be learned. Finally, the output h of the model forward propagation process is determined by the pre-trained model parameters, model mask and low-rank matrix, which can be expressed as:

[0102] h=Z hea / inter ·(Wx+BAx)·Z hidde

[0103] During the back-propagation process, the parameters in the pre-trained model are frozen during the training process and are not updated. Only the low-rank matrices A and B and the parameter α of the mask Z are updated.

[0104] And in the actual training process, the coefficients λ1 and λ2 will continue to increase, thus forcing the sparsity of the model to be equal to the target sparsity. The target sparsity S is gradually reduced until the actual target sparsity is reached, so that the model can gradually adapt to the reduction of sparsity during training and achieve better results.

[0105] (7) Fix the values ​​of the low-rank matrix and the structure mask Z, and determine the fine-grained parameters of the frozen student model based on the learned mask Z.

[0106] After the model training is completed, the values ​​of the low-rank matrix and parameter α have been determined, but the value of the mask Z is still uncertain. At this time, it is calculated in one sampling, and the size is sorted according to the obtained Z value, and then the threshold is calculated according to the target sparsity. Z below the threshold is truncated to 0, indicating that the structure will be injected with knowledge, and Z above the threshold is truncated to 1, indicating that the structure is frozen.

[0107] (8) After the knowledge distillation training is completed, the model is evaluated. If the model evaluation result does not meet expectations, return to step (6) to continue training.

[0108] The effect of the model is evaluated based on the fixed mask Z and the low-rank matrices A and B. If the model evaluation result does not meet expectations, the model should continue to be trained. If the model evaluation result is good, the next step can be taken.

[0109] (9) Perform model fusion on the trained model, fuse the obtained low-rank matrices A and B and the mask Z into the pre-trained large model to obtain the final compressed large model, and deploy the obtained large model in the vehicle system to achieve efficient human-vehicle interaction.

[0110] According to the low-rank matrix obtained through training and the determined mask, model fusion can be expressed as updating the weight of the pre-trained model according to the low-rank matrix. The new weight W′ can be obtained by directly adding two low-rank matrices at different granularities distinguished by the determined mask, expressed as:

[0111] W′=W+(1-Z)·BA

[0112] It can be seen from the above that the knowledge injected into the low-rank matrix will only inject information at the position where the mask confirms that it is sparse, and will not cause knowledge conflicts in the original model.

[0113] The large model that has completed the fusion has learned new knowledge and retained its original general capabilities. It is more suitable for deployment in resource-constrained vehicle systems and can perform well in special areas on the vehicle side.

[0114] The embodiments described above provide a detailed description of the technical solutions and beneficial effects of the present invention. It should be understood that the above are only specific embodiments of the present invention and are not intended to limit the present invention. Any modifications, supplements and equivalent substitutions made within the scope of the principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for deploying large vehicle models based on mixed granularity knowledge distillation, characterized in that: The following steps are involved: (1) Prepare a pre-trained large model and initialize the mask parameters of its coarse-grained and fine-grained structures. The mask is represented by Z∈{0,1}; (2) For matrices that need parameter update, two low-rank matrices A and B are initialized respectively, and only the two low-rank matrices are updated during the process of updating the original parameters of the large model; (3) Prepare a teacher model and a student model, and generate soft labels by the teacher model; (4) Define the comprehensive distillation loss function L cd , including the traditional loss L of the student model and hard labels CE , distillation loss L of soft labels of student model and teacher model KD , and the loss of the intermediate layer features L IF ; (5) Set the mixed particle size loss, which includes the original comprehensive distillation loss L cd With regularization loss (6) Perform mixed granularity training on the large model. During the training process, the parameters of the pre-trained large model are not updated, and only the parameters of the low-rank matrices A and B and the mask Z are updated; (7) Fix the values ​​of the low-rank matrix and the structure mask Z, and determine the fine-grained parameters of the frozen student model based on the learned mask Z; (8) After the knowledge distillation training is completed, the model is evaluated. If the model evaluation result does not meet expectations, return to step (6) to continue training; (9) Perform model fusion on the trained model, fuse the obtained low-rank matrices A and B and the mask Z into the pre-trained large model to obtain the final compressed large model, and deploy the obtained large model in the vehicle system to achieve efficient human-vehicle interaction.

2. The method for deploying a large vehicle model based on mixed granularity knowledge distillation according to claim 1 is characterized in that: In step (1), each structure of the large model is covered with a mask Z∈{0,1} representing whether the structure is to be knowledge injected, where Z=1 represents that the structure is frozen, and Z=0 represents that the structure is to be knowledge injected; The large model is based on the Transformer model. After the embedding layer, it is composed of multiple layers with the same structure stacked together. Each stacked layer contains an attention layer and a feedforward layer. Each attention layer has multiple attention heads, and each feedforward layer has multiple fully connected linear layers. The input dimension is first projected up and then down. The structure that needs to cover the mask includes the hidden dimension mask Z. hidden , attention head mask Z head , the intermediate dimension mask Z of the feedforward layer inter and a coarser layer mask Z layer , attention layer mask Z att and the feed-forward layer mask Z ffn .

3. The method for deploying a large vehicle model based on mixed granularity knowledge distillation according to claim 2 is characterized in that: In addition to the hidden dimension mask Z hidden In addition to being shared by all layers, each structure has its own separate mask representation; masks at different levels overlap to control whether a certain structure is frozen or not.

4. The method for deploying a large vehicle model based on mixed granularity knowledge distillation according to claim 1 is characterized in that: In step (2), the original matrix is ​​defined as The corresponding two low-rank matrices are and Then the matrix update is expressed as: Among them, x is the input value; after adding two low-rank matrices A and B, the updated parameter amount changes from the original d×k to r×d+r×k. Since the rank r of the matrix is ​​much smaller than min(d,k), the number of parameter updates is greatly reduced.

5. The method for deploying a large vehicle model based on mixed granularity knowledge distillation according to claim 1 is characterized in that: In step (3), the teacher model selects a large-scale pre-trained language model; the student model selects a relatively small language model, which has a more compact structure than the teacher model and has lower computing and storage requirements than the teacher model.

6. The method for deploying a large vehicle model based on mixed granularity knowledge distillation according to claim 1 is characterized in that: In step (4), the comprehensive distillation loss function L cd The formula is: L cd =αL CE +βL KD +γL IF Among them, α, β, and γ are weight coefficients used to balance the contribution of different losses; Traditional loss L for student model and hard labels CE for: In the formula, y i is the true label, i.e., hard label; p i is the predicted probability of the student model; Distillation loss L of soft labels of student model and teacher model KD for: In the formula, and are the output probability distributions of the teacher model and the student model respectively; The loss of the intermediate layer features L IF for: In the formula, and are the feature representations of the lth layer of the teacher model and the student model respectively, and L is the total number of layers.

7. The method for deploying a large vehicle model based on mixed granularity knowledge distillation according to claim 6 is characterized in that: In step (5), the overall expression of mixed particle size loss is: Regularization Loss The formula is: in, is the current sparsity of the model, calculated based on the current mask Z; S is the sparsity of the model target, and λ1 and λ2 are coefficients.

8. The method for deploying a large vehicle model based on mixed granularity knowledge distillation according to claim 7 is characterized in that: In step (6), when the large model is trained with mixed granularity, during the back propagation process, the parameters in the pre-trained model are frozen during the training process and are not updated. Only the low-rank matrices A and B and the parameter α of the mask Z are updated. Moreover, during the training process, the coefficients λ1 and λ2 will continue to increase, forcing the sparsity of the model to be equal to the target sparsity; while the target sparsity S is gradually reduced until the actual target sparsity is reached, so that the model gradually adapts to the reduction in sparsity during training.

9. The method for deploying a large vehicle model based on mixed granularity knowledge distillation according to claim 1, characterized in that: In step (7), the value of the structure mask Z is determined as follows: After the model training is completed, the Z value is calculated in one sampling, and the size is sorted according to the obtained Z value, and the threshold is calculated according to the target sparsity; Truncating Z below the threshold to 0 means that the structure will be injected with knowledge, and truncating Z above the threshold to 1 means that the structure is frozen.

10. The method for deploying a large vehicle model based on mixed granularity knowledge distillation according to claim 1, characterized in that: In step (9), model fusion is represented by updating the weights of the pre-trained model according to the low-rank matrix. The new weight W′ is obtained by directly adding the two low-rank matrices at different granularities for determining mask distinction, and is expressed as: W′=W+(1-Z)·BA Where W represents the original matrix that needs parameter update in the large model.

Images