A multi-modal data quality evaluation method of distillation algorithm

Abstract

The application discloses a kind of distillation algorithm multi-modal data quality evaluation method.The present application is by including data quality adjustment coefficient in model training, the sensitivity of model degree data quality, data compliance is strengthened, the training efficiency of model is improved, the professionalism of model is enhanced;The training results of student model are fed back to teacher model through knowledge reflux, so that the teacher model can be better optimized, and the new student model is distilled through the optimized teacher model, forming a positive cycle mechanism of coevolution, so that the teacher model and student model can be continuously optimized, and the accuracy of future data quality evaluation is improved;And student model is obtained by distillation, and is actually deployed to customer data end for data quality evaluation, which consumes limited resources, and as localized deployment, data does not need to be transmitted through the Internet, solving the problem of excessive data transmission and low data quality evaluation efficiency.

Description

TECHNICAL FIELD

[0001] The present application relates to a multi-modal data quality evaluation method of distillation algorithm. BACKGROUND

[0002] In recent years, China has clearly defined the national strategy of developing data elements, and actively promotes data asset evaluation and table entry. Data quality evaluation is the premise of data asset evaluation and table entry. Currently, the main method of data quality evaluation is to pre-set an index system and rules, that is, to calculate the evaluation value by pre-setting the state function of each index or to pre-set the expected evaluation data set according to the data set before evaluation, in order to evaluate the data quality. However, these evaluation methods cannot meet the diversified needs of data quality evaluation in actual evaluation work, and the influence of human factors is difficult to eliminate, resulting in inaccurate evaluation results. Therefore, some evaluation methods begin to try to introduce large model technologies such as adversarial networks or vertical fine-tuning to overcome the above shortcomings, but there are still the following problems in actual operation:

[0003] Training large models requires a large amount of computing resources, but most data sets are core assets of enterprises and institutions, which are physically isolated and stored, and the training resources are limited;

[0004] As core assets, the information transmission between databases and large models is highly sensitive. The introduction of large models requires a large amount of information transmission, which can lead to excessive data transmission and low evaluation efficiency;

[0005] Therefore, it is necessary to invent a multi-modal data quality evaluation method of distillation algorithm to solve the above problems. SUMMARY

[0006] The purpose of the present application is to provide a multi-modal data quality evaluation method of distillation algorithm, which can better optimize the teacher model by returning the training results of the student model to the teacher model, and can distill new student models from the optimized teacher model to form a positive cycle mechanism of co-evolution, thereby continuously optimizing the teacher model and student model, improving the accuracy of future data quality evaluation, and obtaining student models through distillation. The actual deployment to the customer data end is used for data quality evaluation, which consumes limited resources and is deployed locally, without the need to transmit data through the Internet, solving the problems of excessive data transmission and low data quality evaluation efficiency.

[0007] In order to achieve the above purpose, the present application provides the following technical scheme: a multi-modal data quality evaluation method of distillation algorithm, comprising the following steps:

[0008] Step 1, constructing a teacher model M t ;

[0009] Step 2, constructing a student model M sand based on the teacher model M t distilling the student model parameter set φ from the teacher model M

[0010] Step 3, connecting the student model to the customer's database, evaluating the customer data quality, and training the student model parameter set using the customer data to obtain the data quality score and the student model parameter residual;

[0011] Step 4, feeding back the data quality score and student model parameter residual knowledge to the teacher model, optimizing the teacher model, and dividing the data quality score by 100 as the result of this evaluation.

[0012] The aforementioned multi-modal data quality evaluation method of distillation algorithm, in step 1, constructs the teacher model M t , including the following steps:

[0013] 1.1, define the teacher model structure based on the Transformer architecture, including:

[0014] the number of teacher model layers a1;

[0015] the number of teacher model attention heads b1;

[0016] the teacher model hidden layer dimension c1;

[0017] initialize the parameters θ pre to generate the initial teacher model M t , the calculation formula is as follows:

[0018]

[0019] wherein, is the solution of the inner function, and the parameter vector θ that makes the global minimum value is obtained;

[0020] θ is the parameter set of the teacher model;

[0021] i is the index of the current sample, and the specific value is 1-N;

[0022] N is the total number of training samples;

[0023] x ti is the feature vector of the i-th training sample when the teacher model is initialized;

[0024] y ti is the true label of the i-th training sample when the teacher model is initialized;

[0025] P(y ti |x ti ; θ) is the probability of the true label y iThe teacher model predicts the probability that a sample belongs to the true label y under the condition of parameter θ ti ;

[0026] log is the natural logarithm;

[0027] is the forward calculation process of the Transformer architecture-based teacher model under the condition of parameter θ pre ;

[0028] x t is the feature vector set of the training sample when the teacher model is initialized;

[0029] 1.2, fine-tune the teacher model based on data quality awareness, and obtain θ fin , the specific formula is as follows:

[0030]

[0031] where θ fin is the parameter set of the fine-tuned teacher model;

[0032] x' t is the feature vector set of the training sample when the teacher model is fine-tuned;

[0033] y' t is the true label set of the training sample when the teacher model is fine-tuned;

[0034] D domain is the feature vector and true label set of all samples during fine-tuning, and D domain = {(x' t1 , y' t1 ), (x' t2 , y' t2 ),...};

[0035] |D domain | is the total number of samples in the data set D domain ;

[0036] η(x' t ) is the data quality adjustment coefficient;

[0037] M θ (x' t ) is the output of the teacher model when the parameter is θ;

[0038] λ is the regularization coefficient;

[0039] is the square of the L2 norm;

[0040] 1.3, calculate and output structured knowledge K t (x't ), specifically as follows:

[0041]

[0042] wherein, is the output of the teacher model for fine-tuning;

[0043] M t (x) is the probability distribution of the teacher model output;

[0044] AvgPool(·) is spatial dimension feature compression;

[0045] Softmax is a probability normalization function;

[0046] W t is the teacher model weight matrix.

[0047] The aforementioned multi-modal data quality evaluation method of the distillation algorithm, in step 2, constructs a student model M s based on the teacher model M t distillation to obtain a student model parameter set φ, including the following steps:

[0048] 2.1, define the student model structure based on the Transformer architecture, including:

[0049] the number of student model layers a2,

[0050] the number of student model attention heads b2,

[0051] the student model hidden layer dimension c2;

[0052] the student model parameter set φ pre ;

[0053]

[0054] wherein, x s0 is the feature vector set of the training sample during distillation;

[0055] 2.2, knowledge distillation of the teacher model M t (x) to obtain a student model parameter set φ t , specifically as follows:

[0056]

[0057] wherein, L task is the loss function;

[0058] a probability distribution output by the teacher model after temperature softening;

[0059] a probability distribution output by the student model;

[0060] T e a dynamic temperature;

[0061] K t (x s ) is a probability distribution output by the teacher model;

[0062] x s is a feature vector set of the training sample during distillation;

[0063] α1 is a distillation weight, and α1 ∈ [0.5, 0.8], and here α1 = 0.7;

[0064] D KL is a KL divergence function;

[0065] W s is a student model weight matrix;

[0066] M φ (x s ) is an output of the student model;

[0067] H is a total number of categories, and H ≥ 2;

[0068] y h is a real label,

[0069] is the hth component of the vector K s (x s ).

[0070] The aforementioned multi-modal data quality evaluation method of the distillation algorithm, in step 3, connects the student model to the customer's database, evaluates the customer data quality, and uses the customer data to train the student model parameter set, to obtain the data quality score and the student model parameter residual, including the following steps:

[0071] 3.1, deploy the student model on the edge device, and use the student model to evaluate the customer data quality to obtain the data quality score score, specifically:

[0072] Based on the prediction confidence standard deviation and the main indicators of data quality, the score is calculated together, and the specific formula is as follows:

[0073]

[0074] Where β1 is the weight of the weight of the prediction confidence standard deviation;

[0075] x g is a set of customer data feature vectors;

[0076]

[0077] M g is the number of customer data;

[0078] c i is the single prediction confidence degree;

[0079] is the M g prediction average confidence degree;

[0080] λ g is an experimentally optimized scaling factor;

[0081] I2 is a quality score;

[0082] 3.2, using customer data to train the student model parameter set, fine-tune the student model, get φ t+1 , the specific formula is as follows:

[0083]

[0084] wherein, φ t+1 is the parameter residual generated by self-supervised training;

[0085] α2 is the self-supervised training coefficient, and the specific value is α2 = 0.7;

[0086] is the KL divergence of the teacher model output probability distribution and the student model output probability distribution after temperature softening;

[0087] L con is the self-supervised training loss function;

[0088] N b is the number of samples for single training;

[0089] x gi is the feature vector of the i-th data in the customer data set;

[0090] x gk is the feature vector of the k-th data in the same batch of customer data (k≠i);

[0091] is the feature vector of the customer data generated by random mask;

[0092] is the feature vector of the customer data generated by different random masks;

[0093] the output of the student model calculated on the customer data generated by the random mask;

[0094] the output of the student model calculated on the customer data generated by the random mask;

[0095] z k = M φ (x gk ) the output of the student model calculated on other customer data in the same batch;

[0096] the sample exclusion indicator;

[0097] the vector inner product of the output of the student model calculated on the sample generated by the random mask and the output of the student model calculated on the sample generated by the different random mask;

[0098] ||.|| is the L2 norm;

[0099] T c is the adjustment coefficient, and the specific value is T c = 0.1;

[0100] 3.3, based on φ t+1 , calculate the student model parameter residual △φ t , the specific formula is as follows:

[0101] △φ t = φ t+1 - φ t

[0102] Where, φ t+1 is the student model parameter set after self-supervised training.

[0103] The aforementioned multi-modal data quality evaluation method of the distillation algorithm, in step 4, the data quality score and the student model parameter residual knowledge are fed back to the teacher model, the teacher model is optimized, and the data quality score is percentized as the result of this evaluation, including the following steps:

[0104] 4.1, the student model newly learned unpercentized data quality score and student model parameter residual knowledge are fed back to the teacher model, forming knowledge backflow, correcting the teacher model parameters, and obtaining a new parameter set θ (k+1) of the teacher model, the specific formula is as follows:

[0105]

[0106]

[0107] Where B is a cross-model knowledge alignment tensor;

[0108] In B l,j,k here, l is the student model layer traversal index, j is the hidden dimension feature index, and k is the teacher attention head traversal index;

[0109] θ (k) is the parameter set of the teacher model after fine-tuning, which has undergone k times of knowledge reflux;

[0110] θ (k+1) is the new parameter set of the teacher model, which is the parameter set of the teacher model after this (k+1) time of knowledge reflux;

[0111] ε1 is the fusion residual rate;

[0112] ε2 is the quality attenuation rate;

[0113] W is a weight matrix, and W = {W out ,W a ,W g};

[0114] W a is an attention alignment matrix;

[0115] W out is a teacher parameter space conversion matrix;

[0116] W g is a knowledge screening gating matrix;

[0117] ⊙ is the Hadamard product;

[0118] Υ(·) is a knowledge screening gating function;

[0119] σ(·) is a Sigmoid activation function, and

[0120] where x* is the input vector;

[0121] e is a natural constant;

[0122] vec(·) is a tensor vectorization transformation;

[0123] ×3 is a tensor-matrix product;

[0124] Ib1 represents a unit matrix with dimension b1;

[0125] 4.2, establish a re-fine-tuning threshold, and record the data quality score of this round as get the average score of the data quality evaluation of this batch, and the specific formula is as follows:

[0126]

[0127] Wherein, t is the t batches of data quality evaluation that have been passed before;

[0128] Tau is the total of tau rounds of data quality evaluation of this batch, take tau = 10;

[0129] t <= i tre <= t + tau;

[0130] 4.3, the score of data quality score is converted into percentage, and the result is output as the result of this data quality evaluation, the specific formula is as follows:

[0131] score = 100 * (1 - score*) 0.3 + 15 * score.

[0132] Compared with the prior art, the beneficial effects of the present application are:

[0133] 1、The present application adjusts the data quality adjustment coefficient into the model training, strengthens the sensitivity of the model degree data quality, data compliance, improves the training efficiency of the model, and enhances the professionalism of the model;

[0134] 2、The present application considers the difference between the early teacher model and the mature teacher model, introduces dynamic temperature, and realizes differentiated distillation for teacher models with different training times, so that it meets the characteristics of different stages, ensures that the distilled student model can quickly establish basic ability and deeply understand the decision boundary, and improves the ability of the student model to evaluate data quality;

[0135] 3、The present application obtains a student model through distillation, which is actually deployed to a customer data end and used for data quality evaluation, and the resource consumption is limited, and as a localized deployment, it does not need to transmit data through the Internet, solving the problems of large data transmission and low data quality evaluation efficiency;

[0136] 4、Since only the student model can really touch or train the evaluated data in actual data quality evaluation business, by first building a teacher model and training fine-tuning, then getting a student model through distillation, the student model can be carried by an edge device and deployed to a physically isolated customer database environment to evaluate the data quality of customer data and train the student model, and finally the training results of the student model are fed back to the teacher model through knowledge reflux, so that the teacher model can be better optimized, and a new student model can be distilled from the optimized teacher model, forming a positive cycle mechanism of cooperative evolution, so that the teacher model and the student model can be continuously optimized, and the accuracy of future data quality evaluation can be improved;

[0137] 5、The present application constructs a weight matrix W hThe knowledge representation space can be adjusted for different data types, such as text, image and time sequence, so that the quality of multi-modal data can be evaluated, and the adaptability of the model is improved. BRIEF DESCRIPTION OF DRAWINGS

[0138] Figure 1 The flowchart of the present application. DETAILED DESCRIPTION

[0139] In order for those skilled in the art to better understand the technical solutions of the present application, the present application will be further described in detail below with reference to the accompanying drawings.

[0140] The present application provides a multi-modal data quality evaluation method of distillation algorithm as shown in Figure 1 The method comprises the following steps:

[0141] Step 1, constructing a teacher model M t , the specific steps are as follows:

[0142] 1.1, defining the teacher model structure based on the Transformer architecture, including:

[0143] the number of teacher model layers a1;

[0144] the number of teacher model attention heads b1;

[0145] the teacher model hidden layer dimension c1;

[0146] Initialize the parameters θ pre through the deep learning framework to generate the initial teacher model M t , the calculation formula is as follows:

[0147]

[0148] wherein, is the solution of the inner function, and the parameter vector θ is obtained to make the global minimum value;

[0149] θ is the parameter set of the teacher model;

[0150] i is the index of the current sample, and the specific value is 1-N;

[0151] N is the total number of training samples;

[0152] x ti is the feature vector of the i-th training sample when the teacher model is initialized;

[0153] y ti is the true label of the i-th training sample when the teacher model is initialized;

[0154] P(y ti |xti ; θ) is the probability that the teacher model predicts the sample to belong to the true label y i under the condition of given input x ti and parameter θ;

[0155] log is the natural logarithm;

[0156] is the forward calculation process based on the Transformer architecture with parameter θ pre ; θ) is the probability that the teacher model predicts the sample to belong to the true label y t under the condition of given input x

[0157] x fin is the feature vector set of the training sample when the teacher model is initialized;

[0158] 1.2, fine-tune the teacher model based on data quality perception to obtain θ fin , and the specific formula is as follows:

[0159]

[0160] where θ fin is the parameter set of the fine-tuned teacher model;

[0161] x' t is the feature vector set of the training sample when the teacher model is fine-tuned;

[0162] y' t is the true label set of the training sample when the teacher model is fine-tuned;

[0163] D domain is the feature vector set and the true label set of all samples when fine-tuned, and D domain = {(x' t1 , y' t1 ), (x' t2 , y' t2 ),...};

[0164] |D domain | is the total number of samples of the data set D domain ;

[0165] η(x' t ) is the data quality adjustment coefficient;

[0166] M θ (x' t ) is the output of the teacher model when the parameter is θ;

[0167] λ is the regularization coefficient;

[0168] is the square of the L2 norm;

[0169] 1.3, Calculate and output structured knowledge by completing the fine-tuned teacher model, get K t (x t ), The specific formula is as follows:

[0170]

[0171] Wherein, is the output of the fine-tuned teacher model;

[0172] M t (x) is the probability distribution of the teacher model output;

[0173] AvgPool(·) is the spatial dimension feature compression;

[0174] Softmax is a probability normalization function;

[0175] W t is the teacher model weight matrix;

[0176] In this step 1, by using the AvgPool function to compress the spatial dimension feature, the redundant information is eliminated, so that the student model is easier to learn the decision essence characteristics of the teacher model, and the constructed weight matrix W h , For different data types, such as text, image, time sequence, adjust the knowledge representation space, so as to evaluate the quality of multi-modal data.

[0177] Step 2, build student model M s , And based on the teacher model M t Distillation gets the student model parameter set φ, The specific steps are as follows:

[0178] 2.1, Define the student model structure based on the Transformer architecture, including:

[0179] The number of student model layers a2,

[0180] The number of student model attention heads b2,

[0181] The dimension of the student model hidden layer c2;

[0182] Student model parameter set φ pre ;

[0183]

[0184] Wherein, x s0 is the feature vector set of the training sample during distillation;

[0185] 2.2, to the teacher model M t(x) knowledge distillation, a student model parameter set φ is distilled t , and the specific formula is as follows.

[0186]

[0187] wherein, L task is a loss function;

[0188] is a probability distribution output by the teacher model after temperature softening;

[0189] is a probability distribution output by the student model;

[0190] T e is a dynamic temperature;

[0191] K t (x s ) is a probability distribution output by the teacher model;

[0192] x s is a feature vector set of the training sample during distillation;

[0193] α1 is a distillation weight, and α1∈[0.5, 0.8], and herein α1=0.7;

[0194] D KL is a KL divergence function;

[0195] W s is a student model weight matrix;

[0196] M φ (x s ) is an output of the student model;

[0197] H is a total number of categories, and H≥2;

[0198] y h is a real label,

[0199] is an hth component of the vector K s (x s );

[0200] In this step 2, considering that the distillation mode of the early teacher model is different from the distillation mode of the late mature teacher model, a dynamic temperature is proposed. Specifically, the teacher model in the early training (small number of iterations) is trained, the ability is initially established, and through a lower temperature (T e value is smaller), the distillation process is relatively sharpened, so that the student model can quickly establish the basic ability;

[0201] The teacher model in the later training stage (when the number of iterations is large) has relatively mature ability, and through a higher temperature (T e A larger), the distillation process is relatively smooth, allowing the student model to deeply understand the decision boundary and improve the student model's ability to assess data quality.

[0202] Step 3, connect the student model to the customer's database, assess the customer data quality, and train the student model parameter set using customer data to obtain data quality scores and student model parameter residuals, including the following steps:

[0203] 3.1, deploy the student model on the edge device, use the student model to assess the customer data quality, and obtain the data quality score score, specifically:

[0204] Based on the prediction confidence standard deviation and the main data quality indicators to calculate the score, the specific formula is as follows:

[0205]

[0206] Where β1 is the weight of the weight of the prediction confidence standard deviation;

[0207] x g is the customer data feature vector set;

[0208]

[0209] M g is the number of customer data;

[0210] c i is the single prediction confidence;

[0211] is the average prediction confidence of M g ;

[0212] λ g is the scaling factor optimized by experiment;

[0213] I2 is the quality score;

[0214] 3.2, train the student model parameter set using customer data, fine-tune the student model, and obtain φ t+1 , the specific formula is as follows:

[0215]

[0216] Where φ t+1 is the parameter residual generated by self-supervised training;

[0217] α2 is the self-supervised training coefficient, specifically α2 = 0.7;

[0218] is the KL divergence of the teacher model output probability distribution and the student model output probability distribution after temperature softening;

[0219] L con is the self-supervised training loss function;

[0220] N b is the number of samples for a single training;

[0221] x gi is the feature vector of the i-th data in the customer data set;

[0222] x gk is the feature vector of the k-th data in the same batch of customer data (k≠i);

[0223] is the feature vector of the customer data generated by random masking;

[0224] is the feature vector of the customer data generated by different random masking;

[0225] is the output of the student model after calculating the customer data generated by random masking;

[0226] is the output of the student model after calculating the customer data generated by different random masking;

[0227] z k = M φ (x gk ) is the output of the student model after calculating other customer data in the same batch;

[0228] is the sample exclusion indicator;

[0229] is the vector inner product of the output of the student model after calculating the sample generated by random masking and the output of the student model after calculating the sample generated by different random masking;

[0230] ||.|| is the L2 norm;

[0231] T c is the adjustment coefficient, and the specific value is T c = 0.1;

[0232] 3.3, based on φ t+1 , calculate the student model parameter residual △φ t , the specific formula is as follows:

[0233] △φ t = φ t+1- φ t

[0234] wherein φ t+1 is the student model parameter set after self-supervised training;

[0235] In this step 3, by obtaining the formula of the non-differentiated data quality score based on the prediction confidence standard deviation and the data quality main indicators, the data quality evaluation score can be quickly calculated, at the same time, the student model parameter set is trained using customer data, the student model is fine-tuned, and the parameter residual can be obtained through the student model parameters before and after fine-tuning, so as to prepare intermediate data for the next knowledge backflow.

[0236] Step 4, the data quality score and the student model parameter residual knowledge backflow are fed back to the teacher model, the teacher model is optimized, and the data quality score is differentiated as the result of this evaluation, including the following steps:

[0237] 4.1, the non-differentiated data quality score newly learned by the student model and the student model parameter residual knowledge backflow are fed back to the teacher model to form knowledge backflow, correct the teacher model parameters, and obtain a new parameter set θ (k+1) of the teacher model, the specific formula is as follows:

[0238]

[0239] wherein B is a cross-model knowledge alignment tensor;

[0240] In B l,j,k , l is the student model layer traversal index, j is the hidden dimension feature index, and k is the teacher attention head traversal index;

[0241] θ (k) is the parameter set of the teacher model after k times of knowledge backflow after fine-tuning;

[0242] θ (k+1) is the new parameter set of the teacher model, which is the teacher model parameter set after this (k+1) times of knowledge backflow;

[0243] ε1 is the fusion residual rate;

[0244] ε2 is the quality attenuation rate;

[0245] W is a weight matrix, and W = {W out , W a , W g};

[0246] W a is an attention alignment matrix;

[0247] W out is a teacher parameter space conversion matrix;

[0248] W g is a knowledge filter gating matrix;

[0249] is a Hadamard product;

[0250] Y(·) is a knowledge filter gating function;

[0251] σ(·) is a Sigmoid activation function, and

[0252] where x* is an input vector;

[0253] e is a natural constant;

[0254] vec(·) is a tensor vectorization transform;

[0255] is a tensor-matrix product;

[0256] represents a unit matrix of dimension b1;

[0257] 4.2, establish a re-tuning threshold, record this round of data quality score score* as get the average score of this batch of data quality assessment, the specific formula is as follows:

[0258]

[0259] where t is the t batch of data quality assessment has been passed;

[0260] τ is the total of τ rounds of data quality assessment, take τ = 10;

[0261] t≤i tre ≤t+τ;

[0262] 4.3, the score of data quality score is converted into percentage system, and the result is output as the result of this data quality assessment, the specific formula is as follows:

[0263] score = 100 x (1-score*) 0.3 + 15·score*.

[0264] In step 4.2, in order to prevent the basic ability of the teacher model from being adversely affected after multiple knowledge backflow, a closed-loop control module is needed, that is, when the stability of the teacher model evaluation is insufficient, the model re-tuning training is triggered;

[0265] In this step 4, by constructing a knowledge filter gating function, the student model parameter residual error △φ tThe space transformation is performed, and the teacher model parameter θ is corrected, thereby realizing knowledge backflow, so that when the student model performs data quality evaluation work at the customer, the teacher model can be continuously optimized, so that the newly generated student model has stronger ability and higher evaluation accuracy when performing data quality evaluation.

[0266] An edge device of a multi-modal data quality evaluation method using a distillation algorithm, comprising a teacher model hardware system and a student model hardware system;

[0267] The teacher model hardware system comprises a high-performance computing server, a distributed storage array, and a gigabit Ethernet module.

[0268] The high-performance computing server and the distributed storage array are interconnected through a PCIe4.0 data bus, and are connected to an external network interface through the gigabit Ethernet module.

[0269] The student model hardware system comprises a portable computing unit, an isolated execution environment, an encrypted storage card, and a multi-protocol interface.

[0270] The portable computing unit and the isolated execution environment are interconnected through an on-board integrated bus, and are directly connected to the encrypted storage card through an NVMe protocol and the multi-protocol interface integrated in the computing unit.

[0271] Further, the teacher model hardware system is arranged to be deployed in a central computer room.

[0272] The high-performance computing server is configured with ≥4 Xeon Platinum 8380 processors for three-stage training of the teacher model.

[0273] The distributed storage array uses a RAID 60 architecture SSD storage system with a capacity of ≥1PB, which stores the teacher model parameter θ and the training data set.

[0274] The gigabit Ethernet module is used to realize a secure communication interface with each mobile terminal.

[0275] The student model hardware system is arranged as a mobile device.

[0276] The portable computing unit integrates an NVIDIA Jetson AGX Orin module with a power consumption of ≤60W.

[0277] The isolated execution environment is arranged as a hardware-level TrustZone security module that supports physically isolated data processing.

[0278] The encrypted storage card is arranged as a detachable NVMe SSD (capacity ≥2TB) that stores the student model parameter φ and the evaluation algorithm.

[0279] Multi-protocol interface supports USB4, Thunderbolt 4 and PCIe 4.0, realizing rapid deployment of customer room;

[0280] The teacher model hardware system and the student model hardware system work in cooperation, and the cooperation work flow is:

[0281] A. The central computer room generates a lightweight student model and transmits it to the mobile device after encryption;

[0282] B. The mobile device enters the customer isolation environment, loads local data to perform dynamic quality assessment, and performs parameter fine-tuning, and the specific formula is:

[0283] Dynamic quality assessment:

[0284] score = 100 x (1 - score*) 0.3 + 15 · score*

[0285] Parameter fine-tuning: △φ t = φ t+1 - φ t ;

[0286] C. Only the parameter residual △φ τ and the score are transmitted to the central computer room;

[0287] D. The central computer room updates the teacher model, and the specific formula is:

[0288]

[0289] The device adopting the multi-modal data quality evaluation method of the above distillation algorithm has the following effects:

[0290] Guaranteeing safety isolation, realizing physical isolation data processing through a hardware-level security module, and ensuring that customer sensitive data (such as financial transactions and medical records) do not leave the local environment;

[0291] Edge computing efficiency, the portable computing unit is equipped with a lightweight student model, and real-time quality assessment is realized on the customer site, which is more efficient than traditional centralized evaluation;

[0292] Two-way evolution capability, through distillation and knowledge backflow, realize the positive cycle of continuous optimization of the model;

[0293] Multi-modal adaptability, adapting to structured data, text data, picture data, audio and video data and other multi-modal data.

[0294] Verification experiment

[0295] Taking financial risk control data optimization as an example, specifically:

[0296] 1) Build a teacher model:

[0297] Basic architecture: The BERT-Large pre-trained model architecture is adopted, with a1=24 layers, b1=16 attention heads, and h1=1024 hidden dimensions.

[0298] Fine-tuning data: A bank's 100,000 risk control data entries, including structured data containing sensitive information such as customer names, ID numbers, mobile phone numbers, bank card numbers, savings amounts, total loan amounts, and repayment status, have been annotated. The specific formula for fine-tuning the data is as follows:

[0299]

[0300] 2) Distillation Student Model:

[0301] Compression architecture: The TinyBERT knowledge distillation architecture is adopted, with a2=6 layers, b2=4 attention heads, and h2=384 hidden dimensions.

[0302] 3) Dynamic temperature distillation:

[0303]

[0304] Temperature: T e =5×(1-e -0.2k When k1 = 10 and k2 = 0, k = k1 + k2 = 10, T e =4.32;

[0305] 4) Dual task execution:

[0306] Quality rating:

[0307] We can get a score of 0.79, and after percentage division, we can get a score of 73.5.

[0308] Self-supervised training of student models can yield the model parameters φ after training. t+τ And by △φ t =φ t+1 -φ t The parameter residual Δφ can be obtained. t ,||△φ t ||2 = 0.021;

[0309] 5) Teacher model optimization:

[0310] The optimized teacher model parameters are obtained: θ (k+1) =θ (k) +0.3×Υ(△φ t ;W)⊙e -0.5×0.79 , ||θ (k+1) -θ(k) ||2 = 0.017;

[0311] 6) Effect verification:

[0312] 5000 bank risk control test sample data (including 1,200 high-risk samples) with completed labeling are used to verify the effect of the teacher model after knowledge backflow, and the specific results are as shown in Table 1:

[0313] Indicator Before reflux After reflux Lift Accuracy 92.1％ 93.6％ +1.5％ Recall 88.3％ 90.7％ +2.4％ Inference latency (ms) 42.7 41.2 -3.5％

[0314] Among them, the index explanation of Table 1 is as follows:

[0315] Accuracy: The proportion of samples correctly predicted by the statistical model;

[0316] Recall: Calculate the recognition coverage for high-risk samples;

[0317] Inference delay: Measure the average time consumption of 100 inferences under Xeon Gold 6348 CPU single thread;

[0318] As can be seen from Table 1, the student model of the present application is trained by self-supervision, and the teacher model is backfed by knowledge backflow, which can comprehensively improve the ability of the teacher model, including accuracy, recall and improving the work efficiency of the teacher model;

[0319] In summary, the application strengthens the sensitivity of the model to data quality and data compliance by incorporating the data quality adjustment coefficient into the model training, improves the training efficiency of the model, and enhances the professionalism of the model; considering the difference in distillation calculation requirements between the early teacher model and the mature teacher model, a dynamic temperature is introduced to implement differentiated distillation for teacher models with different training times, so that it meets the characteristics of different stages, ensures that the distilled student model can quickly establish basic ability and deeply understand the decision boundary, and improves the ability of the student model to evaluate data quality; the student model obtained through distillation is actually deployed to the customer data end for data quality evaluation, which consumes limited resources, and as a localized deployment, it does not need to transmit data through the Internet, solving the problems of excessive data transmission and low data quality evaluation efficiency; since only the student model can actually touch or train the evaluated data in the actual data quality evaluation business, by first building a teacher model and fine-tuning the training, then obtaining a student model through distillation, the student model can be carried by an edge device and deployed to a physically isolated customer database environment to evaluate the data quality of customer data and train the student model, and finally the training results of the student model are fed back to the teacher model through knowledge reflux, so that the teacher model can be better optimized, and a new student model is distilled from the optimized teacher model, forming a positive cycle mechanism of co-evolution, so that the teacher model and the student model can be continuously optimized, and the accuracy of future data quality evaluation can be improved; the application constructs a weight matrix W h , which can adjust the knowledge representation space for different data types such as text, image and time sequence, so as to evaluate the quality of multi-modal data and improve the adaptability of the model.

[0320] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the above-described drawings needed to be used are only some embodiments described in the present application, and obviously, for those skilled in the art, other drawings can also be obtained from these drawings.

[0321] The above only describes some exemplary embodiments of the application by way of illustration, and it is self-evident that those skilled in the art can modify the described embodiments in various ways without departing from the spirit and scope of the application. Therefore, the above drawings and descriptions are illustrative in nature and should not be construed as limiting the scope of the claims of the present application.

Claims

1. A method for multi-modal data quality assessment of distillation algorithms, characterized in that: Comprising the following steps: Step 1, constructing a teacher model M t ; Step 2, construct student model M s and based on the teacher model M t distill the student model parameter set φ; Step 3, connecting the student model to the customer's database, evaluating the customer data quality, and training the student model parameter set using the customer data to obtain the data quality score and the student model parameter residual, the specific steps are as follows: 3.1, deploy the student model on the edge device, and use the student model to evaluate the customer data quality to obtain the data quality score score, specifically: Based on the prediction confidence standard deviation and the main data quality indicators to calculate the score, the specific formula is as follows: wherein β1 is a weight of a weight of a prediction confidence standard deviation; x g is a customer data feature vector set; M g is a customer data number; c i is a single prediction confidence; is M g times prediction average confidence; λ g is an experimentally optimized scaling factor; I2 is a quality score; 3.2, using the customer data to train the student model parameter set, fine-tuning the student model, obtaining φ t+1 The specific formula is as follows: wherein φ t+1 is the parameter residual generated by self-supervised training; a2 is the self-supervised training coefficient, and the specific value is a2=0.7; is the KL divergence of the output probability distribution of the teacher model after temperature softening and the output probability distribution of the student model; L con is the self-supervised training loss function; N b is the number of samples for single training; x gi is the feature vector of the i-th data in the customer data set; x gk is the feature vector of the k-th data in the same batch of customer data (k≠i); is the feature vector of the customer data generated by the random mask; is the feature vector of the customer data generated by different random masks; is the output of the student model after the customer data generated by the random mask is calculated; is the output of the student model after the customer data generated by different random masks is calculated; is the output of the student model after the other customer data in the same batch is calculated; is the sample exclusion indicator; is the vector inner product of the output of the student model after the sample generated by the random mask is calculated and the output of the student model after the sample generated by different random masks is calculated; ||.|| is the L2 norm; T c is the adjustment coefficient, and the specific value is T c =0.1; 3.3, based on φ t+1 , the student model parameter residual error Δφ t , the specific formula is as follows: Δφ t = φ t+1 - φ t ; where φ t+1 is the set of student model parameters after self-supervised training; Step 4, the data quality score and the student model parameter residual knowledge back to the teacher model, optimize the teacher model, and the data quality score percentage, as the result of this evaluation, including the following steps: 4.1, The newly learned undifferentiated data quality score of the student model and the student model parameter residual knowledge backflow to the teacher model, forming knowledge backflow, correcting the teacher model parameters, and obtaining a new parameter set θ of the teacher model (k+1) The specific formula is as follows: where B is a cross-model knowledge alignment tensor; in B l,j,k where l is a student model layer traversal index, j is a hidden dimension feature index, and k is a teacher attention head traversal index. θ (k) is the parameter set of the teacher model after fine-tuning, which has experienced k times of knowledge distillation; θ (k+1) is the new parameter set of the teacher model, which is the parameter set of the teacher model after this (k+1) time of knowledge distillation; ε1 is a fusion residual rate; ε2 is a mass attenuation rate; W is a weight matrix, and W = {W out , W a , W g} ; W a is an attention alignment matrix; W out is a teacher parameter space conversion matrix; W g is a knowledge screening gating matrix; is a Hadamard product; Y(·) is a knowledge screening gating function; σ(·) is a sigmoid activation function, and where x* is the input vector; e is the natural constant; vec(·) is the tensor vectorization transform; x3 is the tensor-matrix product; denotes the identity matrix of dimension b1. 4.2, Establishing the re-tuning threshold, let the data quality score of this round be score* The average score of the data quality assessment of this batch is obtained, and the specific formula is as follows: Wherein, t is the t batches of data quality evaluation that have been passed before; τ is the total of τ rounds of data quality evaluation that have been passed in this batch, and τ = 10 is taken; t≤i tre ≤t+τ; 4.3, the score of the data quality score is converted to percentage, and the result is output as the result of this data quality evaluation, the specific formula is as follows: score = 100 x (1 - score*) 0.3 + 15 • score*.

2. The method of claim 1, wherein: In step 1, a teacher model M is constructed t including the following steps: 1.1, define the teacher model structure based on the Transformer architecture, including: Teacher model layer number a1; Teacher model attention head number b1; Teacher model hidden layer dimension c1; Initialize the parameters θ through a deep learning framework pre , generate an initial teacher model M t , the calculation formula is as follows: wherein, is solved for the inner function, resulting in a parameter vector θ that attains a global minimum. θ is the parameter set of the teacher model; i is the index of the current sample, the specific value is 1-N; N is the total number of training samples; x ti a feature vector for the i-th training sample at initialization of the teacher model; y ti a true label of the i-th training sample when initializing the teacher model; P(y ti |x ti ; θ) is the probability that the teacher model predicts the sample to belong to the true label y i given the input x ti and parameters θ. log is the natural logarithm; is a parameter for the forward computation process based on the Transformer architecture pre is a parameter for the forward computation process based on the Transformer architecture x t a set of feature vectors for training samples for initializing the teacher model; 1.2, fine-tune the teacher model based on data quality awareness, get θ fin The specific formula is as follows: where θ fin is the fine-tuned teacher model parameter set; x' t a set of feature vectors of training samples for fine-tuning the teacher model; y' t a true label set of training samples for fine-tuning the teacher model; D domain For fine-tuning, all sample feature vectors, true label set, and D domain = {(x' t1 ,y' t1 ), (x' t2 ,y' t2 ),...}; |D domain | is the total number of samples of the dataset D domain ; η(x t ) is a data quality adjustment coefficient; M θ (x' t ) is the output of the teacher model when the parameter is θ; λ is the regularization coefficient; is the L2 norm square; 1.

3. Calculate and output structured knowledge by completing the fine-tuned teacher model, get K t (x' t ), the specific formula is as follows: wherein, output of the fine-tuned teacher model; M t (x) is a probability distribution output by the teacher model; AvgPool(·) is the spatial dimension feature compression; Softmax is the probability normalization function; W t is the teacher model weight matrix.

3. The method of claim 1, wherein: In step 2, the student model M s is constructed, based on the teacher model M t The distillation of the student model parameter set φ is obtained, including the following steps: 2.1, define the student model structure based on the Transformer architecture, including: student model layer number a2, student model attention head number b2, Student model hidden layer dimension c2; student model parameter set φ pre ; wherein x s0 is a set of feature vectors of the training samples at distillation; 2.2, on the teacher model M t (x) knowledge distillation, distilling a student model parameter set φ t The specific formula is as follows; where L task is a loss function; a probability distribution output by a teacher model after temperature softening; output a probability distribution for the student model; T e is the dynamic temperature; K t (x s ) is a probability distribution output by the teacher model; x s is a set of feature vectors of the training samples at the distillation time; α1 is the distillation weight, and α1∈[0.5,0.8], here α1=0.7; D KL is the KL divergence function; W s student_model_weights_matrix; M φ (x s ) is the output of the student model; H is the total number of categories, and H≥2; y h is true label, is vector K s (x s )th component of h.

Images