Data quality scoring method, device, equipment, storage medium and product

Abstract

The present disclosure relates to a data quality scoring method, device, equipment, storage medium and product. The method includes: in response to obtaining a data set, taking each data in the data set as data to be scored; training multiple preset benchmark models based on each data to be scored, and obtaining model update parameters corresponding to each data to be scored for each preset benchmark model; determining the comprehensive sensitivity of each data to be scored corresponding to the multiple preset benchmark models based on the model initial parameters of each preset benchmark model and the model update parameters corresponding to each data to be scored for each preset benchmark model; and determining the quality score of each data to be scored based on the comprehensive sensitivity. The present disclosure can achieve more comprehensive and efficient data quality assessment, thereby improving the accuracy and efficiency of subsequent data filtering.

Description

Technical Field

[0001] The present disclosure relates to the field of data processing technology, and in particular to a data quality scoring method, device, equipment, storage medium and product. Background Art

[0002] In fields such as machine learning, data quality assessment and screening have always been a key link in model training. With the continuous expansion of data scale and the increasing diversity of data sources, how to effectively identify and filter low-quality data has become a key issue that needs to be solved. Traditional data filtering methods are often one-sided and unstable, and it is difficult to meet the needs of accurately identifying and screening high-quality data, which in turn affects the effect and efficiency of subsequent model training. Summary of the invention

[0003] In view of this, the present disclosure proposes a data quality scoring method, device, equipment, storage medium and product to solve the above technical problems.

[0004] In order to achieve the above objectives, the technical solution adopted by the present disclosure is:

[0005] According to a first aspect of an embodiment of the present disclosure, a data quality scoring method is proposed, the method comprising:

[0006] In response to obtaining the data set, each piece of data in the data set is used as data to be scored;

[0007] Based on each of the data to be scored, a plurality of preset benchmark models are trained respectively to obtain model update parameters of each preset benchmark model corresponding to each of the data to be scored, wherein the scales of the plurality of preset benchmark models are different;

[0008] Determining the comprehensive sensitivity of the plurality of preset benchmark models to each of the data to be scored based on the model initial parameters of each of the preset benchmark models and the model update parameters of each of the preset benchmark models corresponding to each of the data to be scored, wherein the model initial parameters include the initial parameters of the preset benchmark models;

[0009] A quality score for each piece of the data to be scored is determined based on the comprehensive sensitivity.

[0010] In some embodiments, the training of multiple preset benchmark models based on each piece of the data to be scored to obtain model update parameters of each preset benchmark model corresponding to each piece of the data to be scored includes:

[0011] Perform forward propagation calculation based on each piece of data to be scored and each preset benchmark model to obtain a model prediction value;

[0012] Constructing a loss function based on the difference between the model prediction value and the actual label value;

[0013] Back propagation is performed based on the gradient of the loss function with respect to the parameters, the preset learning rate, and the initial parameters of the model to obtain model update parameters of each preset benchmark model corresponding to each piece of the data to be scored.

[0014] In some embodiments, the model initial parameters and the model update parameters are parameters of a target key layer, and the target key layer is a key layer that needs to be monitored and is pre-specified in each preset benchmark model.

[0015] In some embodiments, the parameters of the target key layer include a weight matrix and a bias vector of the target key layer.

[0016] In some embodiments, the determining the comprehensive sensitivity of the plurality of preset benchmark models to each of the data to be scored based on the model initial parameters of each of the preset benchmark models and the model update parameters of each of the preset benchmark models corresponding to each of the data to be scored comprises:

[0017] Determine the model parameter change amount of each preset benchmark model corresponding to each of the data to be scored based on the model initial parameters of each preset benchmark model and the model update parameters of each preset benchmark model corresponding to each of the data to be scored;

[0018] A sensitivity feature vector is constructed based on the model parameter changes of the multiple preset benchmark models corresponding to each of the data to be scored, and the sensitivity feature vector is used to characterize the comprehensive sensitivity of the multiple preset benchmark models to each of the data to be scored.

[0019] In some embodiments, the determining the model parameter change amount of each preset benchmark model corresponding to each piece of data to be scored based on the model initial parameters of each preset benchmark model and the model update parameters of each preset benchmark model corresponding to each piece of data to be scored includes:

[0020] For each layer of each preset benchmark model, the weight change matrix of each preset benchmark model corresponding to each data to be scored is determined based on the initial weight matrix and the updated weight matrix corresponding to each data to be scored, wherein the initial weight matrix is ​​the weight matrix of each preset benchmark model before training, and the updated weight matrix is ​​the weight matrix of each preset benchmark model after training.

[0021] In some embodiments, constructing a sensitivity feature vector based on the model parameter change amount corresponding to each piece of the to-be-scored data based on the multiple preset benchmark models includes:

[0022] For each piece of the data to be scored, respectively determine the average value of the model weight change matrix of each layer of each preset benchmark model;

[0023] The sensitivity feature vector is determined based on a weighted sum of average values ​​of model weight change matrices of each layer of the multiple preset reference models.

[0024] In some embodiments, determining the quality score of each piece of the to-be-scored data based on the comprehensive sensitivity includes:

[0025] Calculating a plurality of preset index values ​​for the sensitivity feature vectors respectively, wherein the plurality of preset index values ​​include at least two of a norm, a standard deviation, and a stability index value;

[0026] The quality score of each piece of the to-be-scored data is determined based on a weighted sum of the multiple preset indicator values.

[0027] In some embodiments, the method further comprises:

[0028] Sorting the quality scores of each piece of data in the data set in descending order to obtain a sorting result;

[0029] The data exceeding the preset ratio threshold in the sorting results are filtered out to obtain filtered data.

[0030] According to a second aspect of an embodiment of the present disclosure, a data quality scoring device is provided, the device comprising:

[0031] A scoring data determination module, configured to, in response to acquiring a data set, use each piece of data in the data set as data to be scored;

[0032] An update parameter acquisition module, used to train a plurality of preset benchmark models based on each piece of the data to be scored, respectively, to obtain a model update parameter of each preset benchmark model corresponding to each piece of the data to be scored, wherein the scales of the plurality of preset benchmark models are different;

[0033] A sensitive information determination module, configured to determine the comprehensive sensitivity of the plurality of preset benchmark models to each of the data to be scored based on the model initial parameters of each of the preset benchmark models and the model update parameters of each of the preset benchmark models corresponding to each of the data to be scored, wherein the model initial parameters include the initial parameters of the preset benchmark models;

[0034] A quality score determination module is used to determine the quality score of each piece of the data to be scored based on the comprehensive sensitivity.

[0035] In some embodiments, the update parameter acquisition module includes:

[0036] A forward propagation calculation unit, used to perform forward propagation calculation based on each piece of data to be scored and each preset benchmark model to obtain a model prediction value;

[0037] A loss function construction unit, used to construct a loss function based on the difference between the model prediction value and the actual label value;

[0038] The back propagation unit is used to perform back propagation based on the gradient of the loss function with respect to the parameters, the preset learning rate and the initial parameters of the model to obtain the model update parameters of each preset benchmark model corresponding to each of the data to be scored.

[0039] In some embodiments, the model initial parameters and the model update parameters are parameters of a target key layer, and the target key layer is a key layer that needs to be monitored and is pre-specified in each preset benchmark model.

[0040] In some embodiments, the parameters of the target key layer include a weight matrix and a bias vector of the target key layer.

[0041] In some embodiments, the sensitive information determination module includes:

[0042] a variable determination unit, configured to determine a model parameter change amount of each preset benchmark model corresponding to each piece of data to be scored based on the model initial parameters of each preset benchmark model and the model update parameters of each preset benchmark model corresponding to each piece of data to be scored;

[0043] A vector construction unit is used to construct a sensitivity feature vector based on the model parameter changes of the multiple preset benchmark models corresponding to each of the data to be scored, and the sensitivity feature vector is used to characterize the comprehensive sensitivity of the multiple preset benchmark models to each of the data to be scored.

[0044] In some embodiments, the variable determination unit is also used to determine, for each layer of each preset benchmark model, a weight change matrix corresponding to each piece of data to be scored, based on an initial weight matrix and an updated weight matrix corresponding to each piece of data to be scored, wherein the initial weight matrix is ​​the weight matrix of each preset benchmark model before training, and the updated weight matrix is ​​the weight matrix of each preset benchmark model after training.

[0045] In some embodiments, the vector construction unit is further configured to:

[0046] For each piece of the data to be scored, respectively determine the average value of the model weight change matrix of each layer of each preset benchmark model;

[0047] The sensitivity feature vector is determined based on a weighted sum of average values ​​of model weight change matrices of each layer of the multiple preset reference models.

[0048] In some embodiments, the quality score determination module includes:

[0049] An index calculation unit, used to calculate a plurality of preset index values ​​for the sensitivity feature vectors respectively, wherein the plurality of preset index values ​​include at least two of a norm, a standard deviation and a stability index value;

[0050] A score determination unit is used to determine the quality score of each piece of the to-be-scored data based on a weighted sum of the multiple preset indicator values.

[0051] In some embodiments, the apparatus further comprises a data filtering module;

[0052] The data filtering module comprises:

[0053] A sorting unit, used to sort the quality scores of each piece of data in the data set in descending order to obtain a sorting result;

[0054] The filtering unit is used to filter out the data with a preset ratio threshold in the sorting result to obtain filtered data.

[0055] According to a third aspect of an embodiment of the present disclosure, an electronic device is provided, the electronic device comprising:

[0056] processor;

[0057] memory for storing computer programs;

[0058] Wherein, the processor is configured to implement any of the above-mentioned data quality scoring methods when executing the computer program.

[0059] According to a fourth aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided, on which a computer program is stored, and when the program is processed by a processor, the data quality scoring method described in any one of the above items is implemented.

[0060] According to a fifth aspect of an embodiment of the present disclosure, a computer program product is proposed, on which a computer program is stored, and when the program is processed by a processor, the data quality scoring method described in any one of the above items is implemented.

[0061] Compared with the prior art, the present disclosure, in response to acquiring a data set, takes each data in the data set as data to be scored, and trains multiple preset benchmark models based on each data to be scored, to obtain model update parameters of each preset benchmark model corresponding to each data to be scored, the scales of the multiple preset benchmark models are different, and then determines the comprehensive sensitivity of the multiple preset benchmark models to each data to be scored based on the model initial parameters of each preset benchmark model and the model update parameters of each preset benchmark model corresponding to each data to be scored, the model initial parameters include the initial parameters of the preset benchmark model, and then determines the quality score of each data to be scored based on the comprehensive sensitivity, so that the quality score of each data to be scored can be determined by using the comprehensive sensitivity of multiple preset benchmark models to each data to be scored, which can avoid the subjectivity of manually formulating rules and can achieve more comprehensive and efficient data quality assessment, thereby improving the accuracy and efficiency of subsequent data filtering. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] Figure 1 A flow chart showing a data quality scoring method according to an exemplary embodiment of the present disclosure is shown;

[0063] Figure 2 It shows how to obtain the model update parameters of each preset benchmark model corresponding to each piece of data to be scored according to an exemplary embodiment of the present disclosure;

[0064] Figure 3 A schematic diagram showing how to determine the comprehensive sensitivity of the multiple preset benchmark models to each piece of data to be scored according to an exemplary embodiment of the present disclosure;

[0065] Figure 4 A schematic diagram showing how to determine the quality score of each piece of data to be scored according to an exemplary embodiment of the present disclosure;

[0066] Figure 5 A schematic diagram showing how to filter each piece of data in the data set according to an exemplary embodiment of the present disclosure is shown;

[0067] Figure 6 A flow chart of a method for data quality scoring and filtering according to an exemplary embodiment of the present disclosure is shown;

[0068] Figure 7 A structural block diagram of a data quality scoring device according to an exemplary embodiment of the present disclosure is shown;

[0069] Figure 8 A structural block diagram of a data quality scoring device according to another exemplary embodiment of the present disclosure is shown;

[0070] Fig. 9 A structural block diagram of an electronic device according to an exemplary embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0071] The present disclosure will be described in detail below in conjunction with the specific embodiments shown in the accompanying drawings. However, these embodiments do not limit the present disclosure, and any structural, methodological, or functional changes made by a person of ordinary skill in the art based on these embodiments are included in the protection scope of the present disclosure. In addition, the technical features involved in the different embodiments of the present disclosure described below can be combined with each other as long as they do not conflict with each other.

[0072] In fields such as machine learning, data quality assessment and screening have always been a key link in model training. With the continuous expansion of data scale and the increasing diversity of data sources, how to effectively identify and filter low-quality data has become a key issue that needs to be solved urgently.

[0073] At present, the commonly used data filtering methods in the industry can be mainly divided into the following categories: the first category is rule-based filtering methods, which judge the quality of data through artificially pre-defined rules, such as checking the integrity or consistency of data; the second category is statistical feature-based filtering methods, which identify abnormal data by calculating the statistical features of data (such as mean, variance, etc.); the third category is model-based filtering methods, which distinguish high-quality and low-quality data by training simple discriminant models. Although these methods can work in specific scenarios, they are all one-sided and unstable, and it is difficult to meet the needs of accurately identifying and screening high-quality data, which will affect the effect and efficiency of subsequent model training.

[0074] Based on the defects of the above-mentioned related technologies, the embodiments of the present application provide a data quality scoring method, device, equipment, storage medium and product, which directly evaluates the contribution of the data to be scored to the model training by analyzing the parameter changes of the model during the training process, and achieves comprehensive, efficient and accurate data quality scoring. It can not only avoid the subjectivity of manually formulating rules, but also make full use of the internal information in the model training process to capture the deep-level features and complex relationships in the data, thereby improving the accuracy and efficiency of data filtering, and there is no need to rely on external discriminant models, which can reduce computational overhead.

[0075] Figure 1 A flow chart of a data quality scoring method according to an exemplary embodiment of the present disclosure is shown. The method can be performed by a data quality scoring device, which can be configured in an electronic device. This embodiment does not limit the type of electronic device, which can be any device type suitable for implementation, such as a terminal device or a server.

[0076] like Figure 1 As shown, the method includes the following steps S101-S104:

[0077] In step S101, in response to obtaining a data set, each piece of data in the data set is used as data to be scored.

[0078] In this embodiment, after the electronic device obtains the data set whose data quality is to be evaluated, each piece of data in the data set can be used as data to be evaluated.

[0079] It is worth noting that the above datasets can be set based on actual business needs, or ready-made datasets can be selected from related technologies, such as a mixed dataset of Wikipedia and Common Crawl, etc. This embodiment does not limit this.

[0080] For example, if the electronic device receives a data set D to be evaluated for data quality input by a user, it can extract the i-th data d from the data set. i(i＝1,2,…,n) , as the data to be scored, where n is the total number of data in the data set.

[0081] In step S102, a plurality of preset benchmark models are trained based on each piece of the data to be scored, respectively, to obtain model update parameters of each preset benchmark model corresponding to each piece of the data to be scored.

[0082] In this embodiment, after each piece of data in the data set is used as data to be scored, the electronic device can train multiple preset benchmark models based on each piece of data to be scored to obtain model update parameters of each preset benchmark model corresponding to each piece of data to be scored.

[0083] The scales of the multiple preset benchmark models are different, that is, the structures of the multiple preset benchmark models are the same, but the parameter quantities are different.

[0084] It is worth noting that the types of the above-mentioned multiple preset benchmark models can be selected from relevant technologies based on actual business needs. For example, a model of a Transformer architecture that only includes a decoder part (such as a Liama ​​model or a Qwen model, etc.) can be selected. This embodiment does not limit this.

[0085] For example, three models of Transformer architectures of different sizes can be selected, denoted as M L (Large-Scale Model), M M (Medium scale model), M S (Small-scale model). Among them, the parameter quantities of these three models can show a difference of about 10 times, namely:

[0086] |θ L |≈10*|θ M |≈100*|θ S |;(1-1)

[0087] Among them, |θ k | represents the number of parameters of the corresponding model (wherein k represents the model scale, that is, k = L, M or S).

[0088] For example, the above M L 、M M 、M S The LIama2-70B model (with 70B parameters), the LIama2-7B model (with 7B parameters), and the LIama2-0.7B model (with 700M parameters) can be selected respectively.

[0089] In some embodiments, in order to ensure fairness and accuracy in quality assessment of each piece of data in a data set, the objects to be trained for each piece of data to be scored can be obtained by cloning the above-mentioned multiple preset benchmark models (i.e., creating copies that are exactly the same as the multiple preset benchmark models respectively). In other words, the multiple models trained for one piece of data to be scored are independent of the multiple models trained for another piece of data.

[0090] In step S103, the comprehensive sensitivity of the plurality of preset benchmark models to each of the data to be scored is determined based on the model initial parameters of each of the preset benchmark models and the model update parameters of each of the preset benchmark models corresponding to each of the data to be scored.

[0091] In this embodiment, after obtaining the model update parameters of each preset benchmark model corresponding to each of the data to be scored, the comprehensive sensitivity of the multiple preset benchmark models to each of the data to be scored can be determined based on the model initial parameters of each of the preset benchmark models and the model update parameters of each of the preset benchmark models corresponding to each of the data to be scored.

[0092] The above-mentioned model initial parameters may include the initial parameters of the preset benchmark model.

[0093] In some embodiments, the above-mentioned model initial parameters and model update parameters may both be parameters of the target key layer, wherein the target key layer is a key layer that needs to be monitored and is pre-specified in each preset reference model. Exemplarily, the above-mentioned parameters of the target key layer may include a weight matrix W and a bias vector b of the target key layer.

[0094] For example, the above-mentioned target key layer can be a feed-forward layer and / or a self-attention layer in a neural network model. It is understandable that in a deep learning model, not all layers are equally important. Some layers (such as layers close to the input layer) may contribute less to the final output. In this embodiment, by focusing on those layers that have a greater impact on model performance (i.e., target key layers), it is helpful to deeply understand the comprehensive sensitivity of multiple preset benchmark models to each of the data to be scored, thereby improving the rationality and accuracy of subsequent data quality evaluation.

[0095] That is to say, the comprehensive sensitivity of the above-mentioned multiple preset benchmark models to each of the data to be scored can reflect the comprehensive contribution of each of the data to be scored to the training of the multiple preset benchmark models. That is, the higher the comprehensive sensitivity, the higher the comprehensive contribution of each of the data to be scored to the training of the multiple preset benchmark models.

[0096] In some other embodiments, the above method of determining the comprehensive sensitivity of the multiple preset benchmark models to each of the data to be scored can refer to the following Figure 3 The illustrated embodiment will not be described in detail here.

[0097] In step S104, a quality score of each piece of the to-be-scored data is determined based on the comprehensive sensitivity.

[0098] In this embodiment, after the comprehensive sensitivity of the multiple preset benchmark models to each piece of the data to be scored is determined, the quality score of each piece of the data to be scored can be determined based on the comprehensive sensitivity.

[0099] It is understandable that the comprehensive sensitivity of the above-mentioned multiple preset benchmark models to each of the data to be scored can be positively correlated with the quality score of each of the data to be scored. That is, the higher the comprehensive sensitivity, the higher the comprehensive contribution of each of the data to be scored to the training of the multiple preset benchmark models, and thus the higher the quality score of the data to be scored.

[0100] In some embodiments, the above method for determining the quality score of each piece of data to be scored can refer to the following Figure 4 The illustrated embodiment will not be described in detail here.

[0101] On the other hand, after determining the quality score of each piece of data to be scored, each piece of data in the data set can also be filtered according to the score of each piece of data and a preset ratio threshold. Figure 5 The illustrated embodiment will not be described in detail here.

[0102] As can be seen from the above description, the method of this embodiment, in response to obtaining a data set, takes each data in the data set as data to be scored, and trains multiple preset benchmark models based on each data to be scored, to obtain model update parameters of each preset benchmark model corresponding to each data to be scored, the scales of the multiple preset benchmark models are different, and then the comprehensive sensitivity of the multiple preset benchmark models to each data to be scored is determined based on the model initial parameters of each preset benchmark model and the model update parameters of each preset benchmark model corresponding to each data to be scored, the model initial parameters include the initial parameters of the preset benchmark model, and then the quality score of each data to be scored is determined based on the comprehensive sensitivity, which can realize the use of the comprehensive sensitivity of multiple preset benchmark models to each data to be scored to determine the quality score of each data, can avoid the subjectivity of manually formulating rules, and can achieve more comprehensive and efficient data quality assessment, thereby improving the accuracy and efficiency of subsequent data filtering.

[0103] Figure 2 It shows how to obtain the model update parameters of each preset benchmark model corresponding to each of the data to be scored according to an exemplary embodiment of the present disclosure; based on the above embodiment, this embodiment uses the example of how to obtain the model update parameters of each preset benchmark model corresponding to each of the data to be scored as an example to illustrate.

[0104] like Figure 2 As shown, the above step S102, based on each of the data to be scored, trains a plurality of preset benchmark models respectively to obtain the model update parameters of each preset benchmark model corresponding to each of the data to be scored, which may include the following steps S301-S303:

[0105] In step S201, forward propagation calculation is performed based on each piece of data to be scored and each preset benchmark model to obtain a model prediction value.

[0106] In this embodiment, after each piece of data in the data set is used as the data to be scored, the i-th piece of data d i To M k Training is performed (where k represents the model size, i.e., k = L, M, or S), including forward propagation calculation based on the following formula (2-1):

[0107] y i =M k (d i ); (2-1)

[0108] Among them, y i is the model prediction value, which is the output of the forward propagation calculation.

[0109] In step S302, a loss function is constructed based on the difference between the model prediction value and the actual label value.

[0110] In this embodiment, when the model prediction value y is obtained i Then, the value y can be predicted based on the model i and the actual label value y true The loss function is constructed by taking the difference between .

[0111] For example, the above loss function can be expressed as follows:

[0112] L = Loss(y i ,y true ); (2-2)

[0113] In step S303, back propagation is performed based on the gradient of the loss function with respect to the parameters, the preset learning rate and the initial parameters of the model to obtain the model update parameters of each preset benchmark model corresponding to each piece of the data to be scored.

[0114] In this embodiment, after constructing a loss function based on the difference between the model prediction value and the actual label value, back propagation can be performed based on the gradient of the loss function with respect to the parameters, the preset learning rate, and the initial parameters of the model to obtain the model update parameters of each preset benchmark model corresponding to each piece of data to be scored.

[0115] For example, suppose the gradient of the above loss function with respect to the parameters is The preset learning rate is α and the initial model parameter is θ init , then the model updates the parameter θ new It can be shown as the following formula (2-3):

[0116]

[0117] It is worth noting that the value of the above preset learning rate α can be set based on actual business needs, such as setting it to 1*10 -5 Etc., and this embodiment does not limit this.

[0118] As can be seen from the above description, this embodiment performs forward propagation calculations based on each of the data to be scored and each preset benchmark model to obtain a model prediction value, and constructs a loss function based on the difference between the model prediction value and the actual label value, and then performs back propagation based on the gradient of the loss function with respect to the parameters, the preset learning rate, and the initial parameters of the model to obtain the model update parameters of each preset benchmark model corresponding to each of the data to be scored. This can accurately obtain the model update parameters of each preset benchmark model corresponding to each of the data to be scored, so that the comprehensive sensitivity of the multiple preset benchmark models to each of the data to be scored and the quality score of each of the data to be scored can be determined based on the model update parameters. This can determine the quality score of each data by using the comprehensive sensitivity of multiple preset benchmark models to each of the data to be scored, avoid the subjectivity of manually formulating rules, and achieve more comprehensive and efficient data quality assessment, thereby improving the accuracy and efficiency of subsequent data filtering.

[0119] Figure 3 A schematic diagram is shown of how to determine the comprehensive sensitivity of the multiple preset benchmark models to each of the data to be scored according to an exemplary embodiment of the present disclosure; based on the above embodiment, this embodiment uses an example of how to determine the comprehensive sensitivity of the multiple preset benchmark models to each of the data to be scored.

[0120] like Figure 3 As shown, the step S103 described above of determining the comprehensive sensitivity of the multiple preset benchmark models to each of the data to be scored based on the model initial parameters of each of the preset benchmark models and the model update parameters of each of the preset benchmark models corresponding to each of the data to be scored may include the following steps S301-S302:

[0121] In step S301, the model parameter change amount of each preset benchmark model corresponding to each piece of data to be scored is determined based on the model initial parameters of each preset benchmark model and the model update parameters of each preset benchmark model corresponding to each piece of data to be scored.

[0122] In some embodiments, when determining the model parameter change amount of each preset benchmark model corresponding to each of the data to be scored, the weight change matrix of each preset benchmark model corresponding to each of the data to be scored can be determined for each layer of each preset benchmark model based on the initial weight matrix and the updated weight matrix corresponding to each of the data to be scored, thereby using it as the above-mentioned model parameter change amount.

[0123] The initial weight matrix is ​​the weight matrix of each preset reference model before training, and the updated weight matrix is ​​the weight matrix of each preset reference model after training.

[0124] For example, when using the i-th data d i To M k (k represents the model size, i.e. k = L, M or S) After training, for the model M k Each layer l is based on the initial weight matrix And update the weight matrix Determine the weight change matrix ΔW k .

[0125] Among them, the above initial weight matrix The weight matrix of the above preset benchmark model before training can be updated The weight matrix of the preset benchmark model after training may be used.

[0126] Exemplarily, the weight change matrix ΔW can be determined based on the following formula (2-1): k :

[0127]

[0128] In step S302, a sensitivity feature vector is constructed based on the model parameter changes of the multiple preset benchmark models corresponding to each piece of the data to be scored.

[0129] In this embodiment, after determining the model parameter change amount of each preset benchmark model corresponding to each piece of the data to be scored, a sensitivity feature vector can be constructed based on the model parameter change amounts of the multiple preset benchmark models corresponding to each piece of the data to be scored.

[0130] The sensitivity feature vector is used to characterize the comprehensive sensitivity of the multiple preset benchmark models to each piece of data to be scored.

[0131] In some embodiments, when constructing the above-mentioned sensitivity feature vector, the average value of the model weight change matrix of each layer of each preset baseline model can be determined for each piece of data to be scored, and then the sensitivity feature vector can be determined based on the weighted sum of the average values ​​of the model weight change matrices of each layer of the multiple preset baseline models.

[0132] For example, suppose M L 、M M and M S The weight change matrices are ΔW L , ΔW M and ΔW S, then the corresponding average values ​​can be respectively mean(ΔW L )、mean(ΔW M ) and mean(ΔW S ). Among them, mean() is the mean value of the matrix.

[0133] On this basis, the sensitivity feature vector can be determined based on the following formula (2-2):

[0134] S i =[ω L *mean(ΔW L ),ω M *mean(ΔW M ),ω S *mean(ΔW S )];

[0135] (2-2)

[0136] Among them, ω L ,ω M ,ω S is the weight coefficient and satisfies the following relationship:

[0137] ω L +ω M +ω S =1; (2-3)

[0138] For example, ω L =0.5,ω M =0.3,ω S =0.2.

[0139] As can be seen from the above description, this embodiment determines the model parameter change amount of each preset benchmark model corresponding to each of the data to be scored based on the model initial parameters of each preset benchmark model and the model update parameters of each preset benchmark model corresponding to each of the data to be scored, and constructs a sensitivity feature vector based on the model parameter change amount of the multiple preset benchmark models corresponding to each of the data to be scored. It can be achieved by characterizing the comprehensive sensitivity of the multiple preset benchmark models to each of the data to be scored based on the sensitivity feature vector, and then the quality score of each of the data to be scored can be subsequently determined based on the comprehensive sensitivity. Since the quality score of each data is determined by utilizing the comprehensive sensitivity of multiple preset benchmark models to each of the data to be scored, the subjectivity of manually formulating rules can be avoided, and a more comprehensive and efficient data quality assessment can be achieved, thereby improving the accuracy and efficiency of subsequent data filtering.

[0140] Based on the above embodiments, Figure 4A schematic diagram is shown of how to determine the quality score of each piece of data to be scored according to an exemplary embodiment of the present disclosure; based on the above embodiment, this embodiment uses how to determine the quality score of each piece of data to be scored as an example for exemplary description.

[0141] like Figure 4 As shown, the determination of the quality score of each piece of the to-be-scored data based on the comprehensive sensitivity described in the above step S104 may include the following steps S401-S402:

[0142] In step S401, a plurality of preset index values ​​are calculated for the sensitivity feature vectors respectively.

[0143] In this embodiment, after the sensitivity feature vector is obtained, a plurality of preset index values ​​may be calculated for the sensitivity feature vector respectively.

[0144] Among them, the multiple preset index values ​​include at least two of the norm, standard deviation and stability index value.

[0145] Taking multiple preset index values ​​including norm, standard deviation and stability index value as an example, when the i-th data d i The sensitivity feature vectors S corresponding to the plurality of preset reference models i After that, the norm of the vector norm(S i )、Standard deviation std(S i ) and stability index stability (S i ).

[0146] It is worth noting that the calculation method of the norm and standard deviation of the above sensitivity feature vector can refer to the standard formula recorded in the relevant technology, which will not be elaborated in this embodiment.

[0147] For example, the stability index stability (S i ) can be calculated as shown in the following formula (4-1):

[0148]

[0149] in, The calculation formulas are as follows:

[0150]

[0151] It is understandable that stability(S i ) The larger the value, the better the stability.

[0152] In step S402, the quality score of each piece of the to-be-scored data is determined based on the weighted sum of the multiple preset indicator values.

[0153] In this embodiment, after a plurality of preset index values ​​are calculated for the sensitivity feature vectors respectively, the quality score of each piece of the to-be-scored data may be determined based on a weighted sum of the plurality of preset index values.

[0154] Still taking the above-mentioned multiple preset index values ​​including norm, standard deviation and stability index value as an example, when calculating the norm norm (S i )、Standard deviation std(S i ) and stability index stability (S i ), the quality score Score (d i ):

[0155] Score(d i )=β1*norm(S i )+β2*std(S i )+β3*stability(S i ); (4-5)

[0156] Among them, β1, β2, and β3 are weight coefficients and satisfy the following relationship:

[0157] β1+β2+β3=1; (4-6)

[0158] Exemplarily, β1=0.25, β2=0.25, β3=0.5.

[0159] From the above description, it can be seen that this embodiment can and is able to achieve a more comprehensive and efficient data quality assessment by calculating multiple preset indicator values ​​for the sensitivity feature vector respectively, and determining the quality score of each data to be scored based on the weighted sum of the multiple preset indicator values, thereby improving the accuracy and efficiency of subsequent data filtering.

[0160] Figure 5 A schematic diagram is shown of how to filter each piece of data in the data set according to an exemplary embodiment of the present disclosure; based on the above embodiment, this embodiment uses how to filter each piece of data in the data set as an example to exemplify.

[0161] like Figure 5 As shown, the data quality scoring method of this embodiment may also include filtering each piece of data in the data set based on the following steps S501-S502:

[0162] In step S501, the quality scores of the data in the data set are sorted in descending order to obtain a sorting result.

[0163] In step S502, data exceeding a preset ratio threshold in the sorting result is removed to obtain filtered data.

[0164] For example, after the quality score of each piece of data to be scored is determined based on the model parameter variation corresponding to each piece of data to be scored, a filtering rule may be set based on a preset ratio threshold.

[0165] It is worth noting that the above threshold can be set based on actual business needs, such as being set to 0.25 (ie, retaining the first 75% of the data in the sorting result and removing the last 25% of the data), etc. This embodiment does not limit this.

[0166] On this basis, when the above filtered data set D is obtained filtered Afterwards, the filtering effect can be verified. For example, the filtered data set D filtered To train a new model M new , and evaluate the performance of the new model on the test set, and calculate the performance improvement of the model on the downstream test set.

[0167] From the above description, it can be seen that this embodiment uses the degree of influence of the data to be scored on the model training to determine the quality score of the data, which can avoid the subjectivity of manually formulating rules and achieve efficient and accurate data quality assessment. Therefore, based on the preset ratio threshold and the quality score, each data in the data set is filtered, which can improve the accuracy and efficiency of data filtering. Compared with the traditional statistical value-based solution, it can greatly improve the dirty data identification results, which can bring guarantee for the performance of the subsequent trained model.

[0168] The following is a complete example to introduce the data quality scoring and filtering method flow of the present disclosure, but it is not intended to limit the scope of the present disclosure. Figure 6 A flowchart of a method for data quality scoring and filtering according to an exemplary embodiment of the present disclosure is shown; wherein the specific explanation of each step can be found in the above Figures 1 to 5 The embodiments shown are not described in detail here. Figure 6 As shown, the method includes the following steps S601-S605:

[0169] In step S601, a multi-scale model is constructed.

[0170] In this embodiment, multiple pre-trained models with the same structure can be constructed. For example, three models of Transformer architectures of different scales can be constructed, denoted as M L (Large-Scale Model), M M (Medium scale model), M S (Small-scale model). Among them, the parameter quantities of these three models can show a difference of about 10 times, namely:

[0171] |θ L |≈10*|θ M |≈100*|θ S |;(6-1)

[0172] Among them, |θ k | represents the number of parameters of the corresponding model (wherein k represents the model scale, that is, k = L, M or S).

[0173] In step S602, a comprehensive sensitivity analysis is performed.

[0174] When establishing M L 、M M and M S After that, the initial parameters θ of each model can be recorded init , and then the key layer parameters that need to be monitored can be determined, including the weight matrix W and the bias vector b.

[0175] Exemplarily, the key layer may be selected as a feed-forward layer and / or a self-attention layer.

[0176] On this basis, for the i-th data d in the data set D to be evaluated i(i＝1,2,…,n) , you can perform the following steps:

[0177] a) Single data training, that is, using a single data d i For model M k (k represents the model size, i.e. k = L, M or S) for training, including:

[0178] a1) Forward propagation calculation output:

[0179] y i =M k (d i ); (6-2)

[0180] a2) Calculate the loss function:

[0181] L = Loss(y i ,y true ); (6-3)

[0182] a3) Back propagation updates parameters:

[0183]

[0184] Among them, α is the preset learning rate, for example, 1*10 -5 ; is the gradient of the loss function with respect to the parameters.

[0185] b) Calculate the changes of three model parameters:

[0186] For each model, the parameter change matrix ΔW of the key layer can be calculated k :

[0187]

[0188] Wherein, k represents the model scale, that is, k=L, M or S.

[0189] c) Establish a weighting mechanism:

[0190] Setting up L ,ω M ,ω S M L 、M M and M S The corresponding weight coefficients satisfy the following relationship:

[0191] θ L +ω M +ω S =1; (6-6)

[0192] For example, ω L =0.5,ω M =0.3,ω S =0.2.

[0193] d) Create sensitivity feature vector:

[0194] Assume M L 、M M and M S The weight change matrices are ΔW L , ΔW M and ΔW S , then the corresponding average values ​​can be respectively mean(ΔW L )、mean(ΔW M ) and mean(ΔW S ). Among them, mean() is the mean value of the matrix.

[0195] On this basis, the sensitivity feature vector can be determined based on the following formula (6-7):

[0196] S i =[ω L *mean(ΔW L ),ω M *mean(ΔW M ),ω S *mean(ΔW S )];

[0197] (6-7)

[0198] The above sensitivity feature vector S i It is used to characterize the above three different scales for data d i The overall sensitivity.

[0199] In step S603, the quality score is calculated.

[0200] When the i-th data d is obtained i(i＝1,2,…,n) The sensitivity eigenvector S i After that, multiple preset index values ​​can be calculated for the sensitivity feature vector, such as the norm norm(S i )、Standard deviation std(S i ) and stability index value stability(S i ), and then obtain the data d based on the weighted sum operation i The quality score can be scored by performing the following steps:

[0201] a) Establish a scoring system:

[0202] Score(d i )=β1*norm(S i )+β2*std(S i )+β3*stability(S i ); (6-8)

[0203] Among them, β1, β2, and β3 are weight coefficients and satisfy the following relationship:

[0204] β1+β2+β3=1; (6-9)

[0205] Exemplarily, β1=0.25, β2=0.25, β3=0.5.

[0206] b) Output quality score: After the above scoring system is established, the quality score of each data in the data set D can be calculated separately.

[0207] c) Sort the data according to the score:

[0208] After the quality score of each piece of data in the data set D is calculated, the data in the data set D can be sorted in descending order to obtain a sorting result.

[0209] In step S604, a high-quality data set is output.

[0210] After the sorting results are obtained, the last 25% of the data in the sorting results can be regarded as dirty data and discarded (i.e., not used in the subsequent model training process) according to a preset ratio threshold (e.g., 25%), so that the filtered high-quality data set D filtered Output.

[0211] From the above description, it can be seen that, on the one hand, this embodiment constructs three different scales of models, large, medium and small, and analyzes the parameter changes of these models before and after training to establish a complete sensitivity analysis mechanism. This multi-scale model integration method can ensure that the data quality assessment process simultaneously considers feature information at different levels, thereby providing more comprehensive and accurate data quality assessment results.

[0212] On the other hand, this embodiment proposes a new data quality scoring scheme, which comprehensively considers various preset indicator values ​​of the comprehensive sensitivity of multi-scale models, and calculates the data quality score by weighted summation. This scoring mechanism not only considers the overall impact of data on models of different scales, but also can dynamically adjust the evaluation criteria according to the data type and task characteristics, which can significantly improve the reliability and adaptability of data quality assessment, and greatly improve the accuracy and efficiency of data quality assessment.

[0213] Based on the cooperation of the above two aspects, this embodiment can improve the effect and efficiency of data filtering. Compared with the traditional calculation method based on statistical values, it can greatly improve the recognition effect of dirty data, thereby ensuring the performance of subsequent training models.

[0214] Figure 7 A structural block diagram of a data quality scoring device according to an exemplary embodiment of the present disclosure is shown; the data quality scoring device of this embodiment can be configured in an electronic device. This embodiment does not limit the type of electronic device, which can be any type of device suitable for implementation, such as a terminal device or a server.

[0215] like Figure 7 As shown, the device includes: a scoring data determination module 110, an update parameter acquisition module 120, a sensitive information determination module 130 and a quality score determination module 140, wherein:

[0216] A scoring data determination module 110 is used for, in response to acquiring a data set, taking each piece of data in the data set as data to be scored;

[0217] An update parameter acquisition module 120, configured to train a plurality of preset benchmark models based on each piece of the data to be scored, respectively, to obtain a model update parameter of each preset benchmark model corresponding to each piece of the data to be scored, wherein the plurality of preset benchmark models have different scales;

[0218] A sensitive information determination module 130, configured to determine the comprehensive sensitivity of the plurality of preset benchmark models to each of the data to be scored based on the model initial parameters of each of the preset benchmark models and the model update parameters of each of the preset benchmark models corresponding to each of the data to be scored, wherein the model initial parameters include the initial parameters of the preset benchmark models;

[0219] The quality score determination module 140 is used to determine the quality score of each piece of the to-be-scored data based on the comprehensive sensitivity.

[0220] As can be seen from the above description, the device of this embodiment, in response to acquiring a data set, takes each data in the data set as data to be scored, and trains multiple preset benchmark models based on each data to be scored, to obtain model update parameters of each preset benchmark model corresponding to each data to be scored, and the scales of the multiple preset benchmark models are different. Then, based on the model initial parameters of each preset benchmark model and the model update parameters of each preset benchmark model corresponding to each data to be scored, the comprehensive sensitivity of the multiple preset benchmark models to each data to be scored is determined, and the model initial parameters include the initial parameters of the preset benchmark model. Then, based on the comprehensive sensitivity, the quality score of each data to be scored is determined, and the quality score of each data to be scored can be determined by using the comprehensive sensitivity of multiple preset benchmark models to each data to be scored, which can avoid the subjectivity of manually formulating rules and can achieve more comprehensive and efficient data quality assessment, thereby improving the accuracy and efficiency of subsequent data filtering.

[0221] Figure 8 A structural block diagram of a data quality scoring device according to another exemplary embodiment of the present disclosure is shown; the data quality scoring device of this embodiment can be configured in an electronic device, and this embodiment does not limit the type of electronic device, which can be any type of device suitable for implementation, such as a terminal device or a server, etc.

[0222] The scoring data determination module 210, the update parameter acquisition module 220, the sensitive information determination module 230 and the quality score determination module 240 are the same as those described above. Figure 7 The functions of the score data determination module 110, the update parameter acquisition module 120, the sensitive information determination module 130 and the quality score determination module 140 in the illustrated embodiment are the same and will not be described in detail herein.

[0223] like Figure 8 As shown, the update parameter acquisition module 220 may include:

[0224] A forward propagation calculation unit 221 is used to perform forward propagation calculation based on each piece of the to-be-scored data and each preset benchmark model to obtain a model prediction value;

[0225] A loss function construction unit 222, configured to construct a loss function based on the difference between the model prediction value and the actual label value;

[0226] The back propagation unit 223 is used to perform back propagation based on the gradient of the loss function with respect to the parameters, the preset learning rate and the initial parameters of the model to obtain the model update parameters of each preset benchmark model corresponding to each of the data to be scored.

[0227] In some embodiments, the above-mentioned model initial parameters and the model update parameters may both be parameters of the target key layer, and the target key layer may be a key layer that needs to be monitored and is pre-specified in each preset reference model.

[0228] In some embodiments, the parameters of the target key layer may include a weight matrix and a bias vector of the target key layer.

[0229] In some embodiments, the sensitive information determination module 230 may include:

[0230] A variable determination unit 231 is used to determine a model parameter change amount of each preset benchmark model corresponding to each of the data to be scored based on the model initial parameters of each preset benchmark model and the model update parameters of each preset benchmark model corresponding to each of the data to be scored;

[0231] The vector construction unit 232 is used to construct a sensitivity feature vector based on the model parameter changes of the multiple preset benchmark models corresponding to each of the data to be scored, and the sensitivity feature vector is used to characterize the comprehensive sensitivity of the multiple preset benchmark models to each of the data to be scored.

[0232] In some embodiments, the above-mentioned variable determination unit 231 can also be used to determine the weight change matrix of each preset benchmark model corresponding to each data to be scored for each layer of each preset benchmark model based on the initial weight matrix and the updated weight matrix corresponding to each data to be scored, wherein the initial weight matrix is ​​the weight matrix of each preset benchmark model before training, and the updated weight matrix is ​​the weight matrix of each preset benchmark model after training.

[0233] In some embodiments, the vector construction unit 232 may also be used to:

[0234] For each piece of the data to be scored, respectively determine the average value of the model weight change matrix of each layer of each preset benchmark model;

[0235] The sensitivity feature vector is determined based on a weighted sum of average values ​​of model weight change matrices of each layer of the multiple preset reference models.

[0236] In some embodiments, the quality score determination module 240 may include:

[0237] An index calculation unit 241, configured to calculate a plurality of preset index values ​​for the sensitivity feature vectors, wherein the plurality of preset index values ​​include at least two of a norm, a standard deviation, and a stability index value;

[0238] The score determination unit 242 is used to determine the quality score of each piece of the to-be-scored data based on the weighted sum of the multiple preset indicator values.

[0239] In some embodiments, the above apparatus may further include a data filtering module 250;

[0240] The data filtering module 250 may include:

[0241] The sorting unit 251 is used to sort the quality scores of the data in the data set in descending order to obtain a sorting result.

[0242] The filtering unit 252 is used to filter out the data with a preset ratio threshold in the sorting result to obtain filtered data.

[0243] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can refer to the partial description of the method embodiments. The device embodiments described above are only schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the disclosed solution. A person of ordinary skill in the art can understand and implement it without paying any creative work.

[0244] Fig. 9 1 is a schematic diagram of a structure of an electronic device according to an exemplary embodiment of the present disclosure. The electronic device may be, for example, a mobile phone, a computer, a digital broadcast terminal, a message transceiver device, a game console, a tablet device, a personal digital assistant, a server, a smart home appliance, etc. Fig. 9At the hardware level, the electronic device includes a processor, an internal bus, a network interface, a memory, and a non-volatile memory, and of course may also include hardware required for other services. The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it, forming an inspection device for the grayscale environment at the logical level. Of course, in addition to the software implementation, the present disclosure does not exclude other implementation methods, such as logic devices or a combination of software and hardware, etc., that is to say, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0245] The embodiment of the present disclosure also provides a computer-readable storage medium on which a computer program is stored. When the program is processed by a processor, the above-mentioned data quality scoring method is implemented.

[0246] The computer-readable storage medium may be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc., and the present disclosure is not limited thereto.

[0247] The embodiment of the present disclosure also provides a computer program product, on which a computer program is stored, and when the program is processed by a processor, the above-mentioned data quality scoring method is implemented.

[0248] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. The present disclosure is intended to cover any variations, uses or adaptations of the present disclosure, which follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The description and examples are to be considered exemplary only, and the true scope and spirit of the present disclosure are indicated by the claims of the present disclosure.

[0249] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A data quality scoring method, characterized in that: The method comprises: In response to obtaining the data set, each piece of data in the data set is used as data to be scored; Based on each of the data to be scored, a plurality of preset benchmark models are trained respectively to obtain model update parameters of each preset benchmark model corresponding to each of the data to be scored, wherein the scales of the plurality of preset benchmark models are different; Determining the comprehensive sensitivity of the plurality of preset benchmark models to each of the data to be scored based on the model initial parameters of each of the preset benchmark models and the model update parameters of each of the preset benchmark models corresponding to each of the data to be scored, wherein the model initial parameters include the initial parameters of the preset benchmark models; A quality score for each piece of the data to be scored is determined based on the comprehensive sensitivity.

2. The method according to claim 1, characterized in that The step of training a plurality of preset benchmark models based on each piece of the data to be scored to obtain a model update parameter of each preset benchmark model corresponding to each piece of the data to be scored comprises: Perform forward propagation calculation based on each piece of data to be scored and each preset benchmark model to obtain a model prediction value; Constructing a loss function based on the difference between the model prediction value and the actual label value; Back propagation is performed based on the gradient of the loss function with respect to the parameters, the preset learning rate, and the initial parameters of the model to obtain model update parameters of each preset benchmark model corresponding to each piece of the data to be scored.

3. The method according to claim 1, characterized in that The model initial parameters and the model update parameters are parameters of the target key layer, and the target key layer is a key layer that needs to be monitored and is pre-specified in each preset benchmark model.

4. The method according to claim 3, characterized in that The parameters of the target key layer include a weight matrix and a bias vector of the target key layer.

5. The method according to claim 1, characterized in that The determining, based on the model initial parameters of each preset benchmark model and the model update parameters of each preset benchmark model corresponding to each piece of the data to be scored, the comprehensive sensitivity of the multiple preset benchmark models to each piece of the data to be scored comprises: Determine the model parameter change amount of each preset benchmark model corresponding to each of the data to be scored based on the model initial parameters of each preset benchmark model and the model update parameters of each preset benchmark model corresponding to each of the data to be scored; A sensitivity feature vector is constructed based on the model parameter changes of the multiple preset benchmark models corresponding to each of the data to be scored, and the sensitivity feature vector is used to characterize the comprehensive sensitivity of the multiple preset benchmark models to each of the data to be scored.

6. The method according to claim 5, characterized in that The determining the model parameter change amount of each preset benchmark model corresponding to each piece of the data to be scored based on the model initial parameters of each preset benchmark model and the model update parameters of each preset benchmark model corresponding to each piece of the data to be scored includes: For each layer of each preset benchmark model, the weight change matrix of each preset benchmark model corresponding to each data to be scored is determined based on the initial weight matrix and the updated weight matrix corresponding to each data to be scored, wherein the initial weight matrix is ​​the weight matrix of each preset benchmark model before training, and the updated weight matrix is ​​the weight matrix of each preset benchmark model after training.

7. The method according to claim 5, characterized in that The constructing a sensitivity feature vector based on the model parameter change amount corresponding to each of the to-be-scored data based on the multiple preset benchmark models includes: For each piece of the data to be scored, respectively determine the average value of the model weight change matrix of each layer of each preset benchmark model; The sensitivity feature vector is determined based on a weighted sum of average values ​​of model weight change matrices of each layer of the multiple preset reference models.

8. The method according to claim 5, characterized in that Determining the quality score of each piece of the to-be-scored data based on the comprehensive sensitivity includes: Calculating a plurality of preset index values ​​for the sensitivity feature vectors respectively, wherein the plurality of preset index values ​​include at least two of a norm, a standard deviation, and a stability index value; The quality score of each piece of the to-be-scored data is determined based on a weighted sum of the multiple preset indicator values.

9. The method according to claim 1, characterized in that: The method further comprises: Sorting the quality scores of each piece of data in the data set in descending order to obtain a sorting result; The data exceeding the preset ratio threshold in the sorting results are filtered out to obtain filtered data.

10. A data quality scoring device, characterized in that: The device comprises: A scoring data determination module, configured to, in response to acquiring a data set, use each piece of data in the data set as data to be scored; An update parameter acquisition module, used for training a plurality of preset benchmark models based on each piece of the data to be scored, respectively, to obtain a model update parameter of each preset benchmark model corresponding to each piece of the data to be scored, wherein the scales of the plurality of preset benchmark models are different; A sensitive information determination module, configured to determine the comprehensive sensitivity of the plurality of preset benchmark models to each of the data to be scored based on the model initial parameters of each of the preset benchmark models and the model update parameters of each of the preset benchmark models corresponding to each of the data to be scored, wherein the model initial parameters include the initial parameters of the preset benchmark models; A quality score determination module is used to determine the quality score of each piece of the data to be scored based on the comprehensive sensitivity.

11. An electronic device, characterized in that: The electronic device comprises: processor; memory for storing computer programs; Wherein, the processor is configured to implement the data quality scoring method described in any one of claims 1 to 9 when executing the computer program.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is processed by a processor, the data quality scoring method described in any one of claims 1 to 9 is implemented.

13. A computer program product having a computer program stored thereon, characterized in that: When the computer program is processed by a processor, the data quality scoring method according to any one of claims 1 to 9 is implemented.

Images