Data screening method, device and equipment

Abstract

The invention provides a data screening method, apparatus and device. The method comprises the steps of performing feature extraction on each sample data in to-be-evaluated data; calculating and determining representative parameters of the sample data based on the extracted features, wherein the representative parameters are related to the confidence of the features; calculating and determining weights of the features based on prior data; calculating a Sharpley value of the sample data based on the representative parameter, the weight and an association relationship among features in the sample data; determining a quality evaluation result of the sample data based on the Sharpley value of each piece of sample data; and screening the to-be-evaluated data based on the quality evaluation result of each piece of sample data to obtain high-quality data. The data screening method provided by the invention is used for carrying out rapid and effective screening operation with small processing load on the data to obtain different quality data.

Description

Technical Field

[0001] The present invention belongs to the field of computer technology, and in particular relates to a data screening method, device and equipment. Background Art

[0002] With the advancement of machine learning, data quality has attracted more and more attention from researchers. High-quality data will save a lot of resources for model learning. On the contrary, using low-quality data for model training will lead to degraded performance. Therefore, the process of selecting high-quality data while discarding low-quality data is a key step for the model to learn effectively and efficiently. The main methods for selecting high-quality samples and excluding low-quality samples from a dataset usually include two steps. First, researchers evaluate the samples. Second, researchers rank the samples according to their value and filter out high-value samples. Samples with incorrect labels or noise are usually regarded as low-quality data. However, there are many types of noise, and these methods cannot distinguish them all, especially in medical scenarios. Clinicians may inadvertently record incorrect information when filling out case reports. Pathologists may inadvertently overstain a microscopic section while processing specimens. Medical equipment may produce blurred images when operated by technicians, especially in violation of established regulations or protocols. The above data are considered low-quality data with noise. In addition, due to the inherent complexity of the field, existing evaluation methods are rarely used in the medical field.

[0003] One of the main methods for evaluating samples is to calculate the Shapley value of the sample. This is the expected value of the utility value traditionally generated by the utility function. The core of the utility function is to quantify the difference between the output of a model after learning a specific sample and the output before learning the sample. Many existing evaluation methods, such as TMCShapley and GTB Shapley, estimate the expected value of specific data by repeatedly sampling the utility value of the data. These methods require repeated training of the model. However, these models have strong robustness. Evaluation methods based on these repeatedly trained models may ignore the details of the noise. Therefore, these methods are not general when evaluating data, especially in the medical field. It has been verified experimentally that the model is more sensitive to changes in the relative position of pixels than to changes in pixel color. It can be seen that the reason for this phenomenon is that the model uses relevant information to correct erroneous information. These evaluation methods have relatively poor performance when used to perform abnormal sample detection tasks with special noise. Unfortunately, there are many special noises in medicine, and the model learns from their relevant information and corrects them. C-reactive protein is associated with leukocytes in various pathophysiological processes such as inflammatory response or tumor microenvironment. In the context of microscopic sections, if they are overstained, the shapes of cells within the sections usually remain unchanged. Features extracted by the model from overstained microscopic sections usually show little difference compared to features extracted from normally stained microscopic sections. In addition, the evaluation method of repeatedly training models consumes a lot of resources and time. Summary of the invention

[0004] The technical problem to be solved by the present invention is to provide a data screening method, device and equipment for performing fast and effective screening operations on data with a small processing load to obtain data of different qualities.

[0005] The present invention provides a data screening method, comprising: Extracting features from each sample data in the data to be evaluated, wherein the data to be evaluated includes at least image data and / or text data, and the data to be evaluated is used for analysis, reference, and prediction and evaluation of a specified event, and the extracted features are related to the specified event; Determining representative parameters of the sample data based on the extracted features, wherein the representative parameters are related to the confidence of the features; Determining the weight of the feature based on prior data calculation; Calculate the Shapley value of the sample data based on the representative parameter, the weight and the correlation between the features in the sample data; Determine a quality assessment result of the sample data based on the Shapley value of each of the sample data; The data to be evaluated are screened based on the quality evaluation results of each of the sample data to obtain high-quality data.

[0006] In some embodiments, extracting features from each sample data in the data to be evaluated includes: Encoding and indexing each sample data in the data to be evaluated; Perform feature extraction on the sample data under each index.

[0007] In some embodiments, the method further comprises: Analyze and determine the correlation between each of the features; Based on the association relationship, target features that have an impact on each other are determined.

[0008] In some embodiments, the calculating and determining the representative parameters of the sample data based on the extracted features includes: The features in each sample are compared with the features in other samples, and the representative parameters of each sample data are calculated by the following formula: : y i Represents the label of the sample data with index i, y j Represents the label of the sample data with index j, v j (i) indicates the degree to which the i-th sample data confirms its representativeness to the j-th sample data, represents the abnormal probability of sample data, is the indicator function symbol.

[0009] In some embodiments, the representative degree indicates that the confidence level of the j-th sample data is determined by the i-th sample data according to the distance between the two, and the formula is expressed as: β j is a hyperparameter, indicating the balance point between normal sample data and abnormal sample data, m indicates the number of features of sample data, and d i,k represents the kth feature of the i-th sample data, d j,k Represents the kth feature of the jth sample data.

[0010] In some embodiments, the method further comprises: The representative parameters of all the sample data are normalized to obtain the representative parameters p corresponding to the sample data. j : Said Indicates the trustworthiness level of the sample data, Representation and The label of the corresponding sample data, ⊙ represents the Hadamard matrix calculator.

[0011] In some embodiments, the calculating the Shapley value of the sample data based on the representative parameter, the weight, and the correlation between the features in the sample data includes: Based on the representative parameters, weights and the correlation between the features in the sample data, the Shapley value of the sample is calculated by combining the following formula: : Said is a constant obtained by summing the weights of all features, q is a hyperparameter indicating the degree of association between features, and T s Features with associated relationships.

[0012] In some embodiments, the screening of the data to be evaluated based on the quality evaluation results of each of the sample data to obtain high-quality data includes: sorting each sample data based on the quality assessment result of each sample data; Screening all sample data in the sequence based on the target use of the sample data to determine high-quality data, usable data, and low-quality data; The method further comprises: When the target use is model training, extract part of the available data and fuse it with high-quality data to obtain fused data with diversity; The model is trained using high-quality data and fused data.

[0013] Another embodiment of the present invention also provides a data screening device, including: A feature extraction module is used to extract features from each sample data in the data to be evaluated, wherein the data to be evaluated includes at least image data and / or text data, and the data to be evaluated is used for analysis, reference, prediction and evaluation of a specified event, and the extracted features are related to the specified event; A first calculation module, used to calculate and determine a representative parameter of the sample data according to the extracted features, wherein the representative parameter is related to the confidence of the feature; A second calculation module, used to calculate and determine the weight of the feature according to prior data; A third calculation module, used for calculating the Shapley value of the sample data according to the representative parameter, the weight and the correlation relationship between the features in the sample data; A determination module, configured to determine a quality assessment result of the sample data according to the Shapley value of each sample data; The screening module is used to screen the data to be evaluated according to the quality evaluation results of each sample data to obtain high-quality data.

[0014] Another embodiment of the present invention further provides an electronic device, comprising: one or more processors; a memory configured to store one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the data screening method as described in any one of the embodiments above.

[0015] The beneficial effect of the present invention is that the use of prior data to assist in solving the Shapley value of sample data can greatly save resources and reduce data processing load and processing time. Moreover, by introducing prior data, there is no need to perform complex utility value calculations and repeated sampling on sample data when solving the Shapley value, which further reduces the processing load and improves processing efficiency, so that when screening data, the screening process can be completed more quickly and effectively.

[0016] Other features and advantages of the present application will be described in the following description, and partly become apparent from the description, or be understood by practicing the present application. The purpose and other advantages of the present application can be realized and obtained by the structures specifically pointed out in the written description, claims, and drawings.

[0017] The technical solution of the present application is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings in the following description are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0019] Figure 1 Schematic diagram of the flow of the data screening method in an embodiment of the present invention.

[0020] Figure 2 FIG. 4 is a flow chart of a data screening method in another embodiment of the present invention.

[0021] Figure 3 Schematic diagram of the application process of the data screening method in an embodiment of the present invention.

[0022] Figure 4 4 is a structural block diagram of a data screening device in an embodiment of the present invention. DETAILED DESCRIPTION

[0023] The specific embodiments of the present invention are described in detail below with reference to the accompanying drawings, but are not intended to limit the present invention.

[0024] It should be understood that various modifications may be made to the embodiments disclosed herein. Therefore, the following description should not be considered as limiting, but merely as an example of an embodiment. Other modifications within the scope of the present disclosure will occur to those skilled in the art.

[0025] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate embodiments of the present disclosure and, together with the general description of the present disclosure given above and the detailed description of the embodiments given below, serve to explain the principles of the present disclosure.

[0026] These and other characteristics of the invention will become apparent from the following description of a preferred form of embodiment given as a non-limiting example, with reference to the accompanying drawings.

[0027] It should also be understood that, although the invention has been described with reference to some specific examples, those skilled in the art will be able to realize many other equivalent forms of the invention that have the characteristics recited in the claims and are therefore within the scope of protection defined thereby.

[0028] The above and other aspects, features and advantages of the present disclosure will become more apparent from the following detailed description when taken in conjunction with the accompanying drawings.

[0029] Specific embodiments of the present disclosure are described hereinafter with reference to the accompanying drawings; however, it should be understood that the disclosed embodiments are merely examples of the present disclosure, which may be implemented in a variety of ways. Well-known and / or repeated functions and structures are not described in detail to avoid obscuring the present disclosure with unnecessary or redundant details. Therefore, the specific structural and functional details disclosed herein are not intended to be limiting, but merely serve as a basis and representative basis for the claims to teach those skilled in the art to use the present disclosure in a variety of ways with substantially any suitable detailed structure.

[0030] This specification may use the phrases "in one embodiment," "in another embodiment," "in a further embodiment," or "in other embodiments," all of which may refer to one or more of the same or different embodiments according to the present disclosure.

[0031] The embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0032] One of the main methods for evaluating samples is to calculate the Shapley value of the sample. This is the expected value of the utility value traditionally generated by the utility function. The core of the utility function is to quantify the difference between the output of a model after learning a specific sample and the output before learning the sample. Many existing evaluation methods, such as TMCShapley and GTB Shapley, estimate the expected value of specific data by repeatedly sampling the utility value of the data. However, these methods all require repeated training of the model. When evaluating these repeatedly trained models, the details of the noise may often be ignored. Therefore, these methods are not general when evaluating data, especially in the medical field. This model is more sensitive to changes in the relative position of pixels than to changes in pixel color. The reason for this phenomenon is that the model uses relevant information to correct erroneous information. Therefore, these evaluation methods have relatively poor performance when used to perform abnormal sample detection tasks with special noise. However, medical data in the medical field generally have various special noises, so the model application effect is poor. For example, in various pathological and physiological processes such as inflammatory response or tumor microenvironment, C-reactive protein is related to white blood cells. In the context of microscopic sections, if they are overstained, the shapes of cells within the sections usually remain unchanged. Features extracted by the model in overstained microscopic sections usually show little difference compared to features extracted from normally stained microscopic sections. In addition, the evaluation method of repeatedly training the model consumes a lot of resources and time. All these phenomena indicate that the models in the existing schemes cannot be applied to data with special noise, such as medical data.

[0033] To solve the above problems, Figure 1 As shown, an embodiment of the present invention provides a data screening method, comprising: S1: extracting features from each sample data in the data to be evaluated, wherein the data to be evaluated includes at least image data and / or text data, and the data to be evaluated is used for analysis, reference, prediction and evaluation of a specified event, and the extracted features are related to the specified event; S2: Determine representative parameters of the sample data based on the extracted features, where the representative parameters are related to the confidence of the features; S3: Determine the weight of the feature based on the prior data calculation; S4: Calculating the Shapley value of the sample data based on the representative parameter, the weight and the correlation between the features in the sample data; S5: Determine the quality assessment result of the sample data based on the Shapley value of each of the sample data; S6: Screening the data to be evaluated based on the quality evaluation results of each sample data to obtain high-quality data.

[0034] The specific content and field involved in the specified event are not limited. The above method can process the relevant data of any event in any field to obtain more reference-oriented high-quality data for the event. The data to be evaluated in this embodiment is medical data, and of course it can also be data in other fields, which is not specific. The medical data can be, for example, but not limited to, medical imaging data, diagnostic treatment data, etc. The scheme in this embodiment is to first perform a quality assessment on the data to be evaluated, and then screen the data according to the assessment results. The screened data can at least eliminate low-quality data and identify high-quality data. When performing data quality assessment, the scheme selected in this embodiment is also to calculate the Shapley value, but this embodiment extracts features, calculates representative parameters, and introduces prior data to determine the weights of each sample data feature, and finally calculates the Shapley value based on the representative parameters, weights and the correlation between features. This embodiment can greatly save resources and reduce data processing load and processing time by utilizing prior data to assist in solving the Shapley value of sample data. Furthermore, by introducing prior data, there is no need to perform complex utility value calculations and repeated sampling on sample data when solving the Shapley value, which further reduces the processing load and improves processing efficiency, so that when screening data, the screening process can be completed more quickly and effectively.

[0035] Further, based on the above content, it can be known that in the process of solving the Shapley value, the solution of this embodiment actually redefines the utility function of the sample. Its utility value is generated by prior data, which completely avoids repeated training of the model. Since the definition of the Shapley value requires repeated sampling of utility values, this embodiment combines the synergy function to propose a new method for calculating the Shapley value through the utility function, which is more efficient than previous methods. In addition, the method of this embodiment does not need to manually create a verification set to calculate the utility function, so it can significantly improve the data processing efficiency while ensuring the validity of the data.

[0036] In one embodiment, if Figure 2 As shown, in order to ensure accurate feature management, this embodiment preprocesses the sample data, such as extracting features from each sample data in the data to be evaluated, including: S7: Encode and index each sample data in the data to be evaluated; S8: Extract features from the sample data under each index.

[0037] That is, the sample data is first encoded and indexed, and then each sample data can be retrieved through the index and features extracted.

[0038] After extracting the features, the system will evaluate their quality, so the utility function is needed to calculate the Shapley value. The utility function is different for different scenarios, for example: Take the scenario where features interact with each other, such as in supervised machine learning, where data usually consists of multiple features and one label. The importance of these features varies, and the same feature may have different values ​​for different labels. To represent the characteristics of sample data, its element d i represents the i-th feature of the data. When the value of is equal to the sum of the values ​​of its constituent elements, the following utility function assumption can be obtained: .z i It corresponds to d i The weight of the i-th feature of . p represents the confidence of data j, which can be explained by real-world scenarios. Data from different institutions should have different confidence levels. For example, data collected by authoritative institutions are more credible, and such data should intuitively have a higher confidence level p. and any feature d in the label y i , S is the feature set, it can be inferred that the coordination function ω(S) is: .

[0039] When evaluating data based on examples, data from an authoritative source should carry more weight than data from an inexperienced source. In addition, rare cases should receive more attention. Therefore, the confidence level p should vary across different examples within a group. Use To represent the instance set, a The group has n instances and the instance is independent of the features. The utility function of the group can be defined as: , represents a constant value, m represents the number of features of the sample data, For instance set Any instance of g j , we can infer that the cooperative function is: .

[0040] Taking the scenario where features interact with each other as an example, assume that one feature in a data has the same degree of influence on another feature. That is, in addition to the utility value, the feature set will have a further impact on the instance during the evaluation process. At the same time, their influence is equal, and their utility function can be defined as: .

[0041] When evaluating interaction data based on instances, assuming that confidence is independent of features, then

[0042] The second term in the above equation has a synergistic effect.

[0043] When evaluating interactive data under limited conditions based on features, that is, some features in the sample data have mutual influence, while some features are independent, for this scenario, use It represents a set of features that are expected to influence each other to a certain extent q. There are |T in the feature set S. s | features influence each other, and the total feature set D={d1, d2, .., d m} and the influence set T s ,Label y The relationship is as follows: , the utility function is .

[0044] When evaluating interaction data under limited conditions based on examples, , total feature set and |T s |The features interact with each other in the set S, where . The utility function of this group can be defined as: .

[0045] Based on the above content about utility functions in different scenarios, it can be seen that the use of the synergy function in this embodiment simplifies the calculation process of the Shapley value, improves the calculation efficiency, and saves resources. Therefore, the simplified solution method of the Shapley value only requires the parameter (sum of feature weights), p j (representative parameter, related to credibility) and q (degree of association / influence between features). When calculating the Shapley value of a single case, the parameter p j plays a more important role than other parameters. In this embodiment, the parameter and q are given by known prior data.

[0046] Exemplarily, the method further includes: S9: Analyze and determine the correlation between each of the features; S10: Determine target features that have an impact on each other based on the association relationship.

[0047] That is, the feature set with correlation is determined by analysis, and the correlation degree q between the features in the set is determined by calculation.

[0048] Prior data on feature weights and the degree of mutual influence between features are easy to obtain, and the credibility of the data is jFor data sets from different institutions, there are different levels of confidence. However, it is often challenging to determine the exact source of a data set, so this parameter is relatively quicker to obtain by actual solution. The farther a single data point is from its dataset, the more likely it is to be abnormal, that is, the more likely it is to be of low quality. Therefore, the parameter p j should be negatively correlated with its distance from other data points. Figure 3 As shown, in this embodiment, when calculating and determining the representative parameters of the features based on the extracted features, the algorithm proposed is: S11: Compare the features in each sample with the features in other samples, and calculate the representative parameters of each sample data by the following formula: : y i Represents the label of the sample data with index i, y j Represents the label of the sample data with index j, v j (i) indicates the degree to which the i-th sample data confirms its representativeness to the j-th sample data, represents the abnormal probability of sample data, is the indicator function symbol.

[0049] The representative degree indicates that the credibility level of the j-th sample data is determined by the i-th sample data according to the distance between the two, and its formula is expressed as: β j is a hyperparameter, indicating the balance point between normal sample data and abnormal sample data, m indicates the number of features of sample data, and d i,k represents the kth feature of the i-th sample data, d j,k Represents the kth feature of the jth sample data.

[0050] In this embodiment, and β j It can be obtained through prior knowledge. Given the estimated proportion of noise data in the dataset ε and the estimated proportion of unlabeled data in the dataset δ, the calculation formula of the two parameters can be expressed as: in, Represents j and the set of Percentile distance. The mislabeled data can also be considered as noise data of the class it is incorrectly assigned to.

[0051] Furthermore, the method further comprises: S12: Normalize the representative parameters of all the sample data to obtain the representative parameter p corresponding to the sample data. j : Said Indicates the trustworthiness level of the sample data, Representation and The label of the corresponding sample data, ⊙ represents the Hadamard matrix calculator.

[0052] The calculating the Shapley value of the sample data based on the representative parameter, the weight and the correlation between the features in the sample data includes: S13: Based on the representative parameters, weights and the correlation between the features in the sample data, the Shapley value of the sample is calculated by combining the following formula: : Said is a constant obtained by summing the weights of all features, q is a hyperparameter indicating the degree of association between features, and T s Features with associated relationships.

[0053] After determining the Shapley value of each sample data, the system can screen the sample data. The data to be evaluated is screened based on the quality evaluation results of each sample data to obtain high-quality data, including: S14: sorting each sample data based on the quality assessment result of each sample data; S15: Screening all sample data in the sequence based on the target use of the sample data to determine high-quality data, usable data, and low-quality data; The method further comprises: S16: When the target use is model training, extract part of the available data and fuse it with the high-quality data to obtain fused data with diversity; S17: Train the model using high-quality data and fused data.

[0054] For example, when preparing training data for a medical model, or when classifying or predicting a disease based on a medical model, the method of the above embodiment can be used to screen the medical data, and then put them into different uses. Assuming that during an imaging examination in a medical scenario, some patients may become restless and uncooperative due to their condition, resulting in low-quality imaging data for the patient. If used for learning models such as deep neural networks, it will seriously affect the convergence speed and performance of the model. Therefore, before model training, or when predicting or classifying a disease, it is necessary to evaluate the collected data, discard low-quality data, and reduce its impact on model training or model application analysis. In addition, in order to improve sample diversity, especially when training a model, high-quality data and available data can be fused, so that even if more low-quality data is discarded, the diversity of the samples will not be affected.

[0055] In addition, since low-quality data is discarded, taking model training as an example, this can improve the training efficiency and quality of the model and improve the model accuracy. For example, there is no need to repeatedly train the model in large quantities. The samples to be evaluated can be directly used as training data and verification sets without manual screening. The overall data samples are more in line with the real data distribution, which is more conducive to model training.

[0056] like Figure 4 As shown, another embodiment of the present invention also provides a data screening device 100, including: A feature extraction module is used to extract features from each sample data in the data to be evaluated, wherein the data to be evaluated includes at least image data and / or text data, and the data to be evaluated is used for analysis, reference, prediction and evaluation of a specified event, and the extracted features are related to the specified event; A first calculation module, used to calculate and determine a representative parameter of the sample data according to the extracted features, wherein the representative parameter is related to the confidence of the feature; A second calculation module, used to calculate and determine the weight of the feature according to prior data; A third calculation module, used for calculating the Shapley value of the sample data according to the representative parameter, the weight and the correlation relationship between the features in the sample data; A determination module, configured to determine a quality assessment result of the sample data according to the Shapley value of each sample data; The screening module is used to screen the data to be evaluated according to the quality evaluation results of each sample data to obtain high-quality data.

[0057] In some embodiments, extracting features from each sample data in the data to be evaluated includes: Encoding and indexing each sample data in the data to be evaluated; Perform feature extraction on the sample data under each index.

[0058] In some embodiments, the method further comprises: Analyze and determine the correlation between each of the features; Based on the association relationship, target features that have an impact on each other are determined.

[0059] In some embodiments, the calculating and determining the representative parameters of the sample data based on the extracted features includes: The features in each sample are compared with the features in other samples, and the representative parameters of each sample data are calculated by the following formula: : y i Represents the label of the sample data with index i, y j Represents the label of the sample data with index j, v j (i) indicates the degree to which the i-th sample data confirms its representativeness to the j-th sample data, represents the abnormal probability of sample data, is the indicator function symbol.

[0060] In some embodiments, the representative degree indicates that the confidence level of the j-th sample data is determined by the i-th sample data according to the distance between the two, and the formula is expressed as: β j is a hyperparameter, indicating the balance point between normal sample data and abnormal sample data, m indicates the number of features of sample data, and d i,k represents the kth feature of the i-th sample data, d j,k Represents the kth feature of the jth sample data.

[0061] In some embodiments, the method further comprises: The representative parameters of all the sample data are normalized to obtain the representative parameters p corresponding to the sample data. j : Said Indicates the trustworthiness level of the sample data, Representation and The label of the corresponding sample data, ⊙ represents the Hadamard matrix calculator.

[0062] In some embodiments, the calculating the Shapley value of the sample data based on the representative parameter, the weight, and the correlation between the features in the sample data includes: Based on the representative parameters, weights and the correlation between the features in the sample data, the Shapley value of the sample is calculated by combining the following formula: : Said is a constant obtained by summing the weights of all features, q is a hyperparameter indicating the degree of association between features, and T s Features with associated relationships.

[0063] In some embodiments, the screening of the data to be evaluated based on the quality evaluation results of each of the sample data to obtain high-quality data includes: sorting each sample data based on the quality assessment result of each sample data; Screening all sample data in the sequence based on the target use of the sample data to determine high-quality data, usable data, and low-quality data; The device also includes: A fusion module, used for extracting part of the available data and fusing it with high-quality data to obtain fused data with diversity when the target use is model training; The training module is used to train the model using high-quality data and fused data.

[0064] Another embodiment of the present invention further provides an electronic device, comprising: one or more processors; a memory configured to store one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the data screening method as described in any one of the embodiments above.

[0065] Furthermore, an embodiment of the present invention also provides a storage medium on which a computer program is stored, and when the program is executed by a processor, the data screening method described above is implemented. It should be understood that each solution in this embodiment has the corresponding technical effect in the above method embodiment, which will not be repeated here.

[0066] Furthermore, an embodiment of the present invention also provides a computer program product, which is tangibly stored on a computer-readable medium and includes computer-readable instructions, which, when executed, cause at least one processor to execute a network attack defense method such as that in the embodiment described above.

[0067] It should be noted that the computer storage medium of the present invention may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, system or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access storage medium (RAM), a read-only storage medium (ROM), an erasable programmable read-only storage medium (EPROM or flash memory), an optical fiber, a portable compact disk read-only storage medium (CD-ROM), an optical storage medium, a magnetic storage medium, or any suitable combination of the above. In the present invention, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, system or device. In the present invention, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can send, propagate or transmit a program configured to be used by or in conjunction with an instruction execution system, system or device. The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, antenna, optical cable, RF, etc., or any suitable combination of the above.

[0068] In addition, it should be understood by those skilled in the art that the embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage and optical storage, etc.) containing computer-usable program codes.

[0069] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1A process or multiple processes and / or boxes Figure 1 A system that specifies the functions of a box or multiple boxes.

[0070] These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory produce a product including an instruction system, which is implemented in the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0071] A person skilled in the art should understand that the discussion of any of the above embodiments is merely illustrative and is not intended to imply that the scope of protection of the present application is limited to these examples. In line with the concept of the present application, the technical features in the above embodiments or different embodiments may be combined, the steps may be implemented in any order, and there are many other variations of different aspects of one or more embodiments of the present application as described above, which are not provided in detail for the sake of simplicity.

Claims

1. A data screening method, characterized in that: include: Extracting features from each sample data in the data to be evaluated, wherein the data to be evaluated includes at least image data and / or text data, and the data to be evaluated is used for analysis, reference, and prediction and evaluation of a specified event, and the extracted features are related to the specified event; Determining representative parameters of the sample data based on the extracted features, wherein the representative parameters are related to the confidence of the features; Determining the weight of the feature based on prior data calculation; Calculate the Shapley value of the sample data based on the representative parameter, the weight and the correlation between the features in the sample data; Determine a quality assessment result of the sample data based on the Shapley value of each of the sample data; The data to be evaluated are screened based on the quality evaluation results of each of the sample data to obtain high-quality data.

2. The data screening method according to claim 1, characterized in that: The feature extraction of each sample data in the data to be evaluated includes: Encoding and indexing each sample data in the data to be evaluated; Perform feature extraction on the sample data under each index.

3. The data screening method according to claim 1, characterized in that: The method further comprises: Analyze and determine the correlation between each of the features; Based on the association relationship, target features that have an impact on each other are determined.

4. The data screening method according to claim 1, characterized in that: The step of calculating and determining representative parameters of the sample data based on the extracted features includes: The features in each sample are compared with the features in other samples, and the representative parameters of each sample data are calculated by the following formula: : y i Represents the label of the sample data with index i, y j Represents the label of the sample data with index j, v j (i) indicates the degree to which the i-th sample data confirms its representativeness to the j-th sample data, represents the abnormal probability of sample data, is the indicator function symbol.

5. The data screening method according to claim 4, characterized in that: The representative degree means that the credibility level of the j-th sample data is determined by the i-th sample data according to the distance between the two, and its formula is expressed as: β j is a hyperparameter, indicating the balance point between normal sample data and abnormal sample data, m indicates the number of features of sample data, and d i,k represents the kth feature of the i-th sample data, d j,k Represents the kth feature of the jth sample data.

6. The data screening method according to claim 4, characterized in that: The method further comprises: The representative parameters of all the sample data are normalized to obtain the representative parameters p corresponding to the sample data. j : Said Indicates the trustworthiness level of the sample data, Representation and The label of the corresponding sample data, ⊙ represents the Hadamard matrix calculator.

7. The data screening method according to claim 6, characterized in that: The calculating the Shapley value of the sample data based on the representative parameter, the weight and the correlation between the features in the sample data includes: Based on the representative parameters, weights and the correlation between the features in the sample data, the Shapley value of the sample is calculated by combining the following formula: : Said is a constant obtained by summing the weights of all features, q is a hyperparameter indicating the degree of association between features, and T s Features with associated relationships.

8. The data screening method according to claim 1, characterized in that: The step of screening the data to be evaluated based on the quality evaluation results of each sample data to obtain high-quality data includes: sorting each sample data based on the quality assessment result of each sample data; Screening all sample data in the sequence based on the target use of the sample data to determine high-quality data, usable data, and low-quality data; The method further comprises: When the target use is model training, extract part of the available data and fuse it with high-quality data to obtain fused data with diversity; The model is trained using high-quality data and fused data.

9. A data screening device, characterized in that: include: A feature extraction module is used to extract features from each sample data in the data to be evaluated, wherein the data to be evaluated includes at least image data and / or text data, and the data to be evaluated is used for analysis, reference, prediction and evaluation of a specified event, and the extracted features are related to the specified event; A first calculation module, used to calculate and determine a representative parameter of the sample data according to the extracted features, wherein the representative parameter is related to the confidence of the feature; A second calculation module, used to calculate and determine the weight of the feature according to prior data; A third calculation module, used for calculating the Shapley value of the sample according to the representative parameter, the weight and the correlation relationship between the features in the sample data; A determination module, configured to determine a quality assessment result of the sample data according to the Shapley value of each sample data; The screening module is used to screen the data to be evaluated according to the quality evaluation results of each sample data to obtain high-quality data.

10. An electronic device, characterized in that: include one or more processors; a memory configured to store one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the data screening method according to any one of claims 1 to 8.

Images