Poison data identification method, storage medium and system
By calculating attribute values and performing dimensionality reduction, feature vectors are constructed to identify poisoning data. This solves the problems of high computational overhead and decreased accuracy in detecting poisoning data in high-dimensional space, achieving efficient and stable poisoning data identification, and is suitable for various application scenarios.
Patent Information
- Application Number
- CN202511540536.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-10-27
- Publication Date
- 2026-02-06
Smart Images
Figure CN121479480A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to a method, storage medium, and system for identifying poisoned data based on attribute permutation and combination features. It belongs to the fields of machine learning security, data quality, and model maintenance technology. The goal is to identify poisoned data in model training datasets. It is applicable to scenarios such as machine learning, deep learning, and data mining that require protection against data poisoning attacks. Background Technology
[0002] With the widespread application of machine learning and deep learning in fields such as intelligent security, financial risk control, medical diagnosis, and autonomous driving, the security and reliability of training data have become crucial factors affecting model performance and stability. In recent years, data poisoning attacks have gradually become one of the major security threats faced by artificial intelligence systems. The basic principle is to inject carefully crafted malicious samples into the model training dataset, subtly introducing biases during the model training process. This causes the model to make classification errors, decrease accuracy, or even produce abnormal outputs expected by the attacker under specific triggering conditions. These attacks are characterized by high concealment, strong harm, and wide-ranging impact, posing a serious threat to the robustness and reliability of artificial intelligence systems.
[0003] Existing data poisoning detection methods can be broadly categorized into model-dependent and model-independent methods. Model-dependent methods typically require access to model parameters, gradient information, or prediction results, locating suspicious samples by analyzing gradient changes, loss curves, or output probability distributions during training. Model-independent methods, on the other hand, primarily rely on the statistical characteristics and structural information of the data, identifying anomalous records by calculating distances, densities, or similarity metrics between samples. However, these methods generally face several challenges in practical applications: model-dependent methods often fail to obtain necessary internal information in black-box scenarios or when sharing data across institutions; traditional algorithms based on statistical characteristics are susceptible to the "curse of dimensionality" in high-dimensional spaces, resulting in high computational costs and decreased detection accuracy; furthermore, many existing methods lack adaptability to different data distributions and attack methods. Summary of the Invention
[0004] The purpose of this invention is to provide a method, storage medium, and system for identifying poisoning data. It achieves effective dimensionality reduction of high-dimensional datasets through attribute value calculation, and then constructs feature vectors and feature values using global statistical features of attribute permutations and combinations. Finally, it identifies suspected poisoning data based on the distribution characteristics of the feature values. This invention does not rely on prior models, can adapt to various types of poisoning, and still has high detection accuracy and operating efficiency in high-dimensional data environments, providing a stable and reliable technical means for the security of artificial intelligence systems.
[0005] To achieve the above objectives, the present invention is implemented using the following technical solution.
[0006] On one hand, the present invention provides a method for identifying poisoning data, comprising:
[0007] Obtain the original dataset;
[0008] Each data record in the original dataset is numbered sequentially to obtain a new dataset with numbers;
[0009] The new dataset is dimensionality reduced to obtain a dimensionality-reduced dataset.
[0010] Perform feature calculations on the dimensionality-reduced dataset and sort it according to the aggregated feature values of all data records;
[0011] Poisoning data is identified based on the sorting results of the aggregated feature values of the data records.
[0012] Furthermore, the original dataset and the new dataset each include m data records, and each data record includes n attributes;
[0013] Let the domain of discourse be U = {x1, ..., x}. m}, attribute set A={a1,…,a n};
[0014] Where, x m Let a represent the m-th data record in the universe of discourse. n This represents the nth attribute in the attribute set.
[0015] Furthermore, the new dataset D' is subjected to dimensionality reduction processing to obtain the dimensionality-reduced dataset D. k ,include:
[0016] By partitioning the universe of discourse, data records with the same attribute values are grouped into the same category, resulting in the partition Π of the attribute set A on the universe of discourse U. A ={C1,…,Cg}, where Cg represents data records with completely identical attribute values in the g-th class;
[0017] Calculate the entropy H(A) of attribute set A with respect to the universe of discourse U;
[0018] Remove the column containing each attribute a∈A from the new dataset D' in turn, and calculate the attribute value V(a) for each attribute;
[0019] Sort the attribute values of each attribute from largest to smallest, and select the k attributes with the largest values to form a dimensionality-reduced attribute set A. k And generate a dimensionality-reduced dataset D k , where 1≤k≤n.
[0020] Furthermore, the entropy H(A) of the attribute set A with respect to the universe of discourse U is expressed as: ;
[0021] in, The relative frequencies are obtained by partitioning. For data records with identical attribute values, g represents the number of partitions of U based on attribute set A.
[0022] Furthermore, the formula for calculating the attribute value V(a) is: ;
[0023] Where H(A) represents the entropy of attribute set A with respect to the universe of discourse U. Let represent the entropy of attribute set A with respect to the domain U after removing attribute a.
[0024] When dealing with low-dimensional datasets, the original attribute set is preserved; when dealing with high-dimensional datasets, dimensionality reduction is used to automatically select the subset of attributes that contribute the most to poison detection, thus preserving key discriminative information while effectively reducing computational complexity and mitigating the "curse of dimensionality" problem caused by high-dimensional data.
[0025] Furthermore, feature calculations are performed on the dimensionality-reduced dataset, and the dataset is sorted according to the aggregated feature values of all data records, including:
[0026] For the dimensionality reduction dataset D k The attribute values of each data record in the dataset are combined in an unordered manner to generate a set of attribute value combinations;
[0027] Statistical Dimensionality Reduction Dataset D k The frequency mapping table is obtained by counting the number of times each attribute value combination appears in the global data.
[0028] Set an upper bound for the combinatorial order In the dimensionality reduction dataset D k Perform a global statistical analysis on all data records: for any non-empty subset and and the values of arbitrary attribute subvectors ; Calculate any combination of attribute values Obtain a count table of global attribute value combinations and their occurrence counts.
[0029] For each data record x∈U, generate a set of attribute value combinations:
[0030] ;
[0031] Where 1≤R≤k, (S,x[S]) represents the combination of attribute values indexed by S and equal to the projection of x onto S.
[0032] For any combination of attribute values (S, v) S The number of times it appears in the global dataset D is denoted as:
[0033] ;
[0034] Based on the frequency mapping table, query the occurrence count of each combination in the attribute value set, construct a feature vector, and calculate the corresponding feature value.
[0035] Arrange the global occurrence counts of each combination (S, x[S]) in C(x) in the counting table sequentially to obtain the feature vector.
[0036] ; Eigenvalues ;
[0037] Where c(S,x[S]) characterizes the co-occurrence frequency of x relative to the whole, and s(x) is its aggregation scale.
[0038] Furthermore, the processes of "generating a set of attribute value combinations" and "obtaining a frequency mapping table" can be executed in parallel or step by step.
[0039] Furthermore, based on the sorting results of the aggregated feature values of the data records, poisoning data is identified, including:
[0040] Sort all data records in ascending order by their aggregate feature values, and select the q data records with the smallest aggregate feature values as suspected poisoning data, where 1≤q≤m;
[0041] Alternatively, if the aggregated feature value is less than a given threshold, the data record corresponding to this aggregated feature value is identified as suspected poisoning data.
[0042] In a second aspect, the present invention provides a computer-readable storage medium having a computer program / instructions stored thereon, which, when executed by a processor, implements the steps of the poisoning data identification method described in any one aspect.
[0043] Thirdly, the present invention provides a computer system, comprising:
[0044] Memory, used to store computer programs / instructions;
[0045] A processor for executing the computer program / instructions to implement the steps of the poisoning data identification method described in any one of the first aspects.
[0046] Compared with the prior art, the beneficial effects achieved by the present invention are as follows:
[0047] This invention is applicable to large-scale, high-dimensional data poisoning detection, significantly improving detection efficiency and scalability. Furthermore, this invention does not rely on prior knowledge of the dataset distribution, nor does it require domain knowledge or pre-training estimation models on the dataset. This saves a significant amount of time in model building and parameter tuning when identifying poisoned data, lowers the barrier to entry, and facilitates rapid deployment in various application scenarios. At the same time, it can maintain stable detection performance under various poisoning attack modes. Attached Figure Description
[0048] Figure 1 This is a flowchart of a poisoning data identification method provided in an embodiment of the present invention. Detailed Implementation
[0049] The technical solution of the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the embodiments of the present invention and the specific features in the embodiments are detailed descriptions of the technical solution of the present invention, rather than limitations thereof. In the absence of conflict, the embodiments of the present invention and the technical features in the embodiments can be combined with each other.
[0050] It should be noted that the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. Additionally, the character " / " generally indicates that the preceding and following related objects have an "or" relationship.
[0051] Example 1
[0052] This embodiment introduces a method for identifying poisoning data, such as... Figure 1 As shown, it includes:
[0053] Obtain the original dataset;
[0054] Each data record in the original dataset is numbered sequentially to obtain a new dataset with numbers;
[0055] The new dataset is dimensionality reduced to obtain a dimensionality-reduced dataset.
[0056] Perform feature calculations on the dimensionality-reduced dataset and sort it according to the aggregated feature values of all data records;
[0057] Poisoning data is identified based on the sorting results of the aggregated feature values of the data records.
[0058] The original dataset and the new dataset each contain m data records, and each data record contains n attributes; let the domain of discourse be U={x1,…,x}. m}, attribute set A={a1,…,a n};
[0059] Where, x m Let a represent the m-th data record in the universe of discourse. n This represents the nth attribute in the attribute set.
[0060] The method provided in this embodiment includes four steps: data labeling, dimensionality reduction, feature calculation, and sorting analysis.
[0061] By partitioning the universe of discourse, data records with the same attribute values are grouped into the same category, resulting in the partition of the universe of discourse U by attribute set A;
[0062] Calculate the entropy H(A) of attribute set A with respect to the universe of discourse U;
[0063] Remove the column containing each attribute a∈A from the new dataset D' in turn, and calculate the attribute value V(a) for each attribute;
[0064] Sort the attribute values of each attribute from largest to smallest, and select the k attributes with the largest values to form a dimensionality-reduced attribute set A. k And generate a dimensionality-reduced dataset D k , where 1≤k≤n.
[0065] Specifically, the entropy H(A) of attribute set A with respect to the universe of discourse U is expressed as: ;
[0066] in, The relative frequencies are obtained by partitioning. For data records with identical attribute values, g represents the number of partitions of U based on attribute set A.
[0067] For any candidate attribute 'a', the attribute contribution (attribute value V(a)) is defined as the difference between the entropy after removing the attribute and the entropy of the entire set:
[0068] ;
[0069] Where H(A) represents the entropy of attribute set A with respect to the universe of discourse U. Let represent the entropy of attribute set A with respect to the domain U after removing attribute a.
[0070] In practical applications, the Sig value of each attribute is calculated using the formula described above to measure its contribution to discriminative power. Then, the decision on whether to remove the attribute is based on the size of the Sig. The smaller the Sig, the weaker the attribute's contribution to discriminative power, and the more likely it is to be removed to reduce dimensionality.
[0071] For the dimensionality reduction dataset D k Perform feature calculations and output the data record number and the feature value of that data record, including:
[0072] For the dimensionality reduction dataset D kThe attribute values of each data record in the dataset are combined in an unordered manner to generate a set S of attribute value combinations. i ;
[0073] Statistical Dimensionality Reduction Dataset D k The frequency mapping table `count` is obtained by counting the number of times each attribute value combination appears in the global data.
[0074] Query S based on frequency mapping table i Calculate the frequency of each combination, construct a feature vector, and compute the corresponding feature values:
[0075] ;
[0076] Where c represents the set of attribute value combinations S i Each combination in the list.
[0077] Sort all data records in ascending order by their aggregate feature values, and select the data records with the smallest aggregate feature values among the first p records as suspected poisoning data, where 1≤p≤m;
[0078] Alternatively, if the aggregated feature value is less than a given threshold, the data record corresponding to this aggregated feature value is identified as suspected poisoning data.
[0079] This embodiment provides a set of original datasets for training the classification model, as shown in Table 1:
[0080] Table 1. Original dataset used to train the classification model
[0081]
[0082] Applying the above method to the data in this embodiment, the data records have been numbered. The entropy of the entire set is calculated according to the partitions {{1,2,3,4},{5,6,7,8},{9}}: .
[0083] After removing "device type" and re-categorizing, the entropy remains unchanged. .
[0084] The entropy obtained after removing "data consistency" and "device status" is the same, therefore their contributions are equal and they have non-zero gains: .
[0085] Therefore, in the dimensionality reduction step, if Sig (device type) < Sig (data consistency) = Sig (device status), the "device type" attribute can be deleted, and "data consistency" and "device status" can be retained as the discriminant attributes for subsequent poisoning detection.
[0086] After retaining the attributes, the global frequency of each individual attribute and attribute sub-combinations is calculated for the entire set; the key statistical results in this example are:
[0087] (1) The number of occurrences of each attribute value or attribute sub-combination in the entire dataset:
[0088] Stable: 4; Volatile: 5
[0089] Normal: 5; Abnormal: 4.
[0090] Stable, normal: 4; Fluctuating, normal: 1; Fluctuating, abnormal: 4.
[0091] The above frequency is the global count. This is used to subsequently construct the record feature vector.
[0092] (2) List the set of attribute sub-combinations under the reserved attributes for each record, as shown in Table 2.
[0093] Table 2. Set of attribute sub-combinations for each record under the reserved attributes
[0094]
[0095] (3) Calculate the eigenvalues and eigenvectors of each data record.
[0096] The frequency of occurrence of each attribute sub-combination in the entire set will be recorded. The feature vector components of the record are used as the aggregate features of the record. The sum of these components is then taken as the aggregate features of the record. An equal-weighted summation is used as the aggregation strategy. The feature vectors and feature values of each data record obtained according to this rule are shown in Table 3.
[0097] Table 3 Feature vectors and feature values for each data record
[0098]
[0099] Each component in the table above represents the global frequency of the corresponding sub-combination. The aggregated characteristic value is the sum of the components.
[0100] (4) Feature value sorting and poisoning determination: The results of sorting the feature values in ascending order are shown in Table 4.
[0101] Table 4 Results of ascending sorting of feature values
[0102]
[0103] All records are sorted by their aggregate feature values. Record 9 has an S9 value of 11, which is the minimum among all records. The aggregate feature values of the remaining records are all 13. According to the aggregation scale determination rule described in this invention, records with significantly low aggregate feature values are considered suspected poisoning data. Therefore, in this example, record number 9 is identified as a suspected poisoning sample and added to the candidate list for manual review or automatic cleaning.
[0104] Example 2
[0105] Based on the same inventive concept as Embodiment 1, this embodiment introduces a computer-readable storage medium storing a computer program / instructions thereon, which, when executed by a processor, implements the steps of the poisoning data identification method described in any one of Embodiment 1.
[0106] Example 3
[0107] Based on the same inventive concept as other embodiments, this embodiment describes a computer system, including:
[0108] Memory, used to store computer programs / instructions;
[0109] A processor is configured to execute the computer program / instructions to implement the steps of the poisoning data identification method described in any one of Embodiment 1.
[0110] In summary, this invention performs value calculations on each attribute and automatically selects the subset of attributes that contribute most to poisoning detection. This effectively reduces computational complexity while preserving key discriminative information and mitigates the "curse of dimensionality" problem caused by high-dimensional data. It enables poisoning data identification on large-scale training datasets, significantly improving detection efficiency and scalability. This method does not rely on prior knowledge of the dataset distribution, nor does it require domain knowledge or pre-trained estimation models on the dataset. Therefore, it saves significant time on model building and parameter tuning when identifying poisoning data, lowers the barrier to entry, and facilitates rapid deployment in various application scenarios.
[0111] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0112] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.
[0113] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.
[0114] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.
[0115] The embodiments of the present invention have been described above with reference to the accompanying drawings. However, the present invention is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of the present invention without departing from the spirit and scope of the claims. All of these forms are within the protection scope of the present invention.
Claims
1. A method for identifying poisoning data, characterized in that, include: Obtain the original dataset; Each data record in the original dataset is numbered sequentially to obtain a new dataset with numbers; The new dataset is dimensionality reduced to obtain a dimensionality-reduced dataset. Perform feature calculations on the dimensionality-reduced dataset and sort it according to the aggregated feature values of all data records; Poisoning data is identified based on the sorting results of the aggregated feature values of the data records.
2. The poisoning data identification method according to claim 1, characterized in that, The original dataset and the new dataset each contain m data records, and each data record contains n attributes; Let the domain of discourse be U = {x1, ..., x}. m }, attribute set A={a1,…,a n }; Where, x m Let a represent the m-th data record in the universe of discourse. n This represents the nth attribute in the attribute set.
3. The poisoning data identification method according to claim 2, characterized in that, The new dataset D' is dimensionality reduced to obtain the dimensionality-reduced dataset D. k ,include: Data records with the same attribute values are grouped into the same category, resulting in the partition Π of the domain U by attribute set A. A ={C1,…,Cg}, where Cg represents data records with completely identical attribute values in the g-th class; Calculate the entropy H(A) of attribute set A with respect to the universe of discourse U; Remove the column containing each attribute a∈A from the new dataset D' in turn, and calculate the attribute value V(a) for each attribute; Sort the attribute values of each attribute from largest to smallest, and select the k attributes with the largest values to form a dimensionality-reduced attribute set A. k And generate a dimensionality-reduced dataset D k , where 1≤k≤n.
4. The poisoning data identification method according to claim 3, characterized in that, The entropy H(A) of attribute set A with respect to the universe of discourse U is expressed as: ; in, The relative frequencies are obtained by partitioning. For data records with identical attribute values, g represents the number of partitions of U based on attribute set A.
5. The poisoning data identification method according to claim 4, characterized in that, The formula for calculating the attribute value V(a) is: ； Where H(A) represents the entropy of attribute set A with respect to the universe of discourse U. Let represent the entropy of attribute set A with respect to the domain U after removing attribute a.
6. The poisoning data identification method according to claim 5, characterized in that, Perform feature calculations on the dimensionality-reduced dataset and sort it according to the aggregated feature values of all data records, including: For the dimensionality reduction dataset D k The attribute values of each data record in the dataset are combined in an unordered manner to generate a set of attribute value combinations: Statistical Dimensionality Reduction Dataset D k The frequency mapping table is obtained by counting the number of times each attribute value combination appears in the global data. Based on the frequency mapping table, query the occurrence count of each combination in the attribute value set, construct a feature vector, and calculate the corresponding feature value.
7. The poisoning data identification method according to claim 6, characterized in that, The processes of "generating a set of attribute value combinations" and "obtaining a frequency mapping table" can be executed in parallel or step by step.
8. The poisoning data identification method according to claim 7, characterized in that, Based on the sorting results of the aggregated feature values of the data records, the poisoning data is identified, including: Sort all data records in ascending order by their aggregate feature values, and select the q data records with the smallest aggregate feature values as suspected poisoning data, where 1≤q≤m; Alternatively, if the aggregated feature value is less than a given threshold, the data record corresponding to this aggregated feature value is identified as suspected poisoning data.
9. A computer-readable storage medium having a computer program / instructions stored thereon, characterized in that, When the computer program / instruction is executed by the processor, it implements the steps of the poisoning data identification method according to any one of claims 1 to 8.
10. A computer system, characterized in that, include: Memory, used to store computer programs / instructions; A processor for executing the computer program / instructions to implement the steps of the poisoning data identification method according to any one of claims 1 to 8.