Artificial Intelligence-Based Data Classification and Grading Method and System

Abstract

The present invention relates to the technical field of government affair data processing, and provides a method and system for classifying and grading government affair data based on artificial intelligence, including receiving a government affair data set; cleaning the government affair data set according to industry attributes to obtain a general data set; marking the general data set according to security attributes; clustering the second security level samples to obtain associated features; performing data enhancement on the second security level samples based on the Synthetic Minority Over-sampling Technique (SMOTE) to obtain enhanced second security level samples; wherein, setting the K value of SMOTE according to the associated features; combining the first security level samples, the third security level samples and the enhanced second security level samples to obtain a combined sample set; training a government affair data classification and grading model according to the combined sample set, and classifying and grading government affair data according to the trained government affair data classification and grading model. The method and system of the present invention can improve the accuracy of classifying and grading government affair data under the condition of a small number of samples.

Description

Technical Field

[0001] The present invention relates to the technical field of government data processing, and in particular to an artificial intelligence-based data classification and grading method and system. Background Art

[0002] With the rapid development of information technology, the digitalization degree of government data has been continuously increasing, and relevant institutions have accumulated a large amount of data resources. These data not only cover a wide range of social activity information but also involve sensitive information in many aspects such as personal privacy and public security. Therefore, how to effectively manage, protect, and utilize these data has become an important problem to be solved urgently.

[0003] According to security attributes, government data can be divided into three levels: high security level, medium security level, and low security level; among them, the government data at the medium security level can be further divided into two different levels according to needs, and thus government data can be divided into four different security levels. The data at the high security level contains extremely sensitive personal information and must be strictly controlled and protected; the data at the medium security level may involve some sensitive content, which needs to be protected to a certain extent but allows relatively wide use; the data at the low security level is relatively less sensitive and is suitable for a wider range of application scenarios.

[0004] However, in actual applications, the sample size of government data is often insufficient, which limits its role in model training and data analysis. To address the problem of insufficient sample size, existing data augmentation means (such as Synthetic Minority Over-sampling Technique, abbreviated as SMOTE) are used to augment samples. However, the existing government data augmentation means do not make reasonable distinctions according to the security attributes of the data, but uniformly use the same augmentation method for all government data. This approach may lead to a series of problems: First, for high-security-level data, simple augmentation methods may pose a risk of information leakage; second, using the same strategy for medium-security-level and low-security-level data fails to fully exploit their unique value, affecting the effect of data augmentation and ultimately the accuracy of data classification and grading; at the same time, there is also a risk of leakage of corresponding sensitive information in medium-security-level data. Summary of the Invention

[0005] Therefore, the technical problem to be solved by the present invention is to overcome the problem that the accuracy of government data classification and grading cannot meet the requirements due to various limitations of data augmentation in the prior art, and provide an artificial intelligence-based data classification and grading method and system. When augmenting minority class samples, the security attributes of the samples are fully considered to better capture the potential associations between data, thereby improving the accuracy of data classification and grading to meet the requirements.

[0006] In a first aspect, to solve the above technical problems, the present invention provides an artificial intelligence-based data classification and grading method, including the following steps:

[0007] Receive a government affairs data set, clean the government affairs data set according to industry attributes to obtain a general data set and a non-general data set; the government affairs data set includes multiple groups of government affairs data, and the government affairs data at least includes social security data, real estate data, household register data, and education student status data; the social security data includes a social security account, the personal real estate data includes a house property certificate number, and the household register data includes the ID numbers of the household head and family members;

[0008] Mark the general data set according to security attributes to obtain first-level security samples, second-level security samples, and third-level security samples; among them, the data sensitivity characteristics of the first-level security samples, second-level security samples, and third-level security samples decrease in turn;

[0009] Cluster the second-level security samples to identify the features associated with the first-level security samples in the second-level security samples, and obtain associated features; among them, the associated features represent the association degree between each cluster after clustering of the second-level security samples and the first-level security samples;

[0010] Perform data enhancement on the second-level security samples based on the Synthetic Minority Over-sampling Technique (SMOTE) to obtain enhanced second-level security samples; among them, mark the enhanced second-level security samples according to the associated features, and extract some samples from the enhanced second-level security samples and add them to the first-level security samples; and set the K value of the Synthetic Minority Over-sampling Technique (SMOTE) according to the associated features;

[0011] Merge the first-level security samples, third-level security samples, and enhanced second-level security samples to obtain a merged sample set;

[0012] Train a government affairs data classification and grading model according to the merged sample set, and classify and grade government affairs data according to the trained government affairs data classification and grading model.

[0013] In an embodiment of the present invention, it further includes configuring an associated feature threshold, and adding the samples obtained by performing data enhancement on the second-level security samples whose associated features are greater than or equal to the associated feature threshold to the first-level security samples.

[0014] In an embodiment of the present invention, clustering the second-level security samples to obtain associated features includes:

[0015] Determine the number of clusters m, and cluster the second security level samples into m clusters;

[0016] Extract the sample features of each cluster to obtain the sample feature matrix of all clusters;

[0017] Perform feature overlap analysis on the second security level samples and the first security level samples according to the sample feature matrix to identify the associated features.

[0018] In one embodiment of the present invention, setting the K value of the Synthetic Minority Over-sampling Technique (SMOTE) according to the associated features includes,

[0019] Configure a first feature threshold and a second feature threshold, and the first feature threshold is less than the second feature threshold;

[0020] If the associated feature is less than or equal to the first feature threshold, the K value of the Synthetic Minority Over-sampling Technique (SMOTE) is configured as K = 6 or 7 or 8;

[0021] If the associated feature is greater than the first feature threshold and less than or equal to the second feature threshold, the K value of the Synthetic Minority Over-sampling Technique (SMOTE) is configured as K = 5;

[0022] If the associated feature is greater than the second feature threshold, the K value of the Synthetic Minority Over-sampling Technique (SMOTE) is configured as K = 3 or 4.

[0023] In one embodiment of the present invention, the first feature threshold and the second feature threshold are preset.

[0024] In one embodiment of the present invention, the first feature threshold and the second feature threshold are set according to the distribution of the associated features.

[0025] In one embodiment of the present invention, it further includes performing data enhancement on the third security level samples based on the Synthetic Minority Over-sampling Technique (SMOTE) to obtain enhanced third security level samples;

[0026] Merge the first security level samples, the enhanced second security level samples, and the enhanced third security level samples to obtain the merged sample set.

[0027] In one embodiment of the present invention, merging the first security level samples, the third security level samples, and the enhanced second security level samples to obtain the merged sample set includes,

[0028] Determine the total number of samples N in the merged sample set;

[0029] Determine the number of samples at each security level according to the following method: N = A * N1 + B * N2 + C * N3;

[0030] Among them, A, B, and C are the weights of the first security level samples, the second security level samples, and the third security level samples respectively, satisfying A + B + C = 1; N1, N2, and N3 are the sample numbers of the first security level samples, the second security level samples, and the third security level samples respectively.

[0031] In an embodiment of the present invention, merging the first security level samples, the third security level samples, and the enhanced second security level samples to obtain a merged sample set further includes,

[0032] When merging the sample set, randomly sample the sample data of each security level until the required number of samples is reached.

[0033] In a second aspect, to solve the above technical problems, the present invention provides an artificial intelligence-based data classification and grading system, including,

[0034] A data receiving module, which is used to receive a government affairs data set, the government affairs data set contains multiple groups of government affairs data, and the government affairs data at least includes social security data, real estate data, household register data, and education student status data; the social security data includes social security accounts, the personal real estate data includes real estate certificate numbers, and the household register data includes the ID card numbers of the household head and family members;

[0035] A data cleaning module, which is used to clean the government affairs data set according to industry attributes to obtain a general data set and a non-general data set;

[0036] A security attribute marking module, which is used to mark the general data set according to security attributes to obtain first security level samples, second security level samples, and third security level samples; among them, the data sensitive characteristics of the first security level samples, the second security level samples, and the third security level samples decrease in turn;

[0037] A clustering analysis module, which is used to cluster the second security level samples to identify the characteristics associated with the first security level samples of the second security level samples, and obtain associated characteristics; among them, the associated characteristics represent the association degree between each cluster after clustering of the second security level samples and the first security level samples;

[0038] A data enhancement module, which is used to perform data enhancement on the second security level samples according to the Synthetic Minority Over-sampling Technique (SMOTE) to obtain enhanced second security level samples; among them, mark the enhanced second security level samples according to the associated characteristics, extract some samples from the enhanced second security level samples and add them to the first security level samples; and set the K value of the Synthetic Minority Over-sampling Technique (SMOTE) according to the associated characteristics;

[0039] A sample merging module, which is used to merge the first security level samples, the third security level samples, and the enhanced second security level samples to obtain a merged sample set;

[0040] A model training module, which is used to train a government data classification and grading model according to the merged sample set, and classify and grade government data according to the trained government data classification and grading model.

[0041] In an embodiment of the present invention, it further includes a packaging module, and the packaging module is used to package the trained government data classification and grading model into a terminal device.

[0042] The above technical solution of the present invention has the following beneficial effects compared with the prior art:

[0043] The data classification and grading method and system based on artificial intelligence of the present invention are used to classify and grade government data including social security data, real estate data, household registration data, and education student status data; first, clean the government data set according to industry attributes, eliminate non-universal data sets, and only perform data enhancement on universal data sets. Before data enhancement, mark the universal data set according to security attributes, distinguish samples of three security levels, and perform clustering analysis on the medium security level samples to obtain associated features with high security level samples. According to the associated features, perform data enhancement on the medium security level samples to obtain enhanced medium security level samples. The enhanced medium security level samples are merged with high and low security level samples to form a sample set for training the government data classification and grading model; before data enhancement, mark the universal data set according to security attributes, and do not perform data enhancement on high security level samples to avoid leakage of sensitive data and improve the grading security; for medium security level samples, first perform clustering analysis to extract associated features with high security level samples. The extraction of associated features can uncover the potential association between medium security level sample data and high security level sample data, further improving the grading security; on the other hand, it enables the model to improve the learning ability of high security level sample features without directly using high security level samples during model training, thereby enhancing the accuracy and security of the classification and grading model; perform data enhancement according to the associated features. By setting specific K values and configuring different K values for different associated features, it can effectively solve the problem of data imbalance, increase the quantity and quality of minority class samples, and also retain the diversity of samples, enhancing the generalization ability of the classification and grading model, thereby further enhancing the accuracy of the classification and grading model to meet the requirements. Description of the Drawings

[0044] In order to make the content of the present invention easier to be clearly understood, the following further elaborates on the present invention in detail according to specific embodiments of the present invention and in conjunction with the drawings, where

[0045] Figure 1 It is a flowchart of the data classification and grading method based on artificial intelligence in the preferred embodiment of the present invention;

[0046] Figure 2 It is a flowchart of the data classification and grading method based on artificial intelligence in another embodiment;

[0047] Figure 3 It is a structural block diagram of the data classification and grading system based on artificial intelligence in the preferred embodiment of the present invention. Detailed implementation manners

[0048] The present invention will be further described below in conjunction with the accompanying drawings and specific embodiments, so that those skilled in the art can better understand the present invention and be able to implement it, but the cited embodiments are not intended to limit the present invention. Embodiment

[0049] An embodiment of the present invention discloses a data classification and grading method based on artificial intelligence, which is used to classify and grade government affairs data including social security data, real estate data, household register data, and education student status data; the classification categories include "personal information", "enterprise unit information", "public institution information", and "government department information", etc., and the grading levels include "level one", "level two", "level three", and "level four", etc. The higher the grading level, the higher the data sensitivity. For example, level four data contains extremely sensitive personal information and must be strictly controlled and protected; level two and level three data may involve some sensitive content and need to be protected to a certain extent but relatively extensive use is allowed; level one data is relatively less sensitive and is applicable to a wider range of application scenarios.

[0050] Due to the security (or sensitivity) involved in government affairs data, relevant institutions usually do not actively disclose it, and the data sample size is small. The grading accuracy of the data model trained with small samples does not meet the requirements. In order to overcome the problem of small sample size, existing data augmentation means (such as Synthetic Minority Over-sampling Technique, abbreviated as SMOTE) are used to augment samples, but the existing government affairs data augmentation means do not reasonably distinguish according to the security attributes of the data, but uniformly adopt the same augmentation method for all government affairs data. This approach may lead to a series of problems: First, for high-security-level data, simple augmentation methods may pose a risk of information leakage; second, adopting the same strategy for medium-security-level and low-security-level data fails to fully exploit their unique value, affecting the effect of data augmentation and ultimately affecting the accuracy of data classification and grading; at the same time, there is also a risk of leakage of corresponding sensitive information in medium-security-level data.

[0051] It should be noted that the high, medium, and low security levels are only used to divide data into three different levels in terms of security; among them, the government affairs data at the medium security level can be further divided into two different levels according to needs, so that the government affairs data can be divided into four different security levels. For example, level one, level two, level three, and level four. The higher the classification level, the higher the data sensitivity.

[0052] In response to the above problems, referring to Figure 1 as shown, an artificial intelligence-based data classification and grading method disclosed in an embodiment of the present invention includes the following steps:

[0053] Receive a government affairs data set, clean the government affairs data set according to industry attributes to obtain a general data set and a non-general data set; the government affairs data set includes multiple groups of government affairs data, and the government affairs data includes at least social security data, real estate data, household registration data, and education student status data;

[0054] Mark the general data set according to security attributes to obtain a first security level sample, a second security level sample, and a third security level sample; among them, the data sensitive characteristics of the first security level sample, the second security level sample, and the third security level sample decrease in turn;

[0055] Cluster the second security level sample, identify the characteristics associated with the first security level sample in the second security level sample, and obtain associated characteristics; among them, the associated characteristics represent the degree of association between each cluster after clustering of the second security level sample and the first security level sample;

[0056] Based on the Synthetic Minority Over-sampling Technique (SMOTE), perform data augmentation on the second security level sample to obtain an enhanced second security level sample; among them, mark the enhanced second security level sample according to the associated characteristics, extract some samples from the enhanced second security level sample and add them to the first security level sample; and set the K value of the Synthetic Minority Over-sampling Technique (SMOTE) according to the associated characteristics;

[0057] Merge the first security level sample, the third security level sample, and the enhanced second security level sample to obtain a merged sample set;

[0058] Train a government affairs data classification and grading model according to the merged sample set, and classify and grade government affairs data according to the trained government affairs data classification and grading model.

[0059] The data classification and grading method based on artificial intelligence described in the present invention is used to classify and grade government affairs data including social security data, real estate data, household register data, and education student status data; first, clean the government affairs data set according to industry attributes, eliminate non-general data sets, and only perform data enhancement on general data sets. Before data enhancement, mark the general data set according to security attributes, distinguish samples of three security levels, and perform clustering analysis on samples of the medium security level to obtain associated features with samples of the high security level. According to these associated features, perform data enhancement on samples of the medium security level to obtain enhanced samples of the medium security level. The enhanced samples of the medium security level are merged with samples of the high and low security levels to form a sample set for training the government affairs data classification and grading model; before data enhancement, mark the general data set according to security attributes. Samples of the high security level do not undergo data enhancement to avoid leakage of sensitive data and improve the security of grading. Samples of the medium security level first perform clustering analysis to extract associated features with samples of the high security level. The extraction of associated features can uncover potential associations between data of samples of the medium security level and data of samples of the high security level, further improving the security of grading. On the other hand, it enables the model to improve its learning ability for features of samples of the high security level without directly using samples of the high security level during model training, thereby enhancing the accuracy and security of the classification and grading model; perform data enhancement according to these associated features. By setting specific K values and configuring different K values for different associated features, it can effectively solve the problem of data imbalance, increase the quantity and quality of minority class samples, and retain the diversity of samples, enhancing the generalization ability of the classification and grading model, thereby further enhancing the accuracy of the classification and grading model to meet the requirements.

[0060] In addition, screen out some samples from the enhanced second security level samples according to the associated features and add them to the first security level samples, so as to obtain an increase in the samples of the first security level without performing data enhancement on the first security level samples, that is, obtain an increase in the samples of the first security level without damaging the security of the first security level samples, greatly improving the security of data classification and grading.

[0061] On the other hand, it can make the ratio of the number of samples of the first security level to the number of samples of the second security level after data enhancement of the second security level samples remain the same as before data enhancement; that is to say, due to the association between the first security level samples and the second security level samples, even if only the second security level samples are enhanced and the first security level samples are not enhanced, the final obtained number of samples maintains the same proportional relationship with the true number of samples, further improving the security and accuracy of government affairs data classification and grading.

[0062] Specifically, without data augmentation for the first security level samples, the sample increase of the first security level samples can be implemented based on at least the following three methods:

[0063] Implementation method 1: After all samples of the second security level samples are augmented and then classified, the sample increase of the first security level samples is obtained:

[0064] First, perform data augmentation on the second security level samples to obtain the augmented second security level samples. Then, extract the samples with a high security level from the augmented second security level samples according to the associated features by screening or marking, and add this type of samples to the first security level samples. Of course, in addition to adding this type of samples to the first security level samples, they can also continue to be retained in the augmented second security level samples, or be removed from the augmented second security level samples, so as to achieve the purpose that the ratio of the number of samples of the first security level samples to the number of samples of the second security level samples after the data augmentation of the second security level samples can be kept consistent with that before the data augmentation.

[0065] Implementation method 2: First, extract some samples from the first security level samples, and then perform data augmentation on this part of the samples to obtain the sample increase of the first security level samples:

[0066] Configure the associated feature threshold, compare the relationship between the associated feature and the associated feature threshold, and add the samples obtained by performing data augmentation on the second security level samples whose associated feature is greater than or equal to the associated feature threshold to the first security level samples.

[0067] Implementation method 3: Combine implementation method 1 and implementation method 2.

[0068] In addition, government affairs data has strong industry attributes. Some government affairs data has data levels different from normal values due to industry attributes and is regarded as non-general data, such as the mobile phone numbers and names of legal persons, etc. Before data marking, clean the government affairs data set according to the industry attributes, remove the non-general data set, and only perform data augmentation on the general data set (such as the identity card numbers of legal persons). By removing the non-general data set as noise and only performing data augmentation on the general data, it provides a reliable basis for subsequent data augmentation, mining, and decision-making, so as to improve the accuracy and stability of the classification and grading model.

[0069] It should be noted that the SMOTE algorithm (SMOTE: Synthetic Minority Over-sampling Technique) is a synthetic minority over-sampling algorithm, which is one of the existing algorithms for enhancing sample data. The K value of the SMOTE algorithm refers to the number of the nearest neighbors selected for generating synthetic samples during interpolation synthesis. Specifically, the working principle of the SMOTE algorithm is as follows:

[0070] Calculating the nearest neighbors: For each minority class sample (i.e., the class for which the number needs to be increased), SMOTE calculates the distances between this sample and other minority class samples and finds the K nearest neighbors.

[0071] Generating synthetic samples: Among these K neighbors, randomly select one neighbor, and then SMOTE generates new synthetic samples by performing linear interpolation between the original sample and the selected neighbor. This process generates multiple synthetic samples, and the specific number depends on the expected number of samples to be increased.

[0072] Selection of the K value: The selection of the K value affects the diversity and distribution of the generated synthetic samples. In practical applications, adjust the K value according to the characteristics of the dataset and actual requirements.

[0073] Specifically, clustering the second security level samples to obtain associated features includes,

[0074] Determine the number of clusters m and cluster the second security level samples into m clusters;

[0075] Extract the sample features of each cluster to obtain the sample feature matrix of all clusters;

[0076] Perform feature overlap analysis on the second security level samples and the first security level samples according to the sample feature matrix to identify the associated features.

[0077] In specific application scenarios, some data are marked as second security level samples in terms of security attributes, but the second security level samples may also involve some sensitive content that needs to be protected to a certain extent, such as the social security account number of social security data, the house property certificate number of personal house property data, the household head and the ID card numbers of family members in household register data, etc. Cluster analysis is performed on the second security level samples to extract the associated features that overlap with the features of the first security level samples for subsequent data augmentation.

[0078] Furthermore, setting the K value of the Synthetic Minority Over-sampling Technique (SMOTE) according to the associated features includes,

[0079] Configure a first feature threshold and a second feature threshold, and the first feature threshold is less than the second feature threshold;

[0080] If the associated feature is less than or equal to the first feature threshold, the K value of the Synthetic Minority Over-sampling Technique (SMOTE) is configured as K = 8;

[0081] If the associated feature is greater than the first feature threshold and less than or equal to the second feature threshold, the K value of the Synthetic Minority Over-sampling Technique (SMOTE) is configured as K = 5;

[0082] If the associated feature is greater than the second feature threshold, the K value of the Synthetic Minority Over-sampling Technique (SMOTE) is configured as K = 3.

[0083] In a specific application scenario, when the associated feature is less than or equal to the first feature threshold, it indicates that the sensitivity of the second security level samples is relatively weak. By configuring the K value of SMOTE as K = 8, the diversity of data augmentation can be improved; when the associated feature is greater than the first feature threshold and less than or equal to the second feature threshold, it indicates that the second security level samples have a certain sensitivity. By configuring the K value of SMOTE as K = 5, on the one hand, the newly generated samples tend to be near the original samples, ensuring the quality and security of the augmented data; at the same time, it promotes the diversity of the generated samples to a certain extent; when the associated feature is greater than the second feature threshold, it indicates that the sensitivity of the second security level samples is relatively strong. By configuring the K value of SMOTE as K = 3, the newly generated samples are concentrated near the original samples, ensuring the security of the augmented data.

[0084] Among them, the first feature threshold and the second feature threshold are preset, especially set according to the distribution of the associated feature.

[0085] In another embodiment of the present invention, referring to Figure 2 as shown, it further includes performing data augmentation on the third security level samples based on the Synthetic Minority Over-sampling Technique (SMOTE) to obtain the augmented third security level samples; merging the first security level samples, the augmented second security level samples, and the augmented third security level samples to obtain the merged sample set.

[0086] In a specific application scenario, the third security level samples are relatively less sensitive and applicable to a wider range of application scenarios. Data augmentation is performed based on conventional data augmentation methods. Preferably, the Synthetic Minority Over-sampling Technique (SMOTE) is used to perform data augmentation on the third security level samples to obtain augmented data with qualified data quality.

[0087] Furthermore, merging the first security level samples, the third security level samples, and the augmented second security level samples to obtain the merged sample set includes

[0088] determining the total number of samples N in the merged sample set;

[0089] determining the number of samples at each security level according to the following method: N = A * N1 + B * N2 + C * N3;

[0090] Among them, A, B, and C are the weights of the first security level samples, the second security level samples, and the third security level samples respectively, satisfying A + B + C = 1; N1, N2, and N3 are the sample numbers of the first security level samples, the second security level samples, and the third security level samples respectively.

[0091] In a specific application scenario, the enhanced second security level samples and the enhanced third security level samples need to be merged with the first security level samples to form a merged sample set for subsequent model training, testing, and verification. In the preferred embodiment of the present invention, data is merged based on the weight setting method. For example, different weights are set according to the importance of the security levels to expand the influence of high security level samples during the training process; higher weights can be assigned to high security level samples during model training, which is achieved by setting a loss function and emphasizing the influence of high security samples in the loss calculation.

[0092] Furthermore, merging the first security level samples, the enhanced third security level samples, and the enhanced second security level samples to obtain the merged sample set further includes, when merging the sample set, randomly sampling the sample data of each security level until the required sample number is reached. Randomly sampling the sample data of each level to form a merged sample set can avoid overfitting during model training and further improve the classification accuracy of the classification and grading model. Embodiment

[0093] The difference between this embodiment and Embodiment 1 is only the configuration of the K value of the Synthetic Minority Over-sampling Technique (SMOTE). Specifically,

[0094] Configure a first feature threshold and a second feature threshold, and the first feature threshold is less than the second feature threshold;

[0095] If the associated feature is less than or equal to the first feature threshold, the K value of the Synthetic Minority Over-sampling Technique (SMOTE) is configured as K = 7;

[0096] If the associated feature is greater than the first feature threshold and less than or equal to the second feature threshold, the K value of the Synthetic Minority Over-sampling Technique (SMOTE) is configured as K = 5;

[0097] If the associated feature is greater than the second feature threshold, the K value of the Synthetic Minority Over-sampling Technique (SMOTE) is configured as K = 3. Embodiment

[0098] The difference between this embodiment and Embodiment 1 is only the configuration of the K value of the Synthetic Minority Over-sampling Technique (SMOTE). Specifically,

[0099] Configure a first feature threshold and a second feature threshold, and the first feature threshold is less than the second feature threshold;

[0100] If the associated feature is less than or equal to the first feature threshold, the K value of the Synthetic Minority Over-sampling Technique (SMOTE) is configured as K = 6;

[0101] If the associated feature is greater than the first feature threshold and less than or equal to the second feature threshold, the K value of the Synthetic Minority Over-sampling Technique (SMOTE) is configured as K = 5;

[0102] If the associated feature is greater than the second feature threshold, the K value of the Synthetic Minority Over-sampling Technique (SMOTE) is configured as K = 4.

[0103] According to the different K values of the configured SMOTE, the grading accuracies of the classification and grading models obtained in the first embodiment, the second embodiment, and the third embodiment are as follows: the first embodiment is the highest, the second embodiment is the second highest, and the third embodiment is the worst. Embodiment

[0104] Referring to Figure 3 as shown, this embodiment discloses an artificial intelligence-based data classification and grading system, including,

[0105] A data receiving module, which is used to receive a government affairs data set, where the government affairs data set contains multiple groups of government affairs data, and the government affairs data at least includes social security data, real estate data, household register data, and education student status data;

[0106] A data cleaning module, which is used to clean the government affairs data set according to industry attributes to obtain a general data set and a non-general data set;

[0107] A security attribute marking module, which is used to mark the general data set according to security attributes to obtain first-level security samples, second-level security samples, and third-level security samples; among them, the data sensitivity characteristics of the first-level security samples, second-level security samples, and third-level security samples decrease in turn;

[0108] A clustering analysis module, which is used to cluster the second-level security samples to identify the features associated with the first-level security samples in the second-level security samples, and obtain associated features; among them, the associated features represent the degree of association between each cluster after clustering of the second-level security samples and the first-level security samples;

[0109] A data enhancement module, which is used to enhance the second-level security samples based on the Synthetic Minority Over-sampling Technique (SMOTE) to obtain enhanced second-level security samples; among them, the K value of the Synthetic Minority Over-sampling Technique (SMOTE) is set according to the associated features;

[0110] A sample merging module, which is used to merge the first security level samples, the third security level samples and the enhanced second security level samples to obtain a merged sample set;

[0111] A model training module, which is used to train a government data classification and grading model according to the merged sample set, and classify and grade government data according to the trained government data classification and grading model.

[0112] The artificial intelligence-based data classification and grading system of the present invention is used to classify and grade government data including social security data, real estate data, household register data and education student status data; first, clean the government data set according to industry attributes, eliminate non-universal data sets, and only perform data enhancement on universal data sets. Before data enhancement, mark the universal data set according to security attributes, distinguish samples of three security levels, and perform cluster analysis on the medium security level samples to obtain the associated features with the high security level samples. According to the associated features, perform data enhancement on the medium security level samples to obtain enhanced medium security level samples. The enhanced medium security level samples are merged with the high and low security level samples to form a sample set for training the government data classification and grading model; before data enhancement, mark the government data set according to security attributes, and do not perform data enhancement on the high security level samples to avoid leakage of sensitive data and improve the grading security; first perform cluster analysis on the medium security level samples to extract the associated features with the high security level samples. The extraction of the associated features can uncover the potential association between the medium security level sample data and the high security level sample data, further improving the grading security; on the other hand, it enables the model to improve the learning ability of the high security level sample features without directly using the high security level samples during model training, thereby enhancing the accuracy and security of the classification and grading model; perform data enhancement according to the associated features. By setting specific K values and configuring different K values for different associated features, it can effectively solve the problem of data imbalance, increase the quantity and quality of minority class samples, and also retain the diversity of samples, improving the generalization ability of the classification and grading model, thereby further enhancing the accuracy of the classification and grading model to meet the requirements.

[0113] The government data classification and grading model obtained by the present invention has government data as the input and category and grade as the output.

[0114] Furthermore, it further includes a packaging module, which is used to package the trained government data classification and grading model into a terminal device.

[0115] In a specific application scenario, the government data classification and grading model obtained in the embodiment of the present invention is used to process government data. Since government data has a certain degree of sensitivity, the classification and grading model is not used for deployment to the cloud and is packaged into the user's terminal device according to the usage requirements.

[0116] All user information and data involved in this invention are obtained through legal channels.

[0117] Those skilled in the art should understand that the embodiments of this application can be provided as a method, a system, or a computer program product. Therefore, this application can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, this application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0118] This application is described with reference to the flowcharts and / or block diagrams of methods, apparatuses (systems), and computer program products according to the embodiments of this application. It should be understood that each flow and / or block in the flowchart and / or block diagram can be implemented by computer program instructions, and the combination of flows and / or blocks in the flowchart and / or block diagram can also be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing devices generate means for implementing the functions specified in one Figure 1 one flow or multiple flows and / or blocks Figure 1 one block or multiple blocks.

[0119] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory generate a manufactured article including instruction means, and the instruction means implements the functions specified in one Figure 1 one flow or multiple flows and / or blocks Figure 1 one block or multiple blocks.

[0120] These computer program instructions can also be loaded onto a computer or other programmable data processing device, so that a series of operation steps are executed on the computer or other programmable device to generate a computer-implemented process, and thus the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in one Figure 1 one flow or multiple flows and / or blocks Figure 1 one block or multiple blocks.

[0121] Obviously, the above embodiments are merely examples for clear illustration and are not limitations on the implementation manners. For those of ordinary skill in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to enumerate all the implementation manners here. And the obvious changes or modifications derived therefrom still fall within the protection scope of the present invention.

Claims

1. A data classification and grading method based on artificial intelligence, characterized in that: It includes the following steps: Receive a government affairs data set, clean the government affairs data set according to industry attributes to obtain a general data set and a non-general data set; the government affairs data set includes multiple groups of government affairs data, and the government affairs data at least includes social security data, real estate data, household register data, and education student status data; the social security data includes a social security account number, the personal real estate data includes a house property certificate number, and the household register data includes the identity card numbers of the household head and family members. Mark the general data set according to security attributes to obtain a first security level sample, a second security level sample, and a third security level sample; among them, the data sensitive characteristics of the first security level sample, the second security level sample, and the third security level sample decrease in turn. Cluster the second security level sample to identify the features associated with the first security level sample in the second security level sample, and obtain associated features; among them, the associated features represent the degree of association between each cluster after clustering of the second security level sample and the first security level sample. Perform data augmentation on the second security level sample based on the Synthetic Minority Over-sampling Technique (SMOTE) to obtain an enhanced second security level sample; among them, mark the enhanced second security level sample according to the associated features, and extract some samples from the enhanced second security level sample and add them to the first security level sample; and, set the K value of the Synthetic Minority Over-sampling Technique (SMOTE) according to the associated features; setting the K value includes: configuring a first feature threshold and a second feature threshold, and the first feature threshold is less than the second feature threshold; if the associated feature is less than or equal to the first feature threshold, the K value of the Synthetic Minority Over-sampling Technique (SMOTE) is configured as K = 6 or 7 or 8; if the associated feature is greater than the first feature threshold and less than or equal to the second feature threshold, the K value of the Synthetic Minority Over-sampling Technique (SMOTE) is configured as K = 5; if the associated feature is greater than the second feature threshold, the K value of the Synthetic Minority Over-sampling Technique (SMOTE) is configured as K = 3 or 4. Merge the first security level sample, the third security level sample, and the enhanced second security level sample to obtain a merged sample set. Train a government affairs data classification and grading model according to the merged sample set, and classify and grade government affairs data according to the trained government affairs data classification and grading model.

2. The method for data classification and grading based on artificial intelligence according to claim 1, wherein: It also includes configuring an associated feature threshold, and adding the samples obtained by performing data augmentation on the second security level samples whose associated features are greater than or equal to the associated feature threshold to the first security level sample.

3. The method for classifying and grading data based on artificial intelligence according to claim 1 or 2, characterized in that: Clustering the second security level sample to obtain associated features includes: Determine the number of clusters m, and cluster the second security level sample into m clusters. Extract the sample features of each cluster to obtain the sample feature matrix of all clusters. Perform feature overlap analysis on the second security level sample and the first security level sample according to the sample feature matrix to identify the associated features.

4. The method for data classification and grading based on artificial intelligence according to claim 1, wherein: The first feature threshold and the second feature threshold are set according to the distribution of the associated features.

5. The method for classifying and grading data based on artificial intelligence according to claim 1, wherein: It also includes performing data augmentation on the third security level samples based on the Synthetic Minority Over-sampling Technique (SMOTE) to obtain augmented third security level samples; Combining the first security level samples, the augmented second security level samples, and the augmented third security level samples to obtain the combined sample set.

6. The method for classifying and grading data based on artificial intelligence according to claim 1, wherein: Combining the first security level samples, the third security level samples, and the augmented second security level samples to obtain the combined sample set includes Determining the total number of samples N in the combined sample set; Determining the number of samples for each security level according to the following method: N = A * N1 + B * N2 + C * N3; Where A, B, and C are the weights of the first security level samples, the second security level samples, and the third security level samples respectively, satisfying A + B + C = 1; N1, N2, and N3 are the number of samples of the first security level samples, the second security level samples, and the third security level samples respectively.

7. The method for data classification and grading based on artificial intelligence according to claim 6, wherein: Combining the first security level samples, the third security level samples, and the augmented second security level samples to obtain the combined sample set also includes When combining the sample set, randomly sampling the sample data of each security level until the required number of samples is reached.

8. An artificial intelligence-based data classification and grading system, characterized in that: Including A data receiving module, which is used to receive a government affairs data set. The government affairs data set contains multiple groups of government affairs data, and the government affairs data at least includes social security data, real estate data, household register data, and education student status data; the social security data includes social security accounts, the personal real estate data includes real estate certificate numbers, and the household register data includes the ID numbers of the household head and family members; A data cleaning module, which is used to clean the government affairs data set according to industry attributes to obtain a general data set and a non-general data set; A security attribute marking module, which is used to mark the general data set according to security attributes to obtain first security level samples, second security level samples, and third security level samples; among them, the data sensitivity characteristics of the first security level samples, the second security level samples, and the third security level samples decrease in turn; A clustering analysis module, which is used to cluster the second security level samples to identify the features associated with the first security level samples in the second security level samples and obtain associated features; among them, the associated features represent the degree of association between each cluster after clustering of the second security level samples and the first security level samples; A data augmentation module, which is used to perform data augmentation on the second security level samples according to the Synthetic Minority Over-sampling Technique (SMOTE) to obtain augmented second security level samples; wherein, the augmented second security level samples are labeled according to the associated features, and partial samples are extracted from the augmented second security level samples and added to the first security level samples; and, the K value of the Synthetic Minority Over-sampling Technique (SMOTE) is set according to the associated features; setting the K value includes: configuring a first feature threshold and a second feature threshold, and the first feature threshold is less than the second feature threshold; if the associated feature is less than or equal to the first feature threshold, the K value of the Synthetic Minority Over-sampling Technique (SMOTE) is configured as K = 6 or 7 or 8; if the associated feature is greater than the first feature threshold and less than or equal to the second feature threshold, the K value of the Synthetic Minority Over-sampling Technique (SMOTE) is configured as K = 5; if the associated feature is greater than the second feature threshold, the K value of the Synthetic Minority Over-sampling Technique (SMOTE) is configured as K = 3 or 4; A sample merging module, which is used to merge the first security level samples, the third security level samples and the augmented second security level samples to obtain a merged sample set; A model training module, which is used to train a government data classification and grading model according to the merged sample set, and classify and grade government data according to the trained government data classification and grading model.

9. The data classification and grading system based on artificial intelligence according to claim 8, characterized in that: It further includes a packaging module, and the packaging module is used to package the trained government data classification and grading model into a terminal device.

Images