Data classification and grading method and system based on artificial intelligence

Abstract

The invention relates to the technical field of government affair data processing, and provides a government affair data classification and grading method and system based on artificial intelligence. Cleaning the government affair data set according to industry attributes to obtain a general data set; marking the general data set according to the security attribute; clustering the second security level samples to obtain associated features; performing data enhancement on the second security level sample based on a synthetic minority class oversampling algorithm SMOTE to obtain an enhanced second security level sample; wherein the K value of the SMOTE is set according to the associated features; combining the first security level sample, the third security level sample and the enhanced second security level sample to obtain a combined sample set; and training a government affair data classification and grading model according to the combined sample set, and classifying and grading the government affair data according to the government affair data classification and grading model obtained by training. According to the method and the system, the accuracy of government affair data classification and grading can be improved under the condition of relatively small samples.

Description

Technical Field

[0001] The present invention relates to the field of government data processing technology, and in particular to an artificial intelligence-based data classification and grading method and system. Background Art

[0002] With the rapid development of information technology, the degree of digitalization of government data has continued to increase, and relevant institutions have accumulated a large amount of data resources. These data not only cover a wide range of social activities, but also involve sensitive information such as personal privacy and public safety. Therefore, how to effectively manage, protect and use these data has become an important issue that needs to be solved.

[0003] According to security attributes, government data can be divided into three levels: high security level, medium security level and low security level; among them, government data at the medium security level can be further divided into two different levels as needed, so government data can be divided into four different security levels. High security level data contains extremely sensitive personal information and must be strictly controlled and protected; medium security level data may involve some sensitive content and need to be protected to a certain extent but is allowed to be used relatively widely; low security level data is relatively less sensitive and is suitable for a wider range of application scenarios.

[0004] However, in practical applications, the sample size of government data is often insufficient, which limits its role in model training and data analysis. To address the problem of insufficient sample size, existing data enhancement methods (such as SyntheticMinority Over-sampling Technique, SMOTE for short) are used to enhance samples, but existing government data enhancement methods fail to make reasonable distinctions based on the security attributes of the data, and instead use the same enhancement method for all government data. This approach may lead to a series of problems: first, for data with a high security level, simple enhancement methods may lead to the risk of information leakage; second, the same strategy is used for data with medium and low security levels, which fails to fully tap their unique value, affecting the effect of data enhancement and ultimately affecting the accuracy of data classification and grading; at the same time, medium security level data also has the risk of sensitive information being leaked. Summary of the invention

[0005] To this end, the technical problem to be solved by the present invention is to overcome the problem in the prior art that the accuracy of government data classification and grading cannot meet the requirements due to various limitations of data enhancement, and to provide an artificial intelligence-based data classification and grading method and system, which fully considers the security attributes of samples when enhancing minority class samples, better captures the potential correlations between data, and thereby improves the accuracy of data classification and grading to meet the requirements.

[0006] In the first aspect, in order to solve the above technical problems, the present invention provides a data classification and grading method based on artificial intelligence, comprising the following steps: receiving a government data set, cleaning the government data set according to industry attributes to obtain a general data set and a non-general data set; the government data set includes multiple groups of government data, and the government data includes at least social security data, real estate data, household registration data, and education enrollment data; Marking the general data set according to the security attribute to obtain a first security level sample, a second security level sample and a third security level sample; wherein the data sensitivity characteristics of the first security level sample, the second security level sample and the third security level sample decrease in sequence; Clustering the second security level samples, identifying features of the second security level samples associated with the first security level samples, and obtaining associated features; wherein the associated features represent the degree of association between each cluster after the second security level samples are clustered and the first security level samples; Based on the synthetic minority oversampling algorithm SMOTE, data enhancement is performed on the second security level sample to obtain an enhanced second security level sample; wherein the enhanced second security level sample is marked according to the associated features, and some samples are extracted from the enhanced second security level sample and added to the first security level sample; and the K value of the synthetic minority oversampling algorithm SMOTE is set according to the associated features; Merging the first security level sample, the third security level sample, and the enhanced second security level sample to obtain a merged sample set; A government data classification and grading model is trained according to the combined sample set, and government data is classified and graded according to the government data classification and grading model obtained through training.

[0007] In one embodiment of the present invention, it also includes configuring an associated feature threshold, performing data enhancement on the second security level samples whose associated features are greater than or equal to the associated feature threshold, and adding the obtained samples to the first security level samples.

[0008] In one embodiment of the present invention, clustering the second security level samples to obtain associated features includes: Determine the number of clusters m, and cluster the second security level samples into m clusters; Extract the sample features of each cluster and obtain the sample feature matrix of all clusters; Perform feature overlap analysis on the second security level sample and the first security level sample according to the sample feature matrix to identify the associated features.

[0009] In one embodiment of the present invention, setting the K value of the synthetic minority oversampling algorithm SMOTE according to the association feature includes: Configure a first feature threshold and a second feature threshold, and the first feature threshold is smaller than the second feature threshold; If the associated feature is less than or equal to the first feature threshold, the K value of the synthetic minority oversampling algorithm SMOTE is configured as K=6 or 7 or 8; If the associated feature is greater than the first feature threshold and less than or equal to the second feature threshold, the K value of the synthetic minority class oversampling algorithm SMOTE is configured as K=5; If the associated feature is greater than the second feature threshold, the K value of the synthetic minority oversampling algorithm SMOTE is configured as K=3 or 4.

[0010] In one embodiment of the present invention, the first feature threshold and the second feature threshold are preset.

[0011] In one embodiment of the present invention, the first feature threshold and the second feature threshold are set according to the distribution of the associated features.

[0012] In one embodiment of the present invention, it further includes performing data enhancement on the third security level sample based on a synthetic minority oversampling algorithm SMOTE to obtain an enhanced third security level sample; The first security level sample, the enhanced second security level sample and the enhanced third security level sample are merged to obtain the merged sample set.

[0013] In one embodiment of the present invention, merging the first security level sample, the third security level sample and the enhanced second security level sample to obtain the merged sample set includes: Determine the total number of samples N of the combined sample set; The number of samples at each security level is determined as follows: N=A*N1+B*N2+C*N3; Among them, A, B, and C are the weights of the first security level samples, the second security level samples, and the third security level samples, respectively, satisfying A+B+C=1; N1, N2, and N3 are the number of samples of the first security level samples, the second security level samples, and the third security level samples, respectively.

[0014] In one embodiment of the present invention, merging the first security level sample, the third security level sample and the enhanced second security level sample to obtain the merged sample set further includes: When merging sample sets, randomly sample sample data at each security level until the required number of samples is reached.

[0015] In a second aspect, in order to solve the above technical problems, the present invention provides a data classification and grading system based on artificial intelligence, comprising: A data receiving module, which is used to receive a government data set, wherein the government data set includes multiple sets of government data, and the government data at least includes social security data, real estate data, household registration data, and education and enrollment data; A data cleaning module, which is used to clean the government data set according to industry attributes to obtain a general data set and a non-general data set; A security attribute marking module, which is used to mark the general data set according to the security attribute to obtain a first security level sample, a second security level sample and a third security level sample; wherein the data sensitivity characteristics of the first security level sample, the second security level sample and the third security level sample decrease in sequence; A cluster analysis module, which is used to cluster the second security level samples, identify the features of the second security level samples associated with the first security level samples, and obtain associated features; wherein the associated features represent the degree of association between each cluster after the second security level samples are clustered and the first security level samples; A data enhancement module, which is used to perform data enhancement on the second security level sample according to the synthetic minority oversampling algorithm SMOTE to obtain an enhanced second security level sample; wherein the enhanced second security level sample is marked according to the associated features, and some samples are extracted from the enhanced second security level sample and added to the first security level sample; and the K value of the synthetic minority oversampling algorithm SMOTE is set according to the associated features; A sample merging module, which is used to merge the first security level sample, the third security level sample and the enhanced second security level sample to obtain a merged sample set; A model training module is used to train a government data classification and grading model based on the combined sample training data, and classify and grade the government data based on the government data classification and grading model obtained through training.

[0016] In one embodiment of the present invention, it also includes a packaging module, which is used to encapsulate the government data classification and grading model obtained through training into a terminal device.

[0017] The above technical solution of the present invention has the following beneficial effects compared with the prior art: The artificial intelligence-based data classification and grading method and system described in the present invention are used to classify and grade government data including social security data, real estate data, household registration data and education and enrollment data; first, the government data set is cleaned according to industry attributes, non-general data sets are eliminated, and data enhancement is performed only on general data sets. Before data enhancement, the general data set is marked according to security attributes to distinguish samples of three security levels, and cluster analysis is performed on samples of medium security level to obtain associated features with samples of high security level, and data enhancement is performed on samples of medium security level according to the associated features to obtain enhanced samples of medium security level, and the enhanced samples of medium security level are merged with samples of high and low security levels to form a sample set for training a classification and grading model for government data; before data enhancement, the general data set is marked according to security attributes, and high security level samples are not marked. Data enhancement avoids leakage of sensitive data and improves classification security; medium-security level samples are first clustered and analyzed to extract associated features with high-security level samples. The extraction of associated features can dig out the potential association between medium-security level sample data and high-security level sample data, further improving classification security; on the other hand, it improves the model's learning ability for high-security level sample features without directly using high-security level samples during model training, thereby improving the accuracy and security of the classification and grading model; data enhancement is performed based on the associated features. By setting a specific K value and configuring different K values ​​for different associated features, it can effectively solve the data imbalance problem, increase the quantity and quality of minority class samples, and retain the diversity of samples, thereby improving the generalization ability of the classification and grading model, thereby further improving the accuracy of the classification and grading model to meet the requirements. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to make the content of the present invention more clearly understood, the present invention is further described in detail below according to specific embodiments of the present invention in conjunction with the accompanying drawings, wherein Figure 1 Flow chart of a data classification and grading method based on artificial intelligence in a preferred embodiment of the present invention; Figure 2 A flowchart of a data classification and grading method based on artificial intelligence in another embodiment; Figure 3 It is a structural block diagram of the data classification and grading system based on artificial intelligence in the preferred embodiment of the present invention. DETAILED DESCRIPTION

[0019] The present invention is further described below in conjunction with the accompanying drawings and specific embodiments so that those skilled in the art can better understand the present invention and implement it, but the embodiments are not intended to limit the present invention. Example

[0020] The embodiment of the present invention discloses a data classification and grading method based on artificial intelligence, which is used to classify and grade government data including social security data, real estate data, household registration data and education enrollment data; the classification categories include "personal information", "enterprise unit information", "institution information" and "government department information", etc., and the grading levels include "level one", "level two", "level three" and "level four", etc. The higher the grading level, the higher the data sensitivity. For example, level four data contains extremely sensitive personal information and must be strictly controlled and protected; level two data and level three data may involve some sensitive content, which needs to be protected to a certain extent but is allowed to be used relatively widely; level one data is relatively less sensitive and suitable for a wider range of application scenarios.

[0021] Since government data involves relevant security (or sensitivity), relevant institutions usually do not take the initiative to disclose it, the data sample size is small, and the classification accuracy of the data model trained with small samples cannot meet the requirements. In order to overcome the problem of small sample size, data enhancement methods (such as Synthetic Minority Over-sampling Technique, SMOTE for short) are currently used to enhance samples, but the existing government data enhancement methods fail to make reasonable distinctions based on the security attributes of the data, but instead use the same enhancement method for all government data. This approach may lead to a series of problems: first, for data with high security levels, simple enhancement methods may cause the risk of information leakage; second, the same strategy is used for data with medium and low security levels, which fails to fully tap their unique value, affecting the effect of data enhancement and ultimately affecting the accuracy of data classification and grading; at the same time, medium security level data also has the risk of corresponding sensitive information being leaked.

[0022] It should be noted that high, medium and low security levels are only used to divide data into three different levels from the perspective of security; among them, government data at the medium security level can be further divided into two different levels as needed, thus government data can be divided into four different security levels, for example, level one, level two, level three and level four. The higher the level of the classification, the higher the sensitivity of the data.

[0023] For the above problems, refer to Figure 1 As shown, the embodiment of the present invention discloses a data classification and grading method based on artificial intelligence, comprising the following steps: receiving a government data set, cleaning the government data set according to industry attributes to obtain a general data set and a non-general data set; the government data set includes multiple groups of government data, and the government data includes at least social security data, real estate data, household registration data, and education enrollment data; Marking the general data set according to the security attribute to obtain a first security level sample, a second security level sample and a third security level sample; wherein the data sensitivity characteristics of the first security level sample, the second security level sample and the third security level sample decrease in sequence; Clustering the second security level samples, identifying features of the second security level samples associated with the first security level samples, and obtaining associated features; wherein the associated features represent the degree of association between each cluster after the second security level samples are clustered and the first security level samples; Based on the synthetic minority oversampling algorithm SMOTE, data enhancement is performed on the second security level sample to obtain an enhanced second security level sample; wherein the enhanced second security level sample is marked according to the associated features, and some samples are extracted from the enhanced second security level sample and added to the first security level sample; and the K value of the synthetic minority oversampling algorithm SMOTE is set according to the associated features; Merging the first security level sample, the third security level sample, and the enhanced second security level sample to obtain a merged sample set; A government data classification and grading model is trained according to the combined sample set, and government data is classified and graded according to the government data classification and grading model obtained through training.

[0024] The artificial intelligence-based data classification and grading method described in the present invention is used to classify and grade government data including social security data, real estate data, household registration data and education and enrollment data; first, the government data set is cleaned according to industry attributes, non-general data sets are eliminated, and data enhancement is performed only on general data sets. Before data enhancement, the general data set is marked according to security attributes to distinguish samples of three security levels, and cluster analysis is performed on samples of medium security level to obtain associated features with samples of high security level, and data enhancement is performed on samples of medium security level according to the associated features to obtain enhanced samples of medium security level, and the enhanced samples of medium security level are merged with samples of high and low security levels to form a sample set for training the classification and grading model of government data; before data enhancement, the general data set is marked according to security attributes, and high security level samples are not clustered. The data is enhanced to avoid leakage of sensitive data and improve the security of classification. The medium security level samples are first clustered and analyzed to extract the associated features with the high security level samples. The extraction of associated features can dig out the potential association between the medium security level sample data and the high security level sample data, further improving the security of classification. On the other hand, it makes it possible to improve the model's learning ability for the features of high security level samples without directly using high security level samples during model training, thereby improving the accuracy and security of the classification and grading model. Data enhancement is performed based on the associated features. By setting a specific K value and configuring different K values ​​for different associated features, the problem of data imbalance can be effectively solved, the quantity and quality of minority class samples can be increased, the diversity of samples can be retained, and the generalization ability of the classification and grading model can be improved, thereby further improving the accuracy of the classification and grading model to meet the requirements.

[0025] In addition, some samples are screened out from the enhanced second security level samples according to the associated features and added to the first security level samples, so that the sample increase of the first security level samples is obtained without performing data enhancement on the first security level samples. In other words, the sample increase of the first security level samples is obtained without destroying the security of the first security level samples, which greatly improves the security of data classification and grading.

[0026] On the other hand, the ratio of the sample number of the first security level samples to the sample number of the second security level samples after the second security level sample data enhancement can be made consistent with that before the data enhancement; that is, due to the correlation between the first security level samples and the second security level samples, even if only the second security level samples are enhanced and the first security level samples are not enhanced, the number of samples finally obtained maintains a consistent proportional relationship with the actual number of samples, further improving the security and accuracy of the grading and classification of government data.

[0027] Specifically, the sample increase of the first security level sample without performing data enhancement on the first security level sample can be implemented based on at least the following three methods: Implementation method 1: After all samples of the second security level are enhanced, the samples of the first security level are classified and increased: First, data enhancement is performed on the second security level samples to obtain enhanced second security level samples, and samples with high security levels are extracted from the enhanced second security level samples according to associated features by screening or marking, and such samples are added to the first security level samples. Of course, in addition to being added to the first security level samples, such samples can also continue to be retained in the enhanced second security level samples, or can be removed from the enhanced second security level samples, so as to achieve the purpose that the ratio of the number of samples of the first security level samples to the number of samples of the second security level samples after data enhancement of the second security level samples can be kept consistent with that before data enhancement.

[0028] Implementation method 2: first extract some samples from the first security level samples, and then enhance the data of these samples to obtain the samples of the first security level. Configure an associated feature threshold, compare the relationship between the associated feature and the associated feature threshold, perform data enhancement on the second security level samples whose associated features are greater than or equal to the associated feature threshold, and add the obtained samples to the first security level samples.

[0029] Implementation method three is to combine implementation method one with implementation method two.

[0030] In addition, government data has strong industry attributes. Some government data are considered non-general data because their data levels are different from normal values ​​due to industry attributes, such as the mobile phone number and name of the legal person. Before data labeling, the government data set is cleaned according to industry attributes, non-general data sets are eliminated, and only general data sets (such as the legal person's ID number) are enhanced. By eliminating non-general data sets as noise and only enhancing general data, a reliable basis is provided for subsequent data enhancement, mining and decision-making, thereby improving the accuracy and stability of the classification and grading model.

[0031] It should be noted that the SMOTE algorithm (SMOTESynthetic Minority Over-sampling Technique) algorithm is a synthetic minority oversampling algorithm, which is one of the existing algorithms for enhancing sample data. The K value of the SMOTE algorithm refers to the number of nearest neighbors selected to generate synthetic samples during interpolation synthesis. Specifically, the working principle of the SMOTE algorithm is as follows: Calculate the nearest neighbors: For each minority class sample (i.e., the class whose number needs to be increased), SMOTE calculates the distance between the sample and other minority class samples and finds the nearest K neighbors.

[0032] Generate synthetic samples: Among these K neighbors, a neighbor is randomly selected. Then, SMOTE will generate a new synthetic sample by linearly interpolating between the original sample and the selected neighbor. This process will produce multiple synthetic samples, the specific number of which depends on the expected number of samples to be increased.

[0033] Selection of K value: The selection of K value will affect the diversity and distribution of generated synthetic samples. In practical applications, the K value is adjusted according to the characteristics of the dataset and actual needs.

[0034] Specifically, clustering the second security level samples to obtain associated features includes: Determine the number of clusters m, and cluster the second security level samples into m clusters; Extract the sample features of each cluster and obtain the sample feature matrix of all clusters; Perform feature overlap analysis on the second security level sample and the first security level sample according to the sample feature matrix to identify the associated features.

[0035] In specific application scenarios, some data are marked as second-level security samples based on security attributes, but the second-level security samples may also involve some sensitive content that needs to be protected to a certain extent, such as the social security account number of social security data, the real estate certificate number of personal real estate data, and the ID number of the head of household and family members of the household registration data. Cluster analysis is performed on the second-level security samples to extract related features that overlap with the first-level security sample features for subsequent data enhancement.

[0036] Further, setting the K value of the synthetic minority class oversampling algorithm SMOTE according to the associated features includes: Configure a first feature threshold and a second feature threshold, and the first feature threshold is smaller than the second feature threshold; If the associated feature is less than or equal to the first feature threshold, the K value of the synthetic minority class oversampling algorithm SMOTE is configured as K=8; If the associated feature is greater than the first feature threshold and less than or equal to the second feature threshold, the K value of the synthetic minority class oversampling algorithm SMOTE is configured as K=5; If the associated feature is greater than the second feature threshold, the K value of the synthetic minority oversampling algorithm SMOTE is configured as K=3.

[0037] In specific application scenarios, the associated feature is less than or equal to the first feature threshold, indicating that the sensitivity of the second security full-level samples is weak. The K value of SMOTE is configured to K=8 to improve the diversity of data enhancement; the associated feature is greater than the first feature threshold and less than or equal to the second feature threshold, indicating that the second security level samples have a certain sensitivity. The K value of SMOTE is configured to K=5, on the one hand, the newly generated samples tend to be close to the original samples, ensuring the quality and security of the enhanced data; and at the same time, it promotes the diversification of generated samples to a certain extent; the associated feature is greater than the second feature threshold, indicating that the sensitivity of the second security level samples is strong. The K value of SMOTE is configured to K=3, so that the newly generated samples are concentrated near the original samples to ensure the security of the enhanced data.

[0038] The first feature threshold and the second feature threshold are preset, especially set according to the distribution of the associated features.

[0039] In another embodiment of the present invention, referring to Figure 2 As shown, it also includes performing data enhancement on the third security level samples based on the synthetic minority oversampling algorithm SMOTE to obtain enhanced third security level samples; merging the first security level samples, the enhanced second security level samples and the enhanced third security level samples to obtain the merged sample set.

[0040] In specific application scenarios, the third security level samples are relatively less sensitive and suitable for a wider range of application scenarios. Data enhancement is performed based on conventional data enhancement methods. Preferably, the third security level samples are enhanced based on the synthetic minority oversampling algorithm SMOTE to obtain enhanced data with data quality that meets the requirements.

[0041] Further, the first security level sample, the third security level sample and the enhanced second security level sample are combined to obtain a combined sample set including: Determine the total number of samples N of the combined sample set; The number of samples at each security level is determined as follows: N=A*N1+B*N2+C*N3; Among them, A, B, and C are the weights of the first security level samples, the second security level samples, and the third security level samples, respectively, satisfying A+B+C=1; N1, N2, and N3 are the number of samples of the first security level samples, the second security level samples, and the third security level samples, respectively.

[0042] In specific application scenarios, the enhanced second security level samples and the enhanced third security level samples need to be merged with the first security level samples to form a merged sample set for subsequent model training, testing and verification. In a preferred embodiment of the present invention, data is merged based on weight setting. For example, different weights are set according to the importance of the security level to expand the influence of high security level samples in the training process. High security level samples can be given higher weights during model training, which can be achieved by setting a loss function to emphasize the influence of high security samples in the loss calculation.

[0043] Furthermore, merging the first security level sample, the enhanced third security level sample and the enhanced second security level sample to obtain the merged sample set also includes randomly sampling sample data of each security level when merging the sample set until the required number of samples is reached. Randomly sampling sample data of each level to form a merged sample set avoids overfitting during model training and further improves the classification accuracy of the classification and grading model. Example

[0044] The difference between this embodiment and the first embodiment is that the K value configuration of the synthetic minority class oversampling algorithm SMOTE is different. Specifically, Configure a first feature threshold and a second feature threshold, and the first feature threshold is smaller than the second feature threshold; If the associated feature is less than or equal to the first feature threshold, the K value of the synthetic minority class oversampling algorithm SMOTE is configured as K=7; If the associated feature is greater than the first feature threshold and less than or equal to the second feature threshold, the K value of the synthetic minority class oversampling algorithm SMOTE is configured as K=5; If the associated feature is greater than the second feature threshold, the K value of the synthetic minority oversampling algorithm SMOTE is configured as K=3. Example

[0045] The difference between this embodiment and the first embodiment is that the K value configuration of the synthetic minority class oversampling algorithm SMOTE is different. Specifically, Configure a first feature threshold and a second feature threshold, and the first feature threshold is smaller than the second feature threshold; If the associated feature is less than or equal to the first feature threshold, the K value of the synthetic minority class oversampling algorithm SMOTE is configured as K=6; If the associated feature is greater than the first feature threshold and less than or equal to the second feature threshold, the K value of the synthetic minority class oversampling algorithm SMOTE is configured as K=5; If the associated feature is greater than the second feature threshold, the K value of the synthetic minority oversampling algorithm SMOTE is configured as K=4.

[0046] According to different K values ​​of the configured SMOTE, the classification accuracy of the classification and grading models obtained in Example 1, Example 2 and Example 3 are as follows: Example 1 is the highest, Example 2 is the second, and Example 3 is the worst. Example

[0047] Reference Figure 3 As shown, this embodiment discloses a data classification and grading system based on artificial intelligence, including: A data receiving module, which is used to receive a government data set, wherein the government data set includes multiple sets of government data, and the government data at least includes social security data, real estate data, household registration data, and education and enrollment data; A data cleaning module, which is used to clean the government data set according to industry attributes to obtain a general data set and a non-general data set; A security attribute marking module, which is used to mark the general data set according to the security attribute to obtain a first security level sample, a second security level sample and a third security level sample; wherein the data sensitivity characteristics of the first security level sample, the second security level sample and the third security level sample decrease in sequence; A cluster analysis module, which is used to cluster the second security level samples, identify the features of the second security level samples associated with the first security level samples, and obtain associated features; wherein the associated features represent the degree of association between each cluster after the second security level samples are clustered and the first security level samples; A data enhancement module, which is used to perform data enhancement on the second security level sample based on the synthetic minority oversampling algorithm SMOTE to obtain an enhanced second security level sample; wherein the K value of the synthetic minority oversampling algorithm SMOTE is set according to the associated features; A sample merging module, which is used to merge the first security level sample, the third security level sample and the enhanced second security level sample to obtain a merged sample set; A model training module is used to train a government data classification and grading model based on the combined sample training data, and classify and grade the government data based on the government data classification and grading model obtained through training.

[0048] The artificial intelligence-based data classification and grading system described in the present invention is used to classify and grade government data including social security data, real estate data, household registration data and education and enrollment data; first, the government data set is cleaned according to industry attributes, non-general data sets are eliminated, and data enhancement is performed only on general data sets. Before data enhancement, the general data set is marked according to security attributes, samples of three security levels are distinguished, and cluster analysis is performed on samples of medium security level to obtain associated features with samples of high security level, and data enhancement is performed on samples of medium security level according to the associated features to obtain enhanced samples of medium security level, and the enhanced samples of medium security level are merged with samples of high and low security levels to form a sample set for training the classification and grading model of government data; before data enhancement, the government data set is marked according to security attributes, and high security level samples are not clustered. The data is enhanced to avoid leakage of sensitive data and improve the security of classification. The medium security level samples are first clustered and analyzed to extract the associated features with the high security level samples. The extraction of associated features can dig out the potential association between the medium security level sample data and the high security level sample data, further improving the security of classification. On the other hand, it makes it possible to improve the model's learning ability for the features of high security level samples without directly using high security level samples during model training, thereby improving the accuracy and security of the classification and grading model. Data enhancement is performed based on the associated features. By setting a specific K value and configuring different K values ​​for different associated features, the problem of data imbalance can be effectively solved, the quantity and quality of minority class samples can be increased, the diversity of samples can be retained, and the generalization ability of the classification and grading model can be improved, thereby further improving the accuracy of the classification and grading model to meet the requirements.

[0049] The government data classification and grading model obtained by the present invention has government data as input and categories and grades as output.

[0050] Furthermore, it also includes a packaging module, which is used to encapsulate the government data classification and grading model obtained through training into a terminal device.

[0051] In specific application scenarios, the government data classification and grading model obtained by the embodiment of the present invention is used to process government data. Since government data has a certain degree of sensitivity, the classification and grading model is not deployed to the cloud, but is packaged to the user's terminal device according to usage needs.

[0052] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0053] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0054] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0055] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0056] Obviously, the above embodiments are merely examples for clear explanation and are not intended to limit the implementation methods. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the implementation methods here. The obvious changes or modifications derived from these are still within the protection scope of the invention.

Claims

1. A method for classifying and grading government data based on artificial intelligence, characterized by: The following steps are included: receiving a government data set, cleaning the government data set according to industry attributes to obtain a general data set and a non-general data set; the government data set includes multiple groups of government data, and the government data includes at least social security data, real estate data, household registration data, and education enrollment data; Marking the general data set according to the security attribute to obtain a first security level sample, a second security level sample and a third security level sample; wherein the data sensitivity characteristics of the first security level sample, the second security level sample and the third security level sample decrease in sequence; Clustering the second security level samples, identifying features of the second security level samples associated with the first security level samples, and obtaining associated features; wherein the associated features represent the degree of association between each cluster after the second security level samples are clustered and the first security level samples; Based on the synthetic minority oversampling algorithm SMOTE, data enhancement is performed on the second security level sample to obtain an enhanced second security level sample; wherein the enhanced second security level sample is marked according to the associated features, and some samples are extracted from the enhanced second security level sample and added to the first security level sample; and the K value of the synthetic minority oversampling algorithm SMOTE is set according to the associated features; Merging the first security level sample, the third security level sample, and the enhanced second security level sample to obtain a merged sample set; A government data classification and grading model is trained according to the combined sample set, and government data is classified and graded according to the government data classification and grading model obtained through training.

2. The method for classifying and grading government data based on artificial intelligence according to claim 1 is characterized by: It also includes configuring an associated feature threshold, performing data enhancement on the second security level samples whose associated features are greater than or equal to the associated feature threshold, and adding the samples obtained to the first security level samples.

3. The method for classifying and grading government data based on artificial intelligence according to claim 1 or 2, characterized in that: Clustering the second security level samples to obtain associated features includes: Determine the number of clusters m, and cluster the second security level samples into m clusters; Extract the sample features of each cluster and obtain the sample feature matrix of all clusters; Perform feature overlap analysis on the second security level sample and the first security level sample according to the sample feature matrix to identify the associated features.

4. The method for classifying and grading government data based on artificial intelligence according to claim 3 is characterized by: Setting the K value of the synthetic minority class oversampling algorithm SMOTE according to the associated features includes: Configure a first feature threshold and a second feature threshold, and the first feature threshold is smaller than the second feature threshold; If the associated feature is less than or equal to the first feature threshold, the K value of the synthetic minority class oversampling algorithm SMOTE is configured as K=6 or 7 or 8; If the associated feature is greater than the first feature threshold and less than or equal to the second feature threshold, the K value of the synthetic minority class oversampling algorithm SMOTE is configured as K=5; If the associated feature is greater than the second feature threshold, the K value of the synthetic minority oversampling algorithm SMOTE is configured as K=3 or 4.

5. The method for classifying and grading government data based on artificial intelligence according to claim 4 is characterized in that: The first feature threshold and the second feature threshold are set according to the distribution of the associated features.

6. The method for classifying and grading government data based on artificial intelligence according to claim 1 is characterized by: It also includes performing data enhancement on the third security level sample based on a synthetic minority oversampling algorithm SMOTE to obtain an enhanced third security level sample; The first security level sample, the enhanced second security level sample and the enhanced third security level sample are merged to obtain the merged sample set.

7. The method for classifying and grading government data based on artificial intelligence according to claim 1 is characterized by: The first security level sample, the third security level sample and the enhanced second security level sample are combined to obtain a combined sample set including: Determine the total number of samples N of the combined sample set; The number of samples at each security level is determined according to the following method: N = A*N1+B*N2+C*N3; Among them, A, B, and C are the weights of the first security level samples, the second security level samples, and the third security level samples, respectively, satisfying A+B+C=1; N1, N2, and N3 are the number of samples of the first security level samples, the second security level samples, and the third security level samples, respectively.

8. The method for classifying and grading government data based on artificial intelligence according to claim 7 is characterized by: Combining the first security level sample, the third security level sample and the enhanced second security level sample to obtain the combined sample set further includes: When merging sample sets, randomly sample sample data at each security level until the required number of samples is reached.

9. An artificial intelligence-based government data classification and grading system, characterized by: include, A data receiving module, which is used to receive a government data set, wherein the government data set includes multiple sets of government data, and the government data at least includes social security data, real estate data, household registration data, and education and enrollment data; A data cleaning module, which is used to clean the government data set according to industry attributes to obtain a general data set and a non-general data set; A security attribute marking module, which is used to mark the general data set according to the security attribute to obtain a first security level sample, a second security level sample and a third security level sample; wherein the data sensitivity characteristics of the first security level sample, the second security level sample and the third security level sample decrease in sequence; A cluster analysis module, which is used to cluster the second security level samples, identify the features of the second security level samples associated with the first security level samples, and obtain associated features; wherein the associated features represent the degree of association between each cluster after the second security level samples are clustered and the first security level samples; A data enhancement module, which is used to perform data enhancement on the second security level sample according to the synthetic minority oversampling algorithm SMOTE to obtain an enhanced second security level sample; wherein the enhanced second security level sample is marked according to the associated features, and some samples are extracted from the enhanced second security level sample and added to the first security level sample; and the K value of the synthetic minority oversampling algorithm SMOTE is set according to the associated features; A sample merging module, which is used to merge the first security level sample, the third security level sample and the enhanced second security level sample to obtain a merged sample set; A model training module is used to train a government data classification and grading model based on the combined sample training data, and classify and grade the government data based on the government data classification and grading model obtained through training.

10. The artificial intelligence-based government data classification and grading system according to claim 9 is characterized by: It also includes a packaging module, which is used to package the government data classification and grading model obtained through training into a terminal device.

Images