An Automatic Mining and Classification Method for Private Data

Abstract

This invention discloses an automatic mining and classification method for private data, comprising the following steps: First, receiving private unstructured documents, performing preprocessing and semantic alignment, and using sliding window technology to segment the documents into continuous text blocks; then loading a locally deployed general basic model and an industry-specific model, calculating the perplexity of each text block, and determining the scarcity of the text block by comparing the differences in the model output; next, constructing a statistical model based on the perplexity distribution of the enterprise's historical documents and dynamically setting a threshold, calculating the value score for text blocks exceeding the threshold; finally, projecting the scarcity and value score onto a preset classification decision matrix to automatically determine the secret level of the text block and trigger corresponding security handling strategies. This invention achieves automated and intelligent classification of private data, improving the efficiency and accuracy of data security management.

Description

Technical Field

[0001] This invention relates to a method for automatic mining and grading of private data, belonging to the field of data classification technology. Background Technology

[0002] As enterprises deepen their digital transformation, the number of unstructured documents, such as technical reports, design drawings, meeting minutes, email communications, and code files, is growing exponentially. These documents contain a large amount of commercially valuable or sensitive private data, such as core algorithm parameters, strategic plans, customer lists, and intellectual property information. Effectively identifying and managing this data in a tiered manner has become a crucial aspect of enterprise data security governance.

[0003] Currently, the manual classification method commonly used by enterprises mainly relies on the experience of document owners or security administrators. This method has the following significant drawbacks: it is inefficient and cannot cope with the processing needs of massive amounts of data; the standards are not uniform, and different people may have significantly different classification results for the same document; it is difficult to identify high-value segments hidden in long texts, which can easily lead to the leakage of sensitive information or the underestimation of its commercial value.

[0004] In the field of automated data classification, existing technologies are mostly based on keyword matching, rule engines, or traditional machine learning models. One approach involves matching document content with a predefined sensitive word library to determine its security level. While this method achieves a degree of automation, it struggles with complex semantic expressions and is prone to false positives or false negatives. Another approach uses TF-IDF features and a Naive Bayes classifier for document classification. However, this method relies heavily on the quality and quantity of labeled samples and cannot effectively identify highly scarce information specific to individual companies, lacking the ability to model industry knowledge.

[0005] In recent years, large language models have made significant progress in the field of natural language processing, enabling them to learn rich knowledge representations from massive amounts of internet data. However, directly applying general-purpose large language models to the classification of enterprise private data presents several challenges: the general-purpose model cannot distinguish between enterprise-specific knowledge and industry common knowledge, making it difficult to identify truly scarce information; the perplexity index output by the model lacks a dynamic benchmark, making it difficult to set a unified judgment threshold; and relying solely on perplexity cannot quantify the commercial value of text, requiring a comprehensive evaluation in conjunction with semantic features. Summary of the Invention

[0006] The technical problem to be solved by this invention is that the prior art suffers from low efficiency and inconsistent standards due to reliance on manual labor for private data classification.

[0007] To address the aforementioned technical problems, this invention proposes an automatic mining and grading method for private data, comprising the following steps:

[0008] S1. First, the private unstructured document is received and preprocessed. The local model is used to perform semantic alignment on the preprocessed document. Then, the sliding window technique is used to segment the document into continuous text blocks.

[0009] S2. Then load the locally deployed general base model and industry-specific model, calculate the perplexity of each text block on the base model and industry-specific model respectively, and determine the scarcity of the text block by comparing the output difference between the base model and the industry-specific model.

[0010] S3. Secondly, based on the distribution of confusion in the company's historical documents, a statistical model is constructed and a judgment threshold is dynamically set. When the confusion of a text block exceeds the threshold, a value score is calculated.

[0011] S4. Finally, the scarcity and value scores are projected onto a preset hierarchical decision matrix for comprehensive judgment, and the final secret level of the text block is automatically determined; the corresponding security handling strategy is automatically triggered according to the determined secret level.

[0012] The preprocessing in step S1, as described above, includes:

[0013] The documents are uniformly converted to UTF-8 encoding, irrelevant characters and noise are removed, and paragraphs and sentences are segmented. For non-text content, OCR or speech recognition tools are used to extract text content, and table data is recognized and extracted and converted into structured text for language recognition and translation alignment.

[0014] The semantic alignment in step S1, as described above, includes:

[0015] A pre-built enterprise terminology mapping table aligns internal terms with industry standard terms; a BERT-based semantic similarity model is used to merge synonyms and convert between simplified and traditional Chinese characters, outputting a standardized text stream.

[0016] The sliding window technique in step S1 described above is specifically as follows:

[0017] Set the window length L and the overlap length O, perform overlapping segmentation on the standardized text stream, and output a continuous sequence of knowledge blocks {T1, T2, ..., T}. n}

[0018] In step S2 above, a preliminary classification is performed based on the bi-model perplexity values:

[0019] If PPL_base(T) is less than the base threshold, it is considered common sense.

[0020] If PPL_base(T) is greater than the base threshold and PPL_domain(T) is less than a specific threshold, it is considered industry common knowledge.

[0021] If PPL_base(T) is greater than the base threshold and PPL_domain(T) is greater than a specific threshold, it is marked as a highly scarce knowledge block.

[0022] The dynamic threshold in step S3 described above is specifically as follows:

[0023] Collect non-sensitive historical documents of the enterprise as a baseline corpus, and perform cleaning and segmentation consistent with the main process to obtain baseline text blocks;

[0024] Input the baseline text block into the basic model to calculate the perplexity and obtain the baseline PPL value set;

[0025] The mean μ and standard deviation σ of the benchmark PPL value set are calculated, and the dynamic threshold λ is calculated using the formula λ=μ+k·σ, where k is a sensitivity coefficient set according to the enterprise's risk preference.

[0026] The value scoring calculation in step S3 mentioned above includes extracting semantic value anchors for highly scarce knowledge blocks. The semantic value anchors include parameter anchors, causal anchors, and asset anchors.

[0027] The parameter anchor is used to identify phrases containing numbers, and dependency parsing is used to bind the numerical value to the unit and attribute it modifies;

[0028] The causal anchor point is used to identify logical relation words. By using dependency parsing, cause and effect fragments are extracted and relation triplets of cause, relation, and effect are constructed.

[0029] The asset anchor is a finely tuned NER model used to identify enterprise-defined high-value entity types, including core algorithms, business assets, intellectual property, and strategic information.

[0030] The value score is calculated using the following formula:

[0031] Score_value=α·N_param+β·N_logic+γ·Nasset;

[0032] Where N_param is the number of parameter anchors, N_logic is the number of causal anchors, N_asset is the number of asset anchors, and α, β, γ are the weight coefficients of the three types of anchors, and satisfy α+β+γ=1.

[0033] As mentioned above, determining the final secrecy level in step S4 includes:

[0034] The number and proportion of all knowledge blocks marked as highly scarce in the document are statistically analyzed, and the overall scarcity level of the document is determined based on a preset threshold.

[0035] The maximum or weighted average value score of all knowledge blocks in a document is taken as the representative value score of the document. A value score threshold range is set to classify the document into high-value, medium-value, and low-value levels.

[0036] The perplexity PPL mentioned above is used to quantify the unexpectedness or scarcity of text content in the public knowledge domain; the Base LLM is a general-purpose large model pre-trained on massive public Internet data, without being fine-tuned with private data from specific enterprises.

[0037] Domain LLM, an industry-specific model, is a model that has been fine-tuned based on literature in a specific vertical field and represents common knowledge within the industry.

[0038] This invention has positive effects:

[0039] (1) This invention can effectively distinguish between common sense, industry knowledge and enterprise-specific high-scarcity knowledge by comparing the confusion of the two models, thereby improving the accuracy of scarcity determination.

[0040] (2) The present invention introduces a dynamic threshold mechanism, which enables the grading standard to adapt to the language characteristics and risk preferences of enterprise documents, thereby enhancing the applicability and flexibility of the method.

[0041] (3) The classification results of this invention can automatically trigger corresponding security handling strategies, realize the seamless connection between data protection and business flow, and improve the overall information security level of enterprises. Attached Figure Description

[0042] The invention will now be further described with reference to the accompanying drawings.

[0043] Figure 1 This is a schematic diagram of the process of the present invention;

[0044] Figure 2 This is a schematic diagram of the specific process of Example 1. Detailed Implementation

[0045] Perplexity (PPL) is a metric used in natural language processing to measure the quality of a probabilistic model's predictions. In this invention, PPL is used to quantify the unexpectedness or scarcity of text content in the public knowledge domain. A higher PPL value indicates that the information is rarer in public data.

[0046] Base LLM refers to a general-purpose large model that has been pre-trained on massive amounts of public internet data, such as Llama-Base and Qwen-Base. It has not been fine-tuned with private data from specific companies and is regarded as a projection of the entire set of human public knowledge.

[0047] Domain LLM refers to models that have been fine-tuned in the literature for a specific vertical field, such as pharmaceuticals or automobile manufacturing, and represent the general knowledge within the industry.

[0048] Knowledge Entropy Difference refers to the difference in probability distribution between a general basic model and an industry-specific model, or between model predictions and statistical benchmarks, for the same text. It is used to determine the originality of information.

[0049] Semantic Value Anchors: These refer to syntactic or lexical structures in a text that possess high commercial value, including parameter anchors (core values), causal anchors (logical chains), and asset anchors (highly sensitive vocabulary).

[0050] Sliding window is a text segmentation technique that divides long documents into fixed-length, overlapping chunks to maintain semantic coherence.

[0051] Example 1

[0052] See Figures 1 to 2 This embodiment 1 includes the following steps:

[0053] S1. First, the private unstructured document is received and preprocessed. The local model is used to perform semantic alignment on the preprocessed document. Then, the sliding window technique is used to segment the document into continuous text blocks.

[0054] S2. Then load the locally deployed general base model and industry-specific model, calculate the perplexity of each text block on the base model and industry-specific model respectively, and determine the scarcity of the text block by comparing the output difference between the base model and the industry-specific model.

[0055] S3. Secondly, based on the distribution of confusion in the company's historical documents, a statistical model is constructed and a judgment threshold is dynamically set. When the confusion of a text block exceeds the threshold, a value score is calculated.

[0056] S4. Finally, the scarcity and value scores are projected onto a preset hierarchical decision matrix for comprehensive judgment, and the final secret level of the text block is automatically determined; the corresponding security handling strategy is automatically triggered according to the determined secret level.

[0057] The preprocessing is as follows:

[0058] The system receives unstructured documents from within, including text files, reports, emails, design documents, code files, configuration documents, meeting minutes, video, or audio.

[0059] 1. Standardization and Cleaning

[0060] First, convert the format to UTF-8 encoding. Remove noise by deleting special symbols, garbled characters, advertisements, headers and footers, and duplicate content. Then, segment paragraphs and sentences based on punctuation and line breaks.

[0061] 2. Text Extraction and Enhancement

[0062] Then, OCR or speech recognition tools are used to extract the text content, recognize and extract the table data, and convert it into structured text. If multilingual content is involved, language recognition and translation alignment are performed.

[0063] Semantic alignment is as follows:

[0064] Construct a corporate terminology mapping table to align internal terminology with industry standard terminology. For example, "Customer" → "Customer" and "Beijing Branch".

[0065] “BJOffice” → “Branch_Beijing”

[0066] The semantic similarity model based on BERT is used to merge synonyms and convert between simplified and traditional Chinese characters, and finally output a standardized text stream.

[0067] The document splitting is as follows:

[0068] Using the sliding window technique, the window length L = 512 tokens and the overlap length O = 64 tokens are set.

[0069] Then, overlapping segmentation is performed, with a certain degree of overlap between adjacent text blocks to maintain semantic coherence. The standardized text stream is segmented, outputting a continuous sequence of knowledge blocks {T1,T2,…,T...}. n}

[0070] Sliding window technology is a common segmentation method in data processing, especially in sequence data. It divides a long data sequence, such as text, time series signals, or genomic sequences, into a series of continuous and potentially overlapping shorter segments, i.e., windows, to facilitate segment-by-segment analysis or model processing. It is used to solve the fundamental contradiction of inputting long documents into NLP models with length limitations, such as large language models, which usually have an input limit of 512 or 2048 tokens. At the same time, by setting overlapping regions, it effectively avoids the complete semantic units being abruptly cut between two windows, which could lead to the loss or misunderstanding of key information.

[0071] In this embodiment, a long document is divided into knowledge blocks of 512 tokens each, starting from the beginning. Instead of jumping directly to the beginning of the next 512 tokens, the window goes back 64 tokens and then takes another 512 tokens. This way, there are 64 overlapping tokens between adjacent knowledge blocks, ensuring that any important, cross-boundary semantic information is not lost due to the abrupt segmentation, thus obtaining more accurate and complete analysis results in subsequent steps.

[0072] The perplexity calculation and scarcity determination are as follows:

[0073] 1. For each knowledge block T, input it into the Base LLM and Domain LLM models respectively, and calculate the perplexity PPL_base(T) of the Base LLM and the perplexity PPL_domain(T) of the Domain LLM.

[0074] The formula for calculating perplexity is:

[0075]

[0076] Where t is the number of tokens and P is the conditional probability output by the model;

[0077] 2. Preliminary classification based on the PPL values ​​of the dual-model system:

[0078] If PPL_base(T) < the base threshold: this is considered common sense.

[0079] If PPL_base(T) > base threshold and PPL_domain(T) < specific threshold: this is considered industry common sense.

[0080] If PPL_base(T) > base threshold and PPL_domain(T) > specific threshold: mark as a highly scarce knowledge block.

[0081] The base threshold and specific threshold are configurable parameters, which are obtained by pre-calibration based on the benchmark performance of the selected base model and industry model on their respective representative public corpora.

[0082] The dynamic threshold is as follows:

[0083] 1. Construction of Historical Document Baseline

[0084] We collected non-sensitive documents accumulated from the company's history as a baseline corpus, covering various types such as technical documents, reports, and manuals;

[0085] The baseline document undergoes semantic cleaning and standardization processing consistent with the main process, and the document is divided into text blocks using the same sliding window parameters as in step S1.

[0086] 2. Calculation of baseline perplexity distribution

[0087] Input each text block in the baseline set into the base model, calculate the perplexity value of each text block, and obtain the baseline PPL value set;

[0088] Statistical analysis is performed on the benchmark PPL value set to calculate the central tendency index and dispersion index of the distribution, and to determine whether it conforms to common statistical distribution characteristics.

[0089] Perform a normality test to determine whether the baseline distribution is approximately normal. If it deviates significantly from the normal distribution, consider using data transformation or nonparametric methods for processing.

[0090] 3. Dynamic threshold strategy

[0091] 1. Construction of Historical Document Baseline

[0092] Collect non-sensitive historical documents of the enterprise as a benchmark corpus to ensure coverage of various common document types and that the benchmark set is of sufficient size and representativeness to reflect the linguistic characteristics of the enterprise's common documents.

[0093] The baseline document undergoes semantic cleaning and standardization consistent with the main process. The document is divided into text blocks T using the same sliding window parameters as S1, and excessively short text blocks are excluded to ensure the effectiveness of the analysis.

[0094] 2. Calculation of baseline perplexity distribution

[0095] Input each text block in the baseline set into the base model Base LLM;

[0096] Calculate the perplexity PPL_base value for each text block to obtain the baseline PPL value set {PPL_1,PPL_2,...,PPL_N}.

[0097] Statistical analysis is performed on the benchmark PPL value set to calculate the mean μ and standard deviation σ of the distribution, analyze the distribution shape, and test whether it is approximately a normal distribution.

[0098] 3. Dynamic threshold calculation

[0099] Based on the statistical characteristics of the benchmark distribution, the dynamic threshold λ is calculated using the formula λ=μ+k·σ, where μ is the mean of the benchmark PPL value set, σ is the standard deviation of the benchmark PPL value set, and k is the sensitivity coefficient, which is set according to the company's risk preference;

[0100] 4. Dynamic threshold application

[0101] Calculate the PPL_base(T) value for each knowledge block T, and compare the PPL_base(T) value with the dynamic threshold λ:

[0102] When PPL_base(T) is greater than λ: the knowledge block is determined to have significant "non-public knowledge" and is marked as a "highly scarce knowledge block", and then proceeds to the subsequent semantic value anchor extraction and scoring steps;

[0103] If PPL_base(T) is less than λ, then the knowledge block belongs to the scope of the enterprise's general knowledge and will not be subject to further in-depth value analysis.

[0104] The specific value rating is as follows:

[0105] 1. First, extract the parameter anchor points.

[0106] 1.1 Parameter Anchor Point Extraction

[0107] Numerical entity recognition identifies phrases containing numbers, and binds the numerical value to its modified unit, attribute, and subject through dependency parsing; the extracted numerical parameters are then converted into a unified format.

[0108] For example, the extracted result of the text: "Control the reaction temperature at 150±2℃ and maintain it for 3.5 hours" is: [parameter anchor: {value: 150±2, unit: ℃, attribute: reaction temperature}, {value: 3.5, unit: hours, attribute: maintenance time}].

[0109] 1.2 Causal Anchor Point Extraction

[0110] The system identifies logical relation words such as "cause," "therefore," "make," "because," "so," and "if," and through dependency parsing, analyzes the syntax tree to locate the core arguments governed by the logical connectors, which are usually noun phrases or clauses. This allows the extraction of two segments, cause and effect, to construct a structured representation of a relation triplet consisting of cause, relation, and effect.

[0111] For example, the extraction result of the text "The reaction yield was increased by 15% due to the use of a new catalyst A" is [causal anchor: {cause: new catalyst A, relation: increased, result: reaction yield increased by 15%}].

[0112] 1.3 Asset Anchor Extraction

[0113] By fine-tuning a NER model, we can specifically identify enterprise-defined high-value entity types.

[0114] Define entity types, including core algorithms, business assets, intellectual property, and strategic information;

[0115] For example, the extraction result of the text: "See the attached document for the list of core customers for this year and the corresponding discount price list." is {Type: Commercial Assets, Content: List of Core Customers}, {Type: Commercial Assets, Content: Discount Price List}.

[0116] 2. Value Rating Calculation

[0117] The calculation is performed using the following formula:

[0118] Scorevalue=α⋅Nparam+β⋅Nlogic+γ⋅Nasset;

[0119] Where Score_value is the business value score of the knowledge block; Nparam is the number of parameter anchors identified in the knowledge block; Nlogic is the number of causal anchors identified in the knowledge block; Nasset is the number of asset anchors identified in the knowledge block; α, β, and γ are the weight coefficients of the three types of anchors, and satisfy α+β+γ=1. The weights reflect the degree of importance that enterprises attach to different value dimensions.

[0120] 3. Output: For each highly scarce knowledge block, output a structured result. This structured result includes the specific number of the three types of anchors, the anchor statistics in the list, the calculated value score, the position of the anchor in the original text, and the original data of the content fragment.

[0121] The final secrecy level is as follows:

[0122] The number and proportion of all knowledge blocks marked as highly scarce in the statistical documents are determined by PPL_base>λ.

[0123] As shown in Table 1, the overall scarcity level of the document is first determined based on a preset threshold, and then entropy difference scarcity aggregation is performed:

[0124]

[0125] Among them, "double high" refers to the proportion of highly scarce knowledge blocks that simultaneously satisfy PPL_domain being greater than a specific threshold, which exceeds the set value.

[0126] The proportion of highly scarce knowledge blocks exceeds the set value, but most of them only satisfy PPL_base>λ while PPL_domain does not exceed the specific threshold;

[0127] Low: The proportion of highly scarce knowledge blocks is lower than a set threshold;

[0128] Then, value scoring is aggregated:

[0129] The maximum value score of all knowledge blocks in the document is taken as the representative value score of the document.

[0130] Alternatively, a weighted average can be used: calculated by weighting based on the length of the knowledge block or the information density;

[0131] Set a value threshold range to classify documents into high, medium, and low value levels.

[0132] Example 2

[0133] This embodiment, based on embodiment 1, further introduces a dynamic weight adaptive mechanism for semantic value anchors to solve the problem that fixed weights cannot adapt to changes in different document types, business scenarios, and user feedback. The details are as follows:

[0134] Using the same steps S1 to S3 as in Example 1, the private unstructured document is preprocessed, semantically aligned, and segmented using a sliding window. Highly scarce knowledge blocks are then marked using a dual-model perplexity comparison and dynamic threshold strategy.

[0135] Initialize value rating anchor weights

[0136] Initial weight coefficients α0, β0, and γ0 are set for three types of semantic value anchors: parameter anchors, causal anchors, and asset anchors, satisfying α0 + β0 + γ0 = 1. The initial weights can be set based on the company's historical experience or industry-standard rules.

[0137] For example, α0=0.3, β0=0.3, γ0=0.4; or it can be obtained by pre-training by analyzing the optimal weight combination of historically graded documents.

[0138] Value score calculation based on current weight

[0139] For each highly scarce knowledge block T, extract the number of three types of anchor points N_param, N_logic, and N_asset, and calculate the value score according to the current weights (α,β,γ):

[0140]

[0141] The scoring is then combined with scarcity, and the final secret level of knowledge blocks and documents is determined based on a pre-defined hierarchical decision matrix.

[0142] Feedback signal acquisition

[0143] After the classification results are output, feedback signals from subsequent stages are collected to evaluate the rationality of the current weight configuration. These feedback signals include:

[0144] Manual correction signal: If a security auditor or document owner manually adjusts the automatic classification results, the correction action is recorded;

[0145] Access control logs: Statistical analysis of access frequency to high-secret documents and abnormal visitor identities;

[0146] Leakage event marker: If a document is leaked and confirmed to be of high value, it is marked as positive feedback;

[0147] Business value confirmation: Indirect indicators such as the number of times documents are cited and their contribution in actual projects are recorded through business systems.

[0148] All feedback signals need to be normalized to a single value R, ranging from [−1, 1]. A positive value indicates that the current classification meets expectations, while a negative value indicates that the classification deviation is too large.

[0149] Weight adaptive update

[0150] Based on the collected feedback signals, the weighting coefficients (α, β, γ) are dynamically adjusted to gradually approach the optimal configuration. This embodiment provides three update mechanisms, which are adaptively selected:

[0151] Mechanism 1 involves weight adjustment based on moving averages. For each batch of feedback, the contribution of various anchor points to the grading bias is statistically analyzed. Let the actual value score of a knowledge block be Score1, and the model score be Score2. Define the error Δ = Score1 − Score2. If Δ > 0, it indicates that the model underestimates the value, and the weights corresponding to the types of anchor points with true high value need to be increased; otherwise, they should be decreased.

[0152] Specifically, the contribution ratio of each anchor point type to the error is calculated as follows, and the weights are adjusted proportionally:

[0153]

[0154] Where N_total = N_param + N_logic + N_asset, and η is the learning rate. After the update, the weights are normalized, and the sum is 1.

[0155] Mechanism two involves weight optimization based on reinforcement learning, modeling weight adjustment as a Markov decision process. State s represents the meta-features of the current document, such as document type, length, domain, and historical weight combinations. Action a represents the weight adjustment amounts Δα, Δβ, and Δγ, satisfying Δα + Δβ + Δγ = 0. The reward r is the feedback signal R. A weight adjustment policy network is trained using Q-learning or policy gradient methods. After training, for a new document, the policy network outputs the optimal weight adjustment based on the state, achieving dynamic adaptation.

[0156] Mechanism three involves a weight search based on Bayesian optimization. It periodically collects a batch of historical hierarchical data and its feedback, aiming to maximize the overall feedback mean. Bayesian optimization is then used to search for the optimal weight combination within the weight space. The search results serve as fixed weights for the next period, achieving periodic adaptive updates.

[0157] Iterative optimization

[0158] The updated weights are then applied to the value scoring of subsequent documents, and steps S3 to S5 are repeated to form a closed-loop optimization process. As feedback data accumulates, the weights will gradually converge to the optimal value that aligns with the company's actual business value assessment.

[0159] For example, if a company's technical document contains a large number of parameter anchors and causal anchors, but fewer asset anchors, and the initial weights are α=0.3, β=0.3, γ=0.4, the system classifies the document as "internal public," but a subsequent manual audit determines that its core technical parameters should be classified as confidential, thus providing a feedback reward R=−0.5. According to Mechanism 1, calculate N_param=10, N_logic=5, N_asset=2, N_total=17, Δ=0.5, assuming the actual score should be 0.5 higher than the model score, and the learning rate η=0.1, then the weight adjustment is:

[0160] Δα = 0.1 × 0.5 × (10 / 17) ≈ 0.0294, Δβ ≈ 0.0147, Δγ ≈ 0.0059, where Δα + Δβ + Δγ = 0.05. After updating and normalizing, the new weights are α ≈ 0.33, β ≈ 0.31, γ ≈ 0.36. Subsequent similar documents will receive higher scores and are more likely to be correctly graded.

[0161] Example 3

[0162] Based on Example 2, this embodiment further introduces an adversarial generation mechanism. By using a generative adversarial network (GAN) to perform secondary verification on highly scarce knowledge blocks, noise information with high PPL but no practical significance is filtered out, thereby improving the accuracy of scarcity discrimination.

[0163] Using steps S1 to S3 of Examples 1 and 2, the private unstructured documents are preprocessed, semantically aligned, and segmented using a sliding window. Furthermore, through a dual-model perplexity comparison and a dynamic threshold strategy, a preliminary set of candidate high-scarcity knowledge blocks is identified. At the same time, knowledge blocks that do not exceed the threshold are categorized into the regular knowledge block set. .

[0164] From the company's historical documents, select a batch of labeled samples to construct the dataset required for adversarial training:

[0165] Real positive samples: randomly selected from a historical set of highly scarce knowledge blocks to ensure coverage of various business scenarios.

[0166] True negative samples: randomly selected from a set of regular knowledge blocks, representing low-value information in the daily operations of an enterprise.

[0167] Generating negative samples: Pseudo-highly scarce text generated by generator G is used to train the discriminator to identify noise that appears to be highly scarce but is actually meaningless.

[0168] Construct generator G and discriminator D

[0169] Generator G: Employs a Transformer-based decoder structure, with input random noise vectors. The generator outputs a variable-length text sequence. It is used for adversarial training to generate text that is statistically similar to real, highly scarce text but with random or meaningless content, thus deceiving the discriminator. The generator is trained using policy gradient methods from reinforcement learning or a Gumbel-Softmax-based pass-through estimator to address the discreteness issue in text generation.

[0170] Discriminator D: Employs a BERT-based text classification model, with input text blocks. Output a scalar , representing the probability that the text block belongs to true, highly scarce knowledge. The input to the discriminator can be combined with the semantic features of the text and the PPL_base and PPL_domain values ​​calculated in Example 1 to enhance the discriminative ability.

[0171] The training process involves alternately optimizing the generator G and the discriminator D, and calculating the objective function using the following formula:

[0172]

[0173] in, The data distribution of truly scarce text. For noise distribution, As an optional text reconstruction loss, a pre-trained language model is used to calculate the semantic similarity between the generated text and the real text. This is used to constrain the semantic rationality of the generated text and prevent the generation of completely meaningless gibberish. This is the balance coefficient.

[0174] The specific training steps are as follows:

[0175] 1. Fix the generator G and update the discriminator D: Sample from real high-scarcity samples, regular samples and generated samples, and train D to distinguish real high-scarcity samples from the other two classes.

[0176] 2. Fix the discriminator D and update the generator G: Through policy gradient, make the text generated by the generator G obtain as high a score as possible from the discriminator D, that is, make D mistakenly believe it to be real.

[0177] 3. Repeat until convergence or the preset number of iterations is reached.

[0178] After training, the discriminator D is integrated into the private data grading process. For each new knowledge block... In addition to calculating PPL_base and PPL_domain, the true scarcity probability is obtained by inputting D. Define a comprehensive scarcity index:

[0179]

[0180] in The weights are adjustable and can be initially set to be equal. The fused scarcity index is then compared with a dynamic threshold. The comparison involves setting a threshold based on historical data statistics. If the threshold is exceeded, the knowledge block is identified as a high-value and scarce knowledge block, and then proceeds to the subsequent value scoring step.

[0181] Finally, newly generated classification results and user feedback, such as the feedback signals in Example 2, are collected periodically to retrain or fine-tune the generator G and discriminator D, so that the adversarial discrimination capability continues to improve as the enterprise data evolves.

[0182] For example, suppose a company's historical documents contain a large amount of randomly generated test data in some technical reports. This data has a high PPL_base due to the abundance of rare words, but it is actually worthless. Example 1 would mistakenly mark it as highly scarce. By introducing an adversarial generation mechanism, the discriminator D can effectively identify this type of pseudo-highly scarce noise by learning the difference between genuinely high-scarcity text and generated noise. For example, a random sequence of numbers may have a high PPL_base, but D(T) is close to 0, reducing the overall scarcity index and thus being correctly filtered out.

[0183] Obviously, the above embodiments are merely examples to clearly illustrate the embodiments of the present invention, and are not intended to limit the embodiments of the present invention. Those skilled in the art can make other variations or modifications based on the above description. It is neither necessary nor possible to exhaustively list all embodiments here. However, these obvious variations or modifications derived from the spirit of the present invention are still within the protection scope of the present invention.

Claims

1. A method for automatic mining and grading of private data, characterized in that, Includes the following steps: S1. First, receive the private unstructured document, preprocess the document, and use the local model to perform semantic alignment on the preprocessed document. The document was then divided into continuous text blocks using a sliding window technique. S2. Then load the locally deployed general base model and industry-specific model, calculate the perplexity of each text block on the base model and industry-specific model respectively, and determine the scarcity of the text block by comparing the output difference between the base model and the industry-specific model. S3. Secondly, based on the distribution of confusion in the company's historical documents, a statistical model is constructed and a judgment threshold is dynamically set. When the confusion of a text block exceeds the threshold, a value score is calculated. S4. Finally, the scarcity and value scores are projected onto a preset hierarchical decision matrix for comprehensive judgment, and the final secret level of the text block is automatically determined; the corresponding security handling strategy is automatically triggered according to the determined secret level.

2. The method for automatic mining and grading of private data according to claim 1, characterized in that, The preprocessing in step S1 includes: The documents are uniformly converted to UTF-8 encoding, irrelevant characters and noise are removed, and paragraphs and sentences are segmented. For non-text content, OCR or speech recognition tools are used to extract text content, and table data is recognized and extracted and converted into structured text for language recognition and translation alignment.

3. The method for automatic mining and grading of private data according to claim 1, characterized in that, The semantic alignment in step S1 includes: A pre-built enterprise terminology mapping table aligns internal terms with industry standard terms; a BERT-based semantic similarity model is used to merge synonyms and convert between simplified and traditional Chinese characters, outputting a standardized text stream.

4. The method for automatic mining and grading of private data according to claim 1, characterized in that, The sliding window technique in step S1 is specifically as follows: Set the window length L and the overlap length O, perform overlapping segmentation on the standardized text stream, and output a continuous sequence of knowledge blocks {T1, T2, ..., T}. n } 5. The method for automatic mining and grading of private data according to claim 1, characterized in that, In step S2, preliminary classification is performed based on the bi-model perplexity values: If the perplexity PPL_base(T) is less than the base threshold, it is considered common sense. If the perplexity PPL_base(T) is greater than the base threshold and PPL_domain(T) is less than a specific threshold, it is considered industry common knowledge. If the perplexity PPL_base(T) is greater than the base threshold and PPL_domain(T) is greater than a specific threshold, it is marked as a highly scarce knowledge block.

6. The method for automatic mining and grading of private data according to claim 1, characterized in that, The dynamic threshold in step S3 is specifically as follows: Collect non-sensitive historical documents of the enterprise as a baseline corpus, and perform cleaning and segmentation consistent with the main process to obtain baseline text blocks; Input the baseline text block into the basic model to calculate the perplexity and obtain the baseline PPL value set; The mean μ and standard deviation σ of the benchmark PPL value set are calculated, and the dynamic threshold λ is calculated using the formula λ=μ+k·σ, where k is a sensitivity coefficient set according to the enterprise's risk preference.

7. The method for automatic mining and grading of private data according to claim 1, characterized in that, The value scoring calculation in step S3 includes extracting semantic value anchors from highly scarce knowledge blocks. The semantic value anchors include parameter anchors, causal anchors, and asset anchors. The parameter anchor point is used to identify phrases containing numbers, and the numerical value is bound to the unit and attribute it modifies through dependency parsing. The causal anchor point is identified by logical relation words. Cause and effect fragments are extracted through dependency parsing to construct a relationship triplet of cause, relation, and effect. The asset anchor is a finely tuned NER model used to identify enterprise-defined high-value entity types, including core algorithms, business assets, intellectual property, and strategic information.

8. The method for automatic mining and grading of private data according to claim 7, characterized in that, The value score is calculated using the following formula: Score_value=α·N_param+β·N_logic+γ·Nasset; Where N_param is the number of parameter anchors, N_logic is the number of causal anchors, N_asset is the number of asset anchors, and α, β, γ are the weight coefficients of the three types of anchors, and satisfy α+β+γ=1.

9. The method for automatic mining and grading of private data according to claim 1, characterized in that, Determining the final secrecy level in step S4 includes: The number and proportion of knowledge blocks marked as highly scarce in the document are statistically analyzed, and the overall scarcity level of the document is determined based on a preset threshold. The maximum or weighted average value score of all knowledge blocks in a document is taken as the representative value score of the document. A value score threshold range is set to classify the document into high-value, medium-value, and low-value levels.

10. The method for automatic mining and grading of private data according to claim 1, characterized in that, The perplexity level (PPL) is used to quantify the degree of surprise or scarcity of text content in the public knowledge domain; the base large language model (Base LLM) is a general large model pre-trained on massive public Internet data, without being fine-tuned with specific enterprise private data. The industry-specific model, Domain LLM, is a model that has been fine-tuned based on literature in a specific vertical field, representing common knowledge within the industry.

Images