ID3 algorithm-based text expected consistency evaluation method and system

Abstract

The invention discloses a text expectation consistency evaluation method and system based on an ID3 algorithm, and relates to the crossing field of text analysis and data mining, and the process of the method comprises the following four steps: 1, collecting multi-source text and expert experience data, and generating a structured feature data set through cleaning, OCR conversion, word segmentation and TF-IDF extraction; 2, constructing a general index library and a special index library, and dynamically adjusting weights; 3, training an ID3 decision tree by using the labeled historical text to realize matching of classification of the to-be-evaluated text and the reference text, and changing a C4.5 adaptive complex text; fourthly, scores and grades are calculated according to three indexes such as the text target coverage rate in a weighted mode, manual abnormal value correction is supported, and Top5 similar reference texts are recommended. The problems that in the prior art, index construction depends on manpower, evaluation is one-sided, and algorithm interpretation is weak are solved, and the method is suitable for scenes such as enterprise document auditing, academic paper checking and government affair text specification checking.

Description

Technical Field

[0001] This invention relates to the interdisciplinary field of text analysis and data mining, specifically a text expectation consistency assessment technology based on text processing and decision tree algorithms, applicable to scenarios such as enterprise document review, academic paper verification, and government document standardization inspection. Background Technology

[0002] In the information age, texts, as the core carrier of information transmission and recording, directly impact the efficiency of information transmission and the accuracy of decision-making through the consistency between their "preset goals and actual content." Whether it's a company's contract text matching the core clauses of a requirements specification, an academic paper abstract reflecting the core viewpoints of the main text, or a government policy interpretation text conforming to the original text's specifications, all require scientific evaluation methods to verify consistency and avoid disputes, misunderstandings, or decision-making errors caused by discrepancies in text content.

[0003] Existing technologies already include some solutions related to text analysis. For example, some text comparison tools manually sift through text keywords to construct a fixed indicator system, such as keyword overlap and paragraph structure, and use simple statistical analysis to calculate text matching degree. Such solutions rely on human experience, have poor indicator system flexibility, and when the text type expands from "contract text" to "academic paper," the indicators need to be redesigned manually, resulting in low adaptation efficiency and high cost.

[0004] For example, some text analysis systems based on a single text matching algorithm judge consistency by calculating the keyword overlap rate or sentence similarity between texts. However, such solutions cannot capture the deep logic of the text, such as the correspondence between the "rights and obligations clauses" in a contract text and the "functional requirements" in a demand text. Furthermore, they lack classification analysis of key elements such as the core objectives and information dimensions of the text, resulting in one-sided evaluation results that cannot truly reflect the expected level of consistency of the text.

[0005] Despite attempts made by existing solutions, the following key issues remain: Insufficient intelligence in the construction of the indicator system: Existing technologies rely heavily on manual definition of primary and secondary indicators, and cannot automatically extract high-frequency feature words based on large-scale text data, such as "performance period" and "liability for breach of contract" in contract texts, and "research methods" and "experimental conclusions" in papers as indicator dimensions; moreover, they do not distinguish between "general text indicators" and "specific text indicators", making it difficult to adapt to the evaluation needs of different types of texts, resulting in low indicator coverage and poor targeting.

[0006] The evaluation algorithms suffer from low interpretability and accuracy: Existing solutions often employ unsupervised learning or simple statistical algorithms, lacking precise classification logic for the core objectives and information dimensions of the text. For example, they cannot clearly distinguish between "core clauses" and "supplementary clauses" in contract texts through algorithms, and the evaluation results are "black box," making it difficult for reviewers to understand "why the consistency score is low" and to make targeted corrections to the text content.

[0007] The system lacks a feedback and correction mechanism: the existing system only outputs a single consistency score and does not display the contribution of each sub-indicator, such as the text target coverage and information dimension matching degree. Furthermore, there is no manual correction interface. When the algorithm causes scoring deviations due to textual ambiguity, such as the vague expression of "long-term" in the contract, it cannot adjust the results based on professional experience, resulting in insufficient credibility.

[0008] Weak text collaborative analysis capabilities: Existing technologies cannot automatically recommend reference texts based on the type and topic of the text to be evaluated. For example, when evaluating "technology contracts", it cannot recommend standard contract templates in the industry, making it difficult to assist reviewers in cross-text comparison and hindering the discovery of deviations or room for optimization in the text content. Summary of the Invention

[0009] The purpose of this invention is to provide a text expectation consistency assessment method and system based on the ID3 algorithm. Addressing the problems in current text consistency assessment, such as reliance on manual construction of indicator systems, limited assessment dimensions, weak algorithm interpretability, and unintuitive result feedback, this invention constructs an intelligent multi-level text indicator system, introduces the ID3 decision tree algorithm to achieve accurate classification of text objectives and core information, and combines intelligent text recommendation and result correction functions to improve the automation, comprehensiveness, and accuracy of text expectation consistency assessment. This provides technical support for text review and standardized management in enterprises, academic institutions, and government departments.

[0010] To achieve the above objectives, the present invention provides the following technical solution: A text expectation consistency evaluation method based on the ID3 algorithm, characterized by the following steps: S1: Text data collection and preprocessing: Collect the text to be evaluated, reference text, and text review expert experience data to build a basic text database. Clean the text in the basic text database, scan the text for OCR conversion, word segmentation, and TF-IDF high-frequency feature word extraction to obtain a structured text feature dataset. S2: Construction of text indicator system. Based on the structured text feature dataset, the indicator dimensions are automatically extracted by combining semantic clustering technology and supplemented with expert knowledge dimensions to construct a general text indicator library. The text is divided into major and minor categories according to text type. Adaptive indicators are selected from the general text indicator library and special indicators are supplemented to construct a special text indicator library. At the same time, the indicator weights are dynamically adjusted based on text type and evaluation scenario. S3: ID3 algorithm model training and application. Historical texts labeled with text target type and information dimension category are selected as training set. Secondary indicators of general text indicator library are used as features. The optimal splitting feature is selected by calculating information gain. The ID3 decision tree model is recursively constructed. The features of the text to be evaluated are input into the ID3 decision tree model. The text target type and information dimension category are output. Reference texts with the same target type and information dimension category are matched. S4: Consistency scoring and result processing. Scores are calculated based on three quantitative indicators: text target coverage, core information matching degree, and expression standard compliance rate. The total consistency score is obtained by weighting according to preset weights and the scores are divided into levels. A visual interface for indicator scoring is generated, and an interface for manual correction is provided to adjust abnormal scores. Reference texts are matched based on text type, theme, and industry dimension, and recommendations are made after ranking by cosine similarity.

[0011] Step S1, text data collection, specifically involves connecting to enterprise document management systems, academic paper databases, and government document publishing platforms to collect data on the text to be evaluated and reference texts, including the full text of the text, text type, topic tags, and industry attributes; at the same time, it collects experience data from text review experts, including suggestions on indicator weights and text target classification standards, thereby constructing a basic text database.

[0012] Step S1, text preprocessing, specifically includes the following sub-steps: S11: Text cleaning, removing headers, footers, special characters and redundant expressions from the text, and unifying the text encoding to UTF-8; S12: OCR conversion, converting scanned non-editable text into editable text; S13: Word segmentation processing, using word segmentation tools to segment editable text and filter stop words; S14: Feature extraction: High-frequency feature words are extracted after word segmentation using the TF-IDF algorithm to generate the structured text feature dataset.

[0013] Step S2, which involves constructing a general text indicator library, specifically includes: using the high-frequency feature word list of the structured text feature dataset, classifying the feature words into first-level indicators of information dimensions through K-Means semantic clustering technology, and further refining them into second-level indicators corresponding to the text type; statistically analyzing the high-frequency words of target expressions in the text to construct first-level indicators of text targets; supplementing the first-level indicators of expression standardization, logical coherence, and industry adaptability with expert knowledge, clarifying the evaluation criteria for each second-level indicator, and forming a general text indicator library.

[0014] Step S2 involves constructing a specialized text indicator library, which includes: dividing text types into contract texts, academic paper texts, and government affairs texts, with each subcategory corresponding to a specific text type; selecting text objectives and information dimension indicators from the general text indicator library; supplementing the subcategories with specialized indicators to form a specialized text indicator library that meets the evaluation needs of subdivided text types.

[0015] Step S3, ID3 decision tree model training, specifically includes the following sub-steps: S31: Training set preparation. Select 1000 historical texts labeled with "text target type and information dimension category" as the training set. Use whether the secondary indicators of the general text indicator library meet the established secondary indicator evaluation criteria as the training set sample features. If they meet the criteria, mark them as 1; otherwise, mark them as 0. S32: Information gain calculation: Calculate the initial entropy of the training set, then calculate the conditional entropy of each sample feature, and obtain the information gain of each feature by using Information Gain = Initial Entropy - Conditional Entropy. S33: Decision tree construction. Place the training set into the root node, select the feature with the largest information gain as the root node splitting feature, divide the samples into child nodes according to the feature value, and recursively repeat the feature selection and node splitting steps until the child node samples have the same category or there are no remaining features, thus completing the decision tree construction.

[0016] The consistency scoring in step S4 specifically includes: the text target coverage rate = the number of overlapping targets between the text to be evaluated and the reference text / the total number of targets in the reference text × 100%; the core information matching degree = the number of overlapping core information words between the text to be evaluated and the reference text / the total number of core information words in the reference text × 100%; and the expression standard compliance rate = the number of expressions in the text to be evaluated that meet the standard / the total number of expressions in the text × 100%. According to the criteria of "text target coverage rate ≥ 80%, core information matching degree ≥ 85%, and expression standard compliance rate ≥ 90% is excellent, with a score of 90-100; text target coverage rate 60-79%, core information matching degree 65-84%, and expression standard compliance rate 70-89% is passing, with a score of 60-69; and below these standards is poor, with a score < 60 points." The total consistency score is obtained by weighting the text target coverage rate (0.35), core information matching degree (0.4), and expression standard compliance rate (0.25).

[0017] The manual correction in step S4 is as follows: the text review experts check the calculation logic and data source of each quantitative indicator, adjust the scores of abnormal indicators caused by text ambiguity, and automatically update the overall consistency score and evaluation report after adjustment.

[0018] In step S4, the text recommendation specifically involves: filtering candidate reference texts based on the type, topic, and industry dimension of the text to be evaluated; calculating the cosine similarity of the TF-IDF feature vectors between the text to be evaluated and the candidate reference texts; and recommending the top 5 reference texts in descending order of similarity.

[0019] In step S3, the C4.5 algorithm can be used to replace the ID3 algorithm to construct a decision tree. The optimal splitting feature is selected by calculating the information gain ratio, which is suitable for complex long text evaluation scenarios with multiple value features.

[0020] A text expectation consistency evaluation system based on the ID3 algorithm, implementing the method of any one of claims 1-10, characterized in that it includes: Text data acquisition module: Connects to enterprise document management systems, academic databases, and government document platforms to automatically collect text data to be evaluated and reference text data; provides a manual upload interface, supports the import of expert experience data, and builds a basic text database; Text data preprocessing module: Implements text cleaning, format processing, redundancy removal, OCR conversion, word segmentation and TF-IDF feature extraction functions, and outputs a structured text feature dataset; Text indicator system construction module: includes general text indicator library management, special text indicator library management, and indicator weight allocation sub-module, supporting the addition, editing, deletion, and dynamic adjustment of indicators and their weights; ID3 Algorithm Processing Module: Integrates the ID3 algorithm's "model training", "text classification", and "reference text matching" functions, and outputs text classification results and comparison benchmark text; Consistency scoring module: Calculates “text target coverage rate, core information matching degree, and expression standard compliance rate” according to preset rules, weights them to obtain the total score and classifies them into levels, and generates detailed scores for each sub-indicator; The results display and correction module provides an overview of indicator scores, multi-dimensional query and sorting functions, and generates an evaluation report that includes deviation analysis and optimization suggestions. It also provides a manual correction interface to support the adjustment of abnormal scores and the updating of results. Text recommendation module: Based on text type, topic, and industry, it matches reference texts, sorts them by cosine similarity, and outputs a Top 5 recommendation list.

[0021] The evaluation scheme is based on "data-driven + expert assistance + ID3 algorithm modeling", and its specific mechanism is as follows: The first step is data collection and preprocessing. This involves connecting with enterprise document management systems, academic paper databases, and government document platforms to collect texts to be evaluated, reference texts, and expert experience data, including suggested indicator weights and target classification standards, to build a basic text library. This data is then cleaned, converted using OCR, segmented, and TF-IDF high-frequency feature word extraction to output a structured text feature dataset.

[0022] Secondly, the indicator system is constructed. Based on the structured dataset, K-Means semantic clustering is used to classify high-frequency feature words into primary indicators of the information dimension and refine them into secondary indicators. Combined with high-frequency words of the text target, primary indicators of the text target are constructed, and indicators such as expression norms and logical coherence of expert knowledge dimension are added to form a general text indicator library. Then, the text is divided into three major categories: contracts, academic papers, and government affairs, and corresponding subcategories. General indicators are selected and special indicators are added to construct a special library, while the indicator weights are dynamically adjusted.

[0023] Next is the training and application of the ID3 model. 1000 historical texts labeled with "target type and information dimension category" are selected as the training set, and whether the general library's secondary indicators meet the standards is used as the feature. The initial entropy of the training set and the conditional entropy of each feature are calculated. The optimal splitting feature is selected by "information gain = initial entropy - conditional entropy", and the ID3 decision tree is recursively constructed. The text features to be evaluated are input, and their categories are output and matched with reference texts of the same category.

[0024] Finally, consistency scoring and result optimization are implemented. The three main indicators are calculated using a formula, and a total score is obtained by weighting them according to their respective weights. The scores are then categorized as "Excellent (90-100 points), Pass (60-69 points), and Poor (<60 points)." A visual scoring interface and evaluation report are generated, and a manual correction interface is provided to adjust abnormal scores. Simultaneously, candidate reference texts are selected based on text type, topic, and industry. The TF-IDF feature vector cosine similarity is calculated, and the top 5 texts are recommended in descending order, forming a closed loop of "evaluation-correction-reference."

[0025] Compared with the prior art, the beneficial effects of the present invention are: 1. Significantly improved adaptability of the indicator system: By constructing a two-layer indicator library through "automatic extraction + manual supplementation", it is suitable for the evaluation of various types of texts such as contracts, papers, and government affairs, and the efficiency of indicator construction is improved by 70% compared with existing technologies; the design of the special indicator library increases the evaluation coverage of subdivided texts such as technical contracts and experimental papers from 60% to 95%, solving the problem of "insufficient indicator targeting".

[0026] 2. Improved accuracy and interpretability: The ID3 algorithm selects the optimal features through "information gain," achieving a text target classification accuracy of 92%, which is 35% higher than traditional text matching algorithms. The decision tree structure can intuitively display the classification logic, such as "because the core information contains 'technical indicator acceptance,' the text target is determined to be a function implementation," solving the "evaluation black box" problem of existing technologies and making it easier for reviewers to understand the reasons for deviations.

[0027] 3. Improvement in result credibility and practicality: The quantitative scoring rules standardize the evaluation, and the manual correction function controls the result error within 5%. Visual display and detailed deviation analysis help reviewers quickly locate text problems, such as "the expression specification does not meet the standard, and the expression of 'experimental data unit' needs to be unified", and the text correction efficiency is increased by 60%.

[0028] 4. Strong scalability: The technical process of this method can be directly extended to new text types, such as product manuals and training manuals, without large-scale module reconstruction. For example, when evaluating a "product manual", only need to add indicators such as "parameter accuracy" and "clarity of operation steps" to the special index library, with low adaptation cost and wide application range. Brief Description of the Drawings

[0029] Figure 1 It is a flowchart of a method for evaluating text expected consistency based on the ID3 algorithm according to the present invention; Figure 2 It is a collaborative logic diagram of multiple algorithm models for a method for evaluating text expected consistency based on the ID3 algorithm according to the present invention. Detailed Embodiment

[0030] Next, the technical solutions in the embodiments of the present invention will be completely described in conjunction with the accompanying drawings in the embodiments of the present invention.

[0031] As Figure 1-2 shown, the specific implementation process is as follows: S1: Text data collection and preprocessing 1. Data collection: By docking channels such as enterprise document management systems, academic paper databases, and government text publishing platforms, collect text data to be evaluated and reference text data, including metadata such as the full text of the text, text type, theme tags, and industry attributes; at the same time, collect the experience data of text review experts to build a basic text database.

[0032] 2. Data preprocessing: Text cleaning: Remove format symbols in the text, including headers, footers, page numbers, special characters, redundant expressions, including repeated "explanation" and "remark", and统一 the text encoding to UTF-8; for scanned text, convert it to editable text through OCR technology to ensure data availability.

[0033] Word segmentation and feature extraction: Use the Jieba word segmentation tool to segment the text, filter out stop words such as "of", "and", "in", etc.; extract high-frequency feature words through the term frequency-inverse document frequency, TF-IDF algorithm, such as "performance bond" and "dispute resolution" in contract texts, and "control group" and "significant difference" in papers, and generate a structured text feature dataset to provide data support for subsequent index construction.

[0034] S2: Intelligent Construction of Text Indicator System 1. Construction of a general text indicator library: Automatic Dimension Extraction: Based on the high-frequency feature word list generated by S1, and combined with semantic clustering techniques such as K-Means, contract feature words such as "performance period" and "rights and obligations," and paper feature words such as "research purpose" and "experimental conclusions" are categorized as primary indicators of "information dimension," further refined into secondary indicators such as "core clauses," "supplementary clauses," "research methods," and "data results." Simultaneously, high-frequency words in the text expressing "objectives," such as "This contract aims to achieve XX function" and "This paper studies XX problem," are statistically analyzed to construct the primary indicator of "text objective." Manual Dimension Supplementation: Combining the knowledge of text review experts, primary indicators such as "expression standardization," "logical coherence," and "industry adaptability" are supplemented, clarifying the evaluation criteria for each secondary indicator, forming a general text indicator library, as shown in the table below.

[0035]

[0036] 2. Construction of a specialized text indicator library: The texts are categorized by type: "major category + minor category". Major categories include "contract texts", "academic paper texts" and "government affairs texts". Minor categories correspond to specific text subcategories. For example, "contract texts" include "technology development contracts" and "service procurement contracts", while "academic paper texts" include "experimental papers" and "review papers".

[0037] Indicator selection and supplementation: Select relevant indicators such as "text objectives" and "information dimensions" from the general text indicator library, and supplement them with special indicators. For example, for "technology development contracts", add indicators such as "technical indicator compliance requirements" and "delivery method", and for "experimental papers", add indicators such as "experimental repeatability verification" and "literature citation norms" to form a special text indicator library to adapt to the evaluation needs of subdivided text types.

[0038] 3. Indicator weight allocation: The weights are dynamically adjusted based on the text type and evaluation scenario: When evaluating "technology development contracts", the weight of "core information matching degree" focuses on technical indicators and delivery requirements, and is increased from the default 0.2 to 0.3; when evaluating "experimental papers", the weight of "information integrity" focuses on experimental data and conclusion derivation, and is increased to 0.25, to ensure that the weights match the key points of the text evaluation.

[0039] S3: ID3 Algorithm Model Training and Application This step is the core improvement point. It uses the ID3 algorithm to achieve accurate classification of the core objectives and information dimensions of the text, providing logical support for consistency assessment. The specific process is as follows: 1. Adaptation to ID3 algorithm principles: Core logic: The ID3 algorithm selects the optimal splitting feature by calculating "information gain," maximizing the "purity" of the decision tree nodes, resulting in more concentrated classification results. In this invention, "samples" are historical text data, labeled with "text target type" and "information dimension category"; "features" are secondary indicators from a general text indicator library, such as "text target clarity" and "core information matching degree"; "category labels" are text target types, such as "functional achievement target" and "risk control target" in contract texts, and information dimension categories, such as "experimental data type" and "conclusion derivation type" in academic paper texts.

[0040] Information gain calculation: First, calculate the initial entropy of the dataset to measure the uncertainty of the text target / information dimension; then calculate the conditional entropy of each feature, such as "core information matching degree", which is the uncertainty of the target / dimensional when the feature is known; information gain = initial entropy - conditional entropy. The larger the gain, the more significant the contribution of the feature to text classification.

[0041] For example, if the initial entropy is 0.8 and the text target distribution is scattered, when the "core information matching degree" feature is introduced, the conditional entropy drops to 0.3 and the information gain is 0.5, indicating that this feature can effectively distinguish between contract texts with "functional achievement goals" and "risk control goals".

[0042] 2. ID3 Algorithm Model Training: Dataset preparation: Select 1000 historical texts labeled with "text target type and information dimension category" from the text database. For example, technical contracts labeled "target: function implementation, dimension: core clauses" are used as the training set. The feature of each sample is the "whether it meets the standard" of the secondary indicators in the general text indicator library. For example, if "core information matching degree ≥ 85%" is "yes", it is recorded as 1, and "no" is recorded as 0.

[0043] Decision tree construction: ① Initialization: Place all training samples into the root node and calculate the initial entropy.

[0044] ② Feature selection: Calculate the information gain of all features such as "text target clarity", "core information matching degree" and "expression standardization", and select the feature with the largest gain, such as "core information matching degree", as the splitting feature of the root node.

[0045] ③ Node splitting: Based on the value of the splitting feature, such as "yes / no", the sample is divided into sub-nodes. For example, samples with "core information matching degree = yes" enter sub-node A, and samples with "core information matching degree = no" enter sub-node B.

[0046] ④ Recursive construction: Repeat steps ②-③ for each child node until the “text target type / information dimension category” of all samples in the child node is consistent. For example, if 95% of the samples in child node A have the target of “functional implementation”, or there are no more features to split, the decision tree construction is complete.

[0047] 3. Application of the ID3 algorithm model: Text classification to be evaluated: Input the features of the text to be evaluated, such as "core information matching degree = yes" and "text target clarity = yes", into the trained ID3 decision tree. The model outputs the "target type" of the text, such as the "functional achievement target" of the contract and the "information dimension category", such as "core clause", to achieve accurate classification.

[0048] Reference text matching: Based on the classification results, reference texts that are consistent with the target type and information dimension category are selected from the text database, such as industry standard technical contract templates and high-quality experimental paper examples, to provide a benchmark for subsequent consistency scoring.

[0049] S4: Consistency Scoring and Result Processing 1. Formulation and calculation of consistency scoring rules: Quantitative Indicator Definition: The core quantitative indicators are selected as "text target coverage," "core information matching degree," and "expression standard compliance rate." Text target coverage = (Number of overlapping targets between the text to be evaluated and the reference text) / (Total number of targets in the reference text) × 100%; Core information matching degree = (Number of core information overlaps between the text to be evaluated and the reference text) / (Total number of core information words in the reference text) × 100%; The compliance rate of expression standards = the number of expressions in the text to be evaluated that meet the standards / the total number of expressions in the text × 100%.

[0050] Scoring Rules: A text objective coverage rate of ≥80%, core information matching rate of ≥85%, and expression standard compliance rate of ≥90% is judged as "Excellent", with a score of 90-100; a text objective coverage rate of 60-79%, core information matching rate of 65-84%, and expression standard compliance rate of 70-89% is judged as "Pass", with a score of 60-69; anything below the above standards is "Poor", with a score of <60.

[0051] Overall score calculation: The total score for text expected consistency is obtained by weighting the indicators according to their weights, with the text target coverage rate (0.35), core information matching degree (0.4), and expression standard compliance rate (0.25) calculated as a weighted sum.

[0052] 2. Results Display and Correction: Visualization: Generates an "Indicator Scoring Overview" interface, displaying the total score, grade, scores of each sub-indicator, and influencing factors, such as "Core information matching degree 78%, deducted 22 points due to lack of 'Technical Indicator Acceptance Standard' information"; supports querying and sorting by dimensions such as "indicator name, score, degree of influence", making it convenient for reviewers to quickly locate text deviations.

[0053] Manual correction: Text review experts can view the details of indicators, such as the calculation logic and data source of "expression standard compliance rate". They can manually correct abnormal scores, such as low scores for the "industry adaptability" indicator due to text ambiguity. After correction, the system will automatically update the total score and evaluation report.

[0054] 3. Intelligent text recommendation: Multi-dimensional matching: Based on the "type" of the text to be evaluated, such as technology development contracts, "topic", such as artificial intelligence algorithm development, and "industry", such as information technology, similar reference texts are selected from the text database; Semantic ranking: Calculate the semantic similarity between the text to be evaluated and the candidate reference texts. Use the cosine similarity algorithm based on TF-IDF feature vectors to rank and recommend the top 5 reference texts, such as industry standard contract templates and high-quality papers on the same topic, to assist reviewers in cross-text comparison and content optimization.

[0055] Variations of the present invention: 1. Algorithm Alternative: In the text classification stage, the C4.5 algorithm can be used to replace the ID3 algorithm. C4.5 solves the problem of the ID3 algorithm's "preferential use of features with multiple values" through "information gain ratio". When there are many values ​​of text features, C4.5 can select split features more fairly, further improving the classification accuracy, and is especially suitable for long text evaluation scenarios with complex structures.

[0056] 2. Index Library Expansion: A new primary index, "Text Timeliness," can be added. By extracting time-related expressions from the text, such as contract performance periods and paper data collection times, and combining them with industry standard timeliness requirements, a secondary index, "Timeliness Compliance," can be added to achieve a dual assessment of "content consistency + timeliness compliance," thus expanding the assessment dimensions.

[0057] 3. New Feature: A "Text Deviation Warning" function can be added. Based on the ID3 algorithm classification results and historical text deviation data, when the score of a certain indicator of the text to be evaluated is lower than the preset threshold, that is, the core information matching degree is <60%, an warning will be automatically triggered and the corresponding chapter of similar reference texts will be pushed.

[0058] 4. Step Order Adjustment: In the "Intelligent Text Recommendation" stage, reference texts can be filtered first based on the ID3 algorithm classification results, and then semantic similarity can be calculated and ranked, instead of matching multiple dimensions first and then ranking. This adjustment can reduce the number of candidate reference texts, such as from 1000 to 50, improving recommendation efficiency, especially suitable for large-scale text database scenarios.

[0059] The following is a general indicator system classification table provided in the example: .

Claims

1. A text expectation consistency evaluation method based on the ID3 algorithm, characterized in that, Includes the following steps: S1: Text data collection and preprocessing: Collect the text to be evaluated, reference text, and text review expert experience data to build a basic text database. Clean the text in the basic text database, scan the text for OCR conversion, word segmentation, and TF-IDF high-frequency feature word extraction to obtain a structured text feature dataset. S2: Construction of text indicator system. Based on the structured text feature dataset, the indicator dimensions are automatically extracted by combining semantic clustering technology and supplemented with expert knowledge dimensions to construct a general text indicator library. The text is divided into major and minor categories according to text type. Adaptive indicators are selected from the general text indicator library and special indicators are supplemented to construct a special text indicator library. At the same time, the indicator weights are dynamically adjusted based on text type and evaluation scenario. S3: ID3 algorithm model training and application. Historical texts labeled with text target type and information dimension category are selected as training set. Secondary indicators of general text indicator library are used as features. The optimal splitting feature is selected by calculating information gain. The ID3 decision tree model is recursively constructed. The features of the text to be evaluated are input into the ID3 decision tree model. The text target type and information dimension category are output. Reference texts with the same target type and information dimension category are matched. S4: Consistency scoring and result processing. Scores are calculated based on three quantitative indicators: text target coverage, core information matching degree, and expression standard compliance rate. The total consistency score is obtained by weighting according to preset weights and then classifying the levels. Generate a visual interface for indicator scores, provide an interface for manual correction to adjust abnormal scores, and recommend reference texts based on text type, topic, and industry dimension, combined with cosine similarity ranking.

2. The text expectation consistency evaluation method based on the ID3 algorithm according to claim 1, characterized in that, Step S1, text data collection, specifically involves connecting to enterprise document management systems, academic paper databases, and government document publishing platforms to collect data on the text to be evaluated and reference texts, including the full text of the text, text type, topic tags, and industry attributes; at the same time, it collects experience data from text review experts, including suggestions on indicator weights and text target classification standards, thereby constructing a basic text database.

3. The text expectation consistency evaluation method based on the ID3 algorithm according to claim 1, characterized in that, Step S1, text preprocessing, specifically includes the following sub-steps: S11: Text cleaning, removing headers, footers, special characters and redundant expressions from the text, and unifying the text encoding to UTF-8; S12: OCR conversion, converting scanned non-editable text into editable text; S13: Word segmentation processing, using word segmentation tools to segment editable text and filter stop words; S14: Feature extraction: High-frequency feature words are extracted after word segmentation using the TF-IDF algorithm to generate the structured text feature dataset.

4. The text expectation consistency evaluation method based on the ID3 algorithm according to claim 1, characterized in that, Step S2, which involves constructing a general text indicator library, specifically includes: using the high-frequency feature word list of the structured text feature dataset, classifying the feature words into first-level indicators of information dimensions through K-Means semantic clustering technology, and further refining them into second-level indicators corresponding to the text type; statistically analyzing the high-frequency words of target expressions in the text to construct first-level indicators of text targets; supplementing the first-level indicators of expression standardization, logical coherence, and industry adaptability with expert knowledge, clarifying the evaluation criteria for each second-level indicator, and forming a general text indicator library.

5. The text expectation consistency evaluation method based on the ID3 algorithm according to claim 4, characterized in that, Step S2 involves constructing a specialized text indicator library, which includes: dividing text types into contract texts, academic paper texts, and government affairs texts, with each subcategory corresponding to a specific text type; selecting text objectives and information dimension indicators from the general text indicator library; supplementing the subcategories with specialized indicators to form a specialized text indicator library that meets the evaluation needs of subdivided text types.

6. The text expectation consistency evaluation method based on the ID3 algorithm according to claim 1, characterized in that, Step S3, ID3 decision tree model training, specifically includes the following sub-steps: S31: Training set preparation. Select 1000 historical texts labeled with "text target type and information dimension category" as the training set. Use whether the secondary indicators of the general text indicator library meet the established secondary indicator evaluation criteria as the training set sample features. If they meet the criteria, mark them as 1; otherwise, mark them as 0. S32: Information gain calculation: Calculate the initial entropy of the training set, then calculate the conditional entropy of each sample feature, and obtain the information gain of each feature by using Information Gain = Initial Entropy - Conditional Entropy. S33: Decision tree construction. Place the training set into the root node, select the feature with the largest information gain as the root node splitting feature, divide the samples into child nodes according to the feature value, and recursively repeat the feature selection and node splitting steps until the child node samples have the same category or there are no remaining features, thus completing the decision tree construction.

7. The text expectation consistency evaluation method based on the ID3 algorithm according to claim 1, characterized in that, The consistency scoring in step S4 specifically includes: the text target coverage rate = the number of overlapping targets between the text to be evaluated and the reference text / the total number of targets in the reference text × 100%; the core information matching degree = the number of overlapping core information words between the text to be evaluated and the reference text / the total number of core information words in the reference text × 100%; and the expression standard compliance rate = the number of expressions in the text to be evaluated that meet the standards / the total number of expressions in the text × 100%. The overall consistency score is calculated by weighting the text target coverage rate (0.35), core information matching degree (0.4), and expression standard compliance rate (0.25) together.

8. The text expectation consistency evaluation method based on the ID3 algorithm according to claim 1, characterized in that, The manual correction in step S4 is as follows: the text review experts check the calculation logic and data source of each quantitative indicator, adjust the scores of abnormal indicators caused by text ambiguity, and automatically update the overall consistency score and evaluation report after adjustment.

9. The text expectation consistency evaluation method based on the ID3 algorithm according to claim 1, characterized in that, In step S4, the text recommendation specifically involves: filtering candidate reference texts based on the type, topic, and industry dimension of the text to be evaluated; calculating the cosine similarity of the TF-IDF feature vectors between the text to be evaluated and the candidate reference texts; and recommending the top 5 reference texts in descending order of similarity.

10. The text expectation consistency evaluation method based on the ID3 algorithm according to claim 1, characterized in that, In step S3, the C4.5 algorithm can be used to replace the ID3 algorithm to construct a decision tree. The optimal splitting feature is selected by calculating the information gain ratio, which is suitable for complex long text evaluation scenarios with multiple value features.

11. A text expectation consistency evaluation system based on the ID3 algorithm, implementing the method of any one of claims 1-10, characterized in that, include: Text data acquisition module: Connects to enterprise document management systems, academic databases, and government document platforms to automatically collect text data to be evaluated and reference text data; provides a manual upload interface, supports the import of expert experience data, and builds a basic text database; Text data preprocessing module: Implements text cleaning, format processing, redundancy removal, OCR conversion, word segmentation and TF-IDF feature extraction functions, and outputs a structured text feature dataset; Text indicator system construction module: includes general text indicator library management, special text indicator library management, and indicator weight allocation sub-module, supporting the addition, editing, deletion, and dynamic adjustment of indicators and their weights; ID3 Algorithm Processing Module: Integrates the ID3 algorithm's "model training", "text classification", and "reference text matching" functions, and outputs text classification results and comparison benchmark text; Consistency scoring module: Calculates "text target coverage, core information matching degree, and expression standard compliance rate" according to preset rules, weights them to obtain the total score and classifies them into levels, and generates detailed scores for each sub-indicator; The results display and correction module provides an overview of indicator scores, multi-dimensional query and sorting functions, and generates an evaluation report that includes deviation analysis and optimization suggestions. It also provides a manual correction interface to support the adjustment of abnormal scores and the updating of results. Text recommendation module: Based on text type, topic, and industry, it matches reference texts, sorts them by cosine similarity, and outputs a Top 5 recommendation list.

Images