Intelligent text evaluation system based on artificial intelligence

Abstract

The invention relates to the technical field of intelligent text evaluation. The invention relates to an intelligent text evaluation system based on artificial intelligence. The system comprises a sample collection module, a preset text management module, an evaluation model management module and an evaluation result management module. The sample collection module is used for acquiring evaluation themes and evaluation semantics, and performing semantic requirement classification according to the evaluation semantics; through integration of domain knowledge bases and a multi-dimensional feature extraction and closed-loop iterative optimization mechanism, the problems of non-uniform evaluation standards and insufficient precision in the prior art are effectively solved, the system determines the technical field through evaluation semantic depth analysis, the corresponding domain knowledge base is called to construct a structured evaluation dimension and a to-be-filled template, and the accuracy of the to-be-filled template is improved. The template structure is mined and optimized in combination with the frequent item set, so that the evaluation standard not only conforms to the professional specification of the field, but also can be dynamically adjusted according to the core components of the text, the pertinence and accuracy of evaluation are remarkably improved, and the defect of one-step evaluation in the prior art is avoided.

Description

Technical Field

[0001] This invention relates to the field of intelligent text review technology, and more specifically, to an intelligent text review system based on artificial intelligence. Background Technology

[0002] Text review is a core component in academic research, industry standard implementation, and educational evaluation. Its core purpose is to evaluate texts based on dimensions such as structural integrity, information accuracy, logical coherence, and terminology standardization, thereby selecting high-quality texts, correcting inferior texts, and ensuring that texts meet the usage requirements of specific scenarios.

[0003] Existing technologies have significant shortcomings in practical applications, severely impacting the accuracy and practicality of evaluation. First, the evaluation criteria lack uniformity and flexibility. Existing technologies often employ fixed rules or single features for evaluation, failing to dynamically adjust evaluation dimensions and standards according to the professional requirements of different fields, resulting in insufficient accuracy in evaluating professional texts. Second, they lack deep semantic understanding and closed-loop optimization capabilities. Existing technologies mostly focus on analyzing surface-level text features, neglecting the contextual semantic relationships and logical structure of the text. Furthermore, the evaluation models cannot be dynamically optimized based on the evaluation results after training, making it difficult to continuously improve evaluation quality. The lack of a closed-loop optimization mechanism for the evaluation process leads to weak model adaptability and insufficient credibility of evaluation results. To mitigate these issues, an AI-based intelligent text evaluation system is proposed. Summary of the Invention

[0004] The purpose of this invention is to provide an artificial intelligence-based intelligent text review system to solve the problems mentioned in the background art.

[0005] To achieve the above objectives, an artificial intelligence-based intelligent text review system is provided, including a sample collection module, a preset text management module, a review model management module, and a review result management module; The sample collection module is used to obtain the evaluation topic and evaluation semantics, classify semantic requirements according to the evaluation semantics, establish a corresponding number of templates to be filled according to the semantic requirement classification results, and collect a sample set related to the topic according to the evaluation semantics. The preset text management module is used to segment text units according to the semantics of each sample in the sample set, and set combination rules for text units based on Markov chains. According to the combination rules, the text units are matched with the templates to be filled. The matched templates to be filled are combined with the evaluation semantics and input into the templates to be filled through random selection and recombination methods. The input templates to be filled are combined to form multiple preset texts. The evaluation model management module filters standard texts by the source location of the sample set, and at the same time converts the standard texts into numerical feature vectors and extracts deep semantic feature vectors to form composite feature vectors of standard texts. The composite feature vectors are used as full-score labeled samples, and multi-dimensional evaluation training is carried out in combination with multiple preset texts. Based on the evaluation training results, an artificial intelligence evaluation model is built, and frequent itemset mining is completed on the template to be filled according to the artificial intelligence evaluation model. The evaluation result management module is used to collect the texts to be evaluated and upload them to the artificial intelligence evaluation model for evaluation report output.

[0006] Preferably, in the sample collection module, a blank framework and a network window for the artificial intelligence evaluation model are first established; The evaluation topic is obtained through a network window. After obtaining the evaluation topic, the artificial intelligence evaluation model grants file submission permissions, thereby allowing the evaluation result management module to collect the text to be evaluated. At the same time, the relevant texts are preprocessed in a unified manner. The relevant texts include the evaluation topic, sample set, and text to be evaluated. Unified preprocessing includes: The text is identified for misspellings, and the identified misspellings are summarized and intelligently replaced throughout the text. The text is cleaned of invalid and overlapping characters, retaining only those characters that are helpful for analysis. Perform a unified text format conversion on the text, and convert videos and images to text format only; Based on the preprocessed evaluation topics, the evaluation semantics are deeply analyzed using natural language processing methods to obtain the evaluation semantics of the evaluation topics; the evaluation semantics include core keywords, entities, and semantic relationships; Based on the semantic design of the evaluation, the search terms are combined and the search engine is called to collect samples related to the topic in batches, thereby forming a sample set. At the same time, the search source location of each sample is marked during the collection process.

[0007] Preferably, in the sample collection module, semantic requirements are classified according to the evaluation semantics of the evaluation topic, so that the evaluation semantics are divided into multiple dimensions of semantic requirements; Specifically, the technical field is determined by the evaluation semantics of the evaluation topic, the domain knowledge base corresponding to the technical field is obtained, and then the evaluation semantics is combined with the semantic role labeling method to mine the component steps contained in the evaluation topic. The same number of templates to be filled are initially established with the same number of component steps, so that each component has a template to be filled. The template to be filled contains multiple slots to be filled, and the slots are attached with corresponding evaluation semantic tags. Meanwhile, the evaluation model management module encodes the text units in the standard text set, applies the frequent itemset mining algorithm to mine the combinations of text units that frequently co-occur in the standard text, and optimizes the structure of the template to be filled initially established by the sample collection module based on the mined frequent itemsets. The structure optimization includes adding new slots and adjusting the slot order.

[0008] Preferably, in the preset text management module, the semantics of each sample in the sample set are obtained by using the semantic role labeling method, and the samples are segmented by the text block method based on the sample semantics, so that each sample is segmented into a text unit with independent semantics. Each text unit is treated as an independent state. The transition probabilities between text units in the sample set are statistically analyzed to construct the state transition probability matrix of the Markov chain. Combination rules are set for text units based on the transition probability matrix. The transition probability matrix of the Markov chain is combined with weighted random sampling to form a selection mechanism. Based on the selection mechanism, text units are selected from the set of matched text units and then filled into the slots corresponding to the template to be filled after sequential recombination. Text units can be repeatedly filled in multiple slots. After all slots are filled with text units, the templates to be filled are combined to form the preset text.

[0009] Preferably, in the process of forming preset text, the evaluation model management module scores the formed preset text using an artificial intelligence evaluation model, extracts combination paths based on the scores of each preset text, and obtains the unit combination paths used by high-scoring preset texts and the unit combination paths used by low-scoring preset texts based on the extraction results. The state transition probability matrix of the Markov chain is dynamically adjusted based on two unit combination paths. This increases the probability of unit transition paths that frequently appear in high-scoring texts and decreases the probability of paths that appear in low-scoring texts, thus completing matrix optimization. This updates the quality of subsequently generated preset texts and uses them as new training data to input back to the evaluation model management module to complete the iteration of the artificial intelligence evaluation model.

[0010] Preferably, in the process of forming the preset text, the evaluation model management module sets source authority scoring rules and presets a scoring threshold, wherein the scoring threshold is adjusted according to the user's expected scoring difficulty; the lower the scoring difficulty, the lower the scoring threshold. Then, based on the source location corresponding to the text unit in the preset text and the source authority scoring rules, the source authority score is obtained. The source authority score of each preset text is compared with the score threshold. If the source authority score of a preset text is higher than the score threshold, it is used as the standard text; otherwise, the comparison continues with the next preset text.

[0011] Preferably, the preset text is used as the standard text, and the standard text is transformed into a numerical vector feature using TF-IDF to obtain the numerical feature vector of the standard text. At the same time, the standard text is encoded using BERT, and deep semantic features are obtained based on the text encoding results. The numerical feature vector and the deep semantic feature vector are concatenated to form a composite feature vector of the standard text; in this process, all dimensions of the deep semantic feature vector are added to all dimensions of the numerical feature vector, thus forming a composite feature vector with higher dimensions.

[0012] Preferably, the process of forming a composite feature vector of the standard text involves marking the composite feature vector of the standard text as a perfect score, and then combining it with the composite feature vector of a preset text as training samples. The labels of these samples are pseudo-labels generated from the similarity scores with the standard text. Subsequently, a multi-dimensional evaluation model is trained using the random forest method. The trained model is then used as an artificial intelligence evaluation model for the text to be evaluated. The multi-dimensional evaluation model training is as follows: The composite feature vectors of the generated standard text and the preset text are combined into a feature matrix; For standard text, each dimension of its label vector is a perfect score for that dimension; For a preset text, the value of each dimension of its label vector is obtained by calculating the similarity between the sample and the standard text in the corresponding dimension; Initialize a multi-output random forest regression model consisting of multiple decision trees, each trained to predict all output dimensions. At each node of each decision tree, a subset is randomly selected from all features, and the optimal features and splitting threshold are selected based on the mean squared error until the maximum depth of the tree is satisfied. Once all decision trees are trained, they form the final artificial intelligence evaluation model.

[0013] Preferably, after the multi-dimensional evaluation model has been trained, the text to be evaluated is collected through a network window and input into the artificial intelligence evaluation model; The AI ​​evaluation model outputs the sub-scores and overall scores of the text to be evaluated across various evaluation dimensions. Combined with the key text fragments and their causes that lead to specific scores, which are explained by AI technology, an evaluation report is generated and can be viewed via a web window.

[0014] Compared with the prior art, the beneficial effects of the present invention are as follows: 1. This AI-based intelligent text review system effectively solves the problems of inconsistent review standards and insufficient accuracy in existing technologies by integrating domain knowledge bases, multi-dimensional feature extraction, and closed-loop iterative optimization mechanisms. The system determines the technical field through in-depth semantic analysis of the review, calls the corresponding domain knowledge base to construct structured review dimensions and templates to be filled, and optimizes the template structure by combining frequent itemset mining. This ensures that the review standards not only conform to domain professional norms but can also be dynamically adjusted according to the core components of the text, significantly improving the relevance and accuracy of the review and avoiding the drawbacks of the one-size-fits-all approach in existing technologies. At the same time, the system uses a fusion of TF-IDF numerical features and BERT deep semantic features to construct composite feature vectors, comprehensively capturing the surface statistical information and deep semantic relationships of the text, ensuring that the review results can truly reflect the text quality.

[0015] 2. This AI-based intelligent text review system achieves continuous improvement in review quality through Markov chain transition matrix optimization and model iteration mechanism, overcoming the deficiency of closed-loop optimization in existing technologies. The system dynamically adjusts the transition probability matrix of text units based on the comparison of unit combination paths between high- and low-scoring preset texts, making the subsequently generated preset texts more closely match the structure and semantic features of high-quality texts, providing high-quality samples for model training. Simultaneously, the optimized preset texts are used as new training data to feed back into the model, continuously updating model parameters and forming a closed loop of generation-evaluation-optimization-iteration. This mechanism enables the system to continuously adapt to new review requirements and text features, with review accuracy gradually improving with the number of iterations. Compared to the fixed models of existing technologies, it has stronger adaptability and growth potential. Attached Figure Description

[0016] Figure 1 This is a schematic diagram of the overall structure of an artificial intelligence-based intelligent text review system according to the present invention. Detailed Implementation

[0017] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0018] Please see Figure 1 As shown, the purpose of this embodiment is to provide an artificial intelligence-based intelligent text review system, including a sample collection module, a preset text management module, a review model management module, and a review result management module; The sample collection module is used to obtain the evaluation topics and evaluation semantics, classify semantic requirements according to the evaluation semantics, establish a corresponding number of templates to be filled based on the semantic requirement classification results, and collect a set of samples related to the topics according to the evaluation semantics. The sample collection module is responsible for obtaining the core requirements of the evaluation (topic, semantics), constructing the template to be filled, collecting a high-quality sample set, and completing the unified preprocessing of all texts to provide standardized and structured basic data for subsequent modules. At the same time, it receives optimization feedback from the evaluation model management module and dynamically improves the template structure. In the sample collection module, the blank framework of the artificial intelligence evaluation model is first established (defining the core dimensions of evaluation: structural integrity, terminology standardization, logical coherence, information integrity, and clarity of innovation points) and the network window are established first; A web-based network window is built using the Flask framework, supporting the input of review topics, the upload of text to be reviewed, and the display of review reports. The review topics are obtained through the web window. After obtaining the review topics, the artificial intelligence review model grants file submission permissions, thereby allowing the review results management module to collect the text to be reviewed. Users enter the review topic through the web window, and the system automatically grants file upload permissions, allowing users to subsequently upload the abstract text of the paper to be reviewed; At the same time, the relevant texts are preprocessed in a unified manner. The relevant texts include the evaluation topic, sample set, and text to be evaluated. Unified preprocessing includes: The text is identified for misspellings, and the identified misspellings are summarized and intelligently replaced throughout the text. Combining a domain terminology database, the system identifies and replaces typos using word segmentation and edit distance algorithms. The domain terminology database includes core terms related to the topic and labels them with their definitions, usage scenarios, and specifications. The text is cleaned of invalid and overlapping characters, retaining only those characters that are helpful for analysis. Remove meaningless characters (such as special symbols, redundant spaces, page number markers) and repetitive sentences (such as continuously repeated research) from the text, and retain only the core semantic content; Perform a unified text format conversion on the text, and convert videos and images to text format only; The TesseractOCR tool is used to convert image-formatted abstract text into plain text and extract PDF-formatted text into TXT format, achieving unified textification of all data. Based on the preprocessed evaluation topics, in-depth analysis of the evaluation semantics is performed using natural language processing methods to obtain the evaluation semantics of the evaluation topics. The evaluation semantics includes core keywords, entities, and semantic relationships. The steps are as follows: First, the preprocessed evaluation topic text is input, and the fine-tuned BERT receives the text. The top-N high-frequency, highly relevant core keywords are selected based on the word-level attention weights output by the model, forming a keyword list. At the same time, named entity recognition is performed on the text based on BERT-CRF to identify entities related to the evaluation scenario (such as task entities, method entities, object entities, and evaluation entities), and the entity types are labeled to form an entity list. Then, BERT-based relation extraction is used to extract semantic relationships between entities (such as evaluation-evaluation object, usage-method-task) using the identified entities as arguments, forming a list of semantic relationship triples. Finally, the core keywords, entities, and semantic relationships are integrated as the basic semantic information for subsequent retrieval.

[0019] Based on the semantic design of the evaluation, the search term combination is used to call the search engine to collect samples related to the topic in batches, thereby forming a sample set. At the same time, the search source location of each sample is marked during the collection process. The process involves inputting semantic evaluation results, constructing precise search terms based on semantic relationship triples, connecting core entities using Boolean logic, and combining entity synonym expansion (vector similarity) to form multiple search term combinations. Then, iterates through the search term combinations, calls the target search engine API (academic database API, general search engine API), sets search parameters (number of results, time range, source type), and obtains search results in batches.

[0020] In the sample collection module, semantic requirements are classified according to the evaluation semantics of the evaluation topic, so that the evaluation semantics are divided into multiple dimensions of semantic requirements; The process involves determining the technical field by analyzing the evaluation semantics of the evaluation topic, acquiring the corresponding domain knowledge base (containing standard processes, core elements, and terminology specifications for that domain), and then combining the evaluation semantics with semantic role labeling methods to mine the constituent steps of the evaluation topic. An initial number of templates to be filled is established, equal to the number of constituent steps, ensuring that each component corresponds to one template. Each template contains multiple slots to be filled, with corresponding evaluation semantic tags attached to each slot. The steps are as follows: First, based on the core entities and keywords in the evaluation semantics, the domain classification system is matched to determine the technical field to which the evaluation topic belongs. According to the determined technical field, the corresponding structured domain knowledge base (including domain standard processes, core elements, and terminology specifications) is retrieved. The evaluation semantic text is input into BERT-based semantic role annotation to identify the core predicates and corresponding arguments in the text. Combined with the domain knowledge base, the key component steps required to complete the evaluation topic are mined. According to the number of core component steps mined, an equal number of initial templates to be filled are established, with each template corresponding to one component. Each template contains multiple slots to be filled, and each slot is accompanied by a semantic tag for the evaluation of its constituent parts.

[0021] Meanwhile, the evaluation model management module encodes the text units in the standard text set, applies the frequent itemset mining algorithm to mine the combinations of text units that frequently co-occur in the standard text, and optimizes the structure of the template to be filled initially established by the sample collection module based on the mined frequent itemsets. The structure optimization includes adding new slots and adjusting the slot order. In this process, a unique code is assigned to each text unit to construct a text unit itemset database. Then, a frequent itemset mining algorithm is applied to mine the itemset database, and a minimum support threshold is set to extract frequently co-occurring combinations of text units (i.e., frequent itemsets) in the standard text. Add new slots: Convert core text units in the frequent itemset that were not included in the initial template into new slots; Adjust slot order: Based on the co-occurrence order of text units in the frequent itemset, adjust the slot order in the initial template as follows: ; in, Let X be the support (frequency index) of itemset X, where X is a combination of candidate text units (itemsets), and X is a single sample itemset in the itemset database. This represents the number of samples containing itemsets. The total number of samples in the itemset database D; ; in, This is the set of frequent itemsets after filtering. This is the minimum support threshold (default 0.6, can be adjusted according to the domain). ; in, For slots and The optimal arrangement order The two slots to be adjusted (corresponding to text units in the frequent itemset) , ), For text units Follow closely Co-occurrence relationship, For co-occurrence relationship → The number of times it appears in all frequent itemsets.

[0022] The preset text management module is used to segment text units according to the semantics of each sample in the sample set. At the same time, it sets combination rules for text units based on Markov chains and matches text units with templates to be filled according to the combination rules. The matched templates to be filled are combined with the evaluation semantics and input into the templates to be filled through random selection and recombination methods. The input templates to be filled are combined to form multiple preset texts. The sample set is transformed into standardized training data (preset text) that meets the evaluation requirements. Text unit combination rules are constructed through Markov chains to realize the diversified generation of preset text. At the same time, the scoring feedback from the evaluation model management module is received to optimize the combination rules and continuously improve the quality of preset text, providing high-quality and multi-dimensional sample support for model training. In the preset text management module, the semantics of each sample in the sample set are obtained by using the semantic role labeling method (BERT-based semantic role labeling). The sample semantic recognition and evaluation semantics acquisition methods are consistent here. Simultaneously, based on the sample semantics, the text segmentation method is used to segment the samples, so that each sample is segmented into text units with independent semantics; based on the argument boundaries of the sample semantics, text segmentation is performed according to the principle of semantic integrity, the samples are segmented into the smallest units with independent semantics, and a unique ID is assigned to each text unit, duplicate units are removed, and a structured text unit library is constructed. Each text unit is treated as an independent state. The transition probabilities between text units in the sample set are statistically analyzed to construct the state transition probability matrix of the Markov chain. Combination rules are set for text units based on the transition probability matrix. By extracting the text unit ID sequence of each sample, traversing all sample sequences, counting the transition frequency of adjacent text units, and calculating the transition probability from each unit to other units, a transition probability matrix is ​​constructed as a text unit combination rule. The transition probability matrix of the Markov chain is combined with weighted random sampling to form a selection mechanism. Based on the selection mechanism, text units are selected from the set of matched text units and then filled into the slots corresponding to the template to be filled after sequential recombination. Text units can be repeatedly filled in multiple slots. First slot: Select units by weighted random sampling from the subset of units matched by slot semantic labels, based on the frequency of unit occurrence in the sample; Non-first slot: Extract the corresponding probability distribution from the matrix based on the previous selected unit, filter the units and corresponding probabilities in the unit subset, and perform weighted random sampling to ensure that the logic of the previous selected unit for this slot is correct; After all slots are filled with text units, all filled templates are sorted to optimize language coherence. Then, the templates to be filled are summarized and combined to form the preset text, as shown in the following formula: ; in, For unit arrive The transition probability, For the frequency of transfer, This represents the total number of units in the text unit library. The Laplace smoothing coefficient (default 0.1); ; in, Select unit for the current slot The sampling probability, The unique identifier ID for unit u For the unit already selected in the previous slot, This is a subset of semantic matching units for the current slot.

[0023] The pre-defined text is generated, and the evaluation model management module scores the generated pre-defined text using an artificial intelligence evaluation model. Based on the scores of each pre-defined text, the combination path is extracted. Based on the extraction results, the unit combination path used by the high-scoring pre-defined text and the unit combination path used by the low-scoring pre-defined text are obtained. The Markov chain's state transition probability matrix is ​​dynamically adjusted based on two unit combination paths. This increases the probability of unit transition paths that frequently appear in high-scoring texts and decreases the probability of paths that appear in low-scoring texts, thus optimizing the matrix. This updates the quality of subsequently generated preset texts and uses them as new training data to input back into the evaluation model management module to complete the iteration of the artificial intelligence evaluation model. The specific formula is as follows: in, For the optimized unit arrive The transition probability, These are the weight coefficients of the original matrix. The reward coefficient for high-scoring paths. The penalty coefficient for low-scoring paths. + + =1, For high-scoring paths The frequency of transfers In the low-scoring path The frequency of transfers This is a smoothing coefficient; if the calculation result is negative, the minimum value is taken, thus avoiding the zero probability problem.

[0024] The evaluation model management module filters standard texts by the source location of the sample set, and transforms the standard texts into numerical feature vectors and extracts deep semantic feature vectors to form composite feature vectors of standard texts. The composite feature vectors are used as full-score labeled samples, and multi-dimensional evaluation training is carried out in combination with multiple preset texts. Based on the evaluation training results, an artificial intelligence evaluation model is built, and frequent itemset mining is completed on the template to be filled according to the artificial intelligence evaluation model. Responsible for screening standard texts and training multi-dimensional evaluation models. At the same time, through frequent itemset mining and combinatorial path analysis, we achieve two-way feedback of template optimization and combinatorial rule optimization, which runs through the entire process of data-template-generation-model, ensuring the continuous improvement of the overall system performance. The system generates a pre-defined text and sets source authority scoring rules in the evaluation model management module, such as authoritative database = 10 points, high-quality industry websites = 8 points, ordinary network resources = 5 points, and anonymous sources = 2 points. This forms a scoring rule library and presets a scoring threshold. The scoring threshold is adjusted according to the user's desired scoring difficulty, with a value range of [0,1], where 0 is the lowest difficulty and 1 is the highest difficulty. The lower the scoring difficulty, the lower the scoring threshold. Then, based on the source location corresponding to the text unit in the preset text and the source authority scoring rules, the source authority score is obtained. The source authority score of each preset text is compared with the score threshold. If the source authority score of a preset text is higher than the score threshold, it is used as the standard text; otherwise, the comparison continues with the next preset text.

[0025] The preset text is used as the standard text. The standard text is then transformed into a numerical vector feature vector using TF-IDF. The steps are as follows: A global vocabulary dictionary is constructed based on all standard texts. The TF-IDF weight of each word in each text is calculated to reflect the importance of the word in the text and its distinguishability in the global context. The TF-IDF weights of each text are arranged in order with the vocabulary of the dictionary as the dimension to form a numerical feature vector. Simultaneously, the standard text is encoded using BERT, and deep semantic features are obtained based on the text encoding results. The steps are as follows: The preprocessed text sequence is formatted as input to the BERT model. The formatted input is then fed into the pre-trained BERT model. The token output vector of the last Transformer layer of the model is extracted. This vector contains the global semantic information of the entire text. The token vector is then normalized (L2 normalization) to obtain the deep semantic feature vector.

[0026] The numerical feature vector and the deep semantic feature vector are concatenated to form a composite feature vector of the standard text. In this process, all dimensions of the deep semantic feature vector are added to all dimensions of the numerical feature vector to form a composite feature vector with higher dimensions (which simultaneously includes the surface lexical statistical features and deep semantic association features of the text).

[0027] A composite feature vector of the standard text is generated, and the composite feature vector of the standard text is marked as the full score. Then, the composite feature vector of the preset text is used as a training sample, and its label is a pseudo-label generated by the similarity score with the standard text. Then, a multi-dimensional evaluation model is trained based on the random forest method. The trained model is used as the artificial intelligence evaluation model for the text to be evaluated. The multi-dimensional evaluation model training is as follows: The composite feature vectors of the generated standard text and the preset text are combined into a feature matrix; For standard text, each dimension of its label vector is a perfect score for that dimension; For a preset text, the value of each dimension of its label vector is obtained by calculating the similarity between the sample and the standard text in the corresponding dimension; Initialize a multi-output random forest regression model consisting of multiple decision trees, each trained to predict all output dimensions. At each node of each decision tree, a subset is randomly selected from all features, and the optimal features and splitting threshold are selected based on the mean squared error until the maximum depth of the tree is satisfied. Once all decision trees are trained, they form the final artificial intelligence evaluation model.

[0028] The evaluation results management module is used to collect the texts to be evaluated and upload them to the artificial intelligence evaluation model to output the evaluation report.

[0029] It is responsible for receiving the text to be evaluated from the user, calling the trained evaluation model to perform the evaluation, and outputting a structured and interpretable evaluation report, completing the final closed loop from data input to result output, and providing users with intuitive and usable evaluation feedback; After the multi-dimensional evaluation model is trained, the text to be evaluated is collected through a network window and input into the artificial intelligence evaluation model; The AI-powered evaluation model outputs the individual scores and overall scores of the text to be evaluated across various evaluation dimensions. Combined with an analysis of the key text fragments and their causes that led to the specific scores, an evaluation report is generated and can be viewed via a web interface. The specific steps are as follows: First, following the feature extraction logic of standard text, a composite feature vector is extracted from the text to be evaluated. At the same time, the text to be evaluated is input into the trained artificial intelligence evaluation model, and the model outputs the sub-scores of each evaluation dimension. The comprehensive score is calculated according to the preset dimension weights (the weights are set based on the domain knowledge base or user needs). The LIME tool is called to generate perturbation samples in the local neighborhood of the composite feature vector as the target sample and calculate the prediction results. A local linear model is constructed to fit the prediction function of the original model. Features (corresponding text fragments) that have a significant impact on the scores of each dimension are selected, key deduction points are located, and then the key text fragments that affect the scores and the corresponding reasons are output. Finally, the scores of each evaluation dimension, the weighted composite score, the key text fragments output by LIME, and the causal analysis are integrated to form a structured evaluation report. The evaluation report is then displayed to users through a web window for them to view, archive, and use for subsequent modifications.

[0030] The above are merely preferred embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A text intelligent review system based on artificial intelligence, characterized in that: It includes a sample collection module, a preset text management module, an evaluation model management module, and an evaluation result management module; The sample collection module is used to obtain the evaluation topic and evaluation semantics, classify semantic requirements according to the evaluation semantics, establish a corresponding number of templates to be filled according to the semantic requirement classification results, and collect a set of samples related to the topic according to the evaluation semantics. The preset text management module is used to segment text units according to the semantics of each sample in the sample set, and set combination rules for text units based on Markov chains. According to the combination rules, the text units are matched with the templates to be filled. The matched templates to be filled are combined with the evaluation semantics and input into the templates to be filled through random selection and recombination methods. The input templates to be filled are combined to form multiple preset texts. The evaluation model management module filters standard texts by the source location of the sample set, and at the same time converts the standard texts into numerical feature vectors and extracts deep semantic feature vectors to form composite feature vectors of standard texts. The composite feature vectors are used as full-score labeled samples, and multi-dimensional evaluation training is carried out in combination with multiple preset texts. Based on the evaluation training results, an artificial intelligence evaluation model is built, and frequent itemset mining is completed on the template to be filled according to the artificial intelligence evaluation model. The evaluation result management module is used to collect the texts to be evaluated and upload them to the artificial intelligence evaluation model for evaluation report output.

2. The text intelligent review system based on artificial intelligence according to claim 1, characterized in that: In the sample collection module, a blank framework and network window for the artificial intelligence evaluation model are first established; The evaluation topic is obtained through a network window. After obtaining the evaluation topic, the artificial intelligence evaluation model grants file submission permissions, thereby allowing the evaluation result management module to collect the text to be evaluated. At the same time, the relevant texts are preprocessed in a unified manner. The relevant texts include the evaluation topic, sample set, and text to be evaluated. Unified preprocessing includes: The text is identified for misspellings, and the identified misspellings are summarized and intelligently replaced throughout the text. The text is cleaned of invalid and overlapping characters, retaining only those characters that are helpful for analysis. Perform a unified text format conversion on the text, and convert videos and images to text format only; Based on the preprocessed evaluation topics, the evaluation semantics are deeply analyzed using natural language processing methods to obtain the evaluation semantics of the evaluation topics; the evaluation semantics include core keywords, entities, and semantic relationships; Based on the semantic design of the evaluation, the search term combination is used to call the search engine to collect samples related to the topic in batches, thereby forming a sample set. At the same time, the search source location of each sample is marked during the collection process.

3. The text intelligent review system based on artificial intelligence according to claim 2, characterized in that: In the sample collection module, semantic requirements are classified according to the evaluation semantics of the evaluation topic, so that the evaluation semantics are divided into multiple dimensions of semantic requirements; Specifically, the technical field is determined by the evaluation semantics of the evaluation topic, the domain knowledge base corresponding to the technical field is obtained, and then the evaluation semantics is combined with the semantic role labeling method to mine the component steps contained in the evaluation topic. The same number of templates to be filled are initially established with the same number of component steps, so that each component has a template to be filled. The template to be filled contains multiple slots to be filled, and the slots are attached with corresponding evaluation semantic tags. Meanwhile, the evaluation model management module encodes the text units in the standard text set, applies the frequent itemset mining algorithm to mine the combinations of text units that frequently co-occur in the standard text, and optimizes the structure of the template to be filled initially established by the sample collection module based on the mined frequent itemsets. The structure optimization includes adding new slots and adjusting the slot order.

4. The text intelligent review system based on artificial intelligence according to claim 2, characterized in that: In the preset text management module, the semantics of each sample in the sample set are obtained by using the semantic role labeling method. At the same time, the samples are segmented by the text block method based on the sample semantics, so that each sample is segmented into a text unit with independent semantics. Each text unit is treated as an independent state. The transition probabilities between text units in the sample set are statistically analyzed to construct the state transition probability matrix of the Markov chain. Combination rules are set for text units based on the transition probability matrix. The transition probability matrix of the Markov chain is combined with weighted random sampling to form a selection mechanism. Based on the selection mechanism, text units are selected from the set of matched text units and then filled into the slots corresponding to the template to be filled after sequential recombination. Text units can be repeatedly filled in multiple slots. After all slots are filled with text units, the templates to be filled are combined to form the preset text.

5. The text intelligent review system based on artificial intelligence according to claim 4, characterized in that: The preset text is formed, and the evaluation model management module scores the preset text through an artificial intelligence evaluation model. Based on the scores of each preset text, the module extracts the combination path and obtains the unit combination path used by the high-scoring preset text and the unit combination path used by the low-scoring preset text according to the extraction results. The state transition probability matrix of the Markov chain is dynamically adjusted based on two unit combination paths. This increases the probability of unit transition paths that frequently appear in high-scoring texts and decreases the probability of paths that appear in low-scoring texts, thus completing matrix optimization. This updates the quality of subsequently generated preset texts and uses them as new training data to input back to the evaluation model management module to complete the iteration of the artificial intelligence evaluation model.

6. The text intelligent review system based on artificial intelligence according to claim 4, characterized in that: The preset text is generated, and the evaluation model management module sets source authority scoring rules and presets a score threshold. The score threshold is adjusted according to the user's expected scoring difficulty. The lower the scoring difficulty, the lower the score threshold. Then, based on the source location corresponding to the text unit in the preset text and the source authority scoring rules, the source authority score is obtained. The source authority score of each preset text is compared with the score threshold. If the source authority score of a preset text is higher than the score threshold, it is used as the standard text; otherwise, the comparison continues with the next preset text.

7. The text intelligent review system based on artificial intelligence according to claim 6, characterized in that: The preset text is used as the standard text. The standard text is transformed into a numerical vector feature using TF-IDF to obtain the numerical feature vector of the standard text. At the same time, the standard text is encoded using BERT, and deep semantic features are obtained based on the text encoding results. The numerical feature vector and the deep semantic feature vector are concatenated to form a composite feature vector of the standard text; in this process, all dimensions of the deep semantic feature vector are added to all dimensions of the numerical feature vector, thus forming a composite feature vector with higher dimensions.

8. The text intelligent review system based on artificial intelligence according to claim 7, characterized in that: The composite feature vector of the standard text is formed by marking the composite feature vector of the standard text as full score, and then combining it with the composite feature vector of the preset text as training samples. The labels are pseudo-labels generated by the similarity scores with the standard text. Then, a multi-dimensional evaluation model is trained based on the random forest method. The trained model is used as the artificial intelligence evaluation model for the text to be evaluated. The multi-dimensional evaluation model training is as follows: The composite feature vectors of the generated standard text and the preset text are combined into a feature matrix; For standard text, each dimension of its label vector is a perfect score for that dimension; For a preset text, the value of each dimension of its label vector is obtained by calculating the similarity between the sample and the standard text in the corresponding dimension; Initialize a multi-output random forest regression model consisting of multiple decision trees, each trained to predict all output dimensions. At each node of each decision tree, a subset is randomly selected from all features, and the optimal features and splitting threshold are selected based on the mean squared error until the maximum depth of the tree is satisfied. Once all decision trees are trained, they form the final artificial intelligence evaluation model.

9. The text intelligent review system based on artificial intelligence according to claim 8, characterized in that: Once the multi-dimensional evaluation model is trained, the text to be evaluated is collected through a network window and input into the artificial intelligence evaluation model. The AI ​​evaluation model outputs the sub-scores and overall scores of the text to be evaluated across various evaluation dimensions. Combined with the key text fragments and their causes that lead to specific scores, which are explained by AI technology, an evaluation report is generated and can be viewed via a web window.

Images