Scenario-based essay ai grading improvement system and method

Abstract

This invention relates to the fields of artificial intelligence and education technology, and discloses a scenario-based AI-based essay evaluation improvement system and method. The scenario-based AI-based essay evaluation improvement method includes: acquiring learner basic information and essay text data and performing standardized processing; conducting text statistical analysis on the standardized learner basic information set; identifying scenario types and assessing task difficulty; establishing a matching relationship between learner writing ability and task difficulty assessment results, and obtaining personalized evaluation strategy configuration parameters based on the matching results; processing the personalized evaluation strategy configuration parameters and the essay text to be evaluated; verifying the accuracy of the dimensional scoring results and comprehensive evaluation report based on feedback data, and adjusting the personalized evaluation strategy configuration parameters based on the verification results. This invention, by constructing a scenario-based evaluation mechanism and personalized evaluation strategies, can automatically adjust evaluation standards according to different teaching scenarios, learner characteristics, and essay types.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence and education technology, and more specifically, to a system and method for improving essay AI evaluation based on scenario inference. Background Technology

[0002] With the rapid development of artificial intelligence technology in education, intelligent essay grading has become an important component of educational informatization. Traditional essay grading mainly relies on teachers' subjective judgment, which suffers from problems such as a huge workload, inconsistent evaluation standards, and insufficient personalized guidance, making it difficult to meet the modern education demand for efficient, fair, and personalized evaluation.

[0003] Existing intelligent essay grading technologies primarily rely on natural language processing and machine learning methods, automatically scoring essays by extracting linguistic and structural features from the text. However, these technologies generally suffer from problems such as rigid evaluation standards, a lack of contextual modeling capabilities, and insufficient personalized evaluation capabilities. Specifically, existing systems employ uniform evaluation standards, failing to dynamically adjust based on different writing scenarios and learner characteristics; they lack a deep understanding of the writing context, making it impossible to simulate the diverse evaluation needs of real-world teaching environments; and they ignore individual learner differences, failing to provide truly personalized evaluation and guidance.

[0004] Therefore, there is an urgent need for an intelligent essay evaluation technology that can adaptively adjust according to specific teaching scenarios and learner characteristics, in order to address the shortcomings of existing technologies in terms of scenario adaptability and personalized evaluation, and improve the intelligence level of essay evaluation and the effectiveness of teaching applications. Summary of the Invention

[0005] This invention provides an AI-based essay evaluation system and method based on scenario-based reasoning, which solves the technical problems of single evaluation criteria, lack of scenario-based evaluation, and inability to effectively simulate real teaching environments in related technologies.

[0006] This invention provides a method for improving AI-based essay evaluation based on scenario inference, including:

[0007] Acquire learner basic information and essay text data and perform standardization processing to obtain a standardized learner basic information set and a standardized essay text dataset. The standardized essay text dataset includes current essay task information and essay texts to be reviewed.

[0008] Textual statistical analysis was performed on a standardized set of learners’ basic information to obtain feature vectors of learners’ writing abilities.

[0009] By combining learners’ writing ability feature vectors, we can identify the scenario type and assess the task difficulty of the current writing task information, and obtain the scenario type label and task difficulty assessment results.

[0010] Based on scenario type labels, task difficulty assessment results, and learner writing ability feature vectors, a matching relationship between learner writing ability and task difficulty assessment results is established, and personalized review strategy configuration parameters are obtained based on the matching results.

[0011] Multi-feature fusion technology is used to process personalized review strategy configuration parameters and essay texts to be reviewed, resulting in multi-dimensional scoring results and a comprehensive evaluation report.

[0012] Collect feedback data from teachers on the dimensional scoring results and comprehensive evaluation reports, verify the accuracy of the dimensional scoring results and comprehensive evaluation reports based on the feedback data, adjust the configuration parameters of personalized review strategies according to the verification results, obtain optimized review strategies, and use them as input for the next round of review process.

[0013] In a preferred embodiment, the specific implementation process of the standardization process includes:

[0014] Raw data is obtained through the teaching management system API interface, and the format and integrity of learners' basic information and essay text data are checked and verified.

[0015] Convert raw text data to UTF-8 encoding format and remove special characters and formatting tags;

[0016] Perform range checks and outlier identification on numerical raw data, and use interpolation methods to correct outlier data;

[0017] A unified data storage structure containing student information tables, historical essay tables, and current essay tables was established to obtain a standardized teaching dataset.

[0018] In a preferred embodiment, the specific implementation process of the text statistical analysis includes:

[0019] Obtain historical essay data from the standardized essay text dataset, perform lexical statistical analysis on the historical essay data, calculate the total number of words in each essay and calculate the average number of words to obtain the average vocabulary size.

[0020] Count the number of unique word categories and the total number of times words are used, and calculate the vocabulary richness.

[0021] Syntactic analysis was performed on historical essay data to count the number of words in each sentence and calculate the average sentence length.

[0022] We quantify statistical indicators, including average vocabulary size, vocabulary richness, and average sentence length, to construct a feature vector of learners' writing ability.

[0023] In a preferred embodiment, the specific implementation process of scenario type identification and task difficulty assessment includes:

[0024] Obtain current writing task information, extract keywords and perform semantic analysis on the current writing task information, and identify the writing style through keyword matching;

[0025] Calculate the proportion of abstract concept words in the current writing task information, assess the complexity of the writing requirements, analyze the relationship between the word count requirement and the time limit, and obtain the task difficulty assessment result by weighted summation according to preset weights.

[0026] Based on the assessment results of text type and task difficulty, and combined with the analysis of teaching objectives, the types of teaching objectives are determined, and scenario type labels are output.

[0027] In a preferred embodiment, the process of generating the personalized review strategy configuration parameters includes:

[0028] Obtain the learner's writing ability feature vector and task difficulty assessment results, calculate the absolute difference between the writing ability feature vector and the task difficulty assessment results and divide it by the maximum difficulty value to obtain the relative deviation ratio, and obtain the fit value by reverse calculation of the relative deviation ratio.

[0029] The rigor and tolerance range of the evaluation strategy are determined based on the fit score.

[0030] Select the corresponding evaluation dimension weight configuration based on the text type in the scenario type label;

[0031] Personalized review strategy configuration parameters are generated based on weight configuration and fit value.

[0032] In a preferred embodiment, the specific implementation process of the multi-feature fusion technology includes:

[0033] The algorithm obtains the text of the essay to be reviewed and the configuration parameters of the personalized review strategy. It performs language feature analysis on the text of the essay to be reviewed, calculates the vocabulary richness by counting the total number of words and the number of unique words, identifies the number of grammatical errors by the grammar check algorithm, and calculates the grammar accuracy rate.

[0034] Content quality analysis is conducted, the degree of matching between the essay and the topic is determined through topic relevance analysis, and the originality score is obtained by comparing the essay with sample essays in the knowledge base using word vector similarity calculation method.

[0035] Perform structural organization analysis, check the existence of basic elements, calculate the semantic relevance of keywords in adjacent paragraphs, and assess logical coherence.

[0036] Based on the weight settings in the personalized review strategy configuration parameters, a comprehensive evaluation report is obtained by integrating the scores of three dimensions: language expression, content quality, and structural organization.

[0037] In a preferred embodiment, the specific implementation process of verifying the accuracy of the dimensional scoring results and the comprehensive evaluation report by combining feedback data includes:

[0038] We obtained feedback data and multi-dimensional scoring results, compared and analyzed the teacher scores in the teacher feedback data with the AI ​​scores in the comprehensive evaluation report, and used the Pearson correlation coefficient to calculate the correlation between teacher scores and AI scores to obtain the scoring consistency index.

[0039] Based on the comparison results, the difference between the AI ​​score and the teacher score is calculated to establish an error dataset. Statistical analysis is then used to identify scoring situations and characteristics of misjudged essay types.

[0040] Multiple linear regression analysis was used to analyze the relationship between the error dataset and learner features, task features, and text features to establish an error prediction model.

[0041] Based on the analysis results of the error prediction model, the weight and threshold settings in the personalized review strategy configuration parameters are adjusted to obtain optimized review strategy parameters as input for the next round of review process.

[0042] In a preferred embodiment, the specific implementation of identifying the number of syntax errors and calculating the syntax accuracy using a syntax checking algorithm includes:

[0043] We acquire historical essay data, perform word segmentation and part-of-speech tagging on the historical essay data, and use a part-of-speech tagger based on a hidden Markov model to decompose the text into lexical units and tag grammatical attributes.

[0044] Syntactic structure analysis is performed on the annotated text based on a grammar rule base, and syntax error types are identified by matching context-free grammar rules.

[0045] The number and location of grammatical errors are counted, and the grammatical accuracy rate is calculated by dividing the number of grammatically correct sentences by the total number of sentences, providing a quantitative basis for the subsequent construction of feature vectors.

[0046] In a preferred embodiment, a fit value greater than the relative deviation ratio indicates a high degree of matching, and a fit value less than the relative deviation ratio indicates a low degree of matching; the matching result between the learner's writing ability and the task difficulty assessment result is determined based on the fit value.

[0047] In a preferred embodiment, the scenario-based AI-based essay review improvement system is used to execute the above-described scenario-based AI-based essay review improvement method, including:

[0048] The data acquisition module is used to acquire learner basic information and essay text data and perform standardization processing to obtain a standardized learner basic information set and a standardized essay text dataset. The standardized essay text dataset includes current essay task information and essay text to be reviewed.

[0049] The feature extraction module is used to perform text statistical analysis on a standardized set of learners’ basic information to obtain feature vectors of learners’ writing abilities.

[0050] The context recognition module is used to combine learners’ writing ability feature vectors to identify the context type and assess the task difficulty of the current writing task information, and obtain the context type label and task difficulty assessment results.

[0051] The strategy matching module is used to establish a matching relationship between learners' writing ability and task difficulty assessment results based on scenario type labels, task difficulty assessment results and learners' writing ability feature vectors, and obtain personalized review strategy configuration parameters based on the matching results.

[0052] The scoring and analysis module is used to process personalized review strategy configuration parameters and essay texts to be reviewed using multi-feature fusion technology to obtain multi-dimensional scoring results and comprehensive evaluation reports.

[0053] The optimization control module is used to collect feedback data from teachers on the dimensional scoring results and comprehensive evaluation reports. It combines the feedback data to verify the accuracy of the dimensional scoring results and comprehensive evaluation reports, adjusts the configuration parameters of personalized review strategies based on the verification results, obtains the optimized review strategy, and uses it as input for the next round of review process.

[0054] The beneficial effects of this invention are as follows: by constructing a scenario deduction mechanism and a personalized evaluation strategy, the evaluation criteria can be automatically adjusted according to different teaching scenarios, learner characteristics and essay types, so as to achieve more accurate essay evaluation. Compared with the traditional AI evaluation system, the evaluation accuracy of this invention has been improved, and the consistency with the evaluation results of professional teachers has also been improved, effectively solving the problems of rigid evaluation criteria and insufficient adaptability of existing technologies.

[0055] By modeling learners' writing ability characteristics and generating adaptive strategies, we can identify each learner's strengths and weaknesses, and provide targeted evaluation opinions and improvement suggestions. We not only focus on the evaluation of the writing results, but also pay attention to the analysis of learners' cognitive characteristics and development trajectories, providing a scientific basis for personalized teaching and precise learning, and significantly improving teaching effectiveness and learning experience. Attached Figure Description

[0056] Figure 1 This is a flowchart of the AI-based essay evaluation improvement method based on scenario inference in this invention;

[0057] Figure 2 This is a module diagram of the AI-based essay evaluation improvement system based on scenario deduction in this invention;

[0058] Figure 3 It is a bar chart comparing and analyzing the accuracy of the reviews;

[0059] Figure 4 It is a radar chart for evaluating the effectiveness of personalized review;

[0060] Figure 5 It is a line graph for monitoring system performance indicators. Detailed Implementation

[0061] The subject matter described herein will now be discussed with reference to exemplary embodiments. It should be understood that these embodiments are discussed only to enable those skilled in the art to better understand and implement the subject matter described herein, and changes may be made to the function and arrangement of the elements discussed without departing from the scope of this specification. Various processes or components may be omitted, substituted, or added as needed in the examples. Furthermore, some features described in the examples may be combined in other examples.

[0062] At least one embodiment of the present invention discloses a method for improving AI-based essay evaluation based on scenario inference, such as... Figure 1 As shown, it includes:

[0063] Step 1: Obtain learner basic information and essay text data and perform standardization processing to obtain a standardized learner basic information set and a standardized essay text dataset. The standardized essay text dataset includes current essay task information and essay texts to be reviewed.

[0064] Based on the teaching management system interface, a standardized teaching dataset is obtained using data standardization technology.

[0065] In the specific implementation process, basic learner information is obtained through the standard API interface of the teaching management system. This includes student identification information such as name, grade, class, and student ID, as well as learning performance data such as historical scores, attendance, and homework completion in Chinese language and literature. During data acquisition, the format and completeness of learner basic information and essay text data are checked to ensure that data fields are complete, data types are correct, and required fields are not missing, ensuring that the obtained raw data meets the requirements for subsequent processing. Learners' historical essay records are collected, including essay text content, corresponding topic requirements, teacher's sub-scores and overall evaluation, essay completion time, and other key information. The complete text content of the essay currently awaiting review, the specific requirements of the essay topic, the specified writing time and word limit, and other task parameters are obtained.

[0066] The collected raw data underwent standardization processing, converting text data of different formats to UTF-8 encoding, removing special characters and formatting marks, and retaining only the plain text content. Numerical data underwent range checking and outlier handling to ensure data validity and consistency. A unified data storage structure was established, including student information tables, historical essay tables, and current essay tables, facilitating data retrieval and processing in subsequent steps.

[0067] The output of this step is a standardized teaching dataset, which contains three main components: a set of basic student information, a set of historical essay data, and a set of current essay information. These three sets together constitute the complete foundation of teaching data.

[0068] Furthermore, data augmentation techniques can be used to expand historical essay data. By employing methods such as synonym replacement and sentence structure transformation, more diverse training samples can be generated, improving the effectiveness of subsequent feature extraction and model training. Data augmentation techniques can increase data diversity while preserving the original meaning of the essay, reducing overfitting of the model to specific expressions and enhancing the system's generalization ability.

[0069] Step 2: Perform text statistical analysis on the standardized learner basic information set to obtain learner writing ability feature vectors;

[0070] Based on standardized teaching datasets, text statistical analysis methods were used to obtain multi-dimensional feature vectors reflecting learners' writing abilities.

[0071] In the specific implementation process, a lexical-level statistical analysis is conducted on learners' historical essay data. The essay texts are segmented using Chinese word segmentation algorithms to divide continuous text into independent lexical units. Lexical boundaries are identified through dictionary matching and statistical models, punctuation marks and special characters are removed, and meaningful vocabulary content is retained to obtain the total vocabulary count. The average vocabulary size of historical essays is calculated through the following steps: The total vocabulary size of each historical essay is counted, and a vocabulary count record table is created; the total vocabulary size of all historical essays is summed; the sum is divided by the number of historical essays to obtain the average vocabulary size, which reflects the learner's basic vocabulary usage level.

[0072] The vocabulary richness index is calculated using the type tagging ratio analysis method. The specific steps are as follows: 1. Vocabulary identification and statistics are performed on the essay text to record all words that appear and their frequencies; 2. The number of types of non-repeating words is counted, that is, the total number of unique words after removing repetitions; 3. The total number of times words are used in the essay is counted, including words that are repeated; 4. The number of non-repeating words is divided by the total number of times words are used to obtain the vocabulary richness ratio. The higher the ratio, the more diverse the vocabulary usage.

[0073] Syntactic feature extraction is performed. The specific steps for calculating the average sentence length are as follows: Segment the essay text to identify the boundaries of each complete sentence; count the number of words in each sentence and create a sentence length statistics table; sum the number of words in all sentences; divide the sum by the total number of sentences to obtain the average sentence length value, which reflects the complexity of the learner's sentence expression.

[0074] This study analyzes sentence complexity, statistically analyzes the proportion of compound and complex sentences, and calculates a sentence diversity index. Compound sentence identification is achieved by detecting the presence of coordinating and subordinating conjunctions, such as the keywords "and," "but," "because," and "although." Complex sentence identification is achieved by analyzing the nested structure and hierarchical levels of modifiers, counting the number of sentences containing complex structures such as relative clauses and adverbial clauses. Sentence diversity is measured by calculating the ratio of the number of different sentence types to the total number of sentences, reflecting the richness of learners' sentence usage. Grammatical correctness features are extracted, and a grammar checking tool is used to identify the number and type of grammatical errors, calculating the grammar accuracy rate. The grammar checking tool operates based on a pre-set grammar rule base, automatically detecting common grammatical problems such as subject-verb disagreement, tense errors, and inappropriate word collocation. Statistical records are created for each type of error, and the grammar accuracy rate is calculated as the ratio of the number of sentences without grammatical errors to the total number of sentences.

[0075] Content-level feature analysis is conducted. Thematic consistency of the essay is calculated, and the coherence of content is assessed by calculating the semantic similarity between paragraphs. Thematic consistency analysis employs keyword extraction and semantic matching techniques to extract the core thematic words of each paragraph, calculate the relevance between thematic words in different paragraphs, and assess the unity and coherence of the overall thematic expression. The integrity of the logical structure is analyzed, identifying the presence of structural elements such as introduction, argumentation, and conclusion. Structural element identification is achieved by analyzing the functional and positional characteristics of paragraphs, checking whether the article's beginning includes a thematic introduction, whether the middle section includes argumentation development, and whether the ending includes a summary, assessing the completeness of the article's structure. Sentimental tendency features are extracted, analyzing the type and intensity of emotions expressed in the essay. Sentimental tendency analysis is achieved through sentiment lexicon matching and semantic analysis, identifying sentimental words in the text, judging the overall positive or negative sentiment tendency, calculating the intensity level of emotional expression, and providing a reference for the emotional dimension in personalized evaluation.

[0076] Further analysis of learners' writing development trajectories was conducted. The changing trends in essay quality across different periods were calculated, and a linear regression analysis was used to fit the learners' progress curves. The specific steps were as follows: Essay scoring data was collected from learners at different time points. This data originated from teachers' scores on historical essays, including scores for various dimensions such as language expression, content quality, and structural organization, as well as a comprehensive score. A time-score correspondence table was established. All data points were marked on a coordinate system with time as the horizontal axis and essay score as the vertical axis. The least squares method was used to find the best-fitting straight line, minimizing the sum of squared distances from all data points to the line. The slope and intercept parameters of the line were determined, where the slope reflects the learner's rate of progress, and the intercept reflects the learner's basic writing level. Based on the fitting results, the learners' writing development trends and potential were assessed.

[0077] This study statistically analyzes the performance differences in essays across different writing styles to identify learners' strengths and weaknesses. Based on the statistical results, a learner writing ability feature vector is constructed, encompassing lexical, syntactic, and content features. Lexical features include indicators such as average vocabulary size, vocabulary richness, and usage rate of specialized terms; syntactic features include indicators such as average sentence length, sentence complexity, and grammatical accuracy; and content features include indicators such as thematic consistency, logical structure completeness, and intensity of emotional expression. After numerical processing of these feature indicators, they are combined according to preset feature weights to form a multi-dimensional learner writing ability feature vector. This vector comprehensively reflects the learner's quantitative performance level in different writing abilities.

[0078] The output of this step is a learner's writing ability feature vector, which contains multiple dimensions of writing ability feature values. Each feature corresponds to the learner's quantitative performance in a certain writing ability. The total number of dimensions of the vector is determined according to actual needs.

[0079] Furthermore, principal component analysis (PCA) can be used to reduce the dimensionality of the feature vectors, remove redundant features, and extract the most representative writing ability indicators. Feature importance analysis can then identify the key features that have the greatest impact on essay quality, providing crucial references for subsequent scenario matching.

[0080] Step 3: Combine learners' writing ability feature vectors to identify the scenario type and assess the task difficulty of the current writing task information, and obtain scenario type labels and task difficulty assessment results.

[0081] Based on learners' writing ability feature vectors and current writing task information, a multi-level classification algorithm is used to obtain scenario type labels and task difficulty assessment results.

[0082] In the specific implementation process, the current writing task is classified into different genres. Keyword extraction and semantic analysis are performed on the current writing task information. Keyword extraction uses the TF-IDF algorithm to calculate the importance of words in the question, extracting the keywords with the highest weights as the basis for genre identification. Semantic analysis uses a word vector model to calculate the semantic similarity between the question and different genre standard templates to determine the genre tendency of the question. By analyzing the keywords and expressions in the writing question, basic genre types such as narrative, argumentative, expository, and practical writing are identified. A genre identification rule base is established using a combination of keyword matching and semantic analysis, including genre feature vocabulary, typical sentence patterns, and structural characteristics. For complex questions, the main genre tendency and secondary genre components are identified.

[0083] Conduct a task difficulty level assessment. Analyze the level of abstraction in the essay topic, assessing the complexity of the thinking requirements by calculating the proportion of abstract conceptual vocabulary in the topic. Abstract conceptual vocabulary refers to words that express abstract thinking, philosophical concepts, value judgments, emotional experiences, and other non-concrete things, such as "freedom," "justice," "virtue," "responsibility," "ideal," and "belief." These types of words require learners to have a high level of abstract thinking ability and deep understanding. Assess the specificity of the writing requirements, statistically analyzing the number and level of detail of explicit requirements in the topic. The specificity of writing requirements refers to the explicit restrictions and guidance the topic provides on the content, form, and structure of the writing, such as word limits, genre requirements, argumentation methods, and citation requirements. The diversity of writing requirements refers to the richness of the ability dimensions and assessment elements involved in the topic, including multiple aspects such as language use, logical thinking, innovative expression, and cultural literacy. The complexity level is determined based on the specificity and diversity of the writing requirements. A question with more than three explicit requirements and involving more than four ability dimensions is classified as high complexity; one with two to three explicit requirements and involving two to three ability dimensions is classified as medium complexity; and one with one explicit requirement and involving one ability dimension is classified as low complexity. The target word count requirement and time limit are analyzed to assess the time pressure to complete the task. A difficulty assessment model is established, with the following steps: Quantitatively assessing the abstractness index by statistically analyzing the frequency and proportion of abstract concept words in the question; scoring the requirement complexity index and determining the complexity level based on the specificity and diversity of the writing requirements; calculating the time pressure index, comprehensively considering the ratio between the word count requirement and the allotted time; assigning weight coefficients to each of the three indicators, determining the importance of each indicator based on teaching experience and practical needs; and summing the weighted results of the three indicators to obtain a comprehensive task difficulty assessment value.

[0084] Before performing the difficulty assessment calculation, each indicator needs to be standardized. The abstractness indicator is mapped to a standard range of 0 to 1 using a minimum and maximum value normalization method. The calculation method is to subtract the minimum value from the current value and then divide by the difference between the maximum and minimum values. The complexity indicator is required to use the Z-score standardization method. The calculation method is to subtract the historical data mean from the current value and then divide by the standard deviation. The time pressure indicator is in minutes. It is normalized after logarithmic transformation. The calculation method is to first take the logarithm of the time value, then subtract the minimum logarithm from the logarithm, and divide by the difference between the maximum and minimum logarithms. This ensures that indicators with different dimensions can be reasonably weighted in the calculation.

[0085] Identify the type of teaching objectives. Based on the design intent and requirements of the questions, differentiate between different teaching objectives such as basic training, ability enhancement, and comprehensive assessment. The criteria for identifying basic training questions include: the questions are relatively simple and clear, mainly testing the mastery of basic knowledge and the application of basic skills, such as vocabulary usage, grammar rules, and basic expression. The criteria for identifying ability enhancement questions include: the questions are challenging, focusing on testing learners' analytical, comprehensive, and innovative abilities, such as argumentation analysis, viewpoint articulation, and creative expression. The criteria for identifying comprehensive assessment questions include: the questions are complex and diverse, comprehensively testing learners' multiple abilities, involving multiple levels such as knowledge application, skill demonstration, and thinking development. Analyze whether the questions emphasize the application of basic knowledge, the training of writing skills, or the cultivation of innovative thinking. Determine the emphasis by analyzing the key verbs and requirements in the questions. For example, verbs such as "describe" and "narrate" indicate the application of basic knowledge; verbs such as "analyze" and "argue" indicate the training of writing skills; and verbs such as "innovate" and "hypothesize" indicate the cultivation of innovative thinking. Identify the evaluation focus, such as the degree of emphasis on language expression, content innovation, and logical thinking. The identification of evaluation priorities is achieved by analyzing the core requirements of the questions and the scoring criteria. It determines which dimension is dominant among the three dimensions of language expression, content innovation, and logical thinking, providing a basis for subsequent weighting and forming evaluation priority type labels.

[0086] Determine the applicable evaluation criteria type. Based on the text type and teaching objectives, select appropriate evaluation dimensions and weightings. For narrative texts, focus on dimensions such as plot completeness, character portrayal, and vivid language; for argumentative texts, focus on dimensions such as clarity of argument, sufficiency of reasoning, and logical rigor; for expository texts, focus on dimensions such as accuracy of explanation, logical organization, and scientific language. Generate text type labels based on the text type identification results, and generate teaching objective type labels based on the teaching objective type identification results.

[0087] The output of this step is a scenario type label and a difficulty assessment result. The scenario type label contains three sub-labels: genre type label, teaching objective type label, and evaluation focus type label. The difficulty assessment result is a comprehensive task difficulty value.

[0088] Furthermore, machine learning methods can be used to train the scenario classification model. Through a large amount of labeled essay topic data, the model can automatically learn scenario features and classification rules, improving the accuracy and efficiency of scenario recognition. The scenario classification model adopts a multi-layer neural network structure, including an input layer, two hidden layers, and an output layer. The input layer receives preprocessed word vector representations of the topic text and task difficulty feature vectors. The word vector dimension is set to 300 dimensions, and the difficulty feature vector dimension is set to 10 dimensions, resulting in a total input layer dimension of 310 dimensions. The first hidden layer contains 128 neurons, connected to the input layer via a fully connected approach, and uses the ReLU activation function for non-linear transformation. The second hidden layer contains 64 neurons, also using a fully connected approach and the ReLU activation function. The output layer contains neurons corresponding to the number of scenario types, and uses the softmax activation function to output the probability distribution of each scenario type. The model training process includes a data preprocessing stage, converting the essay topic text into standardized word vector representations; a feature extraction stage, extracting features such as keywords, abstraction level, and complexity from the topic; a batch training stage, using a mini-batch gradient descent algorithm to update network parameters, with a batch size of 32 and a learning rate of 0.001; and a cross-entropy loss function, optimizing model parameters through backpropagation. A hierarchical classification system for scenario types is established, achieving layer-by-layer classification from coarse-grained to fine-grained, providing more accurate scenario localization. The hierarchical classification system adopts a tree structure. The first-layer classifier distinguishes major text categories, including narrative, argumentative, and expository texts; the second-layer classifier further subdivides each major text category into subcategories, such as argumentative texts being divided into argumentative, comparative analysis, and problem-solving texts; and the third-layer classifier performs fine-grained classification based on specific teaching objectives and evaluation focuses. The classifiers are cascaded, with the classification results of the upper layer serving as input features for the lower layer, forming a progressively refined classification process from coarse to fine.

[0089] Step 4: Based on the scenario type label, task difficulty assessment results, and learner writing ability feature vector, establish a matching relationship between learner writing ability and task difficulty assessment results, and obtain personalized review strategy configuration parameters based on the matching results.

[0090] Based on scenario type labels, difficulty assessment results, and learner writing ability feature vectors, a rule-based decision tree algorithm is used to obtain personalized review strategy configuration parameters.

[0091] In the specific implementation process, a matching relationship between learners' abilities and task difficulty is established. The fit between learners' ability levels and task difficulty is calculated through the following steps: obtaining the learner's ability level value and the difficulty assessment result of the current task; calculating the absolute difference between the two, reflecting the degree of deviation between ability and difficulty; dividing the absolute difference by the maximum difficulty value to obtain the relative deviation ratio; subtracting the relative deviation ratio from 1 to obtain the fit value, where the closer the value is to 1, the higher the fit, and the closer it is to 0, the lower the fit; and determining whether the task difficulty is suitable for the learner's current level based on the fit result.

[0092] Before calculating the fit, it is necessary to ensure that learner ability levels and task difficulty use the same quantitative standard. Learner ability levels are derived from a comprehensive assessment of historical writing performance, using a 1 to 10 scale. This needs to be converted to a standardized range of 0 to 1, the same as the task difficulty, through linear mapping. The conversion method is to subtract 1 from the ability level and then divide by 9. The task difficulty has already been standardized in step three, so the standardized value is used directly for fit calculation to ensure that the two indicators are compared under the same scale.

[0093] Based on the fit results, determine the rigor and tolerance range of the evaluation strategy. When the fit score is higher than 0.8, it indicates a good match between the learner's ability and the task difficulty, and a standard evaluation strategy is adopted, with the rigor set to a medium level and the tolerance range set to a normal standard. When the fit score is between 0.5 and 0.8, it indicates a moderate match, and the evaluation strategy needs to be adjusted appropriately. For learners with relatively weaker abilities, the rigor should be reduced and the tolerance range expanded, while for learners with stronger abilities, the rigor should be increased and the tolerance range narrowed. When the fit score is lower than 0.5, it indicates a low match, and the evaluation strategy needs to be adjusted, including the rigor and tolerance range, to suit the learner's actual ability level.

[0094] Select the evaluation criteria configuration for a specific text type. Based on the text type tag, select the corresponding scoring dimensions and weight settings from the evaluation criteria library. For narrative text evaluation, the following weight configuration strategy is adopted: the content vividness weight is set to 0.4, reflecting the characteristics of narrative texts that emphasize plot description and character portrayal; the language expression weight is set to 0.3, emphasizing the requirements of narrative texts for vivid and engaging language; and the structural integrity weight is set to 0.3 to ensure that the narrative text has a complete narrative structure.

[0095] For the evaluation of argumentative essays, different weighting strategies are adopted: the weight for the clarity of the argument is set at 0.35, highlighting the core requirement of clearly expressing the central viewpoint in argumentative essays; the weight for the sufficiency of the argument is set at 0.35, emphasizing that argumentative essays need sufficient supporting evidence and a rigorous argumentation process; and the weight for the rigor of logic is set at 0.3, reflecting the importance that argumentative essays attach to logical reasoning and structural rationality.

[0096] Determine personalized assessment priorities. Based on learners' writing ability characteristics, identify their strengths and weaknesses. For learners with strong language expression skills but relatively weak logical thinking skills, appropriately reduce the weighting of language dimensions and increase the focus on logical thinking. Establish a compensation mechanism to allow performance in strengths to partially offset deficiencies in weaknesses.

[0097] Set adaptive scoring threshold parameters. Dynamically adjust the score thresholds for each evaluation level based on learners' historical performance and the current task difficulty. For learners with relatively weaker abilities, appropriately lower the threshold for the excellent level to encourage their writing enthusiasm; for learners with stronger abilities, appropriately raise the scoring standards to promote further improvement.

[0098] Generate personalized feedback strategy templates. Based on learner characteristics and evaluation results, select appropriate feedback methods and content focus. For learners with weak foundations, focus on providing specific improvement suggestions and examples; for learners with strong abilities, focus on providing in-depth thinking inspiration and innovative guidance.

[0099] The output of this step is the personalized review strategy configuration parameters. This parameter set contains three main components: evaluation dimension weight configuration, rating threshold setting, and feedback strategy parameters. These parameters together constitute a complete personalized review strategy.

[0100] Furthermore, a multi-objective optimization approach can be adopted to maximize personalization and learning incentives while ensuring fairness in evaluation. A strategy effectiveness evaluation mechanism can be established to verify and optimize the effectiveness of strategy selection by tracking learners' writing progress.

[0101] Step 5: Use multi-feature fusion technology to process the personalized review strategy configuration parameters and the essay text to be reviewed, and obtain the multi-dimensional scoring results and comprehensive evaluation report;

[0102] Based on the personalized review strategy configuration parameters and the essay text to be reviewed, multi-feature fusion technology is used to obtain multi-dimensional scoring results and a comprehensive evaluation report.

[0103] In the specific implementation process, a linguistic feature analysis of the essays is conducted. Vocabulary usage characteristics are extracted, and basic indicators such as the total vocabulary, the number of unique words, and the distribution of high-frequency words are statistically analyzed. Vocabulary richness is calculated by dividing the number of unique words by the total number of times each word is used, resulting in a vocabulary richness ratio. A higher ratio indicates more diverse vocabulary usage. The richness and accuracy of the vocabulary are analyzed, calculating the usage of professional terms, emotional terms, and rhetorical terms. The standardization of grammatical structures is evaluated. A grammar checking algorithm is used to identify the types and number of grammatical errors. The steps for calculating grammatical accuracy are as follows: count the total number of sentences in the text; check each sentence for grammatical errors and mark sentences with grammatical problems; count the number of grammatically correct sentences; divide the number of grammatically correct sentences by the total number of sentences to obtain the grammatical accuracy rate.

[0104] The specific implementation of the grammar checking algorithm is as follows: The essay text is segmented and labeled with parts of speech (PTS). A Hidden Markov Model (HMM)-based PTS tagger is used to decompose the text into lexical units and label their grammatical attributes. The HMM comprises three core components: a state set representing all possible PTS tags, including nouns, verbs, adjectives, adverbs, prepositions, etc.; an observation set representing all lexical units in the input text; and a parameter set including the state transition probability matrix and the observation probability matrix. The state transition probability matrix describes the probability distribution of transitioning from one PTS tag to another, reflecting the grammatical rules of the PTS sequence; the observation probability matrix describes the probability distribution of observing a specific word given a PTS tag, reflecting the correspondence between words and PTS. The model training process uses the expectation-maximization algorithm, estimating parameters using an annotated corpus and iteratively optimizing the state transition probability and observation probability parameters. The PTS tagging process uses the Viterbi algorithm, calculating the optimal PTS tag sequence based on the input lexical sequence and model parameters, achieving automatic conversion from text sequence to PTS sequence. Then, based on the grammar rule base, syntactic structure analysis is performed. Syntactic structure is identified through context-free grammar rule matching, and common grammatical error types such as subject-verb agreement, tense errors, and punctuation misuse are detected. The rule base includes subject-verb agreement checking rules (such as matching a singular subject with a singular verb), tense agreement checking rules (such as matching past tense markers with verb tenses), and punctuation usage rules (such as the correct placement of periods, commas, and question marks). Finally, the occurrence frequency and location of each type of error are counted, and an error type location mapping table is established to provide a quantitative basis for subsequent scoring.

[0105] Conduct content quality analysis. Assess the accuracy and depth of the theme expression, and determine whether the essay closely adheres to the topic through theme relevance analysis. Analyze the originality and innovation of the content by comparing its similarity with model essays in the knowledge base to detect its uniqueness. The specific implementation of similarity comparison is as follows: convert the essay text and model essays in the knowledge base into word vector representations. Use pre-trained models such as Word2Vec or BERT to convert the text into a numerical representation in a high-dimensional vector space, with each word corresponding to a fixed-dimensional vector. Then calculate the cosine similarity between the vectors, measuring the degree of similarity by calculating the dot product of two vectors divided by the product of their respective magnitudes. A similarity threshold (usually 0.7-0.8) is set to judge the originality of the content; content with a similarity below the threshold is considered to have high originality, while content with a similarity above the threshold may contain plagiarism or imitation. Assess the sufficiency and persuasiveness of the argument, analyzing the typicality of the evidence and the logic of the argumentation process in argumentative essays. The calculation steps for the content richness index are as follows: assess the depth of the topic by analyzing the level of argumentation and the depth of thinking to obtain a depth score; assess the originality by comparing it with the knowledge base to obtain an originality score; assess the sufficiency of the argument by statistically analyzing the quantity and quality of the arguments to obtain a sufficiency score; and sum the three indicators according to preset weights to obtain the comprehensive content richness index.

[0106] Perform structural organization analysis. Evaluate the overall structural integrity of the essay, checking the existence and function of basic elements such as the introduction, body, and conclusion. Analyze the logical relationships between paragraphs, and assess the naturalness of transitions through semantic coherence calculation. The specific implementation of semantic coherence calculation is as follows: extract the topic words and key phrases of adjacent paragraphs, calculate the keyword weight of each paragraph using the TF-IDF algorithm, and select the top 5-10 words with the highest weights as paragraph topic words; then calculate the semantic relevance between them, and calculate the semantic distance between topic words through a lexical semantic network (such as WordNet) or word vector similarity. The smaller the semantic distance, the higher the relevance; assess the smoothness of transitions between paragraphs by the use of conjunctions (such as the frequency and position of transition words such as "therefore," "however," and "in addition") and theme continuity (the degree of overlap of topic words in adjacent paragraphs). The steps for calculating the coherence score are as follows: Calculate the semantic relevance score, obtained by analyzing the semantic similarity of topic words in adjacent paragraphs; calculate the transition word usage score, statistically analyzing the frequency and appropriateness of transition word usage; calculate the theme continuity score, analyzing the continuity and consistency of themes between paragraphs; and then weight and sum the three sub-scores according to weights of 0.4, 0.3, and 0.3 to obtain the final coherence score. A higher coherence score indicates a more reasonable logical structure in the article. The assessment evaluates the hierarchy and organization of the structure, and checks the rationality of the content arrangement and the degree of emphasis on key points.

[0107] The scores for each dimension are calculated based on personalized strategy parameters. The language expression dimension scoring process includes the following steps: Calculating the grammatical correctness score by counting the number of grammatical errors, subtracting the number of erroneous sentences from the total number of sentences, and then dividing by the total number of sentences to obtain the accuracy rate; calculating the lexical richness score by counting the total number of different words and dividing by the total number of words in the text to obtain the lexical diversity ratio; calculating the sentence structure diversity score by identifying the number of different sentence structure types and dividing by the total number of sentences to reflect the degree of sentence structure variation; and finally, the scores of the three sub-indicators are weighted and summed according to preset weights to obtain the comprehensive score for the language expression dimension.

[0108] Before calculating the scores for the language expression dimension, each sub-indicator needs to be standardized. The grammatical correctness score is already a proportional value, within the range of 0 to 1, and requires no further processing. The vocabulary usage score is calculated by comprehensively considering multiple indicators such as vocabulary richness and the usage rate of specialized terms, and needs to be mapped to the 0 to 1 range using minimum and maximum normalization methods. The language fluency score is calculated based on indicators such as sentence length variation and the use of transition words, and also needs to be normalized to the 0 to 1 range to ensure that the three sub-indicators are weighted under the same units of measurement.

[0109] The content quality dimension scoring calculation process includes the following steps: assessing topic relevance by calculating the degree of matching between the essay content and the given topic, and statistically analyzing the frequency and distribution of topic keywords in the text; analyzing the sufficiency of argumentation by checking whether the arguments are supported by sufficient evidence, and statistically analyzing the quantity and quality of effective evidence; evaluating innovative performance by identifying unique viewpoints, novel expressions, and creative thinking in the text; and comprehensively calculating the three sub-indicators according to preset weights to obtain the final score for the content quality dimension.

[0110] Each sub-indicator of the content quality dimension needs a unified quantitative standard. The topic expression score is obtained through topic relevance analysis, using the cosine similarity calculation method. The result is in the range of 0 to 1 and requires no additional processing. The originality score is based on the similarity comparison with the knowledge base. The calculation method is to subtract the maximum similarity value from 1 to ensure that the higher the originality, the higher the score. The content depth score is evaluated through indicators such as argumentation level and cognitive complexity. It needs to be standardized using Z-scores and mapped to the range of 0 to 1. The calculation method is to divide 1 by 1 and add the natural exponent of the negative Z-value, where Z is the standardized Z-value.

[0111] The scoring process for the structural organization dimension includes the following steps: assessing the overall structural integrity by checking whether the essay has a complete introduction, body, and conclusion, and calculating whether the word count ratio of each part is reasonable; analyzing logical coherence by examining the use of transition words between paragraphs and the continuity of the theme, and calculating the tightness of logical relationships; assessing the clarity of hierarchy by checking whether the paragraph division is reasonable and whether each paragraph has a clear central idea; and weighting the three sub-indicators according to preset weights to obtain the comprehensive score for the structural organization dimension.

[0112] The sub-indicators of the structural organization dimension require a unified scoring standard. The structural integrity score is calculated by checking the integrity of elements such as the introduction, body, and conclusion, and then normalized after using binary scoring. The logical coherence score has already been calculated through semantic coherence, and the result is in the range of 0 to 1, so it can be used directly. The hierarchical clarity score is calculated through paragraph hierarchy analysis and content organization rationality assessment, and needs to be normalized to the range of 0 to 1 after using stratified scoring to ensure that the three sub-indicators are weighted and calculated under a unified standard.

[0113] The specific steps for calculating the overall score of an essay are as follows: ensure that the sum of the weight coefficients for each dimension equals 1; if not, normalization is performed. Multiply the scores for language expression, content quality, and structure organization by their respective weight coefficients. Add the three weighted scores together to obtain the overall score of the essay. Convert the overall score into a corresponding rating system, such as a percentage or a five-point system, as needed to obtain the AI ​​score. The AI ​​score reflects the overall performance level of the essay across multiple dimensions.

[0114] Before calculating the overall score, it is necessary to ensure the rationality and consistency of the dimension weights. The weights for each dimension, including language expression weight, content quality weight, and structural organization weight, must meet the normalization condition, i.e., the sum of the three weights equals 1, ensuring the overall score is within a reasonable range. The scores for each dimension, including language expression score, content quality score, and structural organization score, have already been standardized in the preceding steps and are all values ​​in the range of 0 to 1, allowing for direct weighted calculation. The final overall score will automatically fall within the 0 to 1 range and can be mapped to other scoring systems as needed through linear transformation; for example, the percentage-based calculation method is to multiply the overall score by 100.

[0115] Generate a detailed evaluation report, including specific scores for each dimension, strengths analysis, problem diagnosis, and targeted improvement suggestions. The strengths analysis highlights the excellent aspects of the essay, boosting learners' writing confidence; the problem diagnosis specifically identifies shortcomings and errors, providing clear directions for improvement; and the improvement suggestions, tailored to the learner's abilities, offer actionable methods for enhancement.

[0116] The output of this step is the dimensional scoring results and a comprehensive evaluation report. The dimensional scoring results include scoring data for three dimensions: language expression score, content quality score, and structural organization score.

[0117] Furthermore, deep learning models can be used for semantic understanding of essays, improving the accuracy and depth of content analysis. A scoring consistency verification mechanism should be established to ensure the coordination and rationality of scoring across different dimensions.

[0118] Step 6: Collect feedback data from teachers on the dimensional scoring results and the comprehensive evaluation report; verify the accuracy of the dimensional scoring results and the comprehensive evaluation report based on the feedback data; adjust the personalized review strategy configuration parameters based on the verification results to obtain an optimized review strategy and use it as input for the next round of review process.

[0119] Based on the dimensional scoring results, comprehensive evaluation report and feedback data, quality control technology was used to obtain optimized review strategy parameters.

[0120] In the specific implementation process, the accuracy of the evaluation results is verified. Feedback from teachers on the AI ​​evaluation results is collected, including their level of acceptance of the accuracy of the scoring, their agreement with the evaluation dimensions, and their evaluation of the practicality of the improvement suggestions. A comparative analysis is performed between the teacher scores in the teacher feedback data and the AI ​​scores in the comprehensive evaluation report. A consistency index between the AI ​​scores and teacher scores is calculated, and the Pearson correlation coefficient is used to measure the correlation of the scores.

[0121] To verify the effectiveness of the system's grading, Pearson correlation coefficient analysis was used to analyze the consistency between AI scores and teacher scores. The specific steps of the correlation analysis were as follows: AI scores and teacher scores for the same batch of essays were collected, ensuring complete data pairing; the average AI scores and the average teacher scores were calculated; the deviation of each essay's AI score from its average, and the deviation of each teacher score from its average, were calculated; the sum of the products of the deviations in AI scores and teacher scores for all essays was calculated; the sum of the squares of the deviations in AI scores and teacher scores were calculated; the sum of these products was divided by the square root of the product of the two sums of squares to obtain the Pearson correlation coefficient; the magnitude of the correlation coefficient was used to determine the degree of consistency between AI scores and teacher scores, with a correlation coefficient closer to 1 indicating higher consistency.

[0122] Before calculating the Pearson correlation coefficient, the scoring data needs to be preprocessed to ensure accuracy. AI scoring and teacher scoring need to use unified scoring standards. If they use different scoring systems (e.g., AI systems use 0-1 standardized scoring, while teachers use a percentage system), a standardization conversion is required to map them to the same interval. Outliers (e.g., scores that significantly deviate from the normal range) need to be identified and handled. The 3-standard-deviation criterion or box plot method can be used to detect outliers, and confirmed outliers should be removed or corrected. The integrity of the sample data must be ensured. For missing scoring data, methods such as mean imputation, regression prediction, or direct removal should be used depending on the specific circumstances.

[0123] This study analyzes the causes and patterns of scoring bias. It statistically analyzes instances where AI scores are too high or too low, identifying essay types and characteristics prone to misjudgment. The correlation between scoring errors and factors such as learner characteristics, task difficulty, and essay type is analyzed. An error analysis model is established to identify weaknesses and areas for improvement in the system evaluation. The error analysis model employs a multiple linear regression structure, comprising three core components: a set of independent variables, a dependent variable, and regression coefficients. The set of independent variables includes learner characteristic variables, such as historical average scores, writing ability levels, and years of learning; task characteristic variables, such as essay type coding, difficulty level values, word count requirements, and time limits; and text characteristic variables, such as vocabulary complexity indicators, syntactic complexity indicators, and content richness indicators. The dependent variable is the difference between the AI ​​score and the teacher's score, i.e., the error value. The regression coefficients represent the degree and direction of the influence of each independent variable on the error value. The model training process includes: a data collection phase, extracting AI scores, teacher scores, and related feature data from historical review records; a feature selection phase, screening feature variables that significantly affect the error through correlation analysis and significance testing; a parameter estimation phase, using the least squares method to calculate the regression coefficients of each feature variable to minimize the sum of squared prediction errors; and a model validation phase, evaluating the model's prediction accuracy and generalization ability through cross-validation. The specific implementation of the error analysis model involves: collecting the difference data between AI scores and teacher scores to establish an error dataset, where each error value equals the AI ​​score minus the teacher score; then using statistical analysis methods to identify the distribution patterns and influencing factors of the errors; and using multiple linear regression analysis to analyze the relationship between errors and learner characteristics (such as historical grades and writing ability levels), task characteristics (such as text type, difficulty level, and word count requirements), and text characteristics (such as vocabulary complexity and syntactic complexity). The steps for establishing the error prediction model are as follows: First, determine the independent variables, including influencing factors such as learner characteristics, task characteristics, and text characteristics. Second, establish a linear regression equation, using error as the dependent variable and each influencing factor as an independent variable. Third, estimate the regression coefficients using the least squares method. Fourth, conduct a significance test to identify the main influencing factors. Finally, establish the final error prediction equation to provide a quantitative basis for subsequent parameter adjustments. The model's error prediction results and the evaluation strategy optimization form a closed-loop feedback mechanism. When the prediction error exceeds a set threshold, it automatically triggers adjustments to the corresponding dimension weights and scoring thresholds, achieving continuous optimization of the evaluation strategy.

[0124] Adjust strategy parameters based on feedback. For systematic scoring biases, adjust the scoring weights and threshold settings for the corresponding dimensions. For inaccurate evaluations in specific scenarios, optimize the corresponding strategy selection rules. Update the evaluation standard library, adding new evaluation cases and standard configurations.

[0125] Record and analyze historical data for optimization. Establish a record archive for strategy optimization, tracking the effects and impacts of each adjustment. Analyze optimization trends to identify directions and potential for system improvement. Establish a knowledge accumulation mechanism to transform effective optimization experience into systematic improvement strategies.

[0126] Verify the optimization effect. Use the new evaluation strategy to evaluate the test samples and compare the accuracy and consistency of the evaluation before and after optimization. Collect user feedback on the optimization effect and assess the actual value of the system improvement. Establish a continuous improvement mechanism to form a virtuous cycle of evaluation, feedback, and optimization.

[0127] The output of this step is the optimized review strategy parameters, which will serve as input parameters for the next round of review.

[0128] Furthermore, machine learning methods can be used to build automated optimization models, reducing the workload of manual intervention and improving the efficiency and accuracy of optimization. A multi-layered optimization mechanism can be established, including improvement strategies at different levels such as real-time fine-tuning, periodic optimization, and version updates.

[0129] An AI-based essay evaluation system for improving writing skills based on scenario-based reasoning, such as Figure 2 As shown, the method for improving essay AI evaluation based on contextual inference, as described above, includes:

[0130] The data acquisition module is used to acquire learner basic information and essay text data and perform standardization processing to obtain a standardized learner basic information set and a standardized essay text dataset. The standardized essay text dataset includes current essay task information and essay text to be reviewed.

[0131] The feature extraction module is used to perform text statistical analysis on a standardized set of learners’ basic information to obtain feature vectors of learners’ writing abilities.

[0132] The context recognition module is used to combine learners’ writing ability feature vectors to identify the context type and assess the task difficulty of the current writing task information, and obtain the context type label and task difficulty assessment results.

[0133] The strategy matching module is used to establish a matching relationship between learners' writing ability and task difficulty assessment results based on scenario type labels, task difficulty assessment results and learners' writing ability feature vectors, and obtain personalized review strategy configuration parameters based on the matching results.

[0134] The scoring and analysis module is used to process personalized review strategy configuration parameters and essay texts to be reviewed using multi-feature fusion technology to obtain multi-dimensional scoring results and comprehensive evaluation reports.

[0135] The optimization control module is used to collect feedback data from teachers on the dimensional scoring results and comprehensive evaluation reports. It combines the feedback data to verify the accuracy of the dimensional scoring results and comprehensive evaluation reports, adjusts the configuration parameters of personalized review strategies based on the verification results, obtains the optimized review strategy, and uses it as input for the next round of review process.

[0136] In one embodiment of the present invention, a specific example is provided:

[0137] A 30-day field test was conducted at a middle school in District B of City A. During the test, an AI-based essay evaluation and improvement system based on scenario simulation was deployed, covering a total of 240 students across three grades of junior high school. The test area encompassed the daily essay writing teaching activities of six classes. Multi-dimensional information, including student essay data, teacher evaluation data, and system evaluation data, was collected during the test, providing ample data support for system performance verification.

[0138] Table 1 shows an example of the original data collection for student essays:

[0139] Table 1: Example of raw data collection for student essays;

[0140]

[0141] Table 1 shows the original student essay data samples collected by the system in the field test, including basic information such as data number, student identifier, essay title, text length, submission time, grade information and genre, which provides a data foundation for subsequent scenario analysis and evaluation strategy formulation.

[0142] Table 2 shows an example of data collection for teacher evaluation criteria.

[0143] Table 2: Example of data collection for teacher evaluation criteria;

[0144]

[0145] Table 2 records the evaluation data of professional teachers on student essays, including scores for each dimension (language expression, content quality, and structure), overall scores, and evaluation time. This data serves as a standard reference for system training and validation, ensuring consistency between AI evaluation results and professional teacher evaluation standards.

[0146] like Figure 3As shown, the comparison results of the scenario-based assessment system of this invention and the traditional AI assessment system in terms of accuracy are illustrated. The horizontal axis represents different essay types, and the vertical axis represents the percentage of consistency with professional teacher scores. The results show that the scenario-based assessment system exhibits higher accuracy in all types of essay assessment, with an average consistency of 88.3%, an improvement of 12.8 percentage points compared to the traditional system. The improvement is particularly significant in the assessment of argumentative and expository essays, demonstrating the advantages of the scenario-based assessment strategy.

[0147] like Figure 4 The chart shows the satisfaction levels of students at different ability levels with the effectiveness of personalized feedback. The chart includes five dimensions: feedback relevance, suggestion practicality, motivational effect, learning guidance value, and overall satisfaction. The results indicate that students at all ability levels achieved a satisfaction score exceeding 4.0 (out of 5) for personalized feedback, with students of medium ability achieving the highest satisfaction at 4.6, demonstrating the significant advantages of the scenario-based feedback mechanism in personalized assessment.

[0148] like Figure 5 As shown, the key performance indicators (KPIs) of the system during its six-month operation are displayed. Monitoring indicators include average response time, review accuracy, user satisfaction, and system availability. The results show that the system maintained stable performance during operation, with an average response time controlled within 2.8 seconds, a review accuracy rate consistently above 88%, user satisfaction showing an upward trend, and system availability remaining above 99.2%, validating the stability and practicality of the technical solution.

[0149] The embodiments of the present invention have been described above. However, the embodiments are not limited to the specific implementation methods described above. The specific implementation methods described above are merely illustrative and not restrictive. Those skilled in the art can make more equivalent embodiments under the guidance of the present embodiments, and all of them are within the protection scope of the present embodiments.

Claims

1. A method for improving AI review of composition based on scenario deduction, characterized in that, include: Acquire learner basic information and essay text data and perform standardization processing to obtain a standardized learner basic information set and a standardized essay text dataset. The standardized essay text dataset includes current essay task information and essay texts to be reviewed. Textual statistical analysis was performed on a standardized set of learners’ basic information to obtain feature vectors of learners’ writing abilities. By combining learners' writing ability feature vectors, the current writing task information is analyzed to identify the scenario type and assess the task difficulty, resulting in scenario type labels and task difficulty assessment results. The scenario type identification and task difficulty assessment include: acquiring the current writing task information; performing keyword extraction and semantic analysis on the current writing task information; identifying the genre type through keyword matching; calculating the proportion of abstract concept words in the current writing task information; assessing the complexity of writing requirements; analyzing the ratio of word count requirements to time limits; and obtaining the task difficulty assessment result through weighted summation according to preset weights. Based on the genre type and task difficulty assessment results, the teaching objective type is determined by combining the analysis of teaching objectives, and scenario type labels are output. Based on scenario type labels, task difficulty assessment results, and learner writing ability feature vectors, a matching relationship between learner writing ability and task difficulty assessment results is established. Personalized review strategy configuration parameters are then obtained based on the matching results. The generation process of these personalized review strategy configuration parameters includes: obtaining learner writing ability feature vectors and task difficulty assessment results; calculating the absolute difference between the writing ability feature vectors and task difficulty assessment results and dividing it by the maximum difficulty value to obtain the relative deviation ratio; obtaining the fit value through reverse calculation of the relative deviation ratio; determining the matching result between learner writing ability and task difficulty assessment results based on the fit value; selecting the corresponding evaluation dimension weight configuration based on the text type in the scenario type labels; and generating personalized review strategy configuration parameters based on the weight configuration and fit value. These personalized review strategy configuration parameters include evaluation dimension weight configuration, scoring threshold settings, and feedback strategy parameters. Multi-feature fusion technology is used to process personalized review strategy configuration parameters and essay texts to be reviewed, resulting in multi-dimensional scoring results and a comprehensive evaluation report. Feedback data from teachers on the dimensional scoring results and comprehensive evaluation reports was collected. A comparative analysis was conducted between the teacher scores in the feedback data and the AI ​​scores in the comprehensive evaluation reports. The Pearson correlation coefficient was used to calculate the correlation between teacher scores and AI scores, yielding a scoring consistency index. Based on the comparison results, an error dataset was established by calculating the difference between the AI ​​scores and the teacher scores. Multiple linear regression analysis was used to analyze the relationship between the error dataset and learner characteristics, task characteristics, and text characteristics, establishing an error prediction model. Based on the analysis results of the error prediction model, the weights and threshold settings in the personalized review strategy configuration parameters were adjusted to obtain optimized review strategy parameters, which were then used as input for the next round of review.

2. The method for improving essay AI evaluation based on scenario inference as described in claim 1, characterized in that, The specific implementation process of the standardization process includes: Raw data is obtained through the teaching management system API interface, and the format and integrity of learners' basic information and essay text data are checked and verified. Convert raw text data to UTF-8 encoding format and remove special characters and formatting tags; Perform range checks and outlier identification on numerical raw data, and use interpolation methods to correct outlier data; A unified data storage structure containing student information tables, historical essay tables, and current essay tables was established to obtain a standardized teaching dataset.

3. The method for improving essay AI evaluation based on scenario inference as described in claim 1, characterized in that, The specific implementation process of the text statistical analysis includes: Obtain historical essay data from the standardized essay text dataset, perform lexical statistical analysis on the historical essay data, calculate the total number of words in each essay and calculate the average number of words to obtain the average vocabulary size. Count the number of unique word categories and the total number of times words are used, and calculate the vocabulary richness. Syntactic analysis was performed on historical essay data to count the number of words in each sentence and calculate the average sentence length. We quantify statistical indicators, including average vocabulary size, vocabulary richness, and average sentence length, to construct a feature vector of learners' writing ability.

4. The context-inference-based essay AI review improvement method according to claim 1, wherein, The specific implementation process of the multi-feature fusion technology includes: The algorithm obtains the text of the essay to be reviewed and the configuration parameters of the personalized review strategy. It performs language feature analysis on the text of the essay to be reviewed, calculates the vocabulary richness by counting the total number of words and the number of unique words, identifies the number of grammatical errors by the grammar check algorithm, and calculates the grammar accuracy rate. Content quality analysis is conducted, the degree of matching between the essay and the topic is determined through topic relevance analysis, and the originality score is obtained by comparing the essay with sample essays in the knowledge base using word vector similarity calculation method. Perform structural organization analysis, check the existence of basic elements, calculate the semantic relevance of keywords in adjacent paragraphs, and assess logical coherence. Based on the weight settings in the personalized review strategy configuration parameters, a comprehensive evaluation report is obtained by integrating the scores of three dimensions: language expression, content quality, and structural organization.

5. The context-inference-based essay AI review improvement method according to claim 4, characterized in that, The specific implementation of identifying the number of syntax errors and calculating the syntax accuracy rate through a syntax checking algorithm includes: Acquire historical essay data, perform word segmentation and part-of-speech tagging on the historical essay data, and use a part-of-speech tagger based on a hidden Markov model to decompose the text into lexical units and tag grammatical attributes. Syntactic structure analysis is performed on the annotated text based on a grammar rule base, and syntax error types are identified by matching context-free grammar rules. The number and location of grammatical errors are counted, and the grammatical accuracy rate is calculated by dividing the number of grammatically correct sentences by the total number of sentences, providing a quantitative basis for the subsequent construction of feature vectors.

6. A composition AI review enhancement system based on scenario deduction, characterized in that, The method for improving essay AI evaluation based on scenario inference as described in any one of claims 1-5 includes: The data acquisition module is used to acquire learner basic information and essay text data and perform standardization processing to obtain a standardized learner basic information set and a standardized essay text dataset. The standardized essay text dataset includes current essay task information and essay text to be reviewed. The feature extraction module is used to perform text statistical analysis on a standardized set of learners’ basic information to obtain feature vectors of learners’ writing abilities. The context recognition module is used to combine learners’ writing ability feature vectors to identify the context type and assess the task difficulty of the current writing task information, and obtain the context type label and task difficulty assessment results. The strategy matching module is used to establish a matching relationship between learners' writing ability and task difficulty assessment results based on scenario type labels, task difficulty assessment results and learners' writing ability feature vectors, and obtain personalized review strategy configuration parameters based on the matching results. The scoring and analysis module is used to process personalized review strategy configuration parameters and essay texts to be reviewed using multi-feature fusion technology to obtain multi-dimensional scoring results and comprehensive evaluation reports. The optimization control module is used to collect feedback data from teachers on the dimensional scoring results and comprehensive evaluation reports. It combines the feedback data to verify the accuracy of the dimensional scoring results and comprehensive evaluation reports, adjusts the configuration parameters of personalized review strategies based on the verification results, obtains the optimized review strategy, and uses it as input for the next round of review process.

Images