English writing stream automatic evaluation system

Abstract

The invention discloses an English writing stream automatic evaluation system, relates to the technical field of teaching assistance, and constructs a paragraph association graph with paragraphs as nodes and semantic similarity as edge weights. The consistency score is calculated by searching for the longest coherent path with the maximum edge weight accumulated value in the graph, and a deviation threshold value is set according to the consistency score and preset parameters. The system identifies a deviation paragraph in the graph that has a longest path average semantic association lower than a threshold. And for the deviation paragraph, screening words with the most similar semantics from a preset theme keyword list, generating a specific correction suggestion according to the words, and forming an evaluation report. The key innovation lies in that the feedback regulation module tracks the adoption rate of a user for suggestions, and if the adoption rate is lower than a threshold value, parameters are lowered in proportion, so that the deviation judgment standard is dynamically relaxed in subsequent evaluation, and self-optimization of the system is realized.

Description

Technical Field

[0001] This invention relates to the field of teaching aids technology, specifically to an automatic assessment system for English writing flow. Background Technology

[0002] Developing English writing skills is a crucial aspect of language learning. With the widespread adoption of online education and intelligent learning platforms, automated essay scoring technology has become widely used. Current automated English writing assessment systems primarily focus on grammatical error detection (such as subject-verb agreement and tense errors) and surface feature statistics (such as lexical diversity and sentence length). However, they have the following shortcomings in assessing content coherence:

[0003] Existing methods typically rely on keyword matching or simple vector space models to calculate the similarity between the full text and the topic, and then evaluate the similarity by setting an assessment threshold. However, the existing similarity thresholds for judging deviations are usually pre-set statically. Different writing tasks, topic difficulties, and even differences in learner levels may require different evaluation criteria. Static thresholds lack flexibility and may lead to assessments that are too strict or too lenient, resulting in inaccurate assessment results. Summary of the Invention

[0004] To address the shortcomings of existing technologies, this invention provides an automatic evaluation system for English writing flow.

[0005] To achieve the above objectives, the technical solution of the present invention is as follows:

[0006] An automated assessment system for English writing flow includes:

[0007] The data acquisition module is used to obtain a collection of paragraphs and a list of preset topic keywords for the essay to be evaluated;

[0008] The paragraph association construction module is used to extract the core word set of each paragraph, calculate the semantic similarity between any two paragraphs through the core word set, and construct a paragraph association graph with paragraphs as nodes and semantic similarity as edge weights;

[0009] The semantic consistency calculation module is used to traverse all paths of the paragraph association graph, select the continuous node path with the largest cumulative edge weight value as the longest coherent path, calculate the semantic consistency score of the longest coherent path, and calculate the preset deviation threshold based on the semantic consistency score according to the preset threshold parameter.

[0010] The topic deviation calculation module is used to identify deviating paragraphs in the paragraph association graph whose average semantic relevance to the longest coherent path is lower than a preset deviation threshold.

[0011] The deviation correction module is used to filter out the words with the highest paragraph semantic vector similarity to the deviation paragraph from the preset topic keyword list, and generate correction suggestions based on the words with the highest similarity.

[0012] The evaluation results generation module is used to generate an evaluation report that includes the deviated paragraphs and correction suggestions;

[0013] The feedback adjustment module is used to perform the following operations:

[0014] Obtain the user's adoption flag for the proposed correction and calculate the adoption rate within a preset time period;

[0015] Determine whether the adoption rate is lower than a preset adoption rate threshold. If the determination result is yes, then adjust the preset threshold parameter proportionally and use the corrected preset threshold parameter for the next evaluation.

[0016] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0017] 1. The threshold parameter is adjusted using the adoption rate. Based on actual user feedback, the threshold can be automatically relaxed or expanded to tighten the judgment criteria in the future. This can adapt to different user levels and different tasks / topics. As usage data accumulates, the system parameters tend to be more optimal, the evaluation becomes more accurate, and the suggestions become more effective.

[0018] 2. A graphical model was used to intuitively and quantitatively model the semantic relationship network of the entire text, providing a solid structured foundation for evaluation. By finding the path with the largest weight in the graph, the most coherent core structure of the article was objectively identified. This is more resistant to local noise interference and better reflects the true writing flow than calculating the average similarity of the entire text. It makes the evaluation of abstract concepts such as "writing flow" and "theme consistency" more objective, quantitative, and interpretable, more accurate in locating problems, and more convincing in the evaluation results. Attached Figure Description

[0019] The disclosure of this invention is illustrated with reference to the accompanying drawings. It should be understood that the drawings are for illustrative purposes only and are not intended to limit the scope of protection of this invention. In the drawings, the same reference numerals are used to refer to the same parts. Wherein:

[0020] Figure 1 This is a system module diagram of the present invention. Detailed Implementation

[0021] It is readily understood that, based on the technical solution of this invention, those skilled in the art can propose various interchangeable structural methods and implementations without altering the essential spirit of the invention. Therefore, the following detailed embodiments and accompanying drawings are merely illustrative examples of the technical solution of this invention and should not be considered as the entirety of the invention or as limitations or restrictions on the technical solution of this invention.

[0022] Application Overview

[0023] In traditional, existing automated English writing assessment systems, deviation judgment mechanisms based on statically preset similarity thresholds have adaptive limitations. Due to the varying topic complexity of different writing tasks, the stratification of users' language proficiency, and the dynamic changes in text semantic structure, fixed thresholds cannot accurately reflect the reasonable fluctuation range of semantic connections between paragraphs. When the system uses a uniform threshold to process diverse essay samples, the mismatch between the threshold parameter and the current text semantic features will lead to coherence assessment errors, specifically misjudging reasonable semantic expansions as topic deviations or missing actual logical breaks.

[0024] For example, in the essay grading scenario of an online education platform, the system uses the TF-IDF algorithm to generate paragraph vectors and constructs a paragraph association graph through cosine similarity calculation. When processing advanced argumentative essays on the topic of "artificial intelligence ethics," users may introduce related sub-topics such as "algorithmic bias" and "data privacy" during the argumentation process. If the system uses a preset similarity threshold of 0.65, it may incorrectly filter paragraph edges with reasonable logical progression (such as the semantic association degree of 0.63 from "machine learning model" to "training data bias"), resulting in the missing key connection paths in the association graph structure. At the same time, for paragraphs that actually have a risk of going off-topic (such as suddenly turning to "quantum computing hardware" with an association degree of only 0.41), the system cannot trigger deviation detection due to the threshold setting being too high, ultimately generating an incomplete evaluation report.

[0025] If the aforementioned problems are not addressed, the static threshold mechanism will cause the misjudgment rate of the assessment system to increase exponentially with the increase of text complexity. In cross-topic essay correction tasks, the system cannot establish dynamic semantic association benchmarks, resulting in a continuous decline in the consistency coefficient between the assessment results and human review. This technical deficiency will directly weaken users' trust in automatic correction suggestions, and at the same time, cause the system to lose its ability to optimize assessment strategies based on user feedback, ultimately leading to a degradation of the reliability and validity of the teaching effectiveness assessment system.

[0026] To address the aforementioned challenges, this application first considers establishing a dynamic semantic association benchmark to adapt to different writing scenarios. Traditional static thresholds cannot handle the differences in topic complexity and user skill levels, leading to incomplete association graph construction or detection failures. This application attempts to establish an evaluation benchmark through the automatic identification of semantically coherent paths, using the semantic consistency score of the path as the basis for dynamic threshold calculation. To resolve the mismatch between thresholds and text features, a user feedback mechanism is further introduced, adjusting the threshold parameters in reverse based on the actual adoption of correction suggestions, thus forming a collaborative optimization mechanism between evaluation standards and user perception.

[0027] like Figure 1 As shown, this application proposes: an automatic evaluation system for English writing flow, comprising:

[0028] The data acquisition module is used to obtain a set of paragraphs and a list of preset topic keywords for the essay to be evaluated. Specifically, it can be implemented using a text parsing interface or a file upload interface. Its role is to provide structured input data for subsequent semantic analysis, solving the problem of the existing system relying on single keyword matching, which leads to a single evaluation dimension.

[0029] The paragraph association construction module is used to extract the core word set of each paragraph and calculate the semantic similarity between any two paragraphs using the core word set. It constructs a paragraph association graph with paragraphs as nodes and semantic similarity as edge weights. Specifically, the core words can be selected using the word frequency-inverse paragraph frequency algorithm, and the association weight between paragraph vectors can be calculated using cosine similarity. Its role is to establish a semantic association network between paragraphs and solve the problem that traditional vector space models ignore local semantic coherence.

[0030] The semantic consistency calculation module is used to traverse all paths in the paragraph association graph, select the continuous node path with the largest cumulative edge weight as the longest coherent path, calculate the semantic consistency score of the longest coherent path, and calculate a preset deviation threshold based on the semantic consistency score according to a preset threshold parameter. Specifically, a dynamic programming algorithm can be used to search for the path with the largest cumulative edge weight, and the deviation threshold can be calculated based on the path weight. Its function is to dynamically generate deviation judgment criteria that match the current essay content, solving the problem that static thresholds cannot adapt to different writing tasks.

[0031] The topic deviation calculation module is used to identify deviation paragraphs in the paragraph association graph whose average semantic relevance to the longest coherent path is lower than a preset deviation threshold. Specifically, it can be achieved by calculating the arithmetic mean of the cosine similarity between the deviation paragraph node and the path node. Its function is to accurately locate paragraphs with semantic breaks or topic deviations, thereby improving the interpretability of the evaluation results.

[0032] The deviation correction module is used to filter out the words with the highest paragraph semantic vector similarity to the deviated paragraph from the preset topic keyword list, and generate correction suggestions based on the words with the highest similarity. Specifically, a word vector similarity ranking algorithm can be used to match keywords. Its role is to provide targeted correction guidance and solve the shortcomings of traditional systems that only mark errors and lack improvement suggestions.

[0033] The evaluation result generation module is used to generate an evaluation report that includes deviation paragraphs and correction suggestions. Specifically, it can use natural language generation technology to convert structured data into a text report, and its purpose is to output feedback results that can be directly understood by users.

[0034] The feedback adjustment module is used to perform the following operations:

[0035] Obtain the user's adoption flag for the proposed correction and calculate the adoption rate within a preset time period;

[0036] Determine whether the adoption rate is lower than a preset adoption rate threshold. If the determination result is yes, then adjust the preset threshold parameter proportionally and use the corrected preset threshold parameter for the next evaluation.

[0037] Specifically, the ratio of valid adoptions to total deviations within a preset time period can be statistically analyzed, and parameters can be adjusted based on a linear formula. This approach aims to achieve adaptive optimization of the evaluation criteria and address the lack of flexibility in static thresholds.

[0038] The core innovation of this application lies in quantifying semantic coherence by constructing a paragraph association graph and achieving adaptive optimization of the evaluation criteria based on a dynamic deviation threshold and feedback adjustment mechanism. The semantic consistency calculation module generates a deviation threshold that matches the current essay content, and the feedback adjustment module automatically adjusts the threshold parameters based on user adoption behavior. This allows the system to adapt to different writing task topics and varying user skill levels, effectively solving the evaluation bias problem caused by static thresholds.

[0039] As a preferred embodiment, the solution of this application is specifically implemented as follows:

[0040] The data acquisition module obtains the English essay text to be evaluated from the online writing platform through the API interface, divides the text into paragraph sets, and extracts a list of keywords related to the essay topic from the preset topic library.

[0041] The paragraph association construction module uses the TF-IDF algorithm to extract the top 10 core words from each paragraph as a core word set. It uses the Word2Vec model to convert the core words into word vectors, calculates the cosine similarity between word vectors to obtain the semantic similarity between paragraphs, and constructs an undirected weighted graph as the paragraph association graph.

[0042] The semantic consistency calculation module uses Dijkstra's algorithm to search for the path with the largest sum of edge weights in the paragraph association graph as the longest coherent path, calculates the average edge weight of the path as the semantic consistency score, and multiplies the score by a preset coefficient of 0.8 as a preset deviation threshold.

[0043] The topic deviation calculation module traverses the nodes in the paragraph association graph, calculates the average semantic similarity between each node and the node of the longest continuous path, and marks the nodes below the preset deviation threshold as deviated paragraphs.

[0044] The deviation correction module uses the Word2Vec model to calculate the semantic similarity between the deviating paragraph and the topic keywords, and selects the keyword with the highest similarity as the correction suggestion.

[0045] The evaluation results generation module integrates the deviating paragraphs and corresponding correction suggestions into an evaluation report in JSON format. The feedback adjustment module tracks user adoption of correction suggestions over 7 days, calculates the adoption rate, and lowers the preset threshold parameter by 5% for the next evaluation calculation if the adoption rate is below 60%.

[0046] Through the above-described scheme, this application achieves automatic assessment and dynamic optimization of English writing flow. The system can adaptively adjust the assessment criteria, improving its adaptability to different writing tasks and user levels. By constructing paragraph connection graphs and identifying the longest coherent path, the system can accurately capture the semantic structure and logical coherence of the text. The dynamic deviation threshold calculation and feedback adjustment mechanism makes the assessment results more consistent with actual writing characteristics and user needs. The system can not only identify deviating paragraphs but also provide targeted correction suggestions, effectively improving the practicality of the assessment. This automated and intelligent assessment method greatly improves the efficiency and accuracy of English writing assessment, providing strong technical support for online education platforms.

[0047] This application further proposes the following process for constructing the paragraph association diagram:

[0048] Get the semantic similarity between any two paragraphs :

[0049] ;in, For the first The paragraph semantic vector of a segment. For the first The paragraph semantic vector;

[0050] judge If the similarity exceeds a preset threshold, then the edge between the two paragraphs is preserved.

[0051] Among them, paragraph semantic vector Paragraph text is encoded and generated using a pre-trained language model, such as BERT or Word2Vec. The specific value of the preset similarity threshold is dynamically adjusted according to the writing task type. This threshold can be adjusted based on different topic complexities; for example, it is set to 0.65 for argumentative essays and 0.55 for expository essays. The association graph built in this way only contains paragraph nodes with significant semantic connections, reducing noise interference in the graph structure and improving the computational efficiency of subsequent path search. At the same time, the dynamic threshold setting avoids the problem of insufficient adaptability of fixed thresholds to different writing scenarios. For example, the threshold can be appropriately lowered for novice learners to retain more potential association paths. The edge retention operation is implemented by traversing the pairwise combinations between paragraphs, and the weight of the edges that are not retained is set to zero or directly deleted.

[0052] For example, suppose we have an English essay containing 5 paragraphs. First, calculate the semantic similarity between any two paragraphs: , , , , , , , , , .

[0053] If the preset similarity threshold is set to 0.6, then paragraphs will be retained in the paragraph association diagram. , , , , The corresponding edges are selected, while other edges are not retained. This constructs a graph structure that reflects the semantic relationships between paragraphs.

[0054] Through the above technical solution, this application can effectively construct a graph structure reflecting the semantic relationships between paragraphs, providing a foundation for subsequent semantic consistency analysis and topic deviation detection. This method filters out paragraph connections with weak semantic relationships by setting a preset similarity threshold, retaining those with strong semantic relevance, thereby constructing a more accurate and meaningful paragraph relationship graph. Compared to simple fully connected graphs or fixed-window connection methods, this graph structure construction method based on semantic similarity can better capture the semantic relationships between paragraphs, helping to improve the accuracy and effectiveness of subsequent analysis.

[0055] This application further proposes the following calculation process for semantic consistency score:

[0056] The longest continuous path is the path whose sum of edge weights is maximized in the paragraph association graph. The sum of the weights of all edges in this longest continuous path is then obtained. ;

[0057] Based on the weights of all edges Calculate the semantic consistency score:

[0058] ;in, This represents the number of edges in the path.

[0059] The formula for calculating the preset deviation threshold is:

[0060] ,in This is a preset threshold parameter.

[0061] Among them, semantic consistency score By combining path edge weights and Divide by the number of path edges Standardization is performed to eliminate the influence of path length on the score, which is used to characterize the overall coherence level of the path; a preset deviation threshold is set. Based on semantic consistency score With preset threshold parameters The product is dynamically generated and can be dynamically optimized based on the user's adoption rate of the correction suggestions. The threshold parameter can be dynamically adjusted based on user feedback.

[0062] For example, when When the initial value is 0.6, If the subsequent user adoption rate falls below the preset threshold, The deviation judgment criteria will be relaxed by adjusting the proportionate reduction. This will be achieved through dynamic adjustments. The system can adapt to the evaluation needs of different writing scenarios and avoid evaluation bias caused by static thresholds.

[0063] For example, in an English essay containing five paragraphs, the constructed paragraph association graph contains multiple paths. By calculating the sum of the edge weights of each path, the path with the largest sum of weights is selected as the longest continuous path. Assuming the longest continuous path contains four edges with weights of 0.84, 0.66, 0.88, and 0.62, then... .

[0064] In the above example, ,therefore .

[0065] Assumption ,but .

[0066] Through the above technical solution, this application can calculate the semantic consistency score of an essay based on the structural features of a paragraph association graph, and dynamically determine a preset deviation threshold according to a preset threshold parameter. This method considers the overall semantic coherence of the essay and avoids the inaccuracy problem that may be caused by static thresholds. Therefore, the system can more flexibly adapt to writing tasks of different difficulties and themes, improving the accuracy of assessing the coherence of essay content.

[0067] This application further proposes the following process for adjusting the preset threshold parameter:

[0068] Get existing preset threshold parameters and adoption rate ;

[0069] like Established, The preset adoption rate threshold;

[0070] Then the preset threshold parameter Adjust according to the following formula:

[0071] .

[0072] The parameter adjustment process comprises three core elements. The first element is the adoption rate monitoring mechanism, which ensures the objectivity of data collection by marking user adoption behavior and counting the number of valid adoptions. The second element is the threshold trigger condition, which... Parameter updates are initiated at specific times. The third element is the nonlinear adjustment coefficient, which is adopted... As a regulating factor, it ensures that the magnitude of parameter change is positively correlated with the adoption rate deviation, but the rate of change decreases.

[0073] For example, when Set at 60% but actually When it is 50%, The value will be reduced by 5% of the original value.

[0074] Adjusted The value will directly affect the deviation threshold for the next evaluation task. The calculations ensure that the generated correction suggestions are more closely aligned with the user's actual needs. Through periodic iterative optimization, the system can gradually eliminate evaluation biases caused by differences in writing topics or fluctuations in user skill levels.

[0075] For example, assuming existing preset threshold parameters The adoption rate is 0.8. The preset adoption rate threshold is 0.6. The value is 0.7. Since 0.6 < 0.7, the adjustment condition is met, therefore the following calculation is performed:

[0076] .

[0077] Therefore, the preset threshold parameter was adjusted from 0.8 to 0.76. This dynamic adjustment mechanism allows the system to adaptively adjust the evaluation criteria based on user feedback.

[0078] Through the above technical solution, this application achieves dynamic adjustment of the preset threshold parameter. The system can automatically adjust the strictness of the evaluation criteria based on the actual adoption rate of users' correction suggestions. When the adoption rate is lower than expected, the system will appropriately lower the threshold parameter, thereby relaxing the evaluation criteria. This adaptive mechanism improves the flexibility and applicability of the evaluation system, enabling it to better adapt to differences in writing tasks, topic difficulty, and learner levels.

[0079] Further adoption rate of this application The calculations include:

[0080] Identify correction suggestions marked as "adopted by user" in historical reports and the total number of deviations. ;

[0081] The similarity increase value between subsequent paragraphs in the corresponding essay and the suggested keywords is calculated. If the increase value exceeds a preset threshold, it is counted as a valid adoption. The number of valid adoptions is counted. ;

[0082] Adoption rate The calculation formula is:

[0083] .

[0084] Once a suggested correction is marked as adopted, the system needs to obtain the paragraph semantic vectors before and after the correction. The improvement rate is determined by calculating the difference in similarity between these two vectors and the topic semantic vector. The improvement rate must exceed a preset improvement threshold to be considered a valid adoption. The preset improvement threshold is determined through experimental data; for example, setting it to 0.15 or 0.2 indicates that the similarity improvement must reach 15% or 20% or more. Total number of deviations. This refers to the total number of paragraphs identified as deviating from the intended paragraph, and the number of valid adoptions. The number of correction operations required to meet the improvement conditions.

[0085] For example, in the evaluation reports of the past 100 essays, there were 500 paragraphs marked as deviations.

[0086] The similarity increase between subsequent paragraphs in the corresponding essay and the suggested keywords is calculated. For each revised suggestion marked "Accepted by User," the cosine similarity change between the paragraph and the topic keywords is calculated before and after the revision. If the similarity increase exceeds a preset threshold of 0.1, it is considered a valid adoption. The number of valid adoptions is counted. For example, in 500 instances of deviation from the paragraph, the similarity improvement exceeded 0.1 in 300 of them.

[0087] Adoption rate .

[0088] This adoption rate is used for subsequent adjustments to preset threshold parameters to dynamically optimize the evaluation system's judgment criteria.

[0089] Through the above technical solution, this application can accurately quantify the actual adoption rate of users' suggestions for system correction, providing an objective basis for subsequent adjustment of threshold parameters. This dynamic adjustment mechanism based on user feedback enables the evaluation system to continuously optimize its judgment criteria, improving the accuracy and practicality of the evaluation. Simultaneously, by introducing a similarity-based threshold to determine valid adoption, biases caused by simple statistics are avoided, further enhancing the reliability of the adoption rate calculation.

[0090] This application further proposes the following workflow for the deviation correction module:

[0091] Obtain the paragraph semantic vector that deviates from the corrected paragraph. The original paragraph semantic vector ;

[0092] Calculation of improvement rate :

[0093] ;in, This is a topic semantic vector obtained by vectorizing the words in the preset topic keyword list.

[0094] Among them, the improvement rate The calculation involves three steps:

[0095] First, obtain the paragraph semantic vector that deviates from the corrected paragraph. The original paragraph semantic vector ;

[0096] Secondly, extract the topic semantic vectors obtained by vectorizing the words in the preset topic keyword list. ;

[0097] Finally, the difference in relevance between paragraphs and the topic before and after the correction was based on cosine similarity calculation, and the influence of the initial relevance on the improvement was eliminated by absolute value normalization.

[0098] For example, when the cosine similarity between the paragraph and the topic was 0.3 before correction and increased to 0.6 after correction, the improvement rate is... for This indicates a 100% increase in relevance.

[0099] Specifically, when determining whether users have effectively adopted the suggested improvements, the improvement rate needs to be considered. Compare with the preset increase threshold. If If the threshold is exceeded, it is considered a valid adoption. For example, with a preset improvement threshold of 0.5, only paragraphs whose relevance to the topic has increased by more than 50% compared to those before the correction are counted as valid adoptions. This mechanism avoids statistical errors by quantifying the improvement in relevance, which relies solely on the number of similarity increases. During system operation, the semantic vector of the paragraph before correction... Generated from the original deviated paragraph, and corrected. This is generated based on paragraphs modified after users adopted suggestions. Topic semantic vector. Obtained by averaging the word vectors of all topic keywords, ensuring it represents the overall topic semantics. The improvement rate is then calculated. The system can accurately assess the actual effect of each correction, thereby improving the accuracy of the adoption rate calculation and providing a reliable basis for subsequent parameter adjustments.

[0100] For example, in an English essay on the topic of "environmental protection," a paragraph originally reads "The city has many tall buildings," which is then revised to "The city has implemented green building standards to reduce energy consumption." Using a word vector model, the semantic vectors of these two sentences can be calculated separately. and .

[0101] Furthermore, the deviation correction module calculates the topic semantic vector. Assuming the predefined list of keywords includes words such as "environment," "protection," and "sustainability," the semantic vector of the topic is obtained by averaging the word vectors of these keywords. .

[0102] Therefore, the deviation correction module can calculate and , representing the similarity between the revised and unrevised paragraphs and the topic, respectively.

[0103] For example, calculation , .

[0104] Finally, the deviation correction module calculates the improvement rate according to the formula. : .

[0105] Through the above technical solution, this application can quantitatively evaluate the effectiveness of the revision suggestions. By calculating the change in the similarity between the paragraph and the topic before and after revision, the contribution of the revision suggestions to improving the topic relevance of the paragraph can be objectively measured. This method avoids subjective judgment and improves the accuracy and reliability of the evaluation. At the same time, the calculation of the improvement rate R takes into account the baseline value before revision, making the evaluation results more fair and reasonable. This quantitative evaluation mechanism provides a reliable basis for the system's self-optimization and helps to continuously improve the quality of revision suggestions.

[0106] This application further proposes a process for calculating topic semantic vectors, including the following steps: Topic semantic vector The calculation process is as follows:

[0107] Extract keywords for each topic word vectors ;

[0108] Calculate the arithmetic mean of all word vectors:

[0109] ;in, This represents the number of all word vectors.

[0110] Among them, word vectors Extraction is achieved using a pre-trained language model, where each topic keyword is transformed into a vector representation in a high-dimensional space through a word embedding algorithm. The arithmetic mean is calculated by summing the values ​​of each word vector along the same dimension and then dividing by the total number of keywords. This calculation method eliminates semantic bias of individual keywords through linear combination, enhancing the overall representational power of the topic semantic vectors.

[0111] Specifically, when the preset topic keyword list contains multiple words, each word is independently converted into a word vector. By summing the values ​​of each word vector and calculating the average, a vector that comprehensively reflects the overall semantics of the topic is formed. For example, for a topic list containing the keywords "climate," "change," and "impact," their word vectors are obtained separately, summed dimension by dimension, and then each dimension's value is divided by three to obtain the final topic semantic vector. This vector serves as a benchmark for calculating the similarity between the semantic vectors of subsequent paragraphs and the topic, ensuring that the generation of deviation correction suggestions is based on a comprehensive measurement of the multi-dimensional semantic space.

[0112] Through the above technical solution, this application can effectively integrate the semantic information of multiple topic keywords into a unified vector representation. Therefore, the system can more accurately capture the overall semantics of the topic, providing a more reliable benchmark for subsequent similarity calculations and deviation paragraph identification. Specifically, by calculating the similarity between the paragraph vector and the topic semantic vector, the relevance of the paragraph content to the topic can be more accurately assessed, thereby improving the system's accuracy in identifying topic deviations.

[0113] This application further proposes that, after the paragraph association graph is constructed, the following should also be included:

[0114] Obtain the topic semantic vector ;

[0115] The paragraph semantic vectors of each paragraph are compared with the topic semantic vectors. Calculate the average similarity and filter out paragraphs with an average similarity below the topic threshold.

[0116] The topic semantic vector is obtained by extracting the word vector of each keyword from a preset topic keyword list and then calculating the arithmetic mean of all word vectors. The average similarity between the paragraph semantic vector and the topic semantic vector is calculated using the cosine similarity algorithm. The topic threshold is dynamically adjusted based on the keyword coverage of the preset topic; for example, when the number of topic keywords exceeds a preset number, the topic threshold can be set to 0.6. The filtering operation is performed after the paragraph association graph is constructed to ensure that subsequent semantic consistency calculations are only performed on paragraphs that meet the topic association requirements.

[0117] Specifically, after constructing the paragraph association graph, the system first generates a topic semantic vector using the mean of the word vectors of the topic keywords. For each paragraph, the cosine similarity between its semantic vector and the topic semantic vector is calculated. When this value is lower than a preset threshold, the paragraph is removed from the association graph. For example, when the topic threshold is 0.5, if a paragraph has a similarity of 0.4 to the topic, it is considered off-topic and filtered out. This process, while preserving the coherence between paragraphs, mandates that all nodes must meet a minimum topic relevance, thereby eliminating evaluation errors caused by topic-offending paragraphs in local coherent paths. By dynamically adjusting the topic threshold, the system can adapt to the topic coverage requirements of different writing tasks, ensuring that deviation correction suggestions always revolve around the preset topic.

[0118] Through the above technical solution, this application solves the technical problem that static thresholds cannot dynamically adapt to different topics. By establishing a topic semantic vector benchmark, interfering paragraphs with low topic relevance can be automatically filtered out, allowing subsequent path analysis to focus on effective content, avoiding interference from deviating paragraphs in semantic consistency calculation, and improving the processing efficiency and accuracy of the evaluation system.

[0119] This application further proposes to obtain topic semantic vectors after constructing the paragraph association graph, calculate the average similarity between the paragraph semantic vectors of each paragraph and the topic semantic vectors, and filter paragraphs with an average similarity lower than the topic threshold.

[0120] The topic semantic vector is obtained by extracting the word vector of each keyword in the preset topic keyword list and calculating the arithmetic mean; the average similarity between the paragraph semantic vector and the topic semantic vector of each paragraph is calculated by the cosine similarity algorithm; the topic threshold is dynamically adjusted according to the semantic concentration of the preset topic keywords, for example, set to 70% to 85% of the average similarity value of all paragraphs.

[0121] Specifically, after the paragraph association graph is constructed, the system first extracts the word vectors of each keyword from the preset topic keyword list and calculates their arithmetic mean to generate a topic semantic vector. Then, for each paragraph's semantic vector, the system calculates its cosine similarity value with the topic semantic vector and takes the arithmetic mean of all paragraph similarity values. If the average similarity of a paragraph is lower than a preset topic threshold, the paragraph is deemed insufficiently related to the topic and is filtered out in the subsequent deviation analysis process. For example, when the topic threshold is 0.65, if a paragraph's similarity to the topic semantic vector is 0.58, the system automatically excludes the paragraph to avoid introducing interference during path search and deviation calculation. Through this filtering mechanism, the system can focus on paragraphs with high topic relevance, improving the accuracy of deviation paragraph identification.

[0122] This application further proposes the following process for extracting the core word set:

[0123] Perform part-of-speech filtering on each paragraph, retaining nouns, verbs, and adjectives;

[0124] The first paragraph was selected using a word frequency-inverse paragraph frequency algorithm. The core keywords, among which It is a preset positive integer.

[0125] As a preferred embodiment, the specific implementation of this application's solution is as follows: In the core word set extraction process, firstly, natural language processing tools are used to perform part-of-speech tagging on paragraphs, retaining three categories of content words: nouns, verbs, and adjectives. Then, the weight value of each word is calculated based on the TF-IDF algorithm, where the word frequency statistics are limited to the current paragraph, and the inverse paragraph frequency statistics cover all essays to be evaluated. Finally, the top 5 words with the highest weight values ​​are selected to form the core word set, and each core word must meet the screening condition of a TF-IDF value greater than 0.25.

[0126] By employing the aforementioned technical solution, this application effectively avoids the interference of function words and low-frequency noise words on semantic analysis. By focusing on core words that make substantial semantic contributions, it improves the accuracy of paragraph similarity calculation. This technique solves the problem of semantic feature extraction distortion caused by mixed parts of speech in traditional methods, laying a data foundation for the subsequent construction of high-precision paragraph association graphs.

[0127] This application further proposes a calculation method for the average semantic relevance of the deviating paragraphs, including:

[0128] Traverse all nodes of the longest continuous path and calculate the cosine similarity between the nodes that deviate from the paragraph and the nodes of the path.

[0129] The arithmetic mean of all similarities is taken as the average semantic relevance.

[0130] The traversal process is implemented by visiting the nodes corresponding to each paragraph in the longest coherent path, ensuring that all path nodes participate in the calculation. Cosine similarity is used to quantify the directional consistency between the deviating paragraph and the path node in the semantic vector space, while the arithmetic mean serves as a comprehensive indicator reflecting the degree of association between the deviating paragraph and the overall coherent path. In the specific numerical calculation, the semantic vector of each path node is multiplied by the semantic vector of the deviating paragraph, and then divided by the product of the vector magnitudes to obtain the cosine similarity value. All similarity values ​​are summed and divided by the total number of path nodes to obtain the average semantic association degree.

[0131] Specifically, after the paragraph association graph is constructed, the system locates the nodes of the deviated paragraphs and obtains the set of nodes along the longest coherent path. For each path node, its paragraph semantic vector is extracted, and cosine similarity is calculated between it and the semantic vector of the deviated paragraph. All calculation results are summed and divided by the number of path nodes to generate an average semantic association score. This score is used to compare with a preset deviation threshold to determine the degree of deviation of the deviated paragraph. By introducing the overall semantic association of path nodes as a benchmark, the influence of single nodes or local similarity is avoided, improving the comprehensiveness and accuracy of deviation determination. During the calculation process, the application of the arithmetic mean balances the influence weight of different path nodes on the deviated paragraph, ensuring that the evaluation results reflect the overall deviation trend of the deviated paragraph from the coherent path.

[0132] As a preferred embodiment, the solution of this application is implemented as follows: When calculating the average semantic relevance of the deviating paragraph, firstly, the set of all nodes of the longest coherent path is obtained. For each deviating paragraph node, the cosine similarity between its paragraph semantic vector and the paragraph semantic vector of each node in the path is calculated sequentially. The sum of all obtained cosine similarity values ​​is then divided by the total number of path nodes to obtain the arithmetic mean, which is used as the average semantic relevance index of the deviating paragraph. This index is used for subsequent comparison with a preset deviation threshold to determine whether the paragraph deviates from the topic.

[0133] Through the above technical solution, this application dynamically calculates the average semantic relevance between the deviated paragraph and the coherent path node. The system can flexibly adjust the evaluation criteria according to the actual content structure, avoiding the problem of insufficient adaptability of static thresholds in different writing scenarios, thereby improving the accuracy of deviated paragraph identification and ensuring that the evaluation results are more in line with the learner's actual writing level.

[0134] The technical scope of this invention is not limited to the content described above. Those skilled in the art can make various modifications and variations to the above embodiments without departing from the technical concept of this invention, and all such modifications and variations should fall within the protection scope of this invention.

Claims

1. An automatic evaluation system for English writing flow, characterized in that: include: The data acquisition module is used to obtain a collection of paragraphs and a list of preset topic keywords for the essay to be evaluated; The paragraph association construction module is used to extract the core word set of each paragraph, calculate the semantic similarity between any two paragraphs through the core word set, and construct a paragraph association graph with paragraphs as nodes and semantic similarity as edge weights; The semantic consistency calculation module is used to traverse all paths of the paragraph association graph, select the continuous node path with the largest cumulative edge weight value as the longest coherent path, calculate the semantic consistency score of the longest coherent path, and calculate the preset deviation threshold based on the semantic consistency score according to the preset threshold parameter. The topic deviation calculation module is used to identify deviating paragraphs in the paragraph association graph whose average semantic relevance to the longest coherent path is lower than a preset deviation threshold. The deviation correction module is used to filter out the words with the highest paragraph semantic vector similarity to the deviation paragraph from the preset topic keyword list, and generate correction suggestions based on the words with the highest similarity. The evaluation results generation module is used to generate an evaluation report that includes the deviated paragraphs and correction suggestions; The feedback adjustment module is used to perform the following operations: Obtain the user's adoption flag for the proposed correction and calculate the adoption rate within a preset time period; Determine whether the adoption rate is lower than a preset adoption rate threshold. If the determination result is yes, then adjust the preset threshold parameter proportionally and use the corrected preset threshold parameter for the next evaluation.

2. The automatic evaluation system for English writing flow according to claim 1, characterized in that: The process of constructing the paragraph association diagram is as follows: Get the semantic similarity between any two paragraphs : ;in, For the first The paragraph semantic vector of a segment. For the first The paragraph semantic vector; judge If the similarity exceeds a preset threshold, then the edge between the two paragraphs is preserved.

3. The automatic evaluation system for English writing flow according to claim 1, characterized in that: The semantic consistency score is calculated as follows: The longest continuous path is the path whose sum of edge weights is maximized in the paragraph association graph. The sum of the weights of all edges in this longest continuous path is then obtained. ; Based on the weights of all edges Calculate the semantic consistency score: ;in, This represents the number of edges in the path. The formula for calculating the preset deviation threshold is: ,in This is a preset threshold parameter.

4. The automatic evaluation system for English writing flow according to claim 1, characterized in that: The adjustment process of the preset threshold parameter is as follows: Get existing preset threshold parameters and adoption rate ; like Established, The preset adoption rate threshold; Then the preset threshold parameter Adjust according to the following formula: 。 5. The automatic evaluation system for English writing flow according to claim 4, characterized in that: The adoption rate The calculations include: Identify correction suggestions marked as "accepted by user" in historical reports and the total number of deviations. ; The similarity increase value between subsequent paragraphs in the corresponding essay and the suggested keywords is calculated. If the increase value exceeds a preset threshold, it is counted as a valid adoption. The number of valid adoptions is counted. ; Adoption rate The calculation formula is: 。 6. The automatic evaluation system for English writing flow according to claim 5, characterized in that: The workflow of the deviation correction module is as follows: Obtain the paragraph semantic vector that deviates from the corrected paragraph. The original paragraph semantic vector ; Calculation of improvement rate : ;in, This is a topic semantic vector obtained by vectorizing the words in the preset topic keyword list.

7. An automatic evaluation system for English writing flow according to claim 6, characterized in that: The topic semantic vector The calculation process is as follows: Extract keywords for each topic word vectors ; Calculate the arithmetic mean of all word vectors: ;in, This represents the number of all word vectors.

8. The automatic evaluation system for English writing flow according to claim 7, characterized in that: The constructed paragraph association graph also includes: Obtain the topic semantic vector ; The paragraph semantic vectors of each paragraph are compared with the topic semantic vectors. Calculate the average similarity and filter out paragraphs with an average similarity below the topic threshold.

9. The automatic evaluation system for English writing flow according to claim 1, characterized in that: The extraction process of the core word set is as follows: Perform part-of-speech filtering on each paragraph, retaining nouns, verbs, and adjectives; The first paragraph was selected using a word frequency-inverse paragraph frequency algorithm. The core keywords, among which It is a preset positive integer.

10. An automatic evaluation system for English writing flow according to claim 1, characterized in that: The calculation of the average semantic relevance of the deviated paragraphs includes: Traverse all nodes of the longest continuous path and calculate the cosine similarity between the nodes that deviate from the paragraph and the nodes of the path. The arithmetic mean of all similarities is taken as the average semantic relevance.

Images