A title conformity evaluation method, system, device and medium

Abstract

The application provides a title conformity evaluation method, system, device and medium, the method comprising: obtaining a target composition, extracting a title keyword from the title of the target composition according to a keyword extraction algorithm, and calculating the theme weight of the words in the body of the target composition; expanding the title keyword according to a preset composition keyword expansion knowledge base, the expansion knowledge base being based on the corpus of the target composition, keyword indexing and generating theme-related word clusters by word clustering; extracting the body keyword of the composition according to the word count of the full text of the target composition and a preset dynamic rule; generating a weighted theme vector according to the theme weight of the words, and calculating the cosine similarity between the title keyword and the body keyword of the composition to obtain a title conformity score; selecting an optimal threshold from a candidate threshold range according to a dynamic search algorithm; and outputting the title conformity grade of the target composition according to the title conformity score and the optimal threshold.

Description

Technical Field

[0001] This application relates to the fields of data processing and essay evaluation technology, and in particular to a method, system, device and medium for evaluating the relevance of essays to the topic. Background Technology

[0002] Currently, while automatic essay scoring systems have made good progress in detecting syntax, spelling, and basic language quality, they still mainly rely on a binary judgment of "whether the essay is off-topic" to determine the relevance of the essay to the topic, lacking a fine-grained level evaluation that matches human scoring. Existing methods generally rely on technologies such as LDA, TF-IDF, and Word2Vec to extract themes or expand the vocabulary of the topic, but they are prone to introducing non-theme noise or only generating semantically synonymous words (rather than theme-related words). Furthermore, they are highly sensitive to the manual setting of the number of topics, the number of extracted words, and the judgment threshold, resulting in insufficient generalization ability for essays with new topics or topics with a high degree of divergence. These problems directly restrict the consistency and stability of automatic essay scoring with human scoring in the dimension of "understanding the meaning of the topic".

[0003] The methods proposed in related technologies generally have shortcomings such as requiring manual setting of the number and threshold of topics, bias towards semantically similar rather than topic-related words, and difficulty in making fine-grained classifications of the degree of relevance to the topic. Summary of the Invention

[0004] In view of this, it is necessary to provide a method, system, device and medium for evaluating the conformity of a document with respect to its title, which can at least overcome one of the above-mentioned deficiencies.

[0005] In a first aspect, embodiments of this application provide a method for evaluating the relevance of a text to a topic, applied to an automatic essay scoring system, the method comprising:

[0006] Obtain the target essay, extract title keywords from the title of the target essay using a keyword extraction algorithm, and calculate the topic weight of words in the body of the target essay;

[0007] The title keywords are expanded with related terms based on a pre-set extended knowledge base of essay keywords. The extended knowledge base is generated by keyword indexing based on the corpus of the target essay and word clustering.

[0008] Based on the total word count of the target essay, extract the main theme keywords of the target essay according to preset dynamic rules;

[0009] A weighted topic degree vector is generated based on the topic degree weight of the words, and the cosine similarity between the topic words of the title and the topic words of the target essay is calculated to obtain the title-subject relevance score.

[0010] The optimal threshold is selected from the candidate threshold range based on a dynamic search algorithm;

[0011] Based on the title relevance score and the optimal threshold, the title relevance level of the target essay is output.

[0012] In one embodiment, the method further includes:

[0013] Based on a large-scale classification corpus, the topic degree of words in the corpus is calculated using a word topic degree calculation model.

[0014] The topic weight of the words in the main text of the target essay is calculated based on the topic weight of the words.

[0015] In one embodiment, the extraction of title keywords includes:

[0016] The title text of the target essay is segmented, new words are identified, and stop words are filtered to form a candidate word set;

[0017] The candidate words in the candidate word set are ranked by importance based on their topicality, and the title theme words are selected based on the grammatical position and representational ability of the candidate words in the essay title.

[0018] In one embodiment, the method further includes:

[0019] Keyword indexing of large-scale essay corpora;

[0020] Clustering is performed based on the co-occurrence relationship, semantic similarity, and semantic distance of words in essays on the same topic. The obtained topic-related word clusters are then screened, merged, and denoised through clustering quality assessment.

[0021] The topic-related word clusters are sorted according to the thematic representation strength of the words to form an essay topic-related thesaurus for expanding the title topic words.

[0022] In one embodiment, the dynamic rules include:

[0023] The target essays are divided into short texts and long texts;

[0024] When the target essay is a short text, extract a first preset number of words and select words with high thematic relevance or located in key positions in the text;

[0025] When the target essay is a long text, the configuration is set to extract a larger number of texts to improve topic coverage, and an upper limit is set for the number of texts extracted to prevent dilution of topic features.

[0026] In one embodiment, the dynamic rules further include:

[0027] When the length of the target essay cannot be determined, a fixed number of keywords are extracted, and predefined extraction parameters are selected based on the genre of the target essay.

[0028] In one embodiment, generating a weighted topic degree vector based on word topic degree weights includes:

[0029] The weights of the title keywords and the body keywords are adjusted based on word frequency, topicality value, and credibility of extended sources, respectively.

[0030] Secondly, embodiments of this application provide an automatic essay scoring system, applied to implement the essay-topic relevance assessment method as described in the first aspect, the system comprising:

[0031] The essay acquisition module is used to acquire target essays, extract title keywords from the title of the target essays using a keyword extraction algorithm, and calculate the topic weight of words in the body of the target essays.

[0032] The first extraction module is used to expand the title topic words with topic-related words according to the preset extended knowledge base of essay topic words. The extended knowledge base is generated by keyword indexing based on the corpus of the target essay and word clustering.

[0033] The second extraction module is used to extract the main theme words of the target essay according to the word count of the full text and a preset dynamic rule.

[0034] The analysis module is used to generate a weighted topic degree vector based on the topic degree weight of words, and to calculate the cosine similarity between the topic words of the title and the topic words of the essay body to obtain the title-text relevance score; and to select the optimal threshold from the candidate threshold range according to the dynamic search algorithm.

[0035] The grade output module is used to output the grade of the target essay's relevance to the topic based on the essay's relevance score and the optimal threshold.

[0036] Thirdly, embodiments of this application provide an electronic device, including:

[0037] processor;

[0038] Memory used to store processor-executable instructions;

[0039] The processor is configured to implement the title compliance evaluation method as described in the first aspect when executing the instructions.

[0040] Fourthly, embodiments of this application provide a computer-readable storage medium including instructions that instruct a device to perform the title compliance evaluation method as described in the first aspect.

[0041] This application provides a method, system, device, and medium for evaluating the relevance of essay content to the topic. By constructing a fine-grained evaluation system for essay content relevance, employing a weighted, adjustable word topic degree calculation model, and combining a large-scale essay topic terminology extension knowledge base to expand the topic topic relevance, this method effectively overcomes the shortcomings of existing technologies, which can only perform binary classification judgments of off-topic / on-topic content, inaccurate topic term expansion, and difficulty in determining the judgment threshold. This method achieves accurate and stable evaluation of the degree of relevance between essay content and topic under unsupervised conditions, significantly improving the accuracy and reliability of automatic scoring systems in content-dimensional evaluation. At the same time, it provides usable knowledge base resources for thematic writing instruction and has good generalization ability and practical value. Attached Figure Description

[0042] Figure 1 This is a flowchart illustrating a title conformity assessment method provided in an embodiment of this application.

[0043] Figure 2 This is a flowchart of the essay title compliance level calculation provided in one embodiment of this application.

[0044] Figure 3 This is a schematic diagram showing the calculation results of the title conformity of various schemes provided in an embodiment of this application.

[0045] Figure 4 This is a schematic diagram of the optimal threshold selection result provided in an embodiment of this application.

[0046] Figure 5 This is a schematic diagram of the modules of an automatic essay scoring system provided in an embodiment of this application.

[0047] Figure 6 A schematic diagram of an electronic device provided in an embodiment of this application.

[0048] Explanation of main component symbols

[0049] Detailed Implementation

[0050] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them.

[0051] It should be noted that, in the embodiments of this application, "at least one" refers to one or more, and "more than one" refers to two or more. Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used in the specification of this application is for the purpose of describing particular embodiments only and is not intended to be limiting of this application.

[0052] It should be noted that in the embodiments of this application, the terms "first," "second," etc., are used only for descriptive purposes and should not be construed as indicating or implying relative importance, nor as indicating or implying order. Features specified as "first" or "second" may explicitly or implicitly include one or more of the stated features. In the description of the embodiments of this application, words such as "exemplary" or "for example" are used to indicate examples, illustrations, or explanations. Any embodiment or design scheme described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design schemes. Specifically, the use of words such as "exemplary" or "for example" is intended to present the relevant concepts in a concrete manner.

[0053] Based on the embodiments described in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0054] Currently, while automatic essay scoring systems have made good progress in detecting syntax, spelling, and basic language quality, they still mainly rely on a binary judgment of "whether the essay is off-topic" to determine the relevance of the essay to the topic, lacking a fine-grained level evaluation that matches human scoring. Existing methods generally rely on technologies such as LDA, TF-IDF, and Word2Vec to extract themes or expand the vocabulary of the topic, but they are prone to introducing non-theme noise or only generating semantically synonymous words (rather than theme-related words). Furthermore, they are highly sensitive to the manual setting of the number of topics, the number of extracted words, and the judgment threshold, resulting in insufficient generalization ability for essays with new topics or topics with a high degree of divergence. These problems directly restrict the consistency and stability of automatic essay scoring with human scoring in the dimension of "understanding the meaning of the topic".

[0055] In related technologies, existing methods mainly follow two paths: one is to extract or represent essays and titles based on topic models or text representations (such as LDA, Doc2Vec, TF-IDF + cosine similarity), and then determine relevance based on similarity or classifiers; the other is to expand the title using word vectors (such as Word2Vec) and compare it with the body text to improve coverage. However, these methods generally suffer from shortcomings such as the need to manually set the number and threshold of topics, the bias of expanded words towards semantically similar rather than topic-related words, and the difficulty in making fine-grained classifications of relevance. Therefore, improving the "topic degree" distinguishing ability of topic representations, constructing a topic-related word expansion library for teaching corpora, and designing an evaluation process that can adaptively select level thresholds are key directions for solving the fine-grained evaluation of the relevance of Chinese essays to the topic.

[0056] This application provides a method, system, device, and medium for evaluating the relevance of essay content to the topic. By constructing a fine-grained evaluation system for essay content relevance, employing a weighted, adjustable word topic degree calculation model, and combining a large-scale essay topic terminology extension knowledge base to expand the topic topic relevance, this method effectively overcomes the shortcomings of existing technologies, which can only perform binary classification judgments of off-topic / on-topic content, inaccurate topic term expansion, and difficulty in determining the judgment threshold. This method achieves accurate and stable evaluation of the degree of relevance between essay content and topic under unsupervised conditions, significantly improving the accuracy and reliability of automatic scoring systems in content-dimensional evaluation. At the same time, it provides usable knowledge base resources for thematic writing instruction and has good generalization ability and practical value.

[0057] Figure 1 This is a flowchart illustrating a title conformity assessment method provided in one embodiment of this application, as shown below. Figure 1 The method for evaluating the relevance of a text to a topic includes at least the following steps: S100: Obtain the target essay, extract title keywords from the title of the target essay using a keyword extraction algorithm, and calculate the topic weight of words in the body of the target essay; S200: Expand the title keywords into topic-related words based on a pre-set extended knowledge base of essay topic keywords. The extended knowledge base is based on the corpus of the target essay, which is indexed by keywords and generates topic-related word clusters using word clustering; S300: Extract the body keywords of the essay according to the word count of the full text of the target essay and according to pre-set dynamic rules; S400: Generate a weighted topic vector based on the topic weight of words, and calculate the cosine similarity between the title keywords and the topic keywords of the essay body to obtain the title-to-topic relevance score; S500: Select the optimal threshold from the candidate threshold range using a dynamic search algorithm; S600: Output the title-to-topic relevance level of the target essay based on the title-to-topic relevance score and the optimal threshold.

[0058] S100: Obtain the target essay, extract title keywords from the title of the target essay using the keyword extraction algorithm, and calculate the topic weight of words in the body of the target essay.

[0059] In this embodiment of the application, the topic-response matching evaluation method is applied to an essay scoring system, and the topic-response matching evaluation method includes at least the following in step S100:

[0060] Specifically, in implementing this step, the system first preprocesses the target essay, including unified encoding, sentence segmentation, word segmentation, word form normalization, new word identification, and stop word filtering. Then, it performs a keyword extraction process on the title text. This process may include sub-steps such as word topic degree ranking, weighting based on word frequency and grammatical position within the title (such as subject-predicate headword, object position), and filtering based on the word's discriminative power in similar essay corpora, to determine several title topic words and assign them confidence scores. At the same time, for each candidate word in the body text, the topic degree weight is calculated based on the pre-trained word topic degree model and the word's frequency of occurrence in the body text, and then normalized and scored to form a weight list for subsequent vector construction.

[0061] Understandably, by performing language preprocessing on the words in the title and body text and introducing dual evaluation of topicality and word frequency, it is possible to ensure that key topic words are extracted while providing measurable confidence information for subsequent expansion, matching and weighting, thereby reducing the impact of noisy words on the judgment results.

[0062] S200: Expand the title keywords with related terms based on the pre-set extended knowledge base of essay keywords. The extended knowledge base is based on the corpus of the target essay, which is indexed with keywords and generates topic-related word clusters by word clustering.

[0063] In this embodiment of the application, the title conformity assessment method includes at least the following in step S200:

[0064] Specifically, the system uses the aforementioned title keywords as search seeds and retrieves candidate extended words related to the seed keyword clusters in the extended knowledge base. During the search process, confidence thresholds, source verification, and word form normalization are applied to the candidate extended words, and context disambiguation is performed on polysemous words when necessary. If the candidate extended words come from multiple synonym or near-synonymous clusters, the system removes duplicates and merges them, and sorts them according to source credibility, intra-cluster consistency, and frequency of occurrence in the essay corpus. After the expansion is completed, the system outputs the expanded set of title keywords and records the source tag, cluster identifier, and confidence score for each extended word for subsequent weight adjustment.

[0065] Understandably, title keyword expansion aims to improve the coverage of the title's intent (enhance recall), but expansion inevitably introduces noise. Therefore, using source and confidence controls and retaining traceable information after expansion can help to suppress or amplify the impact of expanded words in a targeted manner when calculating similarity or determining thresholds.

[0066] S300: Based on the total word count of the target essay, extract the main keywords of the essay according to preset dynamic rules.

[0067] In this embodiment of the application, the title conformity assessment method includes at least the following in step S300:

[0068] Specifically, the system first divides the main text into preset length categories (e.g., short, medium, and long text) based on its overall length or number of characters / words, and then selects the corresponding keyword extraction strategy based on the category. In the case of short text, a number of high-confidence words are selected according to the topicality and grammatical position priority to focus on the core topic. In the case of medium and long text, the extraction threshold is dynamically set according to a preset ratio (e.g., Top-N or a certain ratio of the total number of words in the text) combined with the distribution of word frequency steepness and topicality spectrum. If necessary, a semantic redundancy removal mechanism is introduced (e.g., removing synonyms or low-information words). In addition, this step can enable multi-cluster detection to identify multi-topic situations that may occur in the main text, and allows representative keywords to be extracted by cluster. Finally, the system outputs a set of main text keywords sorted by priority and marked with confidence.

[0069] Understandably, the preset dynamic rules adaptively determine the extraction quantity and priority based on the text length and semantic concentration. This allows for the concentrated extraction of the most critical information in short texts and ensures topic coverage in long texts, thereby avoiding under- or over-extraction problems caused by fixed rules.

[0070] S400: Generate a weighted topic degree vector based on the topic degree weight of the words, and calculate the cosine similarity between the topic words of the title and the topic words of the essay to obtain the title-subject relevance score.

[0071] In this embodiment of the application, the title conformity assessment method includes at least the following in step S400:

[0072] Specifically, the title (including extended words) and the set of keywords extracted from the main text are mapped into vector representations respectively. The vector components are determined by a combination of factors such as the topic weight of the corresponding words, word frequency, and confidence of the extended source. In the vector construction, the extended words can be given a lower initial weight or confidence discount to suppress the influence of noise, and the vectors are normalized to eliminate the bias caused by the difference in text length. Then, the cosine similarity between the two vectors is calculated as the basic title-text consistency score, and the original similarity and the normalized and confidence-corrected similarity can be saved at the same time for the threshold selection algorithm to refer to.

[0073] It is understandable that by applying multi-dimensional weight adjustments (topic degree, word frequency, source credibility) to the vector components, and by normalizing and correcting the confidence level before and after calculating the similarity, the obtained similarity score can be made more robust and better reflect the true degree of "subject matter matching", rather than being misled by a small amount of high-frequency or extended noise.

[0074] This application provides a method for automatically selecting the optimal title-subject relevance level threshold based on a dynamic programming algorithm. This method aims to address the key issue of how to objectively and accurately divide continuous title-subject relevance scores into discrete relevance levels. It uses a data-driven approach to find the level boundary point that maximizes consistency between machine and human evaluations.

[0075] Specifically, the threshold selection steps are as follows: First, determine the search range for threshold selection. Based on the correspondence between the title-text matching score and the human evaluation level calculated on the standard set, observe the intervals where levels overlap, and denote the candidate threshold range between level 1 and level 2 as θi[θa1,θa2,θa3,…,θan], and the candidate threshold range between level 2 and level 3 as θj[θb1,θb2,θb3,…,θbn]. Then, using the idea of ​​dynamic programming, systematically traverse all possible threshold combinations (θi,θj). For each combination, apply it to the standard set, calculate the precision, recall, and F1 score between the machine-generated ranking results and the human evaluation ranking results, and finally calculate the key macro F1 score and weighted F1 score. Taking the maximization of the weighted F1 score as the core optimization objective, select the threshold combination [(θa1,θb1)(θa1,θb2)…(θan,θbn)] that maximizes this value as the final optimal threshold. If multiple combinations have the same weighted F1 value, the macro F1 value is used as a secondary evaluation criterion. Furthermore, as an optimization strategy, if the maximum weighted F1 value is found at the endpoint of the initial search range, the search range is automatically expanded until two consecutive decreases in the F1 value are observed, ensuring that the globally optimal solution is found rather than a locally optimal solution.

[0076] Understandably, this embodiment achieves automated and optimal determination of the grading threshold by introducing dynamic programming, an optimization algorithm, into the threshold selection process. This method completely eliminates the limitations and subjectivity of relying on human experience or simple dichotomy to set thresholds, enabling the grading of essay topic relevance to closely align with human scoring standards. This significantly improves the reliability and generalization ability of the automatic scoring results, providing a stable and objective grading benchmark for evaluating essays of different topics and genres.

[0077] S500: Select the optimal threshold from the candidate threshold range based on a dynamic search algorithm.

[0078] In this embodiment of the application, the title conformity assessment method includes at least the following in step S500:

[0079] Specifically, the system traverses and evaluates possible thresholds or combinations of thresholds within a predefined candidate threshold space. During the evaluation process, a prepared validation set (or historical labeled samples) is used to score each candidate threshold using set evaluation metrics (such as weighted F1, a weighted combination of precision and recall, etc.). The dynamic search algorithm can be grid search, heuristic pruning search, or dynamic programming to reduce the search space, and can verify the optimal threshold for different essay genres, grades, or text lengths in parallel. After the search is completed, the system outputs the threshold that optimizes the evaluation metrics or a set of thresholds divided by genre / length.

[0080] Understandably, the threshold is not fixed in one stroke, but is determined by dynamic search on a representative validation set, thereby taking into account the differences in different question types and text features, and improving the robustness and consistency of the rating classification in real-world scenarios.

[0081] S600: Output the title relevance level of the target essay based on the title relevance score and the optimal threshold.

[0082] In this embodiment of the application, the title conformity assessment method includes at least the following in step S600:

[0083] Specifically, the system compares the essay relevance score obtained in step S400 (or after confidence level correction) with the threshold determined in step S500 and maps it to a predetermined set of grades (e.g., off-topic, partially relevant, basically relevant, fairly relevant, completely relevant). At the same time, it generates interpretability output, including a list of matched topic word pairs, uncovered key prompts, score confidence, and a list of extended terms that may affect the judgment. Finally, the grade results and interpretability descriptions are provided to the essay automatic scoring system for comprehensive scoring or presented to teachers / students for review, and the system supports manual review and fine-tuning of thresholds or weights.

[0084] Understandably, in addition to providing a discretized level, the output also includes auxiliary information to facilitate manual verification and correction of misjudgments. This helps to improve system transparency and support interactive teaching application scenarios.

[0085] In this embodiment of the application, the method further includes: calculating the topic degree of words in the corpus using a word topic degree calculation model based on a large-scale classification corpus; and calculating the topic degree weight of words in the main text of the target essay based on the topic degree of the words.

[0086] Specifically, the large-scale classification corpus is first preprocessed (including word segmentation, word form normalization, and low-frequency word removal), and the frequency of occurrence, co-occurrence relationship, and distribution differences of each word in each topic category are statistically analyzed. The statistical results are smoothed and normalized to alleviate the problems of low frequency and uneven distribution, and the topic degree index and confidence score of each word under each topic are output. For each word in the target essay text, the topic degree weight of the word is generated by weighting or linearly combining the word's frequency of occurrence in the text, the topic degree value in this corpus, and the confidence score of the word. The obtained weights are then normalized for use in subsequent vector construction.

[0087] Specifically, a word topic degree calculation model is applied for core calculation. This model determines the topic degree of a word by comprehensively measuring its "neighborhood domain" and "domain unevenness". The neighborhood domain is calculated using a formula... This formula effectively filters out mid-frequency words that characterize the topic, while weakening the influence of high-frequency functional words and low-frequency rare words; domain unevenness is addressed through the formula. This quantifies the dispersion of word distribution across different topic categories; a higher value indicates a stronger ability of the word to represent a specific topic. Ultimately, the topicality of a word is calculated using the following formula.

[0088]

[0089] in, , Is class The sum of the frequencies of all the words contained. It is the first The word in the category Number of times it appears; ,in Number of categories; This indicates the words that appear in the training corpus. Number of times, It is the sum of the occurrences of all words in the training corpus.

[0090] Understandably, this step aims to transfer robust topic information obtained from offline corpora to specific essays, so that the topic relevance of each word in the text reflects both the topic tendency at the corpus level and the importance of the word in the current text, thereby providing a reliable weighting basis for subsequent topic matching.

[0091] In an embodiment of the present application, a method for training the topic degree of words based on a large-scale classified corpus is provided. First, a large-scale and multi-topic classified corpus is constructed as the training basis. Through systematic processing of this corpus, a quantitative topic degree value is assigned to each word in the corpus using a word topic degree calculation model, thereby establishing a knowledge base of word topic degrees with wide coverage and high discrimination.

[0092] Specifically, the training process is as follows: The training corpus used in this embodiment has a scale of 6 billion words and contains approximately 600,000 documents. According to the principle of combining "topic division" and "life priority", the entire corpus is systematically divided into 15 major topic categories and further subdivided into 244 minor topic categories, forming a hierarchical and comprehensive topic system. The major topic categories and the corresponding number of documents are shown in Table 1:

[0093]

[0094] Table 1: Topic Classification Table of the Corpus

[0095] Based on this classification system, the entire corpus is input into the above-mentioned word topic degree calculation model for analysis. The model calculates the distribution of each word in the corpus in the above 244 minor topic categories, and comprehensively uses the calculation of the adjacent domain and domain non-uniformity, and finally outputs a stable topic degree value for each word. Examples of the training results of the topic degrees of some words are as follows in the table:

[0096]

[0097] Table 2: Examples of Word Topic Degrees

[0098] It can be understood that through the training of the above large-scale and multi-topic corpus in this embodiment, the word topic degree calculation model can effectively learn and quantify the representational differences of words in different fields. As shown in Table 2, the training results significantly distinguish functional function words (such as "de", "zai", with low topic degrees) from domain entity words (such as "value-added tax", "radio telescope", with high topic degrees), which verifies the model's ability to capture the topic relevance of words. The knowledge base of word topic degrees obtained from this training provides a reliable and quantifiable semantic basis for subsequent composition content analysis, ensuring the accuracy and robustness of the extraction and weight calculation of topic words in the evaluation of the compliance of the composition title.

[0099] In an embodiment of the present application, a method for dynamically calculating the topic degree weight of words for a single composition is provided. This method aims to solve the problem of the difference in the topic importance of the same word in different compositions. By combining the global word topic degree with local text features, a weight value that can accurately reflect the topic representation intensity of the word in a specific composition is calculated.

[0100] Specifically, for target essays Each word in The calculation of its topic weight follows these steps: First, obtain the global topic weight value of the word obtained through training on a large-scale corpus. Next, statistics were compiled on the frequency of this word in essays. Number of times in And essays Total number of words And calculate its relative word frequency. Finally, the global topic degree and local word frequency are expressed using the formula:

[0101]

[0102] The formula integrates the words, where n is a preset adjustable parameter (n≥1). This formula uses the parameter n to flexibly control the influence of word frequency: when n is small, word frequency has a strong influence on the final weight, which is beneficial to highlighting high-frequency words in the essay; when n is large, the influence of word frequency is weakened, so that keywords that appear infrequently but have high overall topicality can also obtain significant weight.

[0103] Understandably, this embodiment cleverly balances the inherent thematic attributes of words with their prominence in a specific context through the aforementioned calculation method. This dynamic weighting calculation mechanism overcomes the "interference from common words" that may result from relying solely on word frequency, or the "text specificity" problem that may be ignored by relying solely on global thematic degree. This provides a more accurate and representative data foundation for subsequent text vector construction and text-title similarity calculation, ultimately significantly improving the semantic accuracy and contextual adaptability of text-title conformity assessment.

[0104] In this embodiment, the extraction of title keywords includes: segmenting the text of the essay title, identifying new words, and filtering stop words to form a candidate word set. The candidate words in the candidate word set are ranked according to their thematic importance, and the title keywords are selected based on their grammatical position and representational power in the essay title.

[0105] Specifically, the title text is first segmented into fine-grained words and new words are identified. After removing punctuation, function words, and common stop words, a candidate word set is formed. The candidate words are then comprehensively scored and ranked according to their topicality value in the corpus, their distinctiveness as title words in similar essays, and their grammatical position weight within the title (such as subject-verb-object head words and modifier core words). Then, based on a preset strategy (such as by score threshold or Top-K selection), a number of words with the highest scores and strong semantic representation are selected as title topic words. At the same time, words with low confidence or polysemous words are marked for the expansion stage or manual review.

[0106] Understandably, this step aims to accurately extract core terms that represent the writing task's intent from concise titles by combining corpus-level thematic information with the grammar and representational capabilities within the title, thus avoiding misselection based solely on word frequency or position.

[0107] In this embodiment, the construction of the essay topic terminology extension knowledge base includes: keyword indexing of a large-scale essay corpus; clustering based on the co-occurrence relationship, semantic similarity, and semantic distance of words in essays on the same topic, and filtering, merging, and denoising the obtained topic-related term clusters through clustering quality assessment; and sorting the topic-related term clusters according to the topic representation strength of the words to form an essay topic-related terminology database for expanding title topic terms.

[0108] Specifically, firstly, keywords are extracted and indexed from the target essay corpus, recording the context and co-occurrence statistics of each keyword under different themes; then, words are clustered based on multi-source similarity measures such as co-occurrence matrix and word vector similarity to obtain several theme-related word clusters; each word cluster is evaluated for intra-cluster consistency, coverage, and representativeness, low-quality clusters are merged or removed, and noisy words are downweighted, and the source corpus fragments, cluster identifiers, and confidence scores are labeled for the retained extended entries; finally, the word clusters and their entries are ranked based on the representation strength of words within the cluster and their importance in the essay context, forming a searchable extended knowledge base with source and confidence descriptions.

[0109] Understandably, this extended knowledge base not only provides candidate extended words that are semantically related to the title topic to improve coverage, but also controls the extended noise through quality assessment and confidence labeling, making it easier to accurately adjust the influence of extended words in subsequent similarity calculations.

[0110] In this embodiment, the dynamic rules include: dividing the target essay into short text and long text. When the target essay is a short text, a first preset number of words are extracted, and words with high topicality or located in key positions in the text are selected. When the target essay is a long text, the configuration is set to extract a larger number of words to improve topic coverage, and an upper limit is set for the number of words extracted to prevent dilution of topic features.

[0111] Specifically, the text is categorized based on the number of words or characters in the main text (e.g., short text: less than a certain word count threshold; long text: greater than the threshold). For short text, a strict Top-N extraction strategy is adopted, prioritizing words with high topic relevance, those at the beginning of sentences / paragraphs, or those appearing in sentences related to the question stem. For long text, a strategy of extraction based on proportion or segmentation is adopted, ensuring coverage of the main paragraphs' keywords while controlling redundancy through deduplication and synonym merging. At the same time, an upper limit is set to avoid too many non-topic words entering the set. In addition, for boundary text, a smoothing parameter or a confidence threshold can be used to determine whether to increase or decrease the number of extractions.

[0112] Understandably, this dynamic rule, by flexibly adjusting the extraction strategy based on text length and semantic distribution, can maintain accuracy in short texts and comprehensiveness in long texts, avoiding the loss of thematic information or noise expansion caused by a one-size-fits-all rule.

[0113] In this embodiment of the application, the dynamic rules also include: when the length of the target essay cannot be determined, a fixed number of keywords are extracted, and predefined extraction parameters are selected based on the genre of the target essay.

[0114] Specifically, during implementation, when text length information is unavailable or text quality makes length determination unreliable, the system reverts to a set of conservative default parameters (e.g., extracting the Top-M keywords). At the same time, it selects the appropriate extraction strategy and priority weight from a pre-configured parameter library based on the essay genre (e.g., narrative, expository, argumentative). If genre information is also missing, the system adopts default parameters that are common to teaching scenarios, and determines whether to supplement or reduce the extracted keywords in subsequent processing stages based on the topic confidence level.

[0115] Understandably, this fallback mechanism ensures that the system can still operate stably even with insufficient information, and reduces false or missed suggestions through genre awareness or confidence self-adaptation, thereby improving the robustness of the system.

[0116] In this embodiment of the application, a weighted topic degree vector is generated based on the topic degree weight of words, including: adjusting the weights of title topic words and body topic words according to word frequency, topic degree value and credibility of extended sources respectively.

[0117] Specifically, during implementation, initial weights are calculated for title and body text words respectively. The initial weights are jointly determined by the topic degree value of the words and their frequency of occurrence in the current text. Extended words from the extended knowledge base are discounted or gained based on their source confidence to reflect their credibility. Words with high confidence can have their weights appropriately increased, while words with low confidence or suspected noise are attenuated. After weighting, the vector elements are normalized as a whole to eliminate the influence of length and word count differences on similarity calculation, and finally, a weighted topic degree vector for cosine similarity calculation is generated.

[0118] Understandably, by incorporating word frequency, corpus-level topic tendency, and extended word source credibility into the weighting mechanism, the constructed vector can reflect both the topic attributes of words in the global corpus and their actual importance and credibility in the current text, thus making the similarity measurement more reflective of the actual meaning matching relationship.

[0119] This application provides a method for extracting thematic keywords from essay text based on dynamic rules. This method aims to overcome the problems of strong subjectivity and poor adaptability caused by the need for manually pre-setting the number of thematic keywords in traditional topic models (such as LDA). It designs multiple extraction strategies that are adaptively correlated with essay length and conducts combined experiments with thematic keyword weight coefficients to find the optimal thematic keyword extraction scheme in a data-driven manner.

[0120] Specifically, this embodiment designs two key parameters for system optimization: the number of keywords extracted from the main text and the keyword weight coefficient. Regarding the number of keywords extracted from the main text, considering its influence on the total number of words in the text, four dynamic extraction rules are set: (1) extract one-third of the total number of words in the text; (2) extract half of the total number of words in the text; (3) based on the extraction of two keywords, extract one more keyword for every 10 additional words in the essay; (4) based on the extraction of two keywords, extract one more keyword for every 100 additional words in the essay. Meanwhile, to examine the impact of differences in keyword importance on the results, three different keyword weight coefficients of 0.8, 0.5, and 0.33 are set. Furthermore, to compare and verify the effect of title keyword expansion, a baseline case where title keywords are not expanded is also set. Finally, all parameters are combined to form 24 complete parameter combination schemes as shown in Table 3 for system evaluation and optimization.

[0121]

[0122] Table 3. Keyword Parameter Combination Scheme

[0123] The following describes the specific training process of the title conformity assessment method provided in this application using several exemplary embodiments.

[0124] Example 1 (Preferred Example):

[0125] In this embodiment, please refer to the following: Figure 2 , Figure 2This is a flowchart of the essay topic conformity level calculation provided in one embodiment of this application. As shown in the figure, firstly, a word topic degree calculation model is trained using a large-scale classification corpus divided into 15 major categories and 244 subcategories, and this model is deployed as an offline resource in the background of the automatic essay scoring system. For each essay to be evaluated, after preprocessing (word segmentation, new word identification, stop word filtering), the system extracts candidate keywords from the title based on the keyword extraction algorithm and determines several title topic words by combining their topic degree values ​​in the corpus and their grammatical positions in the title. Then, these title topic words are retrieved and expanded in the extended knowledge base to obtain an expanded set of title topics (the source and confidence labels of the expanded words are retained for subsequent weighting). The extraction of topic words in the main text adopts the "one-third of the total number of words" rule and combines word frequency and topic degree. The search algorithm sorts the keywords to output the main text keywords. Then, it constructs weighted topic degree vectors for the title (including extended terms) and the main text keywords (weights are determined and normalized by word frequency, corpus topic degree, and credibility of extended sources). The cosine similarity between the two vectors is used as the title-text relevance score. In the threshold determination stage, a dynamic search algorithm based on the validation set (aiming to maximize the weighted F1 score) is used to select the level mapping threshold. In this embodiment, the optimal combination of "title extension + one-third full-text extraction + weight coefficient 0.8" was used in the experiment. The weighted F1 scores on the test set reached approximately 0.8236 (test set 1) and 0.7738 (test set 2), significantly better than the baseline without extension, indicating that this configuration has good generalization ability in both the same and new question scenarios.

[0126] This application provides a method for calculating the title-text relevance score based on a weighted topic degree vector. The core of this method lies in converting the expanded set of title keywords and the extracted set of body keywords into numerical vectors based on the topic degree weights of the words, and then quantifying the degree of topical relevance between the body text and the title by calculating the cosine similarity between these two vectors.

[0127] Specifically, the calculation process is as follows: First, all title keywords after the knowledge base expansion are considered as a set ti = (t1, t2, ..., tn), and all text keywords extracted according to dynamic rules are considered as another set w. j =(w1, w2,…, w n Next, based on the topic weights of each word calculated in the preceding steps, the two word sets are converted into corresponding numerical vectors. Each dimension of the vector corresponds to a word, and its value is the topic weight of that word. Finally, the cosine similarity formula is applied to calculate the similarity between the two vectors. The formula is:

[0128]

[0129] in, and These are the title vector and the body vector at the [number]th [position]. The value on the dimension (i.e., the topic weight of the corresponding word). The calculated similarity score. This is the final score for the relevance of the text to the topic, with a range of [0,1]. A higher value indicates a higher relevance.

[0130] Understandably, this embodiment achieves an effective mapping from a discrete set of words to a continuous semantic space by vectorizing the keywords and their weights and using cosine similarity for measurement. This method goes beyond simple word matching, capturing the similarity between the title and the body text in the macro-topic distribution from the perspective of directional consistency in the vector space. This provides a deeper and more accurate reflection of whether the essay content revolves around the topic, offering a reliable and interpretable quantitative basis for the automatic scoring system.

[0131] Example 2 (robust variant, long text priority coverage):

[0132] In this embodiment, the overall process is the same as in Embodiment 1, but the extraction rules for the main text keywords are adjusted: when the target essay is determined to be a medium-to-long text, "half of the total number of words" is used as the main text keyword extraction strategy to expand the scope of topic coverage; the title is still expanded using an extended knowledge base, and the vector construction and threshold selection strategies remain the same as in Embodiment 1; experiments show that compared with Embodiment 1, this variant can improve recall and maintain a high weighted F1 (with little overall fluctuation) on long texts or essays with relatively scattered topics, so it is suitable for automatic scoring scenarios for advanced students or long-form writing training.

[0133] Example 3 (Control / Baseline Example: Without Title Extension):

[0134] In this embodiment, the system extracts topic words directly from the title without expansion. The topic word extraction in the main text still uses "one-third of the total number of words in the text". The remaining steps (vector weighting, cosine similarity, threshold search) are consistent with Embodiment 1. This baseline scheme is used as a control group in the experiment. Its weighted F1 on test set 1 and test set 2 is about 0.6185 and 0.6511, respectively, which is significantly lower than the scheme using title expansion. This shows that the topic-related word expansion library built based on the essay corpus has a significant effect on improving the coverage of the topic and the calculation accuracy.

[0135] Example 4 (Enhanced Example of Genre Adaptation and Misjudgment Mitigation):

[0136] This embodiment adds genre recognition, multi-cluster matching, and metaphor / rhetoric recognition modules to the existing embodiment 1. The system first determines the genre of the essay (e.g., narrative, character description, scenery description, object description), selecting different dynamic extraction parameter sets based on the genre. If the text displays multiple thematic features, cluster-level multi-vector matching is used, and inter-cluster similarity is fused to obtain a more robust final similarity score. For numerous metaphorical or personified rhetorical expressions, the system can enable language pattern detection rules to reduce the probability of misjudgment caused by metaphors. During the threshold training phase, independent threshold searches can be performed by genre to account for differences in theme aggregation across genres. After training and testing by genre, the weighted F1 scores for each narrative genre (narrative / character description / scenery description / object description) under the optimal parameter combination are more balanced, indicating that genre adaptation helps improve the consistency and generalization ability of different essay types.

[0137] Specifically, this embodiment uses narrative essays written by elementary school students in Chinese as the experimental subject, and rigorously constructs a standard set and a test set. The standard set is used for parameter optimization and threshold determination, and its construction process is as follows: Narrative essays written by elementary school students (grades 1-6) whose native language is Chinese were selected from a Chinese essay website, with the word count controlled between 400-800 words. According to the four content types of narrative essays (narrative, character description, scenery description, and object description), three different topics were selected for each type, and 10 essays with the same title were selected under each topic, for a total of 120 essays. The topic selection takes into account the divergence of the topics, including both highly divergent topics such as "The Person I Admire Most" and low-divergence topics such as "Cactus," to improve the generalization ability of the method. Subsequently, five essay scoring experts conducted manual evaluations according to the unified standards shown in the table below. The consistency among evaluators was 0.82, and the inconsistencies were resolved through expert discussion to determine the final grade. The distribution of the standard set grades is shown in Table 4.

[0138]

[0139] Table 4: Manual Scoring Criteria for Title Relevance Levels

[0140] The standard set of essays has the following level of matching accuracy to the topic: 72 Level 3 essays, 40 Level 2 essays, and 8 Level 1 essays. The test set is used to assess the generalization ability of the method. Two test sets were constructed: Test Set 1 contains new essays with titles already covered in the standard set (new essays on the same topic), with 5 essays selected for each title, for a total of 60 essays; Test Set 2 contains essays on new topics not covered in the standard set (new essays on new topics), with 3 new topics selected from each of the 4 content types, and 5 essays with the same title selected for each topic, for a total of 60 essays.

[0141] It is understandable that this embodiment provides a reliable and comprehensive experimental basis for the performance verification of the text-topic conformity assessment method by constructing a standardized dataset that combines different content types, topic divergence, and topic novelty, and supplementing it with highly consistent human evaluation results as the gold standard, thus ensuring the scientificity and credibility of the evaluation results.

[0142] Specifically, this embodiment uses precision, recall, and F1 score to perform fine-grained evaluation for each level. Meanwhile, to avoid evaluation bias caused by uneven data distribution across levels in the standard set, weighted precision, weighted recall, and weighted F1 score are used to perform an overall evaluation of the entire dataset. The formula for calculating the weighted F1 score is as follows:

[0143]

[0144] Among them, weight It is determined by the number of real samples at each level.

[0145] Specifically, please refer to the following: Figure 3 , Figure 3 This is a schematic diagram illustrating the title-subject compliance calculation results of various schemes provided in an embodiment of this application. To examine the calculation effect of each parameter combination scheme, this embodiment uses two indicators for analysis: the Spearman rank correlation coefficient and the percentage of title-subject compliance calculation results of 0. The Spearman rank correlation coefficient is used to measure the correlation between the calculation results of each scheme and human evaluation; a higher coefficient indicates that the scheme's effect is closer to human scoring. The percentage of title-subject compliance of 0 is used to assess the stability of the scheme; an excessively high percentage indicates insufficient credibility of the scheme. Figure 3 The results show that, with the same number of extracted keywords from the main text, the smaller the keyword weight coefficient (e.g., 0.33), the lower the Spearman rank correlation coefficient. When the number of extracted keywords is "2 + 1 for every additional 100 words" (corresponding to schemes 4, 8, and 12), the proportion of schemes with a title-text matching degree of 0 is significantly higher than other schemes, indicating poor stability of this extraction strategy. Without expanding the title keywords, the fewer the number of extracted keywords from the main text and the smaller the keyword weight coefficient, the worse the calculation results; more than half of the schemes have a title-text matching degree of 0 with a proportion higher than 0.2. Based on this analysis, this embodiment selects parameter combination schemes with a Spearman rank correlation coefficient greater than 0.5 and a title-text matching degree of 0 with a proportion less than 0.1, selecting a total of 12 high-quality combination schemes to proceed to the next stage of threshold selection.

[0146] Understandably, this embodiment effectively eliminates parameter combinations that may perform well in certain specific situations but lack stability and generalization through a dual-index screening mechanism. This ensures that subsequent threshold selection and model optimization are based on a stable and reliable calculation scheme, thereby improving the robustness and practicality of the text-topic conformity assessment method as a whole.

[0147] Specifically, please refer to Figure 4 , Figure 4 This is a schematic diagram of the optimal threshold selection result provided in an embodiment of this application. Taking Scheme 1 as an example, the threshold selection process is illustrated as follows: First, the threshold search range is determined. The candidate threshold interval between Level 1 and Level 2 is [0.1, 0.11], and the candidate threshold interval between Level 2 and Level 3 is [0.28, 0.32]. Then, the thresholds of the two intervals are combined to obtain 10 different threshold selection schemes. By calculating the precision, recall, F1 score, and weighted F1 score corresponding to each scheme, it is found that the weighted F1 score reaches its maximum when the thresholds for Level 1 and Level 2 are 0.1 and the thresholds for Level 2 and Level 3 are 0.3. Therefore, the level thresholds of Scheme 1 are determined as follows: Level 1 [0, 0.1], Level 2 (0.1, 0.3), and Level 3 [0.3, 1]. The thresholds of the 12 parameter combination schemes selected are optimized according to this method to obtain the optimal thresholds and corresponding weighted F1 scores of each scheme. The specific results are shown in Table 5.

[0148]

[0149] Table 5: Optimal Threshold Selection Results for Each Parameter Combination Scheme

[0150] Based on the above metrics, this embodiment compares three methods: 1) the title keyword expansion method proposed in this application (selecting the optimal parameter combination); 2) the baseline method without title keyword expansion; and 3) the ChatGPT4 model. The core experimental results (weighted F1 scores) on the standard set and two test sets are summarized in Table 6.

[0151]

[0152] Table 6: Experimental Results of Calculating Title-Subject Relevance Level

[0153] This embodiment demonstrates through rigorous experimental comparison that the weighted F1 score of the proposed method on both test sets is significantly higher than that of the baseline method and the ChatGPT4 model. This fully proves the effectiveness of the title keyword expansion strategy, dynamic keyword extraction rules, and dynamic programming-based threshold selection mechanism adopted in this method. Especially noteworthy is its continued excellent performance on new essay topics (Test Set 2), indicating that the method has strong generalization ability and can adapt to unseen new topics, providing reliable technical support for the practical deployment of automatic essay scoring systems.

[0154] Figure 5 This is a schematic diagram of a module of an automatic essay scoring system provided in an embodiment of this application, as shown below. Figure 1The automatic essay scoring system 10 shown includes at least the following components: essay acquisition module 11, first extraction module 12, second extraction module 13, analysis module 14, and grade output module 15.

[0155] In this embodiment, the essay acquisition module 11 is used to acquire the target essay, extract title keywords from the title of the target essay using a keyword extraction algorithm, and calculate the topic weight of words in the body of the target essay. For details on the acquisition method, please refer to [link / reference]. Figures 1 to 4 The details and their corresponding descriptions are not repeated here.

[0156] In this embodiment, the first extraction module 12 is used to expand the title keywords with related terms based on a preset extended knowledge base of essay keywords. The extended knowledge base is generated by keyword indexing based on the target essay corpus and word clustering. Please refer to the specific extraction method for details. Figures 1 to 4 The details and their corresponding descriptions are not repeated here.

[0157] In this embodiment, the second extraction module 13 is used to extract the main theme words of the target essay according to the total word count of the essay and a preset dynamic rule. Please refer to the following for the specific extraction method. Figures 1 to 4 The details and their corresponding descriptions are not repeated here.

[0158] In this embodiment, the analysis module 14 is used to generate a weighted topic degree vector based on the topic degree weight of the words, and calculate the cosine similarity between the topic words of the title and the topic words of the essay body to obtain the title-text relevance score; and select the optimal threshold from the candidate threshold range according to the dynamic search algorithm. For details of the analysis method, please refer to the relevant documentation. Figures 1 to 4 The details and their corresponding descriptions are not repeated here.

[0159] In this embodiment, the grade output module 15 is used to output the grade of the target essay's relevance to the topic based on the essay's relevance score and the optimal threshold. Please refer to the documentation for the specific output method. Figures 1 to 4 The details and their corresponding descriptions are not repeated here.

[0160] Figure 6 This is an electronic device 20 provided in one embodiment of this application. For example... Figure 6 As shown, the electronic device 20 includes at least the following components: a processor 21 and a memory 22.

[0161] In this embodiment, the memory 22 is used to store executable instructions of the processor 21, which, when configured to execute instructions, implement... Figure 1 The method for evaluating the relevance of the text to the title is shown.

[0162] In one embodiment of this application, the program operating in the electronic device 20 may be a program that controls a central processing unit (CPU) or similar device to achieve the functions of the above-described embodiments of the present invention (a program that enables the computer to function). The information processed by these devices is then temporarily stored in random access memory (RAM) during processing, and subsequently stored in various ROMs such as read-only memory (Flash ROM) and hard disk drives (HDDs), and read, corrected, and written by the CPU as needed.

[0163] It should be noted that a portion of the electronic device 20 described above can also be implemented using a computer. In this case, the program for implementing the control function can be recorded on a computer-readable recording medium, and the program recorded on the recording medium can be read into the computer and executed.

[0164] It should be noted that the term "computer" as used here refers to a computer built into electronic device 20, employing hardware including an operating system and peripheral devices. Furthermore, "computer-readable recording media" refers to removable media such as floppy disks, magneto-optical disks, ROMs, and CD-ROMs, as well as storage devices such as hard drives built into the computer.

[0165] Furthermore, a "computer-readable recording medium" can include: a medium that dynamically stores a program for a short period of time, such as a communication line used when transmitting a program via a network such as the Internet or a communication line such as a telephone line; or a medium that stores a program for a fixed period of time, such as volatile memory inside a computer that serves as a server or client in this case. In addition, the aforementioned program can be a program used to implement the above-mentioned functions, or it can be a program that can implement the above-mentioned functions by combining with programs already recorded in the computer.

[0166] Furthermore, the electronic device 20 in the above embodiments can also be implemented as an assembly (device group) composed of multiple devices. Each device constituting the device group can possess some or all of the functions or functional blocks of the electronic device 20 in the above embodiments. As a device group, it is sufficient to have all the functions or functional blocks of the electronic device 20.

[0167] It is understood that the title-content matching assessment method, system 10, electronic device 20, and storage medium provided in this application embodiment achieve accurate and stable assessment of the matching degree between essay content and topic by constructing a fine-grained multi-level evaluation system, employing a weighted adjustable word topic degree calculation model and a topic word extended knowledge base constructed based on a large-scale essay corpus, and combining a dynamic programming algorithm to automatically determine the optimal level threshold. Furthermore, this solution can achieve accurate title-content matching assessment under unsupervised conditions, without relying on sample essays for specific topics, thus maintaining stable performance even when facing new topics, demonstrating strong practical value and promising application prospects. Experimental results show that the weighted F1 score of this method on the test set is significantly higher than the baseline method and the ChatGPT4 model, fully verifying its effectiveness and superiority. Finally, this solution not only provides a reliable content assessment dimension for automatic essay scoring systems, but its constructed topic word extended knowledge base can also provide rich teaching resources for topic-based writing instruction, possessing significant application value and broad application prospects.

[0168] Those skilled in the art should recognize that the above embodiments are only used to illustrate this application and are not intended to limit this application. Any appropriate changes and variations made to the above embodiments within the essential spirit and scope of this application fall within the scope of protection claimed in this application.

Claims

1. A method for evaluating the relevance of a text to a topic, applied to an automatic essay scoring system, characterized in that, The method includes: Obtain the target essay and extract title keywords from its title using a keyword extraction algorithm; The title keywords are expanded with related terms based on a pre-set extended knowledge base of essay keywords. The extended knowledge base is generated by keyword indexing based on the corpus of the target essay and word clustering. Based on a large-scale classification corpus, the topic degree of words in the corpus is calculated using a word topic degree calculation model. The topic degree of a word is calculated using the following formula: ； in, , Is class The sum of the frequencies of all the words contained. It is the first The word in the category Number of times it appears; ,in Number of categories; This indicates the words that appear in the training corpus. Number of times, It is the sum of the occurrences of all words in the training corpus; Based on the total word count of the target essay, the main theme words of the target essay are extracted according to preset dynamic rules. The dynamic rules include: in the case of short text, a number of high-confidence words are selected according to the priority of topic and grammatical position; in the case of medium and long text, the extraction threshold is set according to a preset ratio in combination with word frequency and topic. A weighted topic degree vector is generated based on the topic degree weight of the words, and the cosine similarity between the topic words of the title and the topic words of the target essay is calculated to obtain the title-subject relevance score. The step of generating a weighted topic degree vector based on word topic degree weights includes: The weights of the title keywords and the body keywords are adjusted according to word frequency, topicality value, and credibility of extended sources, respectively. The optimal threshold is selected from the candidate threshold range based on a dynamic search algorithm; Based on the title relevance score and the optimal threshold, the title relevance level of the target essay is output.

2. The title-subject conformity assessment method according to claim 1, characterized in that, The extraction of title keywords includes: The title text of the target essay is segmented, new words are identified, and stop words are filtered to form a candidate word set; The candidate words in the candidate word set are ranked by importance based on their topicality, and the title theme words are selected based on the grammatical position and representational ability of the candidate words in the essay title.

3. The title-subject conformity assessment method according to claim 2, characterized in that, The method further includes: Keyword indexing of large-scale essay corpora; Clustering is performed based on the co-occurrence relationship, semantic similarity, and semantic distance of words in essays on the same topic. The obtained topic-related word clusters are then screened, merged, and denoised through clustering quality assessment. The topic-related word clusters are sorted according to the thematic representation strength of the words to form an essay topic-related thesaurus for expanding the title topic words.

4. The title-subject conformity assessment method according to claim 3, characterized in that, The dynamic rules also include: When the length of the target essay cannot be determined, a fixed number of keywords are extracted, and predefined extraction parameters are selected based on the genre of the target essay.

5. An automatic essay scoring system, applied to implement the essay-topic relevance assessment method as described in any one of claims 1 to 4, characterized in that, The system includes: The essay acquisition module is used to acquire target essays, extract title keywords from the title of the target essays using a keyword extraction algorithm, and calculate the topic weight of words in the body of the target essays. The first extraction module is used to expand the title topic words with topic-related words according to the preset extended knowledge base of essay topic words. The extended knowledge base is generated by keyword indexing based on the corpus of the target essay and word clustering. The second extraction module is used to extract the main theme words of the target essay according to the word count of the full text and a preset dynamic rule. The analysis module is used to generate a weighted topic degree vector based on the topic degree weight of words, and to calculate the cosine similarity between the topic words of the title and the topic words of the essay body to obtain the title-text relevance score; and to select the optimal threshold from the candidate threshold range according to the dynamic search algorithm. The grade output module is used to output the grade of the target essay's relevance to the topic based on the essay's relevance score and the optimal threshold.

6. An electronic device, characterized in that, include: processor; Memory used to store processor-executable instructions; The processor is configured to implement the title compliance evaluation method according to any one of claims 1 to 4 when executing the instructions.

7. A computer-readable storage medium, characterized in that, The instruction includes a command that instructs the device to perform the title compliance assessment method as described in any one of claims 1 to 4.

Images