Composition scoring method, device, computer equipment and computer readable storage medium
By using feature extraction and a random forest model to score essays, the problem of incomplete evaluation in traditional systems is solved, and more accurate essay scoring is achieved.
Patent Information
- Application Number
- CN202010687313.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2020-07-16
- Publication Date
- 2024-11-26
- Estimated Expiration
- 2040-07-16
AI Technical Summary
Existing automatic essay scoring systems suffer from low accuracy due to incomplete evaluation.
We employ a feature extraction and random forest approach to obtain the language, logic, and comprehension features of the essays to be scored. We then use a pre-trained random forest model to score the essays, covering features such as vocabulary, grammar, syntax, length, readability, essay structure, and comparison with high-quality essays.
It improves the accuracy of automatic essay scoring, providing explanations and evaluations for different feature dimensions, and offering comprehensive scoring results.
Smart Images

Figure CN111832281B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of artificial intelligence technology, and in particular to a composition scoring method, apparatus, computer equipment and computer-readable storage medium based on feature extraction and random forest. Background Art
[0002] Automated Essay Scoring (AES) has been widely used in recent years for scenarios such as teacher essay marking, student self-assessment, parent-assisted inspection, and automatic exam scoring. The most widely used systems for automatic essay scoring include E-rater, Project Essay Grader, Intelligent Essay Assessor, IntelliMetric, BiGai.com, and Tencent Cloud Essay Grading.
[0003] The automatic essay scoring systems currently used in industry mainly use linear regression modeling to predict scores, and select a dozen or even more indicators with the highest correlation with essay scores according to different test types to obtain the final score. However, in traditional technologies, each automatic essay scoring system uses different indicators, resulting in each automatic essay scoring system focusing more on certain features, resulting in low accuracy in scoring essays. Summary of the invention
[0004] The embodiments of the present application provide a composition scoring method, apparatus, computer device and computer-readable storage medium based on feature extraction and random forest, which can solve the problem of low scoring accuracy due to incomplete evaluation in traditional automatic scoring of compositions.
[0005] In a first aspect, an embodiment of the present application provides a composition scoring method based on feature extraction and random forest, the method comprising: obtaining a composition to be scored; extracting target features of the composition according to a preset feature extraction method, the target features comprising preset basic language features, preset logical features and preset comprehension features, the preset basic language features comprising preset vocabulary features, preset grammatical features, preset syntactic features, preset length features and preset readability features, the preset logical features comprising preset article structure features, preset article coherence features and preset article element features, the preset comprehension features comprising similarity features with preset articles; inputting the target features into a pre-trained preset random forest model for scoring to obtain a score corresponding to the composition.
[0006] In the second aspect, the embodiment of the present application also provides a composition scoring device based on feature extraction and random forest, including: an acquisition unit, used to obtain a composition to be scored; an extraction unit, used to extract target features of the composition according to a preset feature extraction method, the target features include preset basic language features, preset logical features and preset comprehension features, the preset basic language features include preset vocabulary features, preset grammatical features, preset syntactic features, preset length features and preset readability features, the preset logical features include preset article structure features, preset article coherence features and preset article element features, the preset comprehension features include similarity features with preset articles; a scoring unit, used to input the target features into a pre-trained preset random forest model for scoring, so as to obtain a score corresponding to the composition.
[0007] In a third aspect, an embodiment of the present application further provides a computer device, which includes a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the steps of the composition scoring method based on feature extraction and random forest are implemented.
[0008] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the processor executes the steps of the composition scoring method based on feature extraction and random forest.
[0009] The embodiment of the present application provides a composition scoring method, device, computer equipment and computer-readable storage medium based on feature extraction and random forest. The embodiment of the present application obtains the composition to be scored, extracts the target features of the composition according to the preset feature extraction method, and the target features include preset language basic features, preset logical features and preset comprehension features. The target features are input into the pre-trained preset random forest model for scoring to obtain the score corresponding to the composition. Since the embodiment of the present application implements feature engineering from the three dimensions of language basic ability, logical ability and comprehension, it covers vocabulary, grammar, syntax, composition length, readability, organization and structure of the article, comparison with high-quality articles and other multi-faceted and comprehensive features, thereby extracting comprehensive feature indicators with evaluation significance, and can give explanations and evaluations for the scoring results and different feature dimensions, which can improve the accuracy of the results obtained by automatic composition scoring. BRIEF DESCRIPTION OF THE DRAWINGS
[0010] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.
[0011] Figure 1 A schematic diagram of a flow chart of a composition scoring method based on feature extraction and random forest provided in an embodiment of the present application;
[0012] Figure 2 A schematic diagram of a sub-process in the composition scoring method based on feature extraction and random forest provided in an embodiment of the present application;
[0013] Figure 3 Another sub-process diagram of the composition scoring method based on feature extraction and random forest provided in an embodiment of the present application;
[0014] Figure 4 A schematic block diagram of a composition scoring device based on feature extraction and random forest provided in an embodiment of the present application; and
[0015] Figure 5 A schematic block diagram of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0016] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.
[0017] It should be understood that when used in this specification and the appended claims, the terms "include" and "comprises" indicate the presence of described features, integers, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or combinations thereof.
[0018] See also Figure 1 , Figure 1 A flow chart of a composition scoring method based on feature extraction and random forest provided in an embodiment of the present application. Figure 1 As shown, the method includes the following steps S101-S103:
[0019] S101. Obtain the composition to be graded.
[0020] S102. Extract target features of the composition according to a preset feature extraction method, wherein the target features include preset basic language features, preset logical features and preset comprehension features, the preset basic language features include preset vocabulary features, preset grammar features, preset syntax features, preset length features and preset readability features, the preset logical features include preset article structure features, preset article coherence features and preset article element features, and the preset comprehension features include similarity features with preset articles.
[0021] Specifically, since the basic language ability of an essay includes characteristic indicators such as vocabulary, grammar, syntax, length and readability of the essay, the logical ability of the essay involves characteristic indicators such as the organization and structure of the essay, and the comprehension of the essay includes characteristic indicators such as the comparison between the essay and high-quality articles and the degree of off-topic, the automatic essay scoring method based on feature extraction and random forest provided in the embodiment of the present application implements feature engineering from three dimensions: basic language features (used to describe the basic language ability of the essay), logical features (used to describe the logical ability of the essay), and comprehension features (used to describe the comprehension of the essay), and extracts feature indicators contained in the above three dimensions that have evaluation significance for the essay, which are used to automatically score the essay.
[0022] Before automatically scoring the composition, the composition corpus data is first preprocessed to classify the composition corpus and divide it into different grouping categories. The composition corpus can be divided into different corpus data sets according to the compositions of authors of different age groups, such as high school group, junior high school group and elementary school group. In each group, the composition with too many words or too few words is eliminated. Then the composition is divided into K composition themes according to the theme, and the composition automatic scoring model based on the random forest model is trained for the compositions of different age groups and different themes. Among them, the feature engineering involved in feature extraction of the composition includes the following:
[0023] (1) Language basic features, which are used to describe the language basic ability of the composition, including preset vocabulary features, preset grammatical features, preset syntactic features, preset length features and preset readability features. Specifically, they are divided into the following features:
[0024] 1) Vocabulary, described by vocabulary features, including the following sub-features:
[0025] ① The proportion of Unigram words (characters) is used to describe the richness of vocabulary. It is the ratio of the number of Unigram words to the total number of words in the whole text. Among them, repeated words are counted as only one. Unigram is a concept in natural language processing (NLP) and is used to describe the unigram model. For example, the Unigram form corresponding to Xi'an Jiaotong University is: Xi / An / Jiao / Tong / Da / Xue.
[0026] ② Average number of strokes, which is used to describe the average number of strokes in a single word and can be obtained by counting the strokes.
[0027] ③The standard deviation of the number of strokes is used to describe the variance of the number of strokes in a single word.
[0028] ④The number of word categories (with or without punctuation), which is used to describe the number of Unigram words.
[0029] ⑤The percentage of words with more than n strokes (n=6, 7, 8,...).
[0030] ⑥Mechanics error rate, which is used to describe the percentage of spelling errors, punctuation errors, etc. in the total number of words.
[0031] ⑦ Vocabulary level index, by marking the vocabulary difficulty level of each word, and selecting the level value of the fifth word with the lowest index (i.e. feature) in the composition as the feature;
[0032] ⑧The number of idioms used is used to describe the number of idioms used in the composition.
[0033] 2) Grammar and syntax, which are described by preset grammatical features and preset syntactic features, including the following feature indicators:
[0034] ① The proportion of part-of-speech tags, such as the percentage of nouns, verbs, adjectives, pronouns, loanwords, conjunctions, and function words in the total number of words. You can use the preset language tool to perform word segmentation, then perform part-of-speech tagging, and then calculate the proportion of part-of-speech tags.
[0035] ②Grammar error rate: It is used to describe the percentage of grammatical errors (parts of speech, phrases, etc.) in the total number of words.
[0036] ③ Overly repeated words: used to describe the percentage of overly repeated words or phrases used in the total number of words.
[0037] ④ Syntactic structure parsing tree, which describes the syntactic structure through indicators corresponding to the tree structure such as the depth of the tree.
[0038] ⑤ Clause relations, including the number of subject clauses, object clauses, attributive clauses, adverbial clauses, etc.
[0039] 3) The length of the essay is described by pre-set length characteristics, including the following:
[0040] ① Sentence length distribution, which is used to describe the distribution of the number of words in a sentence.
[0041] ②Average sentence length, used to describe the average number of words in a sentence.
[0042] ③Total number of clauses, used to describe the number of clauses used in the whole text.
[0043] ④Average number of sentences in a paragraph, used to describe the average number of sentences in each paragraph.
[0044] ⑤Total number of sentences, used to describe the number of sentences in the full text.
[0045] ⑥Number of paragraphs, used to describe the number of paragraphs in the full text.
[0046] ⑦ Number of words in the full text, used to describe the total number of words in the full text.
[0047] 4) Readability, described by pre-set readability features, including the following:
[0048] ① Readability index, which is used to describe the evaluation index of the readability of Chinese text. The calculation formula is: N×average number of Chinese strokes+(1-N)×frequency of Chinese difficult words, where N adjusts the weight ratio between the average number of Chinese strokes index and the frequency of Chinese difficult words index.
[0049] (2) Logical ability, which is described by preset logical features, including preset article structure features, preset article coherence features and preset article element features, including the following:
[0050] 1) The organization and structure of the article is described using the preset article structure features, including the following:
[0051] ① Article structure index: classify each sentence into one of the following categories: background, argument, main point, supporting point, and conclusion. Define the largest set of consecutive sentences of a certain category as an element of that category. For example, 3 main points + 4 supporting points + 2 main points correspond to 3 elements. Calculate the number of elements as the article structure index.
[0052] ② Article coherence, which is used to describe the overlap between adjacent sentences.
[0053] ③The average number of words in the article elements is used to describe the mean number of words in all elements.
[0054] (3) Composition comprehension, through the preset comprehension features, the preset comprehension features include similarity features with preset articles, including the following:
[0055] 1) Comparison between composition and high-quality articles.
[0056] ① Most similar article score: divide the corpus articles into n categories according to their scores, and take the score of the article category that is most similar to the article to be evaluated as the indicator.
[0057] ② Similarity with the best papers: the similarity between the article to be evaluated and the article category with the highest score.
[0058] 2) The degree to which the composition deviates from the topic.
[0059] ① Similarity compared with articles on the same topic: the similarity between the article to be evaluated and the category of articles on the same topic.
[0060] After obtaining the composition to be scored, the target features of the composition are extracted according to a preset feature extraction method. For each of the above-mentioned features, a corresponding extraction method can be set in advance for each feature. Then, when performing feature extraction, the above-mentioned features are extracted according to the preset feature extraction methods corresponding to the above-mentioned features, so as to obtain the target features of the composition. The target features include preset basic language features, preset logical features and preset comprehension features. The preset basic language features include preset vocabulary features, preset grammatical features, preset syntactic features, preset length features and preset readability features. The preset logical features include preset article structure features, preset article coherence features and preset article element features. The preset comprehension features include similarity features with preset articles. The automatic essay scoring method based on feature extraction and random forest provided in the embodiment of the present application implements feature engineering from the three dimensions of basic language ability, logical ability, and comprehension, targeting the dimensions and features used for automatic scoring. It covers features such as vocabulary, grammar / syntax, essay length, readability, article organization and structure, comparison with high-quality articles, and degree of off-topicness. It extracts indicators with evaluation significance, can give explanations and evaluations for the scoring results and different feature dimensions, and the automatic essay scoring model has high interpretability.
[0061] S103, inputting the target feature into a pre-trained preset random forest model for scoring to obtain a score corresponding to the composition.
[0062] Among them, Random Forest, in English, is a classifier that uses multiple trees to train and predict samples.
[0063] Specifically, first, a preset random forest model for automatic scoring of compositions is pre-trained through pre-training composition samples, wherein the pre-training composition samples include pre-training compositions and pre-training composition scores corresponding to scoring the pre-training compositions, wherein the scoring of the pre-training composition samples to obtain the pre-training composition scores can be performed manually, wherein the pre-training composition scores can also be the composition score average of the pre-training composition scores obtained by scoring the pre-training compositions multiple times. For example, if the pre-training composition is scored manually, the pre-training composition score can be the composition score average calculated based on the multiple pre-training composition scores obtained after multiple different manual workers each score the pre-training composition based on the target features extracted by the preset random forest model as the composition scoring index, so that the manual scoring of the pre-training composition is as objective and comprehensive as possible. Since the accuracy of the pre-training composition sample data is improved, the trained preset random forest model is also more objective, and the accuracy of the automatic scoring of the composition by the preset random forest model can also be improved. The process of pre-training the preset random forest model includes the following steps:
[0064] Acquire a pre-training composition sample, wherein the pre-training composition sample includes a pre-training composition and a composition score corresponding to the pre-training composition;
[0065] Inputting the pre-trained composition sample into a preset random forest model;
[0066] The preset target features of the pre-trained composition and the composition score are extracted for learning, and the pre-trained composition samples are classified to realize pre-training of the preset random forest model.
[0067] Specifically, since the random forest model is an ensemble learning algorithm based on decision trees as the base learner, it is a classifier that uses multiple trees to train and predict samples. Therefore, when pre-training the preset random forest model, the preset random forest model will extract the preset target features and composition scores of the pre-training composition for learning, and classify the pre-training composition samples according to the extracted preset target features and composition scores of the pre-training composition to obtain different composition categories, thereby realizing the pre-training preset random forest model. Subsequently, when the preset random forest model automatically scores the composition to be scored, it also first extracts the preset target features of the composition to be scored, and classifies the composition to be scored according to the learning results automatically learned during pre-training based on the extracted preset target features of the composition to be scored, that is, the extracted preset target features of the composition to be scored are compared with the preset target features learned for each composition category when classifying the pre-trained composition during pre-training, and the composition to be scored is classified into the composition category with the highest similarity, and the most similar composition to be scored is obtained through the obtained similarity, and the composition score of the most similar composition is used as the basis for scoring the composition to be scored, and the composition score of the composition to be scored is obtained according to the similarity between the composition to be scored and the most similar composition, so as to realize automatic scoring of the composition to be scored.
[0068] Furthermore, when pre-training the preset random forest model, different target groups can also be classified to train different preset random forest models. For example, when pre-processing the composition corpus data to classify the composition corpus and divide it into different grouping categories, the composition corpus can be divided into different corpus data sets according to the compositions of authors of different age groups, for example, divided into different grouping categories such as high school group, junior high school group and elementary school group, etc., and the preset random forest model is trained using the composition corpus and the score corresponding to each composition, so that the preset random forest model can automatically score the composition.
[0069] Furthermore, when training the random forest model, QWK (Quadratic Weighted Kappa), a statistical indicator widely used to measure the consistency of ordered scale evaluation, is used as an evaluation indicator of the automatic scoring model. At the same time, three commonly used indicators, Correlation (measuring the trend consistency between the model prediction results and the actual results), Accuracy (the accuracy of the model prediction results), and RMSE (Root Mean Squared Error), are added as auxiliary output indicator values to debug the random forest model to provide the accuracy of the random forest model in automatic scoring of essays.
[0070] After the pre-training of the preset random forest model is completed and the preset random forest model meets the predetermined goal, the training of the preset random forest model is completed, the composition to be scored is obtained, and after the target feature extraction of the composition is completed, the target feature is input into the random forest model, and the total score of the composition is evaluated through the above three dimensions.
[0071] The embodiment of the present application obtains a composition to be scored, extracts target features of the composition according to a preset feature extraction method, the target features including preset basic language features, preset logical features and preset comprehension features, and inputs the target features into a pre-trained preset random forest model for scoring to obtain a score corresponding to the composition. Since the embodiment of the present application implements feature engineering from the three dimensions of basic language ability, logical ability, and comprehension, it covers multi-faceted and comprehensive features such as vocabulary, grammar, syntax, composition length, readability, organization and structure of the article, and comparison with high-quality articles, thereby extracting comprehensive feature indicators with evaluation significance, and can give explanations and evaluations for the scoring results and different feature dimensions, which can improve the accuracy of the results obtained by automatic composition scoring.
[0072] In one embodiment, the step of extracting the target features of the composition according to a preset feature extraction method includes:
[0073] Extracting the preset basic language features of the composition according to a preset basic language feature extraction method;
[0074] Extracting the preset logical features of the composition according to a preset logical feature extraction method;
[0075] The preset comprehension feature of the composition is extracted according to a preset comprehension feature extraction method.
[0076] Specifically, in order to improve the accuracy of scoring the composition in the embodiment of the present application, relatively comprehensive target features of three dimensions are adopted, and the target features include preset language basic features, preset logical features and preset comprehension features. For each feature of the composition, the corresponding preset feature extraction method is used for feature extraction, that is, the preset language basic features of the composition are extracted according to the preset language basic feature extraction method, the preset logical features of the composition are extracted according to the preset logical feature extraction method, and the preset comprehension features of the composition are extracted according to the preset comprehension feature extraction method, so as to obtain the target features corresponding to the composition, including the preset language basic features, preset logical features and preset comprehension features, and input the target features into the pre-trained preset random forest model for scoring to obtain the score corresponding to the composition, so as to automatically score the composition. Since comprehensive feature indicators with evaluation significance are extracted for the composition, the scoring results and different feature dimensions of the composition can be explained and evaluated through the random forest model, which can improve the accuracy of the results obtained by automatic scoring of the composition.
[0077] In one embodiment, the step of extracting the preset basic language features of the composition according to the preset basic language feature extraction method includes:
[0078] Using a first preset language tool to segment the composition to obtain phrases contained in the composition;
[0079] Based on the phrase, feature extraction is performed according to each preset target language basic sub-feature extraction method to obtain each target language basic sub-feature;
[0080] The obtained plurality of target language basic sub-features are grouped together to obtain the preset language basic feature.
[0081] The first preset language tool may be a language tool that supports word segmentation, such as Stanford Core NLP or HanLP.
[0082] Specifically, since the language tool (such as Stanford Core NLP or HanLP) supports NLP tasks including tokenization, shallow analysis (sentence segmentation), word segmentation, sentence segmentation, segmentation, part-of-speech tagging, named entity recognition and grammatical parsing, the input sentence can be segmented by the preset language tool to obtain phrase division, and then the phrases can be tagged with parts of speech, that is, the single sentence is segmented by the first preset language tool to obtain several phrases contained in the single sentence. Based on the phrases, according to the preset target language basic sub-features contained in the language basic features, according to the above description of the content corresponding to the sub-features, and according to each preset target language basic sub-feature extraction method, feature extraction is performed to obtain each target language basic sub-feature, and the obtained several target language basic sub-features are combined into a set to obtain the preset language basic feature. For example, if the preset target language basic sub-feature is the proportion of Unigram words (characters), the preset target language basic sub-feature is obtained by counting the ratio of the number of Unigram words to the total number of words in the full text; if the preset target language basic sub-feature is the average number of strokes, the target language basic sub-feature is obtained by counting the average number of strokes of a single word; for other preset target language basic sub-features, the corresponding preset target language basic sub-feature is extracted according to each preset target language basic sub-feature extraction method to obtain the preset target language basic sub-feature, and all the obtained target language basic sub-features are combined into a set to obtain the preset language basic feature.
[0083] In one embodiment, the step of extracting the preset logical features of the composition according to the preset logical feature extraction method includes:
[0084] Cutting the composition into a plurality of sentences according to punctuation marks contained in the composition;
[0085] Based on the sentences, the number of elements contained in the composition text corresponding to the preset article structure feature of the composition, the average number of words in all the elements corresponding to the preset article element feature, and the number of overlaps between adjacent sentences corresponding to the preset article coherence feature are counted, wherein the element is the background category, argument category, main point category, supporting point category or conclusion category contained in the composition, and the set of sentences of the largest consecutive occurrence category is called the element of that category.
[0086] Specifically, punctuation marks are generally used to divide sentences in a composition. Therefore, by identifying punctuation marks, the composition can be cut into several independent sentences according to the identified punctuation marks. Based on the above description of the preset article structure features and the sentences obtained by cutting, the number of elements contained in the composition text corresponding to the preset article structure features of the composition, the average number of words in all the elements corresponding to the preset article element features, and the number of overlaps between adjacent sentences corresponding to the preset article coherence features are counted. Among them, the element is the background class, argument class, main point class, supporting point class or conclusion class contained in the composition. The set of sentences of the largest consecutive occurrence of the category is called the element of the category, so that the preset logical features of the composition are extracted according to the preset logical feature extraction method. For example, classify each sentence into one of the following categories: background, argument, main point, supporting point, and conclusion. Define the maximum number of consecutively appearing sentences of a certain category as an element of that category. For example, 3 main points + 4 supporting points + 2 main points correspond to 3 elements. Calculate the number of elements as the composition structure indicator, count the overlap of adjacent sentences in the composition, and obtain the number of overlaps between adjacent sentences corresponding to the preset article coherence feature as the composition structure indicator. Count the average number of words in all the elements corresponding to the preset article element feature as the composition structure indicator, etc.
[0087] See also Figure 2 , Figure 2 A schematic diagram of a sub-process in the composition scoring method based on feature extraction and random forest provided in an embodiment of the present application. In this embodiment, the step of extracting the preset comprehension feature of the composition according to the preset comprehension feature extraction method includes:
[0088] S201, obtaining preset articles contained in a preset corpus.
[0089] S202: Compare the composition with the preset article according to preset characteristic indicators to obtain a preset similarity between the composition and the preset article.
[0090] The preset feature indicators include all or part of the sub-features of the preset language basic features, preset logic features and preset comprehension features.
[0091] Specifically, a preset article included in a preset corpus is obtained, and the preset feature indicators include all or part of the sub-features of a preset language basic feature, a preset logical feature, and a preset comprehension feature.
[0092] Please continue reading Figure 2 ,like Figure 2As shown, further, the preset articles are several articles, and after the step of comparing the composition with the preset articles according to preset feature indicators to obtain a preset similarity between the composition and the preset articles, it also includes:
[0093] S203, obtaining the most similar article with the smallest similarity score difference with the composition from the preset articles.
[0094] S204: Obtain a score index of an article belonging to the same article category as the most similar article as a comparison index.
[0095] S205. Based on the comparison index, the composition is compared with a preset target article with the highest score to obtain a similarity between the composition and the preset target article.
[0096] Specifically, the preset articles included in the preset corpus are several articles, each of which has a corresponding score. The several articles in the corpus can be divided into n categories according to the scores, and the most similar article with the smallest similarity score difference with the composition is obtained from the preset articles, and then the score index of the article category most similar to the article to be evaluated is obtained as the comparison index, and based on the comparison index, the composition is compared with the preset target article with the highest score to obtain the similarity between the composition and the preset target article, so as to compare the composition to be scored with the best quality paper to obtain the similarity between the composition and the best quality paper, so as to obtain the similarity between the composition to be evaluated and the article category with the highest score, thereby achieving a more accurate scoring of the composition.
[0097] See also Figure 3 , Figure 3 Another sub-flow diagram of the composition scoring method based on feature extraction and random forest provided in an embodiment of the present application. In this embodiment, the step of extracting the preset comprehension feature of the composition according to the preset comprehension feature extraction method also includes:
[0098] S301, obtaining the subject corresponding to the composition;
[0099] S302, obtaining articles with the same theme contained in the theme in the preset corpus, wherein the articles with the same theme are articles with the same theme as the composition;
[0100] S303: Compare the similarity between the composition and the article on the same subject, so as to obtain the category similarity between the category corresponding to the composition and the article on the same subject.
[0101] Specifically, the subject corresponding to the composition is obtained, for example, the subject of the composition is obtained from the title of the composition, and articles with the same subject contained in the subject in the preset corpus are obtained, wherein the articles with the same subject are articles with the same subject as the composition, and the composition is compared with the articles with the same subject for similarity to obtain the category similarity of the categories corresponding to the composition and the articles with the same subject, thereby realizing the comparison between the composition and the articles with the same subject, and the degree of off-topicness of the composition can be further judged through the category similarity obtained by comparing the composition with the articles with the same subject, thereby automatically scoring the composition from a more comprehensive dimension, thereby improving the accuracy of automatic scoring of the composition.
[0102] It should be noted that the composition scoring method based on feature extraction and random forest described in the above embodiments can recombine the technical features contained in different embodiments as needed to obtain a combined implementation plan, but they are all within the scope of protection required by this application.
[0103] See also Figure 4 , Figure 4 A schematic block diagram of a composition scoring device based on feature extraction and random forest provided in an embodiment of the present application. Corresponding to the composition scoring method based on feature extraction and random forest described above, an embodiment of the present application also provides a composition scoring device based on feature extraction and random forest. Figure 4 As shown, the composition scoring device based on feature extraction and random forest includes a unit for executing the composition scoring method based on feature extraction and random forest. The composition scoring device based on feature extraction and random forest can be configured in a computer device. Figure 4 The composition scoring device 400 based on feature extraction and random forest includes an acquisition unit 401, an extraction unit 402 and a scoring unit 403.
[0104] The acquisition unit 401 is used to acquire the composition to be graded;
[0105] The extraction unit 402 is used to extract the target features of the composition according to a preset feature extraction method, wherein the target features include preset basic language features, preset logical features, and preset comprehension features, wherein the preset basic language features include preset vocabulary features, preset grammar features, preset syntax features, preset length features, and preset readability features, wherein the preset logical features include preset article structure features, preset article coherence features, and preset article element features, and wherein the preset comprehension features include similarity features with preset articles;
[0106] The scoring unit 403 is used to input the target feature into a pre-trained preset random forest model for scoring to obtain a score corresponding to the composition.
[0107] In one embodiment, the extraction unit 402 includes:
[0108] A language basic feature extraction subunit, used for extracting the preset language basic features of the composition according to a preset language basic feature extraction method;
[0109] A logic feature extraction subunit, used for extracting the preset logic feature of the composition according to a preset logic feature extraction method;
[0110] The comprehension feature extraction subunit is used to extract the preset comprehension feature of the composition according to a preset comprehension feature extraction method.
[0111] In one embodiment, the language basic feature extraction subunit includes:
[0112] A word segmentation subunit, configured to segment the composition using a first preset language tool to obtain phrases contained in the composition;
[0113] A sub-feature extraction unit, configured to extract features based on the phrase according to each preset target language basic sub-feature extraction method to obtain each target language basic sub-feature;
[0114] A collection subunit, used for combining the obtained several target language basic sub-features into a collection to obtain the preset language basic feature;
[0115] In one embodiment, the logical feature extraction subunit includes:
[0116] A cutting subunit, used for cutting the composition into a plurality of sentences according to punctuation marks contained in the composition;
[0117] The statistical subunit is used to count the number of elements contained in the composition text corresponding to the preset article structure feature of the composition, the average number of words in all the elements corresponding to the preset article element feature, and the number of overlaps between adjacent sentences corresponding to the preset article coherence feature based on the sentences, wherein the element is the background class, argument class, main point class, supporting point class or conclusion class contained in the composition, and the set of sentences of the largest consecutive occurrence of the category is called the element of the category.
[0118] In one embodiment, the comprehension feature extraction subunit includes:
[0119] The acquisition subunit is used to acquire the preset articles contained in the preset corpus;
[0120] The comparison subunit is used to compare the composition with the preset article according to preset characteristic indicators to obtain a preset similarity between the composition and the preset article.
[0121] In one embodiment, the preset articles are several articles, and the comprehension feature extraction subunit further includes:
[0122] The most similar article obtaining subunit is used to obtain the most similar article with the smallest similarity score difference with the composition from the preset articles;
[0123] A comparison index obtaining unit, used to obtain a score index of an article belonging to the same article category as the most similar article as a comparison index;
[0124] The comparison subunit is used to compare the composition with a preset target article with the highest score based on the comparison index to obtain the similarity between the composition and the preset target article.
[0125] In one embodiment, the comprehension feature extraction subunit further includes:
[0126] A theme acquisition subunit, used to acquire the theme corresponding to the composition;
[0127] The same-theme article acquisition subunit is used to acquire the same-theme articles contained in the theme in the preset corpus, wherein the same-theme articles are articles with the same theme as the composition;
[0128] The category similarity acquisition subunit is used to compare the similarity between the composition and the article on the same subject, so as to obtain the category similarity between the category corresponding to the composition and the article on the same subject.
[0129] It should be noted that technical personnel in the relevant field can clearly understand that the specific implementation process of the above-mentioned composition scoring device based on feature extraction and random forest and each unit can refer to the corresponding description in the aforementioned method embodiment, and for the convenience and conciseness of the description, it will not be repeated here.
[0130] At the same time, the division and connection methods of each unit in the above-mentioned composition scoring device based on feature extraction and random forest are only used for illustration. In other embodiments, the composition scoring device based on feature extraction and random forest can be divided into different units as needed, and each unit in the composition scoring device based on feature extraction and random forest can also adopt a different connection order and method to complete all or part of the functions of the above-mentioned composition scoring device based on feature extraction and random forest.
[0131] The above-mentioned composition scoring device based on feature extraction and random forest can be implemented in the form of a computer program. The computer program can be used in Figure 5 Runs on the computer device shown.
[0132] See also Figure 5 , Figure 5 1 is a schematic block diagram of a computer device provided in an embodiment of the present application. The computer device 500 may be a computer device such as a desktop computer or a server, or may be a component or part in other devices.
[0133] See also Figure 5 The computer device 500 includes a processor 502 , a memory and a network interface 505 connected via a system bus 501 , wherein the memory may include a non-volatile storage medium 503 and an internal memory 504 .
[0134] The non-volatile storage medium 503 can store an operating system 5031 and a computer program 5032. When the computer program 5032 is executed, the processor 502 can execute the above-mentioned composition scoring method based on feature extraction and random forest.
[0135] The processor 502 is used to provide computing and control capabilities to support the operation of the entire computer device 500 .
[0136] The internal memory 504 provides an environment for the operation of the computer program 5032 in the non-volatile storage medium 503. When the computer program 5032 is executed by the processor 502, the processor 502 can execute the above-mentioned composition scoring method based on feature extraction and random forest.
[0137] The network interface 505 is used to communicate with other devices over the network. Figure 5 The structure shown in the figure is only a block diagram of a part of the structure related to the present application scheme, and does not constitute a limitation on the computer device 500 to which the present application scheme is applied. The specific computer device 500 may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components. For example, in some embodiments, the computer device may only include a memory and a processor. In such an embodiment, the structure and function of the memory and the processor are the same as those of the present application scheme. Figure 5 The embodiments shown are consistent and will not be described again here.
[0138] The processor 502 is used to run a computer program 5032 stored in a memory to implement the following steps: obtaining a composition to be scored; extracting target features of the composition according to a preset feature extraction method, wherein the target features include preset basic language features, preset logical features, and preset comprehension features, wherein the preset basic language features include preset vocabulary features, preset grammatical features, preset syntactic features, preset length features, and preset readability features, wherein the preset logical features include preset article structure features, preset article coherence features, and preset article element features, and the preset comprehension features include similarity features with preset articles; and inputting the target features into a pre-trained preset random forest model for scoring to obtain a score corresponding to the composition.
[0139] In one embodiment, when the processor 502 implements the step of extracting the target feature of the composition according to the preset feature extraction method, the processor 502 specifically implements the following steps:
[0140] Extracting the preset basic language features of the composition according to a preset basic language feature extraction method;
[0141] Extracting the preset logical features of the composition according to a preset logical feature extraction method;
[0142] The preset comprehension feature of the composition is extracted according to a preset comprehension feature extraction method.
[0143] In one embodiment, when the processor 502 implements the step of extracting the preset basic language features of the composition according to the preset basic language feature extraction method, the processor 502 specifically implements the following steps:
[0144] Using a first preset language tool to segment the composition to obtain phrases contained in the composition;
[0145] Based on the phrase, feature extraction is performed according to each preset target language basic sub-feature extraction method to obtain each target language basic sub-feature;
[0146] The obtained plurality of target language basic sub-features are grouped together to obtain the preset language basic feature.
[0147] In one embodiment, when the processor 502 implements the step of extracting the preset logical features of the composition according to the preset logical feature extraction method, the processor 502 specifically implements the following steps:
[0148] Cutting the composition into a plurality of sentences according to punctuation marks contained in the composition;
[0149] Based on the sentences, the number of elements contained in the composition text corresponding to the preset article structure feature of the composition, the average number of words in all the elements corresponding to the preset article element feature, and the number of overlaps between adjacent sentences corresponding to the preset article coherence feature are counted, wherein the element is the background category, argument category, main point category, supporting point category or conclusion category contained in the composition, and the set of sentences of the largest consecutive occurrence category is called the element of that category.
[0150] In one embodiment, when the processor 502 implements the step of extracting the preset comprehension feature of the composition according to the preset comprehension feature extraction method, the processor 502 specifically implements the following steps:
[0151] Get the preset articles contained in the preset corpus;
[0152] The composition is compared with the preset article according to preset characteristic indicators to obtain a preset similarity between the composition and the preset article.
[0153] In one embodiment, after implementing the step of comparing the composition with the preset articles according to preset feature indicators to obtain a preset similarity between the composition and the preset articles, the processor 502 further implements the following steps:
[0154] Obtaining the most similar article with the smallest similarity score difference with the composition from the preset articles;
[0155] Obtaining a score index of an article belonging to the same article category as the most similar article as a comparison index;
[0156] Based on the comparison index, the composition is compared with a preset target article with the highest score to obtain the similarity between the composition and the preset target article.
[0157] In one embodiment, when the processor 502 implements the step of extracting the preset comprehension feature of the composition according to the preset comprehension feature extraction method, the processor 502 further specifically implements the following steps:
[0158] Obtaining the subject corresponding to the composition;
[0159] Obtaining articles on the same subject as the subject in the preset corpus, wherein the articles on the same subject are articles on the same subject as the composition;
[0160] The composition is compared with the article on the same subject in terms of similarity to obtain the category similarity between the composition and the category corresponding to the article on the same subject.
[0161] It should be understood that in the embodiment of the present application, the processor 502 may be a central processing unit (CPU), and the processor 502 may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.
[0162] Those skilled in the art will appreciate that all or part of the processes in the method for implementing the above embodiment can be completed by a computer program, which can be stored in a computer-readable storage medium. The computer program is executed by at least one processor in the computer system to implement the process steps of the embodiment of the above method.
[0163] Therefore, the present application also provides a computer-readable storage medium. The computer-readable storage medium may be a non-volatile computer-readable storage medium, and the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the processor executes the following steps:
[0164] A computer program product, when running on a computer, enables the computer to execute the steps of the composition scoring method based on feature extraction and random forest described in the above embodiments.
[0165] The computer-readable storage medium may be an internal storage unit of the aforementioned device, such as a hard disk or memory of the device. The computer-readable storage medium may also be an external storage device of the device, such as a plug-in hard disk, a smart memory card (SmartMediaCard, SMC), a secure digital (SecureDigital, SD) card, a flash card (FlashCard), etc. equipped on the device. Furthermore, the computer-readable storage medium may also include both an internal storage unit of the device and an external storage device.
[0166] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described equipment, devices and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.
[0167] The storage medium is a physical, non-transient storage medium, for example, it can be a USB flash drive, a mobile hard disk, a read-only memory (ROM), a magnetic disk or an optical disk, etc., which can store computer programs.
[0168] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.
[0169] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of each unit is only a logical function division, and there may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed.
[0170] The steps in the method of the embodiment of the present application can be adjusted in order, combined and deleted according to actual needs. The units in the device of the embodiment of the present application can be combined, divided and deleted according to actual needs. In addition, the functional units in the various embodiments of the present application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.
[0171] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for an electronic device (which can be a personal computer, terminal, or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application.
[0172] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in the present application, and these modifications or replacements should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.
Claims
1. A composition scoring method based on feature extraction and random forest, characterized in that: The method comprises: Get the essays to be graded; Extracting target features of the composition according to a preset feature extraction method, wherein the target features include preset basic language features, preset logical features, and preset comprehension features, wherein the preset basic language features include preset vocabulary features, preset grammar features, preset syntax features, preset length features, and preset readability features, wherein the preset logical features include preset article structure features, preset article coherence features, and preset article element features, and wherein the preset comprehension features include similarity features with preset articles; The similarity features with the preset articles include: the comparison between the composition and the high-quality articles and the degree of deviation from the topic of the composition; Input the target feature into a pre-trained preset random forest model for scoring to obtain a score corresponding to the composition; wherein the preset random forest model is trained based on compositions of authors of different age groups and different themes; The pre-training process of the preset random forest model includes: Acquire a pre-training composition sample, wherein the pre-training composition sample includes a pre-training composition and a composition score corresponding to the pre-training composition; Inputting the pre-trained composition sample into a preset random forest model; Extracting the preset target features of the pre-trained composition and the composition scores for learning, and classifying the pre-trained composition samples to implement pre-training of the preset random forest model; The step of extracting the target features of the composition according to the preset feature extraction method comprises: Extracting the preset basic language features of the composition according to a preset basic language feature extraction method; Extracting the preset logical features of the composition according to a preset logical feature extraction method; Extracting the preset comprehension feature of the composition according to a preset comprehension feature extraction method; The step of extracting the preset logical features of the composition according to the preset logical feature extraction method includes: Cutting the composition into a plurality of sentences according to punctuation marks contained in the composition; Based on the sentences, the number of elements contained in the composition corresponding to the preset article structure feature of the composition, the average number of words in all the elements corresponding to the preset article element feature, and the number of overlaps between adjacent sentences corresponding to the preset article coherence feature are counted, wherein the element is a set of sentences of the largest consecutive occurrence in the background category, argument category, main view category, supporting view category or conclusion category contained in the composition, which is called the element of the category; The step of extracting the preset comprehension feature of the composition according to the preset comprehension feature extraction method comprises: Get the preset articles contained in the preset corpus; Comparing the composition with the preset article according to preset characteristic indicators to obtain a preset similarity between the composition and the preset article; The preset articles are several articles. After the step of comparing the composition with the preset articles according to preset characteristic indicators to obtain a preset similarity between the composition and the preset articles, the method further includes: Obtaining the most similar article with the smallest similarity score difference with the composition from the preset articles; Obtaining a score index of an article belonging to the same article category as the most similar article as a comparison index; Based on the comparison index, the composition is compared with a preset target article with the highest score to obtain the similarity between the composition and the preset target article.
2. The composition scoring method based on feature extraction and random forest according to claim 1 is characterized in that: The step of extracting the preset basic language features of the composition according to the preset basic language feature extraction method comprises: Using a first preset language tool to segment the composition to obtain phrases contained in the composition; Based on the phrase, feature extraction is performed according to each preset target language basic sub-feature extraction method to obtain each target language basic sub-feature; The obtained plurality of target language basic sub-features are grouped together to obtain the preset language basic feature.
3. The composition scoring method based on feature extraction and random forest according to claim 1 is characterized in that: The step of extracting the preset comprehension feature of the composition according to the preset comprehension feature extraction method also includes: Obtaining the subject corresponding to the composition; Obtaining articles on the same subject as the subject in the preset corpus, wherein the articles on the same subject are articles on the same subject as the composition; The composition is compared with the article on the same subject in terms of similarity to obtain the category similarity between the composition and the category corresponding to the article on the same subject.
4. A composition scoring device based on feature extraction and random forest, characterized in that: include: An acquisition unit is used to acquire the composition to be graded; An extraction unit is used to extract target features of the composition according to a preset feature extraction method, wherein the target features include preset language basic features, preset logical features, and preset comprehension features, wherein the preset language basic features include preset vocabulary features, preset grammar features, preset syntax features, preset length features, and preset readability features, wherein the preset logical features include preset article structure features, preset article coherence features, and preset article element features, and wherein the preset comprehension features include similarity features with preset articles; The similarity features with the preset articles include: the comparison between the composition and the high-quality articles and the degree of deviation from the topic of the composition; A scoring unit, used for inputting the target feature into a pre-trained preset random forest model for scoring, so as to obtain a score corresponding to the composition; wherein the preset random forest model is trained based on compositions of authors of different age groups and different themes; The pre-training process of the preset random forest model includes: Acquire a pre-training composition sample, wherein the pre-training composition sample includes a pre-training composition and a composition score corresponding to the pre-training composition; Inputting the pre-trained composition sample into a preset random forest model; Extracting the preset target features of the pre-trained composition and the composition scores for learning, and classifying the pre-trained composition samples to implement pre-training of the preset random forest model; The extraction unit comprises: A language basic feature extraction subunit, used for extracting the preset language basic features of the composition according to a preset language basic feature extraction method; A logic feature extraction subunit, used for extracting the preset logic feature of the composition according to a preset logic feature extraction method; A comprehension feature extraction subunit, used to extract the preset comprehension feature of the composition according to a preset comprehension feature extraction method; The logical feature extraction subunit comprises: A cutting subunit, used for cutting the composition into a plurality of sentences according to punctuation marks contained in the composition; A statistical subunit is used to count the number of elements contained in the composition text corresponding to the preset article structure feature of the composition, the average number of words in all the elements corresponding to the preset article element feature, and the number of overlaps between adjacent sentences corresponding to the preset article coherence feature based on the sentences, wherein the elements are the background class, the argument class, the main view class, the supporting view class or the conclusion class contained in the composition, and the set of sentences of the largest consecutive occurrence of the class is called the element of the class; The comprehension feature extraction subunit includes: The acquisition subunit is used to acquire the preset articles contained in the preset corpus; A comparison subunit, used for comparing the composition with the preset article according to preset characteristic indicators to obtain a preset similarity between the composition and the preset article; The preset articles are a plurality of articles, and the comprehension feature extraction subunit further includes: The most similar article obtaining subunit is used to obtain the most similar article with the smallest similarity score difference with the composition from the preset articles; A comparison index obtaining unit, used to obtain a score index of an article belonging to the same article category as the most similar article as a comparison index; The comparison subunit is used to compare the composition with a preset target article with the highest score based on the comparison index to obtain the similarity between the composition and the preset target article.
5. A computer device, characterized in that: The computer device comprises a memory and a processor connected to the memory; the memory is used to store a computer program; the processor is used to run the computer program to execute the steps of the method according to any one of claims 1 to 3.
6. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 3 can be implemented.
