English article evaluation system based on deep learning

Abstract

The invention provides an English article evaluation system based on deep learning. The system comprises a data acquisition module, a data preprocessing module, a feature extraction module and an article evaluation module. The feature extraction module constructs a soft diagonal structure-Transform model, introduces structural prior, constructs a diagonal attention bias mechanism and a structural supervision loss function, realizes modeling of a paragraph semantic main line, an inter-sentence sequence relationship and discourse coherence, and improves structural sensitivity of feature representation; the article evaluation module constructs a multi-quantile-XGBoost scoring model, proposes a derivable arc tangent pin loss function, solves the problem that a traditional quantile loss function is not derivable in a tree model, and realizes multi-quantile stable prediction of a total score and a dimension score of a composition; the system has remarkable advantages in the aspects of structural modeling capability, scoring precision and interpretability, and is suitable for scenes such as intelligent education evaluation and writing auxiliary feedback.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to an English article evaluation system based on deep learning. Background Art

[0002] With the continuous advancement of educational informatization, the application of AI-based automated assessment systems in language teaching and educational assessment is becoming increasingly widespread. English composition, as an important manifestation of comprehensive language ability, requires assessment not only of the correctness of linguistic expression but also of the completeness of content, logical coherence, and structural standardization. Existing intelligent English composition assessment systems are mostly based on deep neural networks, traditional machine learning, or hybrid model architectures. Although some progress has been made in automating scoring and improving efficiency, they still suffer from the following technical deficiencies: First, most systems construct scoring models based solely on word-level or sentence-level semantic features, ignoring the sequential dependencies between sentences and the paragraph hierarchy within the composition. They lack explicit modeling of structural elements such as discourse coherence and paragraph development logic, resulting in unstable performance when processing compositions with complex structures or diverse expressions. Second, existing systems use pre-trained Transformer models such as BERT and GPT for text modeling. However, these models primarily perform attention calculations based on context windows and lack structural prior constraints. As a result, the generated semantic representations ignore the development patterns of the main semantic thread within a paragraph and fail to capture the overall organizational characteristics of the discourse. Summary of the Invention

[0003] The present invention aims to solve the problems of existing intelligent evaluation systems for English essays, such as insufficient discourse structure modeling capabilities, poor score interpretability, and difficulty in optimizing quantile predictions. A deep learning-based English essay evaluation system is proposed. The system introduces a soft diagonal structure-Transformer model and a multi-quantile-XGBoost scoring model to achieve structured modeling of essay features and stable output of multidimensional scores. In the feature extraction stage, the present invention constructs a diagonal attention bias mechanism, introduces structural prior information, and explicitly models the sequential dependencies between sentences in a paragraph and the discourse coherence features, significantly enhancing the system's ability to perceive the main semantic path. In the scoring stage, in response to the problem that the traditional quantile loss function is non-differentiable in the tree model and difficult to adapt to second-order optimization, an innovative inverse tangent pin loss function is proposed to achieve smooth fitting and efficient learning of quantile targets, thereby improving the distribution fitting capability and scoring stability of the XGBoost model. The system performs superiorly in discourse structure modeling, scoring accuracy improvement, and result interpretability, and is suitable for application scenarios such as educational assessment and intelligent teaching feedback.

[0004] The present invention provides an English article evaluation system based on deep learning, which includes a data acquisition module, a data preprocessing module, a feature extraction module and an article evaluation module;

[0005] The data collection module collects students' original English essays, the corresponding manual scoring records, and the public English corpus to build a unified data sample pool as raw data; the raw data is formatted and labeled to generate structured sample data; the manual scoring records include the total score and dimension scores;

[0006] The data preprocessing module performs standardized text processing on structured sample data, including encoding unification, cleaning of whitespace and abnormal characters, sentence segmentation, part-of-speech tagging, grammatical error correction, and alignment of score labels based on score information to obtain cleaned text data.

[0007] The feature extraction module constructs a soft diagonal structure-Transformer model to extract comprehensive feature data from the cleaned text data. The soft diagonal structure-Transformer model includes a diagonal bias generation unit, a structure enhancement encoding unit, a structure supervision extraction unit, a feature fusion output unit, and a Transformer encoder.

[0008] The article evaluation module constructs a multi-quantile-XGBoost scoring model, combines the cleaned text data and comprehensive feature data to train the multi-quantile-XGBoost scoring model, and outputs the article evaluation results; the multi-quantile-XGBoost scoring model includes a sample construction unit, a loss function generation unit, and a scoring model training unit.

[0009] Furthermore, the process of extracting comprehensive feature data of the cleaned text data through the soft diagonal structure-Transformer model specifically includes the following:

[0010] The diagonal bias generation unit processes the cleaned text data into sentences and inputs each sentence into the word segmenter to generate the corresponding sentence-level token sequence. The sentence-level token sequence is input into the Transformer encoder to obtain the hidden vector representation of each token. The hidden vector representation of each token is aggregated using the average pooling method to calculate the sentence-level initial vector representation to form a node feature matrix. The normalized geometric distance from each sentence node in the node feature matrix to the diagonal of the paragraph main structure is calculated. A soft diagonalization mechanism is introduced to calculate soft structure weights based on the normalized geometric distance. The soft structure weights are combined pairwise to generate a diagonal attention bias matrix.

[0011] The structure-enhanced encoding unit is combined with the diagonal attention bias matrix to optimize the attention mechanism of the Transformer encoder, constructing a structure-enhanced Transformer encoder, strengthening the information interaction between sentences in the main structure area, extracting semantic embedding features, and retaining the intermediate attention matrix;

[0012] The structural supervision extraction unit combines the diagonal attention bias matrix and the intermediate attention matrix to construct the main diagonal consistency loss function and the diagonal separability loss function, applies structural supervision to the intermediate attention matrix, and extracts the focus weight distribution, attention offset index and diagonal attention distribution vector from the intermediate attention matrix to obtain the discourse coherence feature;

[0013] The feature fusion output unit fuses semantic embedding features and discourse coherence features to obtain comprehensive feature data.

[0014] Furthermore, a multi-quantile-XGBoost scoring model is constructed, and the process of training the multi-quantile-XGBoost scoring model by combining the cleaned text data and comprehensive feature data to output the article evaluation results specifically includes the following:

[0015] The sample construction unit combines the cleaned text data and comprehensive feature data to construct a quantile labeled sample set;

[0016] The loss function generation unit constructs a traditional quantile loss function. To address the difficulty in adapting to the XGBoost second-order optimization mechanism, the inverse tangent function is introduced to smoothly reconstruct the traditional quantile loss function and construct an inverse tangent pin loss function. This adapts the XGBoost model based on second-order gradient optimization to achieve efficient fitting and robust learning of quantile targets.

[0017] The scoring model training unit constructs an XGBoost model, inputs the quantile labeled sample set into the XGBoost model for training, and optimizes the training process with the inverse tangent pin loss function to obtain a multi-quantile-XGBoost scoring model, performs multi-dimensional score prediction on the composition, and obtains the article evaluation results.

[0018] By adopting the above scheme, the beneficial effects achieved by the present invention are as follows:

[0019] By introducing the soft diagonal structure-Transformer model, the present invention realizes the explicit modeling of the structural relationship between the semantic main line of paragraphs and sentences in English compositions, thereby improving the structured representation capability of composition features. Compared with the traditional shallow modeling method based only on syntactic or lexical features, the present invention uses the diagonal attention bias matrix to guide the attention mechanism to focus on the main structural path within the paragraph, and combines the main diagonal consistency and diagonal separability loss functions to achieve deep extraction and supervision of discourse coherence. This mechanism significantly enhances the system's perception of the paragraph development logic, structural order and semantic direction, effectively improves the modeling accuracy of the correspondence between composition content and structure, and provides more expressive and interpretable input features for the subsequent scoring module.

[0020] The inverse tangent pin loss function proposed in the scoring stage of the present invention solves the problem that the traditional quantile loss function is not differentiable in the tree model and cannot be adapted to the second-order gradient optimization, thereby improving the stability and convergence efficiency of multi-quantile prediction; by embedding the smooth and differentiable quantile loss function into the XGBoost scoring model, the model can simultaneously fit the total score of the composition and the different quantiles of the scores of each dimension, effectively dealing with the skewness and extreme cases in the distribution of student composition scores; this improvement not only improves the sensitivity of the scoring system to differences in composition quality, but also enhances the applicability and trustworthiness of the evaluation results in actual teaching scenarios.

[0021] In summary, the present invention realizes the intelligent processing of the entire process of English composition from semantic structure modeling to quantitative scoring through the deep integration of the soft diagonal structure-Transformer with strong structural modeling capabilities and the multi-quantile-XGBoost scoring model with excellent optimization and adaptation capabilities, thereby improving the accuracy, stability and interpretability of the scoring results; the system as a whole is suitable for application scenarios such as large-scale teaching evaluation, composition correction and personalized writing feedback, and provides effective technical support for educational intelligence and language evaluation systems. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 A schematic diagram of a module of an English article evaluation system based on deep learning provided by the present invention;

[0023] Figure 2 This is a comparison chart of the XGBoost scoring model evaluation proposed in Examples 3, 4, and 5;

[0024] Figure 3 This is the prediction error distribution density diagram proposed in Examples 3, 4, and 5.

[0025] Figure 2 In , MAE stands for mean absolute error and MSE stands for mean square error. DETAILED DESCRIPTION

[0026] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments; based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0027] Example 1, according to Figure 1 ,The present invention provides an English article evaluation system based on deep learning, which includes a data acquisition module, a data preprocessing module, a feature extraction module and an article evaluation module;

[0028] The data collection module collects students' original English essays, the corresponding manual scoring records, and the public English corpus to build a unified data sample pool as raw data; the raw data is formatted and labeled to generate structured sample data; the manual scoring records include the total score and dimension scores;

[0029] The data preprocessing module performs standardized text processing on structured sample data, including encoding unification, cleaning of whitespace and abnormal characters, sentence segmentation, part-of-speech tagging, grammatical error correction, and alignment of score labels based on score information to obtain cleaned text data.

[0030] The feature extraction module constructs a soft diagonal structure-Transformer model to extract comprehensive feature data from the cleaned text data. The soft diagonal structure-Transformer model includes a diagonal bias generation unit, a structure enhancement encoding unit, a structure supervision extraction unit, a feature fusion output unit, and a Transformer encoder.

[0031] The article evaluation module constructs a multi-quantile-XGBoost scoring model, combines the cleaned text data and comprehensive feature data to train the multi-quantile-XGBoost scoring model, and outputs the article evaluation results; the multi-quantile-XGBoost scoring model includes a sample construction unit, a loss function generation unit, and a scoring model training unit.

[0032] Example 2: This example is based on Example 1. In this example, the process of extracting comprehensive feature data of cleaned text data through the soft diagonal structure-Transformer model specifically includes the following:

[0033] The diagonal bias generation unit processes the cleaned text data into sentences and inputs each sentence into the word segmenter to generate the corresponding sentence-level token sequence. The sentence-level token sequence is input into the Transformer encoder to obtain the hidden vector representation of each token. The hidden vector representation of each token is aggregated using the average pooling method to calculate the sentence-level initial vector representation to form a node feature matrix. The normalized geometric distance from each sentence node in the node feature matrix to the diagonal of the paragraph main structure is calculated. A soft diagonalization mechanism is introduced to calculate soft structure weights based on the normalized geometric distance. The soft structure weights are combined pairwise to generate a diagonal attention bias matrix.

[0034] The paragraph main structure diagonal line is a structural prior, essentially an abstract spatial layout hypothesis. After embedding the sentences in a paragraph into a two-dimensional structural space in their natural order, this diagonal line connects the first and last sentences of the paragraph, representing the main semantic path of the paragraph's semantic development along a temporal or logical sequence. It is used to explicitly model the sequential structural relationship between sentences and discourse coherence.

[0035] The structure-enhanced encoding unit is combined with the diagonal attention bias matrix to optimize the attention mechanism of the Transformer encoder, construct a structure-enhanced Transformer encoder, strengthen the information interaction between sentences in the main structure area, extract semantic embedding features, and retain the intermediate attention matrix. The formula used is as follows:

[0036] Formula for optimizing the attention mechanism of the Transformer encoder:

[0037] ;

[0038] in, represents the Query matrix, represents the Key matrix, represents the Value matrix, represents the attention mechanism, represents the transpose symbol, represents the dot product attention score matrix, represents the scaling factor, represents the weight coefficient, represents the diagonal attention bias matrix, represents the normalization function;

[0039] The structural supervision extraction unit combines the diagonal attention bias matrix and the intermediate attention matrix to construct the main diagonal consistency loss function and the diagonal separability loss function, applies structural supervision to the intermediate attention matrix, extracts the focus weight distribution, attention offset index and diagonal attention distribution vector from the intermediate attention matrix, and obtains the discourse coherence feature. The formula used is as follows:

[0040] ;

[0041] in, represents the main diagonal consistency loss, represents the intermediate attention matrix, Represents the node feature matrix The transpose of represents the Frobenius norm;

[0042] ;

[0043] in, represents the diagonal separability loss, Representation and diagonal attention bias matrix complementary areas of attention;

[0044] The feature fusion output unit fuses semantic embedding features and discourse coherence features to obtain comprehensive feature data.

[0045] Example 3, according to Figure 2 、 Figure 3 This embodiment is based on the second embodiment. In this embodiment, a multi-quantile-XGBoost scoring model is constructed, and the multi-quantile-XGBoost scoring model is trained by combining the cleaned text data and the comprehensive feature data to output the article evaluation results. The process specifically includes the following:

[0046] The sample construction unit combines the cleaned text data and comprehensive feature data to construct a quantile labeled sample set;

[0047] The loss function generation unit constructs the traditional quantile loss function (Pinball Loss). To address the problem of difficulty in adapting to the XGBoost second-order optimization mechanism, the inverse tangent function is introduced to smoothly reconstruct the traditional quantile loss function and construct the inverse tangent pinball loss function; this is adapted to the XGBoost model based on second-order gradient optimization to achieve efficient fitting and robust learning of quantile targets. The formula used is as follows:

[0048] Inverse tangent pin loss function:

[0049] ;

[0050] in, represents the quantile, represents the smoothing factor, represents the prediction error, represents the arctangent pin loss function value, represents the inverse tangent function, represents the normalized inverse tangent value, represents the constant compensation term;

[0051] This loss function is globally differentiable, effectively solving the non-optimization problem of the traditional quantile loss function (Pinball Loss) in XGBoost.

[0052] The scoring model training unit constructs an XGBoost model, inputs the quantile labeled sample set into the XGBoost model for training, and optimizes the training process with the inverse tangent pin loss function to obtain a multi-quantile-XGBoost scoring model, performs multi-dimensional score prediction on the composition, and obtains the article evaluation results.

[0053] Example 4, according to Figure 2 、 Figure 3 This embodiment is based on the second embodiment. In this embodiment, the process of training the XGBoost scoring model by combining the cleaned text data and the comprehensive feature data and outputting the article evaluation results specifically includes the following:

[0054] The sample construction unit combines the cleaned text data and comprehensive feature data to construct a quantile labeled sample set;

[0055] The loss function generation unit constructs the traditional quantile loss function;

[0056] The scoring model training unit constructs an XGBoost model, inputs the quantile labeled sample set into the XGBoost model for training, and optimizes the training process with the traditional quantile loss function to obtain the XGBoost scoring model, perform multi-dimensional score prediction on the composition, and obtain the article evaluation results.

[0057] Example 5, according to Figure 2 、 Figure 3 This embodiment is based on the second embodiment. In this embodiment, the process of training the XGBoost scoring model by combining the cleaned text data and the comprehensive feature data and outputting the article evaluation results specifically includes the following:

[0058] The sample construction unit combines the cleaned text data and comprehensive feature data to construct a quantile labeled sample set;

[0059] The loss function generation unit constructs the mean square error loss function;

[0060] The scoring model training unit constructs an XGBoost model, inputs the quantile labeled sample set into the XGBoost model for training, optimizes the training process with the mean square error loss function, obtains the XGBoost scoring model, performs multi-dimensional score prediction on the composition, and obtains the article evaluation results.

[0061] Example 6. This example is based on Example 3. In this example, the article evaluation module constructs a multi-quantile-XGBoost scoring model, combines the cleaned text data and the comprehensive feature data to train the multi-quantile-XGBoost scoring model, and outputs the article evaluation results.

[0062] In this embodiment,

[0063] Structured sample data:

[0064] ;

[0065] Main diagonal consistency loss: ;

[0066] Diagonal separability loss: ;

[0067] Article evaluation results:

[0068] A001: 215 words in total, clearly structured, describing the natural environment, personal emotions, and life experiences;

[0069] Dimension score (mean):

[0070] Grammar: 4.1;

[0071] Vocabulary: 5.0;

[0072] Logic: 4.2;

[0073] Coherence: 4.1

[0074] Overall rating: 4.4

[0075] Structural coherence index:

[0076] Standard deviation of attention shift: 0.17 (high coherence);

[0077] Main diagonal coverage: 87.5% (main structural area well modeled);

[0078] A002: 189 words in total, describing an experience at a school sports meet. The content is interesting but some of the grammar is poor, and the use of improper conjunctions leads to a slight jump in logic.

[0079] Dimension score (mean):

[0080] Grammar: 3.0;

[0081] Vocabulary: 3.8;

[0082] Logic: 3.1;

[0083] Coherence: 3.2

[0084] Overall rating: 3.3

[0085] Structural coherence index:

[0086] Standard deviation of attention shift: 0.32 (slightly higher, indicating inter-sentence jumping);

[0087] Main diagonal coverage: 69.1% (weak coherent structure);

[0088] A003: 245 words, with rigorous structure, sincere emotions, skillful grammar, diverse sentence patterns, and natural transitions;

[0089] Dimension score (mean):

[0090] Grammar: 5.0;

[0091] Vocabulary: 5.0;

[0092] Logic: 4.9;

[0093] Coherence: 5.0

[0094] Overall rating: 5.0

[0095] Structural coherence index:

[0096] Standard deviation of attentional shift: 0.08 (very focused);

[0097] Main diagonal coverage: 96.4% (almost complete coverage).

[0098] The present invention and its embodiments are described above. Such description is not restrictive. What is shown in the accompanying drawings is only one of the embodiments of the present invention, and the actual structure is not limited thereto. In short, if ordinary technicians in this field are inspired by it and do not depart from the purpose of the invention, they can creatively design structural methods and embodiments similar to the technical solution, which should all fall within the scope of protection of the present invention.

Claims

1. A deep learning-based English article evaluation system, comprising a data acquisition module and a data preprocessing module, wherein the data acquisition module acquires structured sample data; the data preprocessing module preprocesses the structured sample data to obtain cleaned text data; and characterized in that: The system also includes a feature extraction module and an article evaluation module; The feature extraction module constructs a soft diagonal structure-Transformer model to extract comprehensive feature data of the cleaned text data through the soft diagonal structure-Transformer model; the soft diagonal structure-Transformer model includes a diagonal bias generation unit, a structure enhancement encoding unit, a structure supervision extraction unit, a feature fusion output unit and a Transformer encoder; The article evaluation module constructs a multi-quantile-XGBoost scoring model, combines the cleaned text data and comprehensive feature data to train the multi-quantile-XGBoost scoring model, and outputs the article evaluation results; the multi-quantile-XGBoost scoring model includes a sample construction unit, a loss function generation unit, and a scoring model training unit.

2. The deep learning-based English article evaluation system according to claim 1, characterized in that: The diagonal bias generation unit processes the cleaned text data into sentences, generates a sentence-level token sequence, and inputs it into the Transformer encoder to obtain the hidden vector representation of each token. It then aggregates the hidden vectors using the average pooling method and calculates the sentence-level initial vector representation to form a node feature matrix. Calculate the normalized geometric distance of the node feature matrix; A soft diagonalization mechanism is introduced to calculate the soft structure weights based on the normalized geometric distance, and the soft structure weights are combined pairwise to generate a diagonal attention bias matrix.

3. The deep learning-based English article evaluation system according to claim 2, characterized in that: The structure-enhanced encoding unit is combined with the diagonal attention bias matrix to optimize the attention mechanism of the Transformer encoder, construct a structure-enhanced Transformer encoder, extract semantic embedding features, and retain the intermediate attention matrix.

4. The deep learning-based English article evaluation system according to claim 3, characterized in that: The structural supervision extraction unit combines the diagonal attention bias matrix and the intermediate attention matrix to construct the main diagonal consistency loss function and the diagonal separability loss function, and applies structural supervision to the intermediate attention matrix to obtain the discourse coherence features.

5. The deep learning-based English article evaluation system according to claim 4, characterized in that: The feature fusion output unit fuses semantic embedding features and discourse coherence features to obtain comprehensive feature data.

6. The English article evaluation system based on deep learning according to claim 1, characterized in that: The sample construction unit combines the cleaned text data and comprehensive feature data to construct a quantile labeled sample set.

7. The deep learning-based English article evaluation system according to claim 6, characterized in that: The loss function generation unit constructs the traditional quantile loss function, introduces the inverse tangent function to smoothly reconstruct the traditional quantile loss function, and constructs the inverse tangent pin loss function.

8. The deep learning-based English article evaluation system according to claim 7, characterized in that: The scoring model training unit constructs an XGBoost model, inputs the quantile labeled sample set into the XGBoost model for training, and optimizes the training process with the inverse tangent pin loss function to obtain a multi-quantile-XGBoost scoring model, perform multi-dimensional score prediction on the composition, and obtain the article evaluation results.

Images