A cross-prompt automatic composition scoring method, device and electronic device based on category adversarial joint learning

Abstract

The present invention provides a cross-prompt automatic composition scoring method, device and electronic device based on category adversarial joint learning, which belongs to the technical field of composition scoring, and obtains composition data of a composition to be evaluated; inputs the composition data into a feature extractor with a Hi-att network, and outputs composition features; defines a discriminator, and conducts adversarial training on the discriminator and the feature extractor; splices the composition features with manual features, and predicts the score through a plurality of different nonlinear feature transformations and calculates the score loss; uses regression loss, classification loss and adversarial loss to process the score, grade prediction and generalization ability of the composition to be evaluated. The present invention improves the performance of the model. By adopting a category-aware adversarial training method, the classification performance is improved, and the problem of misaligned alignment between source prompts caused by uneven category distribution is effectively avoided, which greatly enhances the practicality and accuracy of the model.

Description

Technical Field

[0001] The present invention belongs to the technical field of composition scoring, and in particular relates to a cross-prompt automatic composition scoring method, device and electronic equipment based on category adversarial joint learning. Background Art

[0002] The development of automatic essay scoring technology has experienced an evolution from the initial manually defined features and traditional machine learning methods to deep learning technology. Although the early methods contain rich expert knowledge, the process of designing and extracting features is time-consuming and labor-intensive.

[0003] With the introduction of deep learning, especially the application of convolutional neural networks (CNN) and long short-term memory networks (LSTM), the development of automatic essay scoring technology has been greatly promoted. Subsequently, the widespread application of attention mechanisms has further improved the effect of automatic essay scoring, which can more finely capture the importance of different parts of the text. Generally, automatic essay scoring methods based on deep learning can be divided into automatic essay scoring based on specific prompts and automatic essay scoring across prompts, depending on whether their training data and test data come from the same prompt.

[0004] 1) Automatic essay scoring for specific prompts

[0005] Automatic essay scoring for specific prompts aims to learn an automatic scoring model based on training and test samples from the same prompt. Automatic scoring models for specific prompts can make full use of specific prompt information in the data to improve the scoring performance of the model. The T&N model is the earliest automatic essay scoring method based on deep learning. This method uses convolutional neural networks (CNNs) and long short-term memory networks (LSTMs) to model local and long-distance dependency features respectively. In the same year, Alikaniotis et al. proposed an enhanced CBOW model to learn word embedding representations for scoring by jointly modeling language models and AES. Wang Yaohua et al. proposed a method to measure semantic differences by analyzing the relationship between sentence vectors and overall essay vectors, and built a deep neural network model based on this difference for essay score prediction. Subsequently, in 2018, Tay et al. proposed the Skipflow mechanism. SKIPFLOW uses LSTM to explicitly capture relevant attributes and uses neural tensor layers for coherent text representation. Fu Ruiji et al. modeled beautiful sentence recognition based on the CNN-LSTM model and used the learned features to improve the performance of automatic essay scoring. In 2021, Zhou Xianbing et al. proposed a neural network based on multi-level semantic features, using CNN to capture local semantic features while using a hybrid neural network to capture global semantic features. In addition, a small number of artificial features were constructed to obtain the linguistic features of the composition from the shallow level.

[0006] With the widespread use of attention mechanisms, many automatic essay scoring methods adopt attention mechanisms to enhance the effectiveness of automatic essay scoring. One of the typical methods is Hi-att, which introduces self-attention into the classic T&N framework to model the importance of different words in automatic essay scoring. Xia Linzhong et al. proposed a model that combines a two-layer LSTM network and an attention mechanism layer to extract text information, and finally produces scoring results through a softmax layer. Another widely adopted paradigm is the Bert-based method. Benefiting from its outstanding achievements in various natural language processing tasks, Bert is used to solve automatic essay scoring. Rodriguez et al. directly adopted pre-trained Bert and XLNet models to predict essay scores, verifying the effectiveness of using pre-trained models. Subsequently, Yang et al. fine-tuned the BERT model by jointly optimizing the regression and ranking losses of automatic essay scoring. In addition, Ormer et al. enhanced the performance of automatic essay scoring by integrating pre-trained models, even when using reduced scale. In addition, Sethi et al. introduced a parameter-efficient method to minimize the number of training parameters by using adapter modules as an alternative to fine-tuning the entire pre-trained model. In 2024, Yu Mingcheng et al. proposed an automatic essay scoring method based on multi-scale context, which uses the XLNet pre-trained model to extract word embeddings and sentence embeddings in the text. These features are then integrated and processed through a linear layer to generate a scoring result.

[0007] 2) Automatic essay scoring across prompts

[0008] Although the automatic essay scoring method for specific prompts has achieved good results, it is difficult to collect a large number of scored essays for specific prompts in actual scenarios, and it is even impossible to obtain essays for the target prompt. At the same time, there are a large number of scored essays belonging to other different prompts (source prompts) that can be used.

[0009] Cross-prompt automatic essay scoring aims to learn a transferable automatic scoring model that can score essays with target prompts well. It mainly includes two scenarios: target prompt samples are visible and invisible.

[0010] (1) Automatic essay scoring with visible target prompt samples.

[0011] Most existing automatic essay scoring methods with visible target cues adopt a two-stage approach. In the first stage, a cue-independent scoring model is trained based on labeled source cue data; in the second stage, the performance of the model is improved based on cue-related information. A representative two-stage method is the TDNN model proposed by Jin et al. This model first trains a cue-independent model based on multiple source cues, and generates pseudo-labels for target-cue essays based on this model. Subsequently, a scoring model for the target cue is trained using the target-cue essays with the highest and lowest scores in the pseudo-labels. On this basis, Li et al. further proposed the SEDNN model, which uses an adversarial network to learn a cue-independent model in the initial stage and adds cue information in the subsequent stage to improve the performance of the cross-cue essay scoring model.

[0012] Another two-stage cross-prompt automatic essay scoring method is mainly based on a pre-trained model, using a pre-training and fine-tuning approach. The pre-trained model allows the Transformer-based model to integrate general knowledge obtained from various natural language texts. Song et al. proposed a Chinese AES method based on a pre-trained model, using one pre-training stage and two fine-tuning stages to improve the scoring performance of the target prompt. Similarly, Cao et al. first learned the shared features between prompts by constructing two prompt-independent self-supervised tasks based on multiple source prompts and target prompts, and then used adversarial training to achieve the scoring of target prompt essays. In addition, in 2023, Li et al. used a contrastive learning-based technique to feature map the source prompts and target prompts to achieve feature alignment of the source prompts and target prompts, thereby improving the generalization performance of the model.

[0013] (2) Automatic essay scoring where target prompt samples are invisible.

[0014] The target prompt visible method attempts to use the target prompt samples to improve the performance of the cross-prompt automatic essay scoring model. However, in the scenario where the target prompt essay is invisible, since the model cannot use the target prompt data, it is difficult to learn the shared features between the source prompt and the target prompt. Therefore, there are relatively few studies on automatic scoring of cross-topic essays for unseen topics, and it is still in the exploratory stage.

[0015] At present, some cross-prompt automatic essay scoring methods where the target prompt is invisible assume that the model trained based on general language features (such as part-of-speech features) has strong generalization ability and can be easily transferred to the target prompt. Phandi et al. tried to extract hand-crafted features from all source prompt data to learn essay scoring that can be used for target prompts, and at the same time took the part-of-speech features of the source prompt essay as input, and concatenated its feature representation with the hand-designed prompt-independent features to improve the generalization performance of the model. Subsequently, in response to the feedback needs for essay attribute scoring in real scenarios, Ridley et al. extended the PAES framework and proposed a cross-prompt automatic essay scoring method CTS for unseen prompts with attributes, which adopted a multi-task learning method based on the attention mechanism to achieve joint scoring of the overall and each attribute. Furthermore, in 2023, Do et al. explored the integration of prompt information into the model based on the CTS model to improve the model's scoring ability for unseen target prompt essays. Summary of the invention

[0016] The present invention provides a cross-prompt automatic composition scoring method based on category adversarial joint learning, which makes full use of the shared features between classification and regression tasks to achieve comprehensive performance improvement.

[0017] Methods include:

[0018] S101: Obtain composition data of the composition to be evaluated;

[0019] S102: Input the composition data into a feature extractor with a Hi-att network and output composition features;

[0020] S103: define a discriminator, and perform adversarial training on the discriminator and the feature extractor;

[0021] S104: Concatenate the composition features with the manual features, predict the score through multiple different nonlinear feature transformations, and calculate the score loss;

[0022] S105: Use regression loss, classification loss and adversarial loss to process the scoring, grade prediction and generalization ability of the essays to be evaluated.

[0023] S106: Use regression loss, classification loss and adversarial loss to process the scoring, grade prediction and generalization ability of the essays to be evaluated.

[0024] It should be further explained that, in S102, it is further defined that the composition data e includes the sentences {S1, S2, ..., S l}, where the sentence S i Contains |w| words {w1,w2,…,w |w|}; Word embeddings applied to each sentence via a convolutional neural network:

[0025] z k =f(W n *[w k :w k+n-1 ]+b n ),

[0026] Among them, w k represents the word embedding of the kth word, f represents the activation function, n represents the size of the sliding window in the convolutional neural network, and W n and b n Represent the weight matrix and bias respectively.

[0027] It should be further explained that in the method, a regression scorer and a classification scorer are also configured;

[0028] The regression scorer includes an overall scorer and M attribute scorers;

[0029] The regression scorer predicts the overall score and attribute score of the composition data and defines manual features. h ;

[0030] After obtaining the composition feature representation e, it is compared with the manual feature e h To perform splicing, that is, [e; e h ], and obtain the feature representation for the overall essay score through nonlinear feature transformation:

[0031]

[0032] in, and are the weight matrix and bias parameters.

[0033] It should be further explained that the attribute attention mechanism defined in the method is used to model the importance of the mutual influence between attributes and between the overall and each attribute, and obtain the composition feature representation t for the mth attribute score m :

[0034] C=[c0,c1,c2,…,c M ]

[0035]

[0036]

[0037] t m =[c m ;p m ]

[0038] Among them, score(·) is a function that calculates the correlation between two attributes. is the attention weight of the kth attribute relative to the mth attribute, C -m,k It means to remove c in C m The composition feature representation of the k-th attribute.

[0039] It should be further explained that in the method, the feature representation for the overall score of the composition and each feature representation for the attribute score are respectively sent to a nonlinear regression layer with a sigmoid activation function, and the method of predicting the score value also includes: predicting the score value v of the mth task j :

[0040] v m =σ(W m t m +b m )

[0041] Among them, W m and b m Predict layer weight matrices and bias parameters for scoring tasks;

[0042] For the predicted value and true value of the composition data score, the score loss is calculated using the mean square error loss:

[0043]

[0044] It should be further explained that the mask function is introduced into the loss function; for the i-th training sample, the attribute label set is a subset of the label set Y, and the mask function is:

[0045]

[0046]

[0047]

[0048] Among them, mask m,i ∈[0,1] represents the mth attribute mask value of the i-th essay; the overall regression loss is:

[0049]

[0050] It should be further explained that in the method, the composition data are classified into four levels: excellent, good, fair, and poor based on the classification scorer;

[0051] Given the composition feature e, the composition feature e is combined with the manual feature e h To perform splicing, that is, [e; e h], the concatenated features are passed through the feature transformation layer to obtain the feature representation for composition classification:

[0052]

[0053] E c It represents the input to the composition classifier and calculates the cross entropy loss between the predicted category and the true category label;

[0054] Hint for the pth source:

[0055]

[0056] Among them, C is the classification scorer, and denote the predicted label and true category label of the i-th sample in the p-th source cue respectively; the overall classification loss is:

[0057]

[0058] It should be further explained that, in step S103, four category discriminators are set;

[0059] The composition feature representations of all source prompts are input into the corresponding category discriminator to discriminate the sample source, and the invariant feature representations between different source prompts are learned through adversarial training between the feature extractor and the discriminator;

[0060] Among them, given the composition feature representation e output by the feature extractor, the composition feature representation e is input into the discriminator D k , discriminator D k For N classifiers, calculate the discriminator D k The cross entropy loss of the predicted label and the true label is obtained:

[0061]

[0062] Among them, N k represents the number of essays whose true labels in all source prompts are the k-th category, and They represent the predicted label and true category label of the i-th sample in the k-th category respectively; the adversarial loss is:

[0063]

[0064] The present invention also provides a device for automatic composition scoring across prompts based on category adversarial joint learning, the device is used to implement the steps of the method for automatic composition scoring across prompts based on category adversarial joint learning;

[0065] The device includes: a data acquisition module, a feature extraction module, an adversarial training module, a feature splicing module and a combination optimization module;

[0066] The data acquisition module is used to obtain the composition data of the composition to be evaluated;

[0067] The feature extraction module is used to input the composition data into the feature extractor with Hi-att network and output the composition features;

[0068] The adversarial training module is used to define the discriminator and conduct adversarial training between the discriminator and the feature extractor;

[0069] The feature concatenation module is used to concatenate composition features with manual features, predict scores through multiple nonlinear feature transformations, and calculate score losses;

[0070] The combined optimization module is used to process the scoring, grade prediction and generalization ability of the composition to be evaluated using regression loss, classification loss and adversarial loss. The present invention also provides an electronic device, including a memory, a processor and a computer program stored in the memory and executable on the processor, and when the processor executes the computer program, the steps of the cross-prompt automatic composition scoring method based on category adversarial joint learning are implemented.

[0071] It can be seen from the above technical solutions that the present invention has the following advantages:

[0072] The cross-prompt automatic composition scoring method based on category adversarial joint learning provided by the present invention realizes cross-prompt automatic composition scoring through a category adversarial joint learning mechanism. The present invention adopts a joint modeling method of regression and classification. On the one hand, the shared features of regression and classification tasks can be learned by joint modeling, thereby achieving a comprehensive improvement in performance; on the other hand, a category-level adversarial network based on classification tasks is introduced to achieve effective alignment of category-level features between source prompts, learn invariant feature representations across prompts, and greatly improve the generalization ability of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0073] In order to more clearly illustrate the technical solution of the present invention, the accompanying drawings required for use in the description will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0074] Figure 1 Flowchart of the cross-prompt automatic essay scoring method based on category adversarial joint learning;

[0075] Figure 2 An example diagram of the cross-prompt automatic essay scoring method based on category adversarial joint learning. DETAILED DESCRIPTION

[0076] The cross-prompt automatic composition scoring method based on category adversarial joint learning provided by the present invention is aimed at improving the automatic composition scoring technology. The method of the present invention combines the joint modeling of classification and regression, and makes full use of the shared features between classification and regression tasks to achieve comprehensive performance improvement. The present invention adopts a category-aware adversarial training strategy: this strategy improves the generalization ability of the model for cross-prompt composition scoring through adversarial training of feature extractors and category-level discriminators, and enhances the robustness of the model when facing different category distributions.

[0077] For the cross-prompt automatic essay scoring method of the present invention, the associated data can be acquired and processed based on artificial intelligence technology. Among them, the cross-prompt automatic essay scoring method uses the theory, method, technology and application device of digital computer machine simulation, extension and expansion of human intelligence to perceive words, acquire knowledge and use knowledge to obtain the best results.

[0078] The cross-prompt automatic essay scoring method can combine computer vision technology, speech processing technology, natural language processing technology, and machine learning / deep learning. Among them, machine learning and deep learning usually include artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and teaching learning. It solves the problem that it is difficult to collect a large number of scored essays with specific prompts in actual scenarios, or even impossible to obtain essays with target prompts.

[0079] The cross-prompt automatic essay scoring method involved in the present invention is applied to one or more electronic devices, wherein the electronic device is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes but is not limited to microprocessors, application specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), digital signal processors (DSP), embedded devices, etc.

[0080] An electronic device can be any electronic product that can interact with a user, such as a personal computer, a tablet computer, a smart phone, a personal digital assistant (PDA), an interactive network television (IPTV), etc.

[0081] The network where the electronic device is located includes but is not limited to the Internet, wide area network, metropolitan area network, local area network, virtual private network (VPN), etc.

[0082] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0083] See also Figure 1 The figure is a flowchart of a method for automatic essay scoring across prompts based on category adversarial joint learning in a specific embodiment, the method comprising:

[0084] Get the composition data of the composition to be evaluated.

[0085] Input the composition data into the feature extractor with Hi-att network and output the composition features;

[0086] According to the embodiment of the present application, the feature extraction part is a feature extractor, and the feature extractor is used to represent the input composition. In this embodiment, the widely recognized Hi-att network is used as the feature extractor. The Hi-att network models the hierarchical structure of the composition, and learns the embedding representation of the composition in a hierarchical manner from the vocabulary level to the sentence level and then to the article level.

[0087] Alternatively, assume that a composition e contains l sentences {S1,S2,…,S l}, where the sentence S i Contains |w| words {w1,w2,…,w |w|}. First a convolutional neural network is applied to the word embeddings of each sentence:

[0088] z k=f(W n *[w k :w k+n-1 ]+b n ) (1)

[0089] Among them, w k represents the word embedding of the kth word, f represents the activation function, n represents the size of the sliding window in the convolutional neural network, and W n and b n Represent the weight matrix and bias respectively.

[0090] Since different words contribute differently in automatic essay scoring, word attention pooling is used to enhance the sentence representation s, which is expressed as:

[0091]

[0092]

[0093]

[0094] Among them, W a and W z represents the weight matrix, b a Indicates bias, and a k They represent the attention vector and attention weight of the kth word respectively.

[0095] h q =LSTM(s q ,h q-1 ) (5)

[0096] where s q and h q They are the input sentence and its hidden state in step q. After that, the feature representation e of the composition is obtained through the attention pooling layer. The specific formula is:

[0097]

[0098]

[0099]

[0100] In this embodiment, all W ha and W h represents the weight matrix, b ha represents the bias term, and ha q are the attention vector and attention weight of the qth sentence respectively.

[0101] According to the embodiment of the present application, the regression scorer part and the classification scorer part are, first, the regression scorer, which includes an overall scorer and M attribute scorers. The regression scorer is intended to predict the overall score and attribute score of the composition. At the same time, in order to further improve the scoring performance, the same manual features as Ridley et al. are used, including, for example, composition length, readability and text complexity, etc., represented by symbol e h express.

[0102] After obtaining the composition feature representation e, it is compared with the manual feature e h To perform splicing, that is, [e; e h ], and transform it through a nonlinear feature transformation to obtain the feature representation for the overall score of the composition:

[0103]

[0104] in, and are the weight matrix and bias parameters.

[0105] All features obtained by this function for the overall score and all attribute score tasks in this embodiment are expressed as [c0, c1, c2, ..., c M ], where C0 is the feature representation used for overall scoring, [C1,C2,…,C M ] is represented as the feature representation for M attribute tasks.

[0106] It can be seen that in the essay scoring task, there is a certain correlation between the overall score of the essay or the scores of each attribute. In order to fully consider the relationship between each attribute and the overall score, the attribute attention mechanism of this embodiment is used to model the importance of the mutual influence between attributes and between the overall score and each attribute, so as to obtain the essay feature representation t for the mth attribute score. m :

[0107] C=[e0,c1,c2,…,c M ] (10)

[0108]

[0109]

[0110] t m =[c m ;p m ] (13)

[0111] Among them, score(·) is a function that calculates the correlation between two attributes. is the attention weight of the kth attribute relative to the mth attribute, C -m,k It means to remove c in C m The composition feature representation of the k-th attribute.

[0112] Finally, the feature representation of the overall score and the feature representation of all attribute scores are sent to their corresponding nonlinear regression layers with sigmoid activation function to predict the score value v of the mth task. j :

[0113] v m =σ(W m t m +b m ) (14)

[0114] Among them, W m and b mPredict the layer weight matrix and bias parameters for the scoring task.

[0115] For each essay's predicted and true rating, the mean square error loss is used to calculate the rating loss:

[0116]

[0117] It should be noted that, since each prompt involves different genres and different genres contain different scoring attributes, not every attribute will appear in every prompt. Considering the situation where some attribute scores are missing, this embodiment introduces a mask function in the loss function. Specifically, for the i-th training sample, its attribute label set is a subset of the label set Y, and the mask function is:

[0118]

[0119]

[0120]

[0121] Among them, mask m,i ∈[0,1] represents the mth attribute mask value of the i-th essay. If the value is missing, then after masking one by one, its corresponding The value is 0.

[0122] Since there are N source topics in total, the overall regression loss is:

[0123]

[0124] In this embodiment, unlike the existing automatic composition scoring methods that only model automatic composition scoring as a classification or regression task, this paper jointly models the regression and classification tasks, and achieves a joint improvement in the performance of the two tasks by learning the shared features between the two tasks.

[0125] For the essay classification task, since each essay has a specific score, the essays are first classified into four levels: excellent, good, fair, and poor according to the interval in which their scores fall. The classification scorer aims to predict the level of each essay.

[0126] Specifically, given the composition feature representation e, it is compared with the manual feature e h To perform splicing, that is, [e; e h ] and pass it through a feature transformation layer to obtain the feature representation for composition classification:

[0127]

[0128] Ec It represents the input to the composition classifier, and the cross entropy loss between its predicted category and the true category label is calculated. For the pth source prompt:

[0129]

[0130] Among them, C is the classification scorer, and They represent the predicted label and true category label of the i-th sample in the p-th source prompt respectively. There are N source prompts in total, so the overall classification loss is:

[0131]

[0132] Finally, the category adversarial part, in order to learn the invariant feature representations between different source cues and improve the generalization performance of the model, this embodiment jointly models regression and classification tasks, and introduces an adversarial training strategy based on the classification task to learn the invariant feature representations between different source cues.

[0133] In this embodiment, although the introduction of domain adversarial learning can learn domain-invariant features between different source prompts, such as Figure 2 As shown, there are large differences in the category distributions of different source prompts. Directly aligning the prompt-level distributions of each source prompt will lead to the problem of misaligned distributions of each category. Therefore, this embodiment takes into account the distribution differences between different source prompt categories and performs category-level feature alignment on different source prompts.

[0134] Specifically, four class discriminators are introduced, and all source-cued compositions are input into the corresponding class discriminators to determine which of the N source cues the sample comes from. Since the features learned by the feature extractor minimize the classification loss on the one hand, and on the other hand, confuse the discriminator so that it is difficult to accurately determine the source cues of the sample, the invariant feature representations between different source cues are learned through adversarial training between the feature extractor and the discriminator. Given the composition feature representation e output by the feature extractor, it is input into its corresponding class discriminator D k , the discriminator is N classifiers, and the cross entropy loss between its predicted labels and true labels is calculated to obtain:

[0135]

[0136] Among them, N k represents the number of essays whose true labels in all source prompts are the k-th category, and They represent the predicted label and true category label of the i-th sample in the k-th category respectively. Since there are C category discriminators, the adversarial loss is:

[0137]

[0138] In summary, the final loss of the present invention is composed of three parts: regression loss, classification loss, and category adversarial loss. The regression loss is used to ensure that the model accurately scores the composition, the classification loss is used to ensure that the model can accurately predict the grade of the input composition, and the adversarial loss is used to confuse the C category discriminators to make it difficult to distinguish which prompt the sample belongs to.

[0139] Among them, during the adversarial process, the gradient reversal layer (GRL) is used to automatically reverse the gradient direction of the adversarial loss to realize the back propagation of the gradient and achieve the effect of adversarial training.

[0140] Therefore, the total loss is a linear combination of three losses:

[0141] L=L reg +αL cls +βL cadv (25)

[0142] Among them, α and β are hyperparameters used to balance the importance of different loss terms.

[0143] The following is an embodiment of a device for automatic composition scoring across prompts based on category adversarial joint learning provided by an embodiment of the present disclosure. The device and the method for automatic composition scoring across prompts based on category adversarial joint learning of the above-mentioned embodiments belong to the same inventive concept. For details not fully described in the embodiment of the device for automatic composition scoring across prompts based on category adversarial joint learning, please refer to the embodiment of the method for automatic composition scoring across prompts based on category adversarial joint learning mentioned above.

[0144] The device comprises: a data acquisition module, a feature extraction module, an adversarial training module, a feature splicing module and a combination optimization module.

[0145] The data acquisition module is used to obtain the composition data of the composition to be evaluated.

[0146] The feature extraction module is used to input composition data into a feature extractor with a Hi-att network and output composition features.

[0147] The adversarial training module is used to define the discriminator and perform adversarial training on the discriminator and the feature extractor.

[0148] The feature concatenation module is used to concatenate composition features with manual features, predict scores through multiple different nonlinear feature transformations, and calculate score losses.

[0149] It should be noted that here, the overall score of the composition can be spliced ​​with the manual score, the attribute score of the composition can be spliced ​​with the manual score, and the classification of the composition can be spliced ​​with the manual classification. Multiple ways of splicing can be achieved.

[0150] The combined optimization module is used to process the scoring, grade prediction and generalization ability of the essays to be evaluated using regression loss, classification loss and adversarial loss.

[0151] In one embodiment of the present invention, the composition data of the composition to be evaluated is obtained; the composition data to be evaluated is input into the trained cross-prompt automatic composition evaluation model, and the evaluation result is output; wherein the trained cross-prompt automatic composition evaluation model is obtained by training with all source prompt compositions with known evaluation results. Wherein, during the training phase, the trained cross-prompt automatic composition evaluation model captures the shared features of all source prompts through a feature extractor, and the model inputs the extracted features into different tasks, namely a classification task for overall composition scoring and M+1 regression tasks for detailed evaluation of various aspects of the composition, including the whole. Here, in addition to the regression task, a classification task can also be performed based on the predicted overall rating level.

[0152] The device involved in the present invention enables the model to not only provide an overall score and grade for the composition, but also give detailed feedback on multiple specific attributes of the composition (such as language standardization, content richness, narrative quality, etc.).

[0153] In order to further improve the accuracy of scoring and the generalization ability of the model, the present invention introduces a category-level discriminator in the system. The discriminator is trained adversarially with the feature extractor used in the classification task, thereby optimizing the feature extractor so that it can learn more robust and discriminative features, thereby improving the model's judgment ability and accuracy in scoring essays of different categories. By comprehensively considering the regression loss, classification loss, and adversarial loss of the training model, the present invention achieves accurate scoring of essays, grade prediction, and improvement of generalization ability through the combined optimization of these three losses.

[0154] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A cross-prompt automatic essay scoring method based on category adversarial joint learning, It is characterized in that Methods include: S101: Obtain composition data of the composition to be evaluated; S102: Input the composition data into a feature extractor with a Hi-att network and output composition features; It also defines that the composition data e contains sentences , where the sentence Contains |w| words ; Word embeddings applied to each sentence via a convolutional neural network: in, represents the word embedding of the 𝑘th word, 𝑓 represents the activation function, 𝑛 represents the size of the sliding window in the convolutional neural network, and Represent the weight matrix and bias respectively; In the method, regression scorer and classification scorer are also configured; The regression scorer includes an overall scorer and M attribute scorers; The regression scorer predicts the overall score and attribute score of the composition data and defines manual features ; Get composition feature representation After that, with the manual features To splice, , the feature representation for the overall essay score is obtained through nonlinear feature transformation: in, and are the weight matrix and bias term parameters; The method also defines an attribute attention mechanism to model the importance of the mutual influence between attributes and between the overall and each attribute, and obtain the composition feature representation for the mth attribute scoring : in, is a function that calculates the correlation between two attributes. is the attention weight of the kth attribute relative to the mth attribute, Indicated in Remove The composition feature representation of the k-th attribute; The feature representation used for the overall score of the composition and each feature representation used for the attribute score are respectively sent to the nonlinear regression layer with the activation function as sigmoid. The way to predict the score value also includes: predicting the score value of the mth task : in, and Predict layer weight matrices and bias parameters for scoring tasks; For the predicted value and true value of the composition data score, the score loss is calculated using the mean square error loss: ； Introduce the mask function in the loss function; for the i-th training sample, the attribute label set is a subset of the label set Y, and the mask function is: in, represents the mth attribute mask value of the i-th composition; the overall regression loss is: ； In the method, the composition data are classified into four levels: excellent, good, fair, and poor based on a classification scorer; Given essay features Then, the composition features With manual features To splice, , the concatenated features are passed through the feature transformation layer to obtain the feature representation for composition classification: Will It represents the input to the composition classifier and calculates the cross entropy loss between the predicted category and the true category label; Hint for the pth source: Among them, F is the classification scorer, and denote the predicted label and true category label of the i-th sample in the p-th source cue respectively; the overall classification loss is: ； S103: define a discriminator, and perform adversarial training on the discriminator and the feature extractor; Set up four category discriminators; The composition feature representations of all source prompts are input into the corresponding category discriminator to discriminate the sample source, and the invariant feature representations between different source prompts are learned through adversarial training between the feature extractor and the discriminator; Among them, given the composition feature representation e output by the feature extractor, the composition feature representation e is input into the discriminator , discriminator For N classifiers, calculate the discriminator The cross entropy loss of the predicted label and the true label is obtained: in, represents the number of essays whose true labels in all source prompts are the k-th category, and They represent the predicted label and true category label of the i-th sample in the k-th category respectively; the adversarial loss is: S104: Concatenate the composition features with the manual features, predict the score through multiple different nonlinear feature transformations, and calculate the score loss; S105: Use regression loss, classification loss and adversarial loss to process the scoring, grade prediction and generalization ability of the essays to be evaluated.

2. A cross-prompt automatic composition scoring device based on category adversarial joint learning, It is characterized in that The device is used to implement the steps of the cross-prompt automatic composition scoring method based on category adversarial joint learning as claimed in claim 1; The device includes: a data acquisition module, a feature extraction module, an adversarial training module, a feature splicing module and a combination optimization module; The data acquisition module is used to obtain the composition data of the composition to be evaluated; The feature extraction module is used to input the composition data into the feature extractor with Hi-att network and output the composition features; The adversarial training module is used to define the discriminator and conduct adversarial training between the discriminator and the feature extractor; The feature concatenation module is used to concatenate composition features with manual features, predict scores through multiple nonlinear feature transformations, and calculate score losses; The combined optimization module is used to process the scoring, grade prediction and generalization ability of the essays to be evaluated using regression loss, classification loss and adversarial loss.

3. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, It is characterized in that When the processor executes the computer program, the steps of the cross-prompt automatic composition scoring method based on category adversarial joint learning as claimed in claim 1 are implemented.

Images