A method, system and medium for automatically scoring cross-topic compositions
By constructing positive and negative example representations through encoder and cross-mapping processing, and combining contrast loss and scoring loss to update model parameters, the problem of insufficient topic diversity in cross-topic essay automatic scoring is solved, thereby improving scoring accuracy and consistency.
Patent Information
- Application Number
- CN202210987627.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-08-17
- Publication Date
- 2025-12-19
- Estimated Expiration
- 2042-08-17
AI Technical Summary
Existing cross-topic essay automatic scoring methods fail to effectively utilize high-level topic-irrelevant features, resulting in insufficient learning of semantic differences between different topics and affecting scoring performance.
The model encodes essays on the source and target topics using an encoder, constructs positive and negative example representations using cross-mapping, and updates model parameters by combining contrast loss and rating loss, thereby reducing topic differences and improving the learning performance of the cross-topic rating model.
It improves the accuracy and consistency of automatic scoring of cross-topic essays, enhances the model's ability to share information across different topics, and improves scoring performance.
Smart Images
Figure CN115455178B_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the field of artificial intelligence, and in particular to a cross-topic composition automatic scoring method and system and a medium. BACKGROUND
[0002] Cross-topic composition automatic scoring refers to a research task of training a scoring model using compositions with labeled source topics and applying it to automatically score compositions of target topics without labels. It is one of the important research tasks in natural language processing research and has important research and application value in the field of education.
[0003] Cross-topic composition automatic scoring uses labeled composition training data in the source topic to train the scoring model and migrates it to the target topic composition for scoring. The core lies in the learning and migration of semantic differences between different topics (source topic and target topic).
[0004] For example: suppose topic 1 is "Please talk about the benefits and drawbacks of human use of computers", and topic 2 is "Please talk about the impact of water resource shortage on humans". It can be seen that the semantic information described by the compositions of the two topics is different, one focuses on the use of computers and involves technological issues, while the other focuses on water resource shortage and involves environmental issues.
[0005] How to reduce the semantic difference between topics so that the learned composition representation can maximize the consistency between topics is the core problem of cross-topic composition automatic scoring. Although existing cross-topic composition automatic scoring research methods have achieved good results, there are still the following shortcomings:
[0006] (1) Existing technologies focus more on obtaining theme-independent handcrafted features through feature engineering and using these theme-independent handcrafted features to represent compositions of different topics to achieve migration between different topics. Although this method can directly use theme-independent features to express compositions, the extracted features are shallow text features such as sentence length, composition length, composition readability indicators, composition complexity indicators, and some high-level theme-independent features cannot be handpicked.
[0007] (2) Some existing technologies extract high-level theme-independent features by inputting source topic and target topic compositions simultaneously and using an adversarial network to obtain theme-shared features between different topics, and using these features to express compositions, thereby achieving better results than handcrafted features. However, this method still does not learn the maximum consistency between different topics from the perspective of theme consistency. SUMMARY
[0008] Therefore, the embodiment of the present application provides a cross-topic composition automatic scoring method, system and medium, which can effectively improve the cross-topic learning effect and further improve the scoring performance of the model in the cross-topic composition automatic scoring task.
[0009] In one aspect, the embodiment of the present application provides a cross-topic composition automatic scoring method, comprising:
[0010] The compositions of the source topic and the target topic are encoded by an encoder to obtain topic representations of the source topic and the target topic; wherein the topic representations include a first representation matrix and a second representation matrix, the first representation matrix is a topic representation matrix of the source topic, and the second representation matrix is a topic representation matrix of the target topic; and the topic representations are taken as first topic representations;
[0011] According to the composition anchor point representations and the first topic representations, first positive example representations of the source topic and second positive example representations of the target topic are obtained through cross mapping processing; according to prior knowledge, first negative example representations of the source topic and second negative example representations of the target topic are constructed; wherein the composition anchor point representations include first composition anchor point representations of the source topic and second composition anchor point representations of the target topic;
[0012] According to the first composition anchor point representations, the first positive example representations and the first negative example representations are combined to construct a first contrastive loss; and according to the second composition anchor point representations, the second positive example representations and the second negative example representations are combined to construct a second contrastive loss;
[0013] The composition anchor point representations of the source topic are scored by a scorer to obtain predicted scores, and the real scores of the composition anchor point representations of the source topic are combined to construct a scoring loss;
[0014] According to the first contrastive loss, the second contrastive loss and the scoring loss, a total loss is constructed; and based on the total loss, model parameters are updated;
[0015] The compositions of the source topic and the target topic are encoded by the updated encoder to obtain second topic representations of the source topic and the target topic; the second topic representations are taken as first topic representations, and then the step of obtaining the first positive example representations of the source topic and the second positive example representations of the target topic through cross mapping processing according to the first topic representations is returned until a preset training round number is reached, and a cross-topic scoring model is obtained;
[0016] The compositions of the target topic are scored by the cross-topic scoring model to obtain target predicted scores.
[0017] Optionally, the encoding, by the encoder, of the writings of the source subject and the writings of the target subject to obtain the subject representations of the source subject and the target subject comprises:
[0018] encoding, by the encoder, of the writings of the source subject to obtain a first representation matrix, taking the first representation matrix as the subject representation of the source subject; the first representation matrix being a vector representation matrix of the writings of the source subject;
[0019] and, encoding, by the encoder, of the writings of the target subject to obtain a second representation matrix, taking the second representation matrix as the subject representation of the target subject; the second representation matrix being a vector representation matrix of the writings of the target subject.
[0020] Optionally, the first positive example representation of the source subject and the second positive example representation of the target subject are obtained by cross-mapping processing according to the writing anchor point representation and the first subject representation, comprising:
[0021] point-multiplying the first writing anchor point representation with the transpose matrix of the second representation matrix to obtain a first projection representation; and performing linear transformation on the first projection representation to obtain the first positive example representation;
[0022] and, point-multiplying the second writing anchor point representation with the transpose matrix of the first representation matrix to obtain a second projection representation; and performing linear transformation on the second projection representation to obtain the second positive example representation;
[0023] wherein, the writing anchor point representation is the encoding representation of the current processing writing sample.
[0024] Optionally, the first contrast loss is constructed according to the first writing anchor point representation, in combination with the first positive example representation and the first negative example representation, comprising:
[0025] the first contrast loss is constructed by calculating the cosine similarity of the first writing anchor point representation, the first positive example representation and the first negative example representation;
[0026] wherein, the first writing anchor point is the writing representation of the first target batch of the source subject.
[0027] Optionally, the second contrast loss is constructed according to the second writing anchor point representation, in combination with the second positive example representation and the second negative example representation, comprising:
[0028] the second contrast loss is constructed by calculating the cosine similarity of the second writing anchor point representation, the second positive example representation and the second negative example representation;
[0029] The second composition anchor point is a composition representation of a second target batch of the target topic.
[0030] Optionally, the total loss is constructed according to the first contrast loss, the second contrast loss and the score loss, and includes:
[0031] The total loss is constructed according to the first contrast loss, the second contrast loss and the score loss, in combination with a contrast weight and a score weight.
[0032] The expression of the total loss is:
[0033]
[0034] In the formula, Ltotal represents the total loss, a represents the contrast weight, l S L1 represents the first contrast loss, l T L2 represents the second contrast loss, β represents the score weight, and l MSE Lscore represents the score loss.
[0035] Optionally, the method further includes:
[0036] Based on the first contrast loss and the second contrast loss, the theme difference between the source topic and the target topic is reduced from a target direction through gradient descent; wherein the target direction includes a first direction from the source topic to the target topic and a second direction from the target topic to the source topic.
[0037] In another aspect, an embodiment of the present application provides a cross-topic composition automatic scoring system, including:
[0038] A first module is configured to encode the compositions of the source topic and the compositions of the target topic through an encoder to obtain theme representations of the source topic and the target topic; wherein the theme representations include a first representation matrix and a second representation matrix, the first representation matrix is a theme representation matrix of the source topic, and the second representation matrix is a theme representation matrix of the target topic; and the theme representations are used as first theme representations.
[0039] A second module is configured to obtain a first positive example representation of the source topic and a second positive example representation of the target topic through cross-mapping processing according to a composition anchor point representation and the first theme representations; and construct a first negative example representation of the source topic and a second negative example representation of the target topic according to prior knowledge; wherein the composition anchor point representation includes a first composition anchor point representation of the source topic and a second composition anchor point representation of the target topic.
[0040] The third module is configured to construct a first contrast loss according to the first composition anchor point representation, in combination with the first positive example representation and the first negative example representation, and construct a second contrast loss according to the second composition anchor point representation, in combination with the second positive example representation and the second negative example representation.
[0041] The fourth module is configured to perform scoring prediction on the composition anchor point representation of the source subject by a scorer to obtain a predicted score, and construct a scoring loss in combination with a real score of the composition anchor point representation of the source subject.
[0042] The fifth module is configured to construct a total loss according to the first contrast loss, the second contrast loss and the scoring loss, and update the model parameters based on the total loss.
[0043] The sixth module is configured to perform encoding processing on the composition of the source subject and the composition of the target subject based on the updated encoder to obtain second subject representations of the source subject and the target subject, and take the second subject representations as first subject representations, and then return to the step of obtaining the first positive example representation of the source subject and the second positive example representation of the target subject by cross mapping processing based on the first subject representations until a preset training round number is reached to obtain a cross-subject scoring model.
[0044] The seventh module is configured to perform scoring on the composition of the target subject by the cross-subject scoring model to obtain a target predicted score.
[0045] In another aspect, an embodiment of the present application provides an electronic device including a processor and a memory.
[0046] The memory is configured to store a program.
[0047] The processor executes the program to implement the method as described above.
[0048] In another aspect, an embodiment of the present application provides a computer readable storage medium, which stores a program. The program is executed by a processor to implement the method as described above.
[0049] The present application also discloses a computer program product or a computer program, which includes computer instructions stored in a computer readable storage medium. A processor of a computer device can read the computer instructions from the computer readable storage medium, and the processor executes the computer instructions to make the computer device execute the method as described above.
[0050] The embodiment of the application first encodes the compositions of the source subject and the compositions of the target subject through an encoder to obtain subject representations of the source subject and the target subject; wherein the subject representations include a first representation matrix and a second representation matrix, the first representation matrix is a subject representation matrix of the source subject, and the second representation matrix is a subject representation matrix of the target subject; the subject representations are taken as first subject representations; first positive example representations of the source subject and second positive example representations of the target subject are obtained through cross mapping processing according to composition anchor point representations and the first subject representations; first negative example representations of the source subject and second negative example representations of the target subject are constructed according to priori knowledge; wherein the composition anchor point representations include first composition anchor point representations of the source subject and second composition anchor point representations of the target subject; a first contrast loss is constructed according to the first composition anchor point representations, in combination with the first positive example representations and the first negative example representations; a second contrast loss is constructed according to the second composition anchor point representations, in combination with the second positive example representations and the second negative example representations; a score loss is constructed by combining a real score of the composition anchor point representations of the source subject with a predicted score obtained by scoring the composition anchor point representations of the source subject through a scorer; a total loss is constructed according to the first contrast loss, the second contrast loss and the score loss; model parameters are updated based on the total loss; the compositions of the source subject and the compositions of the target subject are encoded through the encoder after the update processing, to obtain second subject representations of the source subject and the target subject; the second subject representations are taken as first subject representations, and then the step of obtaining the first positive example representations of the source subject and the second positive example representations of the target subject through cross mapping processing according to the first subject representations is returned, until a preset training round number is reached, to obtain a cross-subject scoring model; the compositions of the target subject are scored through the cross-subject scoring model to obtain a target predicted score. Through cross mapping of the source subject and the target subject, the subject difference between the source subject and the target subject can be reduced, and then the model can learn more common information between the subjects, which makes the model more suitable for cross-subject scoring. BRIEF DESCRIPTION OF DRAWINGS
[0051] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.
[0052] Figure 1 A flowchart of a cross-subject composition automatic scoring method provided by the embodiment of the present application;
[0053] Figure 2A schematic flow diagram of the cross-topic composition automatic scoring method provided by the embodiment of the present application is shown in the figure.
[0054] Figure 3 A schematic flow diagram of the model training of the cross-topic composition automatic scoring method provided by the embodiment of the present application is shown in the figure. DETAILED DESCRIPTION
[0055] In order to make the purpose, technical scheme and advantages of the present application clearer, the present application is further described in detail below in combination with the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and do not limit the present application.
[0056] In one aspect, with reference to Figure 1 The embodiment of the present application provides a cross-topic composition automatic scoring method, comprising:
[0057] The compositions of the source topic and the target topic are encoded and processed by the encoder to obtain the topic representations of the source topic and the target topic; wherein the topic representation comprises a first representation matrix and a second representation matrix, the first representation matrix is a topic representation matrix of the source topic, and the second representation matrix is a topic representation matrix of the target topic; and the topic representation is taken as a first topic representation;
[0058] According to the composition anchor point representation and the first topic representation, a first positive example representation of the source topic and a second positive example representation of the target topic are obtained through cross mapping processing respectively; according to prior knowledge, a first negative example representation of the source topic and a second negative example representation of the target topic are constructed; wherein the composition anchor point representation comprises a first composition anchor point representation of the source topic and a second composition anchor point representation of the target topic;
[0059] According to the first composition anchor point representation, the first positive example representation and the first negative example representation are combined to construct a first contrastive loss; according to the second composition anchor point representation, the second positive example representation and the second negative example representation are combined to construct a second contrastive loss;
[0060] The composition anchor point representation of the source topic is scored and predicted by the scorer to obtain a predicted score, and the real score of the composition anchor point representation of the source topic is combined to construct a scoring loss;
[0061] According to the first contrastive loss, the second contrastive loss and the scoring loss, a total loss is constructed; and based on the total loss, the model parameters are updated;
[0062] The encoder is used for encoding the writings of the source subject and the writings of the target subject after the updating processing, to obtain second subject representations of the source subject and the target subject; the second subject representations are used as the first subject representations, and then the step of obtaining the first positive example representation of the source subject and the second positive example representation of the target subject through cross mapping processing according to the first subject representations is returned until a preset training round number is reached, to obtain the cross-subject scoring model;
[0063] The writings of the target subject are scored through the cross-subject scoring model, to obtain a target predicted score.
[0064] Optionally, the encoder is used for encoding the writings of the source subject and the writings of the target subject, to obtain subject representations of the source subject and the target subject, including:
[0065] The encoder is used for encoding the writings of the source subject, to obtain a first representation matrix, and the first representation matrix is used as the subject representation of the source subject; the first representation matrix is a vector representation matrix of the writings of the source subject;
[0066] And the encoder is used for encoding the writings of the target subject, to obtain a second representation matrix, and the second representation matrix is used as the subject representation of the target subject; the second representation matrix is a vector representation matrix of the writings of the target subject.
[0067] Optionally, the first positive example representation of the source subject and the second positive example representation of the target subject are obtained through cross mapping processing according to the writing anchor point representation and the first subject representation, including:
[0068] The first writing anchor point representation is dot multiplied with a transpose matrix of the second representation matrix, to obtain a first projection representation; the first projection representation is subjected to linear variation, to obtain the first positive example representation;
[0069] And the second writing anchor point representation is dot multiplied with a transpose matrix of the first representation matrix, to obtain a second projection representation; the second projection representation is subjected to linear variation, to obtain the second positive example representation;
[0070] The writing anchor point representation is an encoding representation of a current processing writing sample.
[0071] Optionally, a first contrastive loss is constructed according to the first writing anchor point representation, in combination with the first positive example representation and a first negative example representation, including:
[0072] The first contrastive loss is constructed by calculating cosine similarities of the first writing anchor point representation and the first positive example representation and the first negative example representation;
[0073] The first writing anchor point is a writing representation of a first target batch of the source subject.
[0074] Optionally, according to the second composition anchor point representation, a second contrast loss is constructed in combination with the second positive example representation and the second negative example representation, including:
[0075] The second contrast loss is constructed by calculating the cosine similarity of the second composition anchor point representation and the second positive example representation and the second negative example representation.
[0076] The second composition anchor point is a composition representation of a second target batch of the target topic.
[0077] Optionally, according to the first contrast loss, the second contrast loss and the scoring loss, a total loss is constructed, including:
[0078] According to the first contrast loss, the second contrast loss and the scoring loss, a total loss is constructed in combination with a contrast weight and a scoring weight.
[0079] The expression of the total loss is:
[0080]
[0081] In the formula, Ltotal represents the total loss, a represents the contrast weight, l S L1 represents the first contrast loss, l T L2 represents the second contrast loss, β represents the scoring weight, l MSE Ls represents the scoring loss.
[0082] Optionally, the method further includes:
[0083] Based on the first contrast loss and the second contrast loss, the theme difference between the source theme and the target theme is reduced from a target direction by gradient descent; wherein the target direction includes a first direction from the source theme to the target theme and a second direction from the target theme to the source theme.
[0084] In another aspect, an embodiment of the present application provides a cross-theme composition automatic scoring system, including:
[0085] A first module is configured to encode the compositions of the source theme and the compositions of the target theme by an encoder to obtain theme representations of the source theme and the target theme; wherein the theme representation includes a first representation matrix and a second representation matrix, the first representation matrix is a theme representation matrix of the source theme, and the second representation matrix is a theme representation matrix of the target theme; and the theme representation is taken as a first theme representation.
[0086] The second module is configured to obtain a first positive example representation of the source topic and a second positive example representation of the target topic by cross mapping processing according to the composition anchor point representation and the first topic representation; and construct a first negative example representation of the source topic and a second negative example representation of the target topic according to prior knowledge; wherein the composition anchor point representation comprises a first composition anchor point representation of the source topic and a second composition anchor point representation of the target topic.
[0087] The third module is configured to construct a first contrast loss by combining the first positive example representation and the first negative example representation according to the first composition anchor point representation; and construct a second contrast loss by combining the second positive example representation and the second negative example representation according to the second composition anchor point representation.
[0088] The fourth module is configured to perform score prediction on the composition anchor point representation of the source topic by a scorer to obtain a predicted score, and construct a score loss by combining a true score of the composition anchor point representation of the source topic.
[0089] The fifth module is configured to construct a total loss according to the first contrast loss, the second contrast loss and the score loss; and update the model parameters based on the total loss.
[0090] The sixth module is configured to perform encoding processing on the composition of the source topic and the composition of the target topic based on the updated encoder to obtain a second topic representation of the source topic and the target topic; and use the second topic representation as the first topic representation, and then return to the step of obtaining the first positive example representation of the source topic and the second positive example representation of the target topic by cross mapping processing according to the first topic representation, until a preset training round number is reached, to obtain a cross-topic scoring model.
[0091] The seventh module is configured to score the composition of the target topic by the cross-topic scoring model to obtain a target predicted score.
[0092] The contents of the method embodiments of the present application are applicable to the system embodiments of the present application, the system embodiments of the present application specifically implement the same functions as the above-mentioned method embodiments, and achieve the same beneficial effects as the above-mentioned method.
[0093] Another aspect of the embodiment of the present application further provides an electronic device comprising a processor and a memory.
[0094] The memory is configured to store a program.
[0095] The processor executes the program to implement the method as described above.
[0096] The contents of the method embodiments of the present application are applicable to the electronic device embodiments of the present application, the electronic device embodiments of the present application specifically implement the same functions as the above-mentioned method embodiments, and achieve the same beneficial effects as the above-mentioned method.
[0097] Another aspect of the embodiments of the present application also provides a computer readable storage medium, which stores a program, and the program is executed by a processor to implement the method as described above.
[0098] The contents of the method embodiments of the present application are applicable to the computer readable storage medium embodiments of the present application, the computer readable storage medium embodiments specifically implement the same functions as the above method embodiments, and achieve the same beneficial effects as the above method.
[0099] The embodiments of the present application also disclose a computer program product or a computer program, which comprises computer instructions stored in a computer readable storage medium. A processor of a computer device can read the computer instructions from the computer readable storage medium, and the processor executes the computer instructions to enable the computer device to perform the above method.
[0100] The present application will be further described in detail with reference to some specific embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application, and are not used to limit the present application.
[0101] In order to make the content and technical solutions of the present application clearer and more understandable, the related terms and meanings are described as follows:
[0102] Automatic essay scoring: Automatic essay scoring (AES) refers to the process of using a scoring model to score the quality of an essay. Automatic essay scoring is widely used in the field of education, which can effectively reduce the cost of manual scoring and improve the fairness and consistency of scoring.
[0103] Prompt-dependent automated essay scoring: Prompt-dependent automated essay scoring refers to training a scoring model using labeled essay training data under the same theme, and automatically evaluating other unlabeled essays under the same theme.
[0104] Cross-prompt automated essay scoring: Cross-prompt automated essay scoring refers to training a scoring model using labeled essay training data under different themes, and automatically evaluating the quality of unlabeled essays under new themes. Usually, the different themes used for model training are called source themes (Scource Prompt), and the themes of the essays to be scored are called target themes (Target Prompt).
[0105] Contrastive Learning: Contrastive Learning (CL) refers to the process of bringing together samples with similar semantics in the same space, while distinguishing samples with dissimilar semantics. Recent research has shown that contrastive learning not only makes similar semantics more similar, but also alleviates the problem of over-similar encoding of vocabulary by pre-training language models. The main idea of contrastive learning is to construct positive and negative examples of samples, and bring the anchor sample to be trained closer to the semantic similarity of the positive example sample, while moving away from the semantic similarity of the negative example sample.
[0106] In view of the deficiencies of the prior art method, the present application proposes a cross-topic composition automatic scoring method based on topic mapping contrastive learning. For a composition sample of a source topic, the mapping of the source topic composition on the target topic is taken as the positive example of the sample. Similarly, for a composition sample of a target topic, the mapping of the composition of the target topic on the source topic is taken as the positive example of the sample. The purpose of the present application is to use the advantages of contrastive learning to make the model learn and obtain as much shared information between different topics as possible, improve the consistency between different topics, and thus improve the scoring performance of the model in the cross-topic composition automatic scoring task.
[0107] In order to better learn the consistency of articles of different topics in the semantic space, the present application proposes a cross-topic composition automatic scoring method based on topic mapping contrastive learning. The core is to construct positive examples of contrastive learning through topic mapping, so that the model learns more cross-topic information through contrastive learning, and makes the representation of articles of different topics in the semantic space more consistent.
[0108] Suppose the source topic composition data is: Wherein represents the i-th composition in the source topic, represents the score of the i-th composition, i=1,2,…,m, and m represents the number of source topic composition samples. The target topic composition data is Wherein represents the j-th composition in the target topic, j=1,2,…,n, and n represents the number of all composition samples in the target topic.
[0109] The present application mainly focuses on contrastive learning based on topic mapping. The encoding structure of the composition itself can be freely selected by the user. The following uses a pre-trained language model BERT as a composition encoder to describe in detail.
[0110] Referring to Figure 2This invention first uses all source topic essays with labels and target topic essays without labels as input, and trains a cross-topic scoring model using the cross-topic essay automatic scoring method based on topic mapping contrastive learning proposed in this invention; then, the target topic essays without labels are used as input, and the trained cross-topic scoring model is used to predict the scores of the target topic essays to obtain their scores.
[0111] Specifically, refer to Figure 3 The training of the cross-topic scoring model corresponds to Steps 1 to 8. The detailed steps of this invention are as follows:
[0112] Step 1 (corresponding to S1-1): Obtain the topic representations of the source topic and the target topic.
[0113] (1) For all essays in the source topic Encode using a shared encoder to obtain the vector representation matrix (i.e., the first representation matrix) of all essays on the source topic. Here, m represents the vector representation of the i-th essay in the source topic, and m is the number of essays in the source topic. This vector representation matrix is called the source topic representation:
[0114] (2) All essays in the target topic Similarly, a shared encoder is used for encoding to obtain the vector representation matrix (i.e., the second representation matrix) of all essays on the target topic. in Let n be the vector representation of the j-th essay in the target topic, and n be the number of essay samples in the target topic. We call this vector representation matrix the target topic representation:
[0115] Step 2 (corresponding to S1-2): For sample anchor points in the source topic The following is a regular example of its construction:
[0116] anchor the sample The encoding representation (i.e., the first essay anchor point representation) Projected onto the target subject, as shown in Equation (1), by... With the target theme represented by H t Perform a dot product operation on the transpose of the matrix to obtain its projection onto the target subject. in, H represents the dot product operation. t T This represents the transpose of the target topic representation.
[0117]
[0118] To facilitate the calculation of learning loss, for comparison Perform linear transformation operations Make and Keep the dimensions consistent, Represented as source topic anchor point The positive example representation (i.e., the first positive example representation).
[0119] Step 3 (corresponding to S1-3): For sample anchor points in the target topic The following is a regular example of its construction:
[0120] anchor the sample The encoding representation (i.e., the second essay anchor representation) Projected onto the target subject, as shown in equation (2), by... With the target theme represented by H s Perform a dot product operation on the transpose of the matrix to obtain its projection onto the target subject. in, H represents the dot product operation. s T This represents the transpose of the target topic representation.
[0121]
[0122] To facilitate the calculation of learning loss, for comparison Perform linear transformation operations Make and Keep the dimensions consistent, Represented as target topic anchor point The positive example representation (i.e., the second positive example representation).
[0123] Step 4: Construct a loss function for cross-topic contrastive learning. By constraining the contrastive learning loss function, continuously narrow the difference between the source topic and the target topic.
[0124] Corresponding to S1-4, for the source topic, use This represents the source topic essays in a specific batch, and the representation of a particular essay within that batch. Its positive example is its mapping representation on the target topic. The goal is to ensure that anchor points and their mappings on the target topic are close to each other. For negative examples, there is no special design; for ease of description, existing methods can be used to construct negative examples, such as using the representations of other essays in the same batch and their mappings as negative examples. The cosine similarity to both positive and negative examples is calculated separately, and then the loss function for contrastive learning (i.e., the first contrastive loss) is constructed. S, the similarity with positive examples is increased and the similarity with negative examples is decreased by gradient descent in neural network training, the difference between the two topics can be gradually reduced from the source topic to the target topic.
[0125] Corresponding to S1-5, for the target topic, the representation of the target topic in the batch is represented by , the representation of a certain composition in the batch is represented by , and the positive example is its mapping representation on the target topic The purpose is to make the anchor point close to its mapping on the source topic, and the representations of other compositions in the same batch and their mapping representations are used as negative examples. By respectively calculating and the cosine similarity of positive examples and negative examples, and then constructing the loss function (i.e., the second contrastive loss) l T , the similarity with positive examples is increased and the similarity with negative examples is decreased by gradient descent in neural network training, the difference between the two topics can be gradually reduced from the source topic to the target topic.
[0126] So far, from the source topic to the target topic and from the target topic to the source topic, the loss function of cross-topic contrastive learning is constructed, and by gradient descent, the difference between the two topics can be continuously reduced from both directions.
[0127] Step5 (corresponding to S1-6): input the encoded source topic composition representation into the scorer to predict the composition score, and calculate the scoring loss according to the true score of the composition.
[0128] By using mean-square error (MSE) as the loss function of the scoring task. For the composition representation of a batch in the source topic , the true score is , and the score predicted by the scorer is , the scoring loss l MSE is calculated.
[0129] Step6 (corresponding to S1-7): the total loss of the cross-topic scoring training model is:
[0130]
[0131] Where, α represents the weight of contrastive learning, and β represents the weight of the scoring task. The loss function is used for gradient update, and the parameters of the encoder, the scorer and the linear change are updated at the same time.
[0132] Step7 (corresponding to S1-8): after executing all the training samples of the batch, the updated parameters of the encoder are used to obtain the topic representation matrix H of the next round of source topic and target topics and H t .
[0133] Step 8 (corresponding to S1-9): Steps 2 to 7 are repeated to train for a certain number of rounds to obtain a cross-topic essay scoring model.
[0134] Step 9: input the target topic composition into the trained cross-topic scoring model to obtain the final predicted score.
[0135] In summary, the embodiment of the present application provides a cross-topic composition automatic scoring method based on topic mapping contrast learning. For a composition sample of a source topic, the mapping of the source topic composition on the target topic is taken as the positive example of the sample. Similarly, for a composition sample of a target topic, the mapping of the composition of the target topic on the source topic is taken as the positive example of the sample. The purpose of the present application is to use the advantages of contrast learning to make the model learn and obtain as much shared information between different topics as possible, improve the consistency between different topics, and thus improve the scoring performance of the model in the cross-topic composition automatic scoring task. Compared with the prior art, the present application has the following advantages: by taking the mapping of the source topic composition on the target topic as the positive example and inversely taking the mapping of the composition of the target topic on the source topic as the positive example, the theme difference between the source topic and the target topic composition is continuously reduced, and the model is then made to learn more shared information between the topics, which makes the model better applicable to cross-topic scoring.
[0136] In some alternative embodiments, the functions / operations mentioned in the block diagram can not occur in the order mentioned in the operation diagram. For example, depending on the functions / operations involved, two blocks shown in succession can actually be executed substantially simultaneously or the blocks can sometimes be executed in reverse order. In addition, the embodiments presented and described in the flowcharts of the present application are provided by way of example, with the purpose of providing a more comprehensive understanding of the technology. The disclosed method is not limited to the operations and logical flows presented herein. Alternative embodiments are contemplated in which the order of various operations is changed and in which sub-operations described as part of a larger operation are independently executed.
[0137] Furthermore, although the present application is described in the context of functional modules, it is to be understood that one or more of the described functions and / or features can be integrated in a single physical device and / or software module, or one or more functions and / or features can be implemented in separate physical devices or software modules. It will also be appreciated that detailed discussion of the actual implementation of each module is not necessary to an understanding of the application. Rather, the actual implementation of the modules, in combination with their attributes, functions, and internal relationships, are to be taken as being readily understood by one skilled in the art in view of the property disclosed herein. Accordingly, the present application is not limited to the embodiments illustrated in the figures. Rather, one of ordinary skill in the art will be able to make and use the present application without undue experimentation based on the detailed description of the drawings in combination with the property disclosed herein. It is also to be understood that the specific concepts disclosed are merely illustrative and that the scope of the present application is not limited to the specific concepts presented. The scope of the present application is to be limited only by the claims set forth below together with their equivalents.
[0138] If the functions are implemented in software, the functions can be stored in or implemented as one or more computer program products, which can be incorporated into a computer-readable medium for use by or in connection with an apparatus, method, or system as described herein. The computer-readable medium can be a computer- readable storage medium or a computer-readable signal medium. The computer-readable storage medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a computer-readable storage medium can include, for example, an electrical connection based on one or more lines of wire, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disc read-only memory (CD-ROM), and the Internet or other communication link. In the context of this document, a computer-readable storage medium can also include a computer-readable medium that can be accessible via a communication link from one computer application, system, or device to another, directly or through one or more networks or network element. The computer program product can have one or more computer program components implemented as one or more sequences of instructions (for example, software or firmware) that reside on or in one or more computer-readable media and are executed by the processor of the computer. The software and / or firmware can include, but is not limited to, device drivers, firmware, operating systems, development tools, applications software, data, or any other suitable information including programs desired to be executed by the computer. The computer software product can also reside on or within any computer-readable medium for use by or in connection with an application, system, or device.
[0139] Logic and / or steps represented in flow diagrams or otherwise described herein, for example, can be embodied in computer-readable instructions, instructions configurable to operate processing circuitry, or in any other way that is mapped to the logic functions described. For the purposes of this description, a "computer-readable medium" can be any apparatus that can contain, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution apparatus, system or device. The computer-readable medium can be, for example but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device. More specific examples of a computer-readable medium include, for example, an electrical connection based on one or more lines of wire, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disc read-only memory (CD-ROM), and the Internet or other communication link. The computer-readable medium can also include a computer-readable medium that is accessible via a communication link from one computer application, system, or device to another, directly or through a network or network element. The computer-readable medium can be any combination of these or other computer-readable media.
[0140] More specific examples (a non-exhaustive list) of the computer-readable medium include the following: an electrical connection (electronic) having one or more wires, a portable computer diskette (magnetic), a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, and a portable compact disc read-only memory (CDROM). Additionally, the computer-readable medium can also be paper or another suitable medium upon which the program is printed, as the program can be electronically captured, for example, via optical scanning of the paper or other medium, then compiled, interpreted, or otherwise processed in a suitable manner, if necessary, and then stored in a computer memory.
[0141] It should be understood that aspects of the application can be implemented in hardware, software, firmware or combinations thereof. In the embodiments described above, various steps or methods can be implemented, for example, by software or firmware stored in memory and executed by a suitable instruction execution device. If implemented in hardware, for example, in another embodiment, any of the following techniques can be used in whole or in part: discrete logic circuitry having logic gates for implementing logic functions upon data signals, application specific integrated circuits having appropriate combinational logic gates, programmable gate arrays (PGA), field programmable gate arrays (FPGA), and so forth, as well as or instead of software instructions.
[0142] In the description of the specification, the description of the terms "one embodiment", "some embodiments", "an example", "a specific example", or "some examples" and the like means that the specific feature, structure, material or characteristic being described is included in at least one embodiment or example of the application. The illustrative description of the above terms does not necessarily refer to the same embodiment or example. Moreover, the description of the specific features, structures, materials or characteristics can be combined in any suitable manner in one or more embodiments or examples.
[0143] Although embodiments of the application have been shown and described, it would be recognized by those of ordinary skill in the art that various changes, modifications, substitutions, and alterations can be made hereto without departing from the spirit and scope of the application, which is defined by the following claims and their equivalents.
[0144] The above is a specific description of the preferred embodiments of the present application, but the present application is not limited to the described embodiments, and those skilled in the art can make various equivalent modifications or replacements without departing from the spirit of the present application, and these equivalent modifications or replacements are all included in the scope defined by the claims of the present application.
Claims
1. A method for automatically scoring cross-topic compositions, the method comprising: The method comprises the following steps: encoding the compositions of the source subject and the compositions of the target subject by an encoder to obtain subject representations of the source subject and the target subject; wherein the subject representations comprise a first representation matrix and a second representation matrix, the first representation matrix is a subject representation matrix of the source subject, and the second representation matrix is a subject representation matrix of the target subject; and the subject representations are taken as first subject representations; obtaining first positive example representations of the source subject and second positive example representations of the target subject by cross-mapping processing according to composition anchor point representations and the first subject representations; and constructing first negative example representations of the source subject and second negative example representations of the target subject according to priori knowledge; wherein the composition anchor point representations comprise first composition anchor point representations of the source subject and second composition anchor point representations of the target subject; constructing a first contrast loss by combining the first positive example representations and the first negative example representations according to the first composition anchor point representations; and constructing a second contrast loss by combining the second positive example representations and the second negative example representations according to the second composition anchor point representations; obtaining a predicted score by scoring the composition anchor point representations of the source subject by a scorer, and constructing a scoring loss by combining a true score of the composition anchor point representations of the source subject; constructing an overall loss according to the first contrast loss, the second contrast loss and the scoring loss; and updating model parameters based on the overall loss; obtaining second subject representations of the source subject and the target subject by encoding the compositions of the source subject and the compositions of the target subject by the encoder after the updating processing; taking the second subject representations as first subject representations, and then returning to the step of obtaining the first positive example representations of the source subject and the second positive example representations of the target subject by cross-mapping processing according to the first subject representations until a preset training round number is reached to obtain a cross-subject scoring model; scoring the compositions of the target subject by the cross-subject scoring model to obtain a target predicted score; wherein the step of obtaining the first positive example representations of the source subject and the second positive example representations of the target subject by cross-mapping processing according to the composition anchor point representations and the first subject representations comprises: performing a dot multiplication operation on the first composition anchor point representations and a transposed matrix of the second representation matrix to obtain first projection representations; and performing a linear change operation on the first projection representations to obtain the first positive example representations; and performing a dot multiplication operation on the second composition anchor point representations and a transposed matrix of the first representation matrix to obtain second projection representations; and performing a linear change operation on the second projection representations to obtain the second positive example representations; wherein the composition anchor point representations are the encoding representations of the current processing composition sample.
2. The method of claim 1, wherein, The step of encoding the compositions of the source subject and the compositions of the target subject by the encoder to obtain the subject representations of the source subject and the target subject comprises: encoding the compositions of the source subject by the encoder to obtain a first representation matrix, and taking the first representation matrix as the subject representation of the source subject; wherein the first representation matrix is a vector representation matrix of the compositions of the source subject. and, encoding the writings of the target subject by an encoder to obtain a second representation matrix, taking the second representation matrix as a subject representation of the target subject; the second representation matrix is a vector representation matrix of the writings of the target subject.
3. The method of claim 1, wherein, The first contrast loss is constructed according to the first writing anchor point representation, in combination with the first positive example representation and the first negative example representation, including: The first contrast loss is constructed by calculating the cosine similarity of the first writing anchor point representation and the first positive example representation and the first negative example representation. The first writing anchor point representation is a writing representation of a first target batch of the source subject.
4. The method of claim 1, wherein, The second contrast loss is constructed according to the second writing anchor point representation, in combination with the second positive example representation and the second negative example representation, including: The second contrast loss is constructed by calculating the cosine similarity of the second writing anchor point representation and the second positive example representation and the second negative example representation. The second writing anchor point representation is a writing representation of a second target batch of the target subject.
5. The method of claim 1, wherein, The overall loss is constructed according to the first contrast loss, the second contrast loss and the scoring loss, including: The overall loss is constructed according to the first contrast loss, the second contrast loss and the scoring loss, in combination with a contrast weight and a scoring weight. The expression of the overall loss is: wherein, denotes the overall loss, denotes the contrast weight, denotes the first contrast loss, denotes the second contrast loss, denotes the score weight, denotes the score loss.
6. The method of claim 1, wherein, Further comprising: Based on the first contrast loss and the second contrast loss, the theme difference between the source subject and the target subject is reduced from a target direction by gradient descent; wherein the target direction includes a first direction from the source subject to the target subject and a second direction from the target subject to the source subject.
7. An automatic essay scoring system across topics, characterized by, The system is applied to the cross-subject writing automatic scoring method of claim 1, and the system comprises: A first module is configured to encode writings of a source subject and writings of a target subject by an encoder to obtain subject representations of the source subject and the target subject; wherein the subject representations include a first representation matrix and a second representation matrix, the first representation matrix is a subject representation matrix of the source subject, and the second representation matrix is a subject representation matrix of the target subject; and the subject representations are taken as a first subject representation. A second module is configured to obtain a first positive example representation of the source subject and a second positive example representation of the target subject by cross-mapping processing according to a writing anchor point representation and the first subject representation; and obtain a first negative example representation of the source subject and a second negative example representation of the target subject according to prior knowledge; wherein the writing anchor point representation includes a first writing anchor point representation of the source subject and a second writing anchor point representation of the target subject. A third module is configured to construct a first contrast loss according to the first writing anchor point representation, in combination with the first positive example representation and the first negative example representation; and construct a second contrast loss according to the second writing anchor point representation, in combination with the second positive example representation and the second negative example representation. A fourth module is configured to score the writing anchor point representation of the source subject by a scorer to obtain a predicted score, and construct a scoring loss in combination with a true score of the writing anchor point representation of the source subject. A fifth module configured to construct an overall loss based on the first contrast loss, the second contrast loss, and the scoring loss; and update model parameters based on the overall loss; A sixth module configured to encode the writings of the source topic and the writings of the target topic based on the updated encoder to obtain second topic representations of the source topic and the target topic; and use the second topic representations as the first topic representations, and then return to the step of obtaining the first positive example representation of the source topic and the second positive example representation of the target topic based on the first topic representations through cross mapping until a preset number of training rounds is reached to obtain a cross-topic scoring model; A seventh module configured to score the writings of the target topic through the cross-topic scoring model to obtain a target predicted score. 8.An electronic device, comprising a processor and a memory; The memory is configured to store a program; The processor executes the program to implement the method of any one of claims 1 to 6.
9. A computer-readable storage medium, characterized in that, The storage medium stores a program, and the program is executed by the processor to implement the method of any one of claims 1 to 6.