A test paper structure parsing method based on span information and local attention
Through the analysis method of test paper structure based on span information and local attention, the problems of high segmentation error rate and difficult model training in the existing technology are solved, and efficient and accurate test paper analysis is achieved, which is suitable for storage of a large number of electronic test questions. and analysis.
Patent Information
- Application Number
- CN202210063063.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-01-19
- Publication Date
- 2024-11-12
- Estimated Expiration
- 2042-01-19
AI Technical Summary
The existing technology ignores span segmentation in the analysis of test papers, resulting in high segmentation error rate and classification paradigm make model training difficult and inefficient, making it difficult to cope with the storage and analysis needs of a large number of electronic test questions.
Using the test paper structure analysis method based on span information and local attention, the line semantic representation is extracted through the pre-trained model, combined with the two-way long and short-term memory network and supervisory attention, the auxiliary task modeling segmentation structure information is designed, and the major questions are analyzed first and then split. Score the difficulty of model training and improve accuracy.
It effectively reduces the segmentation error rate, improves the accuracy and efficiency of test paper analysis, and adapts to the storage and analysis needs of a large number of electronic test questions.
Smart Images

Figure CN114429118B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of test question matching, and in particular to a test paper structure parsing method based on span information and local attention. Background Art
[0002] With the vigorous development of the education industry, more and more test papers and questions are appearing on the market, among which electronic test questions account for a very large proportion. How to collect test questions and store them in a structured manner in the corresponding database is of great significance to the education industry. In the past, because test papers were few and easy to collect, the analysis of test papers could be done manually, but manual work is too time-consuming and inefficient, making it difficult to cope with the current situation.
[0003] With the development of artificial intelligence technology, the present invention aims to use a small amount of manually labeled data to train a deep learning model to split out all question types in the test paper and parse out the content of each question, so as to facilitate subsequent database storage and future test paper analysis and mining applications. The previous method was completely based on a classification paradigm, which classified each basic unit of the test paper and then merged them according to the classification results to obtain the final test paper analysis results. However, this type of method overemphasizes classification and ignores the role of span segmentation, resulting in partial segmentation errors in the test paper analysis results, that is, the divided test paper paragraphs may cause the entire segmentation error because of only one missing line or one extra line. Summary of the invention
[0004] The purpose of the present invention is to solve the above-mentioned defects in the prior art and provide a method for parsing the test paper structure based on span information and local attention. The method first uses a pre-trained model to obtain the row semantic representation of the electronic test paper, and then obtains the row representation containing context information through a bidirectional long short-term memory network. Then, through the supervised attention based on span information and an auxiliary task based on span classification to explicitly model the segmentation structure information of the test paper and the feature information based on span, the test paper is split into different types of large question segments and their corresponding types through a classifier; each large question segment obtained is further processed separately, and is split into small questions using a classifier and various feature information extracted before, and the final parsing result is returned. The present invention adopts a two-stage method of parsing the large questions first and then splitting the small questions, which effectively takes into account the efficiency of test paper parsing and reduces the training difficulty of the model to improve the accuracy of test question parsing.
[0005] The purpose of the present invention can be achieved by adopting the following technical solutions:
[0006] A test paper structure parsing method based on span information and local attention, the test question matching method comprises the following steps:
[0007] S1. Obtain electronic test papers and pre-process the multivariate data in the electronic test papers into a unified line-by-line structured text;
[0008] S2, firstly, extract the semantic features of each line of the test paper through a large-scale pre-trained model, and use the bidirectional long short-term memory network to model the line context information;
[0009] S3, using the supervised attention based on span information and designing an auxiliary task based on span classification to explicitly model the segmentation structure information of the test paper and the feature information based on span, and split the test paper into different types of large question segments and their types through the classifier;
[0010] S4. Process each obtained large question segment separately, split it into small questions using the classifier and various feature information extracted previously, and return the final parsing result;
[0011] The specific calculation process of preprocessing the electronic test paper into a line-by-line structured text in step S1 is as follows:
[0012] S1.1. For the text information in the test questions, first use the word segmenter of the pre-trained model BERT to segment the text and filter out the stop words;
[0013] S1.2. For non-textual multivariate data in the test questions, such as formulas, pictures, and tables, a non-text data processing method is designed based on the characteristics of these data. The specific process is as follows:
[0014] First, the data are cleaned to obtain data with less noise. Considering that these data information is useless for analyzing the test paper structure, they are replaced with unified special symbols based on the regular expression replacement method.
[0015] S1.3. The branch structure of the electronic test paper is retained to obtain the final test paper list as follows:
[0016]
[0017] in Representative After preprocessing, Represents the number of rows contained in the electronic test paper.
[0018] The specific calculation process of step S1 using the pre-trained model and the bidirectional long short-term memory network to obtain the row feature representation containing context information is as follows:
[0019] S2.1. Given a preprocessed test paper D, input each row into the pre-trained model to obtain the semantic vector representation of each row:
[0020]
[0021] in Afterwards, the long short-term memory network is used to model the vector sequence The specific calculation process is as follows:
[0022]
[0023] in , is the latitude of the hidden state, is the time step The input data, , , They are the input gate, forget gate and output gate of the long short-term memory network. This is the currently added information. is the information of the memory unit, is a nonlinear activation function, is an element-by-element multiplication, W , U and b is a trainable parameter;
[0024] In order to model the bidirectional characteristics of the test paper structure, the test paper sequence vector is input into the long short-term memory network from the front and back directions respectively, and the vectors in the two directions are directly concatenated as the final test paper hidden layer representation:
[0025]
[0026]
[0027]
[0028] in , They represent the hidden layer representations obtained by forward and backward input into the long short-term memory network, Represents a vector concatenation operation,
[0029] The process of segmenting the large question by integrating the supervised local attention and the auxiliary task based on span classification in step S3 is as follows:
[0030] S3.1. Given a hidden layer representation H of a test paper, construct , calculate the differentiable left and right boundary masks:
[0031]
[0032] in , Represent the left and right boundary masks respectively, is a trainable parameter, Used to ensure that the generated boundaries meet the conditions , then the final local range mask matrix can be calculated:
[0033]
[0034] in is an upper triangular matrix of all 1s, and then the self-attention mechanism is combined with the obtained local range mask matrix to enable the model to focus on semantically related sentences around the target position and eliminate noise aggregation:
[0035]
[0036] in is a trainable parameter, and then introduces segmentation signals to guide the learning of dynamic local attention to capture coherent semantics. Specifically, the binary cross entropy loss is used. To describe the attention matrix and segmented signals The difference between:
[0037]
[0038] in is the sigmoid activation function, Representative Bank With line be within a segment;
[0039] S3.2. In addition, due to the obvious label consistency of sentences within a span, an additional auxiliary task called span-based classification is introduced:
[0040]
[0041]
[0042] in is the output probability at the span level, represents a learnable positional encoding, is the cross entropy loss function, Indicates and A line is the beginning and end of a segment;
[0043] S3.3. Finally, the big question parsing model is trained according to the following loss function:
[0044]
[0045] in It is used for balance and The hyperparameter of loss strength, The guided main classifier can give a category label for each row, and the final analysis result of the big question is obtained after merging the rows of the same category:
[0046]
[0047] Where M represents the number of sections with different categories in the test paper D. Each section contains one or more rows. Represents the category label of the i-th segment.
[0048] The specific process of dividing the obtained large question into small questions in step S4 is as follows:
[0049] S4.1 Given a large question, divide it into sections , which contains one or more rows, trains a sub-question segmentation classifier using the binary cross entropy loss function to obtain the final sub-question segmentation result:
[0050]
[0051] in , 1 means the kth row is a split row;
[0052] Compared with the prior art, the present invention has the following advantages and effects:
[0053] (1) The present invention proposes a test paper structure parsing method based on span information and local attention, which effectively reduces the error rate ratio of partial segmentation errors by considering span information and segmentation information.
[0054] (2) Since the test paper analysis not only needs to analyze each question and its corresponding label, the present invention proposes a two-stage analysis method of first splitting the large questions and then splitting the small questions, which effectively reduces the learning difficulty of the model and improves the accuracy. BRIEF DESCRIPTION OF THE DRAWINGS
[0055] Figure 1 It is a flowchart of a test paper structure analysis method based on span information and local attention disclosed in an embodiment of the present invention. DETAILED DESCRIPTION
[0056] In order to make the purpose, technical solution and advantages of the present invention more clear, the present invention is further described in detail below in combination with specific implementation methods and with reference to the accompanying drawings. In the following description, the description of well-known structures and technologies is omitted to avoid unnecessary confusion of the concept of the present invention.
[0057] Example
[0058] like Figure 1 As shown in FIG. 1 , a test paper structure parsing method based on span information and local attention includes the following steps:
[0059] S1. Obtain the electronic mathematics test paper and pre-process the multivariate data in the electronic test paper into a unified line-by-line structured text;
[0060] S1.1. For the text information in the test questions, first use the word segmenter of the pre-trained model BERT to segment the text and filter out the stop words;
[0061] S1.2. For non-textual multivariate data in the test questions, such as formulas, pictures, and tables, a non-text data processing method is designed based on the characteristics of these data. The specific process is as follows:
[0062] First, the data are cleaned to obtain data with less noise. Considering that these data information is useless for analyzing the test paper structure, they are replaced with unified special symbols based on the regular expression replacement method.
[0063] S1.3. The branch structure of the electronic test paper is retained to obtain the final test paper list as follows:
[0064]
[0065] in Representative After preprocessing, Represents the number of rows contained in the electronic test paper.
[0066] S2, firstly, extract the semantic features of each line of the test paper through a large-scale pre-trained model, and use the bidirectional long short-term memory network to model the line context information;
[0067] S2.1. Given a preprocessed test paper D, input each row into the pre-trained model to obtain the semantic vector representation of each row:
[0068]
[0069] The pre-training model selected is the BERT-base model, , The value is 768 dimensions, and then the long short-term memory network is used to model the vector sequence The specific calculation process is as follows:
[0070]
[0071] in , is the latitude of the hidden state, is the time step The input data, , , They are the input gate, forget gate and output gate of the long short-term memory network. This is the currently added information. is the information of the memory unit, , is a nonlinear activation function, is an element-by-element multiplication, W , U and b It is a trainable parameter, and the size of the hidden layer state dimension d is set to 200;
[0072] In order to model the bidirectional characteristics of the test paper structure, the test paper sequence vector is input into the long short-term memory network from the front and back directions respectively, and the vectors in the two directions are directly concatenated as the final test paper hidden layer representation:
[0073]
[0074]
[0075]
[0076] in , They represent the hidden layer representations obtained by forward and backward input into the long short-term memory network, Represents a vector concatenation operation,
[0077] S3, using the supervised attention based on span information and designing an auxiliary task based on span classification to explicitly model the segmentation structure information of the test paper and the feature information based on span, and split the test paper into different types of large question segments and their types through the classifier;
[0078] S3.1. Given a hidden layer representation H of a test paper, construct , calculate the differentiable left and right boundary masks:
[0079]
[0080] in , Represent the left and right boundary masks respectively, is a trainable parameter, Used to ensure that the generated boundaries meet the conditions , then the final local range mask matrix can be calculated:
[0081]
[0082] in is an upper triangular matrix of all 1s, and then the self-attention mechanism is combined with the obtained local range mask matrix to enable the model to focus on semantically related sentences around the target position and eliminate noise aggregation:
[0083]
[0084] in is a trainable parameter;
[0085] Then, the segmentation signal is introduced to guide the learning of dynamic local attention to capture coherent semantics. Specifically, the binary cross entropy loss is used. To describe the attention matrix and segmented signals The difference between:
[0086]
[0087] in is the sigmoid activation function, Representative Bank With line be within a segment;
[0088] S3.2. In addition, due to the obvious label consistency of sentences within a span, an additional auxiliary task called span-based classification is introduced:
[0089]
[0090]
[0091] in is the output probability at the span level, represents a learnable positional encoding, is the cross entropy loss function, Indicates and A line is the beginning and end of a segment;
[0092] S3.3. Finally, the big question parsing model is trained according to the following loss function:
[0093]
[0094] in It is used for balance and The hyperparameter of loss strength, The guided main classifier can give a category label for each row, and the final analysis result of the big question is obtained after merging the rows of the same category:
[0095]
[0096] Where M represents the number of sections with different categories in the test paper D. Each section contains one or more rows. Represents the category label of the i-th segment. Taking the math test as an example, there are multiple-choice questions, fill-in-the-blank questions, and answer questions.
[0097] S4. Process each obtained large question segment separately, split it into small questions using the classifier and various feature information extracted previously, and return the final parsing result;
[0098] S4.1 Given a large question, divide it into sections , which contains one or more rows, trains a sub-question segmentation classifier using the binary cross entropy loss function to obtain the final sub-question segmentation result:
[0099]
[0100] in , 1 means the kth row is a split row.
Claims
1. A test paper structure parsing method based on span information and local attention, characterized in that: The test paper structure analysis comprises the following steps: S1. Obtain electronic test papers and pre-process the multivariate data in the electronic test papers into a unified line-by-line structured text; S2, firstly, extract the semantic features of each line of the test paper through a large-scale pre-trained model, and use the bidirectional long short-term memory network to model the line context information; S3, using the supervised attention based on span information and designing an auxiliary task based on span classification to explicitly model the segmentation structure information of the test paper and the feature information based on span, and split the test paper into different types of large question segments and their types through the classifier; S4. Process each obtained large question segment separately, split it into small questions using the classifier and various feature information extracted previously, and return the final parsing result; The specific calculation process of preprocessing the electronic test paper into a line-by-line structured text in step S1 is as follows: S1.
1. For the text information in the test questions, first use the word segmenter of the pre-trained model BERT to segment the text and filter out the stop words; S1.
2. For the non-textual multi-dimensional data in the test questions, including formulas, pictures, and tables, a non-textual data processing method is designed based on the characteristics of these data. The specific process is as follows: First, the data are cleaned to obtain data with less noise. Considering that these data information is useless for analyzing the test paper structure, they are replaced with unified special symbols based on the regular expression replacement method. S1.
3. The branch structure of the electronic test paper is retained to obtain the final test paper list as follows: ; in Representative After preprocessing, Represents the number of rows contained in the electronic test paper; The specific calculation process of step S2 using the pre-trained model and the bidirectional long short-term memory network to obtain the row feature representation containing context information is as follows: S2.
1. Given a preprocessed test paper D, input each row into the pre-trained model to obtain the semantic vector representation of each row: ; in , and then use the long short-term memory network to model the vector sequence The specific calculation process is as follows: ; in , is the dimension of the hidden state, is the time step The input data, , , They are the input gate, forget gate and output gate of the long short-term memory network. This is the currently added information. is the information of the memory unit, , is a nonlinear activation function, is an element-by-element multiplication, W , U and b is a trainable parameter; In order to model the bidirectional characteristics of the test paper structure, the test paper sequence vector is input into the long short-term memory network from the front and back directions respectively, and the vectors in the two directions are directly concatenated as the final test paper hidden layer representation: ; ; ; in , They represent the hidden layer representations obtained by forward and backward input into the long short-term memory network, Represents a vector concatenation operation, The process of segmenting the large question by integrating the supervised local attention and the auxiliary task based on span classification in step S3 is as follows: S3.
1. Hidden layer representation of a given test paper , build , calculate the differentiable left and right boundary masks: ; in , Represent the left and right boundary masks respectively, is a trainable parameter, Used to ensure that the generated boundaries meet the conditions , and then calculate the final local range mask matrix: ; in is an upper triangular matrix of all 1s, and then the self-attention mechanism is combined with the obtained local range mask matrix to make the model focus on semantically related sentences around the target position and eliminate noise aggregation: ; in is a trainable parameter; Then, the segmentation signal is introduced to guide the learning of dynamic local attention to capture coherent semantics. Specifically, the binary cross entropy loss is used. To describe the attention matrix and segmented signals The difference between: ; in is the sigmoid activation function, Representative Bank With line be within a segment; S3.
2. In addition, due to the obvious label consistency of sentences within a span, an additional auxiliary task called span-based classification is introduced: ; ; in is the output probability at the span level, represents a learnable positional encoding, is the cross entropy loss function, Indicates Row and A line is the beginning and end of a segment; S3.
3. Finally, the big question parsing model is trained according to the following loss function: ; in It is used for balance and The hyperparameter of loss strength, The guided main classifier can give a category label for each row, and the final analysis result of the big question is obtained after merging the rows of the same category: ; Where M represents the number of sections with different categories in the test paper D. Each section contains one or more rows. Represents the category label of the i-th segment; The specific process of dividing the obtained large question into small questions in step S4 is as follows: S4.1 Given a large question, divide it into sections , which contains one or more rows, trains a sub-question segmentation classifier using the binary cross entropy loss function to obtain the final sub-question segmentation result: ; in , 1 means the kth row is a split row.
