A test reading comprehension answer sentence summarization method fusing CPT and optimization model

Abstract

The application belongs to the technical field of natural language processing, and discloses a test reading comprehension answer sentence summarization method fusing CPT and an optimization model. The application summarizes and generalizes background material answer sentences based on a CPT model, and optimizes the text output by the CPT model by comprehensively using various factors such as semantic authority, question matching degree, dependency syntax and sentence length, to generate an answer sentence which is accurate in information, high in question matching degree, complete in syntax structure and summarizes and generalizes answer points, thereby improving the summarization and generalization capability of a question and answer type reading comprehension system. The achievement of the application can be applied to an education test tutoring platform to provide an effective technical solution for science and education.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of natural language processing, and particularly relates to a test reading comprehension answer sentence summarization method fusing a CPT and an optimization model. BACKGROUND

[0002] Machine reading comprehension aims to enable machines to deeply understand and analyze given background materials and answer related questions like humans, and is an important and challenging task in the field of natural language processing. Thanks to the development of deep learning technology and large-scale data sets, large-scale pre-training models have the advantage of strong text modeling capability. Among them, CPT (Chinese Pre-trained Unbalanced Transformer) is a newly proposed Chinese pre-training model that can obtain rich information in feature extraction and semantic representation.

[0003] The problems of Chinese test reading comprehension are complex and diverse, and the semantics expressed by the background material sentences are rich and long. When the extractive question answering system outputs these sentences as answer sentences, there are many redundancies or irrelevant information in the answers. At the same time, the test reading comprehension question answering question is scored according to the answer points, that is, under the condition of limited word number, the analyzed and summarized text can contain more scoring points. Therefore, it is necessary to summarize the extractive answer sentences and regenerate more concise answer sentences that meet the answer points.

[0004] However, the test reading comprehension answer generation task is different from the ordinary sentence generation task. When measuring the credibility of words, in addition to considering the meaning expressed by the sentence itself, the matching degree of the answer sentence and the question, the accuracy of the information and the integrity of the structure are also considered. In response to the call of the national science and education, technology and education, the application fuses CPT and optimization model to summarize the test reading comprehension answer sentence. The CPT model can fully learn the corresponding relationship between the source sentences in the background material and the reference answer sentences, and generate text that can better summarize the main information of the sentences. The optimization model can further optimize the generated answer sentences. The corresponding achievements of the application can be directly applied to the education examination guidance platform, providing effective technical support for science and education, realizing significant social value and economic value. SUMMARY

[0005] In view of the above problems, the application provides a test reading comprehension answer sentence summarization method fusing a CPT and an optimization model.

[0006] In order to achieve the above purpose, the application adopts the following technical solutions:

[0007] A test reading comprehension answer sentence summarization method fusing a CPT and an optimization model, comprising the following steps:

[0008] Step S1, constructing a summary corpus;

[0009] Step S2, training and fine-tuning the CPT model using the summary corpus;

[0010] Step S3, input the background material answer sentence into the CPT model trained and fine-tuned in step S2 for summarization;

[0011] Step S4, calculating the credibility of the words in the answer sentence of the background material by comprehensively considering the semantic authority, question matching and syntactic structure importance;

[0012] Step S5: Optimize the text output by the CPT model using the optimization model method, regenerate new answer sentences, and complete the summary of the test reading comprehension answer sentences.

[0013] Furthermore, the step S1 is to construct the summary corpus as follows: using corpus (1) real test questions and simulation questions from various provinces in previous years; (2) test reading comprehension questions crawled from the examination website; (3) NLPCC2017 conference Chinese single document summary assessment task data set; first, the sentences in the original text are divided into single sentences according to punctuation marks, and then the similarity between the summary sentence and each sentence in the original text is calculated according to the following formula, and the sentence with the highest similarity in the original text is selected as the corresponding sentence of the summary sentence to construct the summary corpus;

[0014]

[0015] in, Represents the i-th summary sentence S in the abstract i The word collection, Represents the jth sentence P in the original text j The word set, w k Indicates words in a sentence.

[0016] Furthermore, in step S2, the CPT model is trained and fine-tuned using the summary corpus as follows: special tags "CLS" and "SEP" are added to the beginning and end of the source sentence and target sentence in the summary corpus respectively, the source sentence sequence is input into the shared encoder of the CPT model, and the target sentence sequence is input into the generative decoder, and the CPT model is trained and fine-tuned so that the CPT model has the ability to remove redundant information and summarize.

[0017] Furthermore, in step S3, the background material answer sentence is input into the CPT model trained and fine-tuned in step S2 for summarization, specifically: the background material answer sentence S={w1,w2,...,w m} is input into the trained CPT model, m represents the number of words in the answer sentence of the background material, w irepresent the i-th word in the sentence, and generate a more concise and authoritative answer sentence, i.e., a summarized answer sentence T = {t1, t2,..., t n}.

[0018] Further, the step S4 integrates the semantic authority degree, the question matching degree and the syntactic structure importance to calculate the credibility of the words in the background material answer sentence, including the following steps:

[0019] Step S4.1, according to the text generated by the step 3 CPT model to calculate the semantic authority degree of the word α i The text generated by the CPT model expresses the main meaning of the original sentence, i.e., the background material answer sentence, if the word w m} in the original sentence S = {w1, w2,..., w i In the text T = {t1, t2,..., t n} generated by the CPT model, the semantic authority degree α i of the word w i is set to 1, otherwise it is set to 0;

[0020] Step S4.2, combine Word2Vec and HowNet to calculate the matching degree sim i of the word w i with the question Q, which is beneficial to recall the words in the original sentence that are more relevant to the question, and the calculation formula is as follows:

[0021]

[0022]

[0023] where Q * represents the word list of the question Q after removing stop words, q j represents the word in the list, represents the cosine similarity of the word vectors of w i and q j calculated by Word2Vec, and represent the word vectors of w i and q j respectively, simHowNet(w i , q j ) represents the similarity of w i and q j calculated by HowNet, β1 and β2 are positive parameters that need to be set, and β1 + β2 = 1;

[0024] Step S4.3, according to the depth of the word in the dependency syntactic analysis tree to calculate the syntactic structure importance str iThe closer the word to the root node of the tree, the greater the probability that the word becomes the main component of the sentence, and the farther the word is from the root node and the greater the depth, the more likely that the word has no impact on the meaning of the original sentence, and the calculation formula is as shown below:

[0025] str i =-dep i (4)

[0026] Wherein, dep i represents the depth of the word w i in the dependency syntax analysis tree, and the present application provides that the depth of the tree root node is 0, the depth of the direct child node of the root node is 1, and so on.

[0027] Step S4.4, the semantic authority degree alpha i , the question matching degree sim i and the syntactic structure importance str i Three factors are combined to calculate the credibility I i of the word:

[0028] I i =λ1×α i +λ2×sim i +λ3×str i (5)

[0029] Wherein, λ1, λ2 and λ3 are positive parameters that need to be set.

[0030] Further, the text output by the CPT model is optimized in the step S5 to regenerate a new answer sentence, including the following steps:

[0031] Step S5.1, the optimization model objective function is constructed by the credibility of the word, which is used to select the main information and delete the redundant words, and ensure the accuracy of the answer sentence:

[0032]

[0033] Wherein, m represents the number of words in the background material answer sentence, y i represents the label of whether the i-th word w i in the sentence should be deleted or retained, if y i =1, the word w i is retained, if y i =0, the word w i is deleted, I i represents the credibility of the word w i ;

[0034] Step S5.2, introduce the syntactic structure integrity, semantic correctness and coherence constraints to guarantee the fluency of the generated answer sentence, and introduce the sentence length constraint to control the length of the generated answer sentence;

[0035] (1) Syntactic structure integrity constraint

[0036] In the optimized answer sentence, if the child node word is retained, its parent node word should also be retained:

[0037]

[0038] where w p is the parent node word of w i , y p and y i are their corresponding labels, respectively;

[0039] For some dependency relations, such as subject-predicate relations, if the parent node word (predicate) is retained, the child node word (subject) should also be retained:

[0040]

[0041] where R represents a set of dependency relations that are often retained in the optimized answer sentence if the parent node word is retained: R{SBV,VOB,IOB,FOB,POB,ATT,DBL,CMP,WP}, which is constructed according to the characteristics of the test reading comprehension answer sentence summarization task; ip represents the dependency relation between the child node word w i and its parent node word w p ;

[0042] (2) Semantic correctness constraint

[0043] When the child node word and the parent node word are in a parallel relation (COO), and the parent node word is not the core word of the sentence (HED), if the parent node word is retained, the child node word should also be retained, otherwise the semantic expression of the answer sentence is not complete:

[0044]

[0045] where R p represents the dependency relation between w p and its parent node word;

[0046] Negative words affect the semantics of the answer sentence. When the child node word and the parent node word are in an adverbial relation (ADV), and the child node word contains a negative word, if the parent node word is retained, the child node word should also be retained:

[0047]

[0048] wherein no_adv represents an element in the list no_advlist {not, no, none, nor, nought, no, naught, without} ;

[0049] (3) Coherence constraint

[0050] The parallel relation conjunctions "and, with, follow, and, as well as" can guarantee the connectivity of the optimized answer sentence. When the child node word and the parent node word are left-attached relation (LAD), and the child node word is a parallel relation conjunction, if the parent node word is retained, the child node word should also be retained:

[0051]

[0052] wherein coolist represents a set of parallel relation conjunctions: coolist {and, with, follow, and, as well as};

[0053] When the child node word and the parent node word are right-attached relation (RAD), and the child node word is "of" or "ed", if the parent node word is retained, the child node word should also be retained:

[0054]

[0055] (4) Sentence length constraint

[0056] Because efforts are made to compress the sentence to ensure that the optimized answer sentence is shorter than the original sentence, but if the compression rate of the optimized answer sentence is too large, the authoritative information of the original sentence will be lost, so the sentence length constraint needs to be introduced to control the maximum and minimum length of the optimized answer sentence:

[0057]

[0058] wherein m represents the number of words in the original sentence, and ω is a positive parameter that needs to be set, 0 < ω < 1.

[0059] Step S5.3, under the constraints of syntactic structure integrity, semantic correctness, coherence and sentence length, maximize the objective function, convert the word sequence S = {w1, w2,..., w m} of the background material answer sentence into the label sequence y = {y1, y2,..., y m}, y i ∈ {0, 1}; if y i = 1, it means that the word w i is retained; if y i = 0, it means that the word w i is deleted, and finally an answer sentence A = {a1, a2,..., ak},(k<m)。

[0060] Compared with the prior art, the present application has the following advantages:

[0061] (1) The present application constructs a summarization corpus for exam reading comprehension answer generation, effectively makes up for the lack of exam reading comprehension training corpus, and avoids the phenomenon of model overfitting;

[0062] (2) The present application summarizes the background material answer sentence based on the CPT model, and generates an answer sentence that can better summarize the main meaning of the original sentence;

[0063] (3) The present application measures the credibility of words by comprehensively considering semantic authority, question matching degree, and syntactic structure importance, so that the model can more accurately select credible words and delete redundant words;

[0064] (4) The present application introduces syntactic structure integrity constraint, semantic correctness constraint and sentence coherence constraint to ensure the fluency of the generated answer sentence, and through sentence length constraint to ensure that the generated answer sentence is shorter;

[0065] (5) The present application proposes an exam reading comprehension answer generation method that integrates the pre-training model CPT and the optimization model, generates an answer sentence with high question matching degree, accurate information and complete structure, and solves the problem of poor summarization ability of existing models;

[0066] (6) The method proposed by the present application can also be applied to natural language processing tasks such as text summarization, recommendation system and information retrieval. In addition, the present application can be applied to an education examination guidance platform, responds to the call of national science and education, provides effective technical solutions for science and education, and stimulates students' interest in learning. BRIEF DESCRIPTION OF DRAWINGS

[0067] Figure 1 is the flow chart of the answer sentence summarization system in the present application;

[0068] Figure 2 is the specific flow chart of the answer sentence summarization method in the present application;

[0069] Figure 3 is the specific flow chart of training and fine-tuning CPT in the present application;

[0070] Figure 4 is the specific flow chart of optimizing the answer sentence using the optimization model in the present application. DETAILED DESCRIPTION

[0071] The technical solution of the present invention is specifically and detailedly described below in conjunction with the embodiments of the present invention and the accompanying drawings. It should be noted that, for those skilled in the art, several modifications and improvements can be made without departing from the principles of the present invention, and these should also be considered to fall within the scope of protection of the present invention.

[0072] A method for summarizing answer sentences for test reading comprehension by integrating CPT and optimization model

[0073] Step S1: Construct a summary corpus; specifically:

[0074] The corpus used is (1) real and simulated examination questions from various provinces over the years; (2) examination reading comprehension questions crawled from the examination website; (3) the Chinese single document summary evaluation task dataset of the NLPCC2017 conference. First, the sentences in the original text are divided into single sentences according to punctuation marks, and then the similarity between the summary sentence and each sentence in the original text is calculated according to the following formula. The sentence with the highest similarity in the original text is selected as the corresponding sentence of the summary sentence to construct the summary corpus.

[0075]

[0076] in, Represents the i-th summary sentence S in the abstract i The word collection, Represents the jth sentence P in the original text j The word set, w k Indicates words in a sentence;

[0077] Step S2: Use the summary corpus to train and fine-tune the CPT model; specifically:

[0078] Special tags "CLS" and "SEP" are added to the beginning and end of the source and target sentences in the summary corpus, respectively. The source sentence sequence is input into the shared encoder of the CPT model, and the target sentence sequence is input into the generative decoder. The CPT model is trained and fine-tuned to enable it to remove redundant information and summarize.

[0079] Step S3: Input the background material answer sentence into the CPT model trained and fine-tuned in step 2 for summarization; specifically:

[0080] The background material answer sentence S={w1,w2,...,w m} is input into the trained CPT model, m represents the number of words in the answer sentence of the background material, w i Represents the i-th word in the sentence, generates a more concise and authoritative answer sentence, that is, a summarized answer sentence T = {t1, t2, ..., t n};

[0081] Step S4, the credibility of the words in the background material answer sentence is calculated by integrating the semantic authority, the question matching degree and the syntactic structure importance; specifically:

[0082] Step S4.1, the semantic authority a of the words is calculated according to the text generated by the CPT model in step 3 i The text expressed by the CPT model is the summary of the original sentence, i.e., the main meaning of the background material answer sentence. If the words w m in the original sentence S = {w1, w2,..., w i} are in the text T = {t1, t2,..., t n} generated by the CPT model, the semantic authority a i of the word w i is set to 1, otherwise it is set to 0.

[0083] Step S4.2, the matching degree sim of the words and the question is calculated by combining Word2Vec and HowNet i The matching degree of the words and the question is conducive to recalling the words in the original sentence that are more relevant to the question. The calculation formula is as follows:

[0084]

[0085]

[0086] where Q * represents the word list of the question Q after removing stop words, q j represents the words in the list, represents the cosine similarity of the word vectors of w i and q j calculated by Word2Vec, and represent the word vectors of w i and q j , respectively, simHowNet(w i , q j ) represents the similarity of w i and q j calculated by HowNet, and β1 and β2 are positive parameters that need to be set, and β1 + β2 = 1.

[0087] Step S4.3, the syntactic structure importance str i of the words is calculated according to the depth of the words in the dependency syntactic analysis tree; the words closer to the root node of the tree are more likely to be the main components of the sentence, and the words farther away from the root node and deeper are more likely to have no impact on the meaning of the original sentence. The calculation formula is as follows:

[0088] str i=-dep i (4)

[0089] Among them, dep i Expressing word w i The depth in the dependency parse tree is defined as 0 for the root node, 1 for the direct child nodes of the root node, and so on.

[0090] Step S4.4, combining semantic authority α i , question matching sim i and syntactic structure importance str i Three factors to calculate word credibility i :

[0091] I i =λ1×α i +λ2×sim i +λ3×str i (5)

[0092] Among them, λ1, λ2 and λ3 are positive parameters that need to be set;

[0093] Step S5: Optimize the text output by the CPT model using the optimization model, regenerate a new answer sentence, and complete the summary of the answer sentence for the test reading comprehension; specifically:

[0094] Step S5.1: Construct an optimization model objective function based on word credibility to select authoritative information and redundant words to ensure the accuracy of the generated answer sentence information:

[0095]

[0096] Among them, m represents the number of words in the answer sentence of the background material, y i Represents the i-th word w in the sentence i Whether the label should be removed or kept if y i =1, word w i is retained if y i =0, word w i Deleted, I i Expressing word w i credibility;

[0097] Step S5.2: Introduce syntactic structure integrity, semantic correctness, and coherence constraints to ensure the fluency of the generated answer sentence. At the same time, introduce sentence length constraints to control the length of the generated answer sentence.

[0098] (1) Syntactic structure integrity constraints

[0099] In the optimized answer sentence, if a child node word is retained, then its parent node word should also be retained:

[0100]

[0101] Among them, w p It is w i The parent node word, y p and y i They are their corresponding labels respectively;

[0102] For some dependency relationships, such as the subject-predicate relationship, if the parent node word, that is, the predicate, is retained, then its child node word, that is, the subject, should also be retained:

[0103]

[0104] Among them, R represents a set of dependency relations in which the child node words are often retained if the parent node word is retained in the optimized answer sentence: R{SBV,VOB,IOB,FOB,POB,ATT,DBL,CMP,WP}, which is constructed according to the characteristics of the answer sentence summarization task of the test reading comprehension; R ip Represents the child node word w i Its parent node word w p dependencies;

[0105] (2) Semantic Correctness Constraints

[0106] When the child node word and the parent node word are in a parallel relationship, and the parent node word is not the core word of the sentence, if the parent node word is retained, the child node word should also be retained, otherwise the semantics of the answer sentence will be incomplete:

[0107]

[0108] Among them, R p Indicates w p The dependency relationship with its parent node word, COO means that the child node word and the parent node word are in a parallel relationship; HED means that the parent node word is not the core word of the sentence;

[0109] Negative words affect the semantics of the answer sentence. When the child node word and the parent node word are in an adverbial-predicate relationship, and the child node word contains a negative word, if the parent node word is retained, the child node word should also be retained:

[0110]

[0111] Among them, ADV indicates that the child node word and the parent node word are in an adverbial-verbial relationship, and no_adv indicates an element in the negation word list no_advlist {不,没,未,别,勿,否,莫,无};

[0112] (3) Coherence constraints

[0113] The parallel relation conjunctions "and, with, follow, and, as well as" can ensure the connectivity of the optimized answer sentence, when the child node word and the parent node word are left attached relation, and the child node word is a parallel relation conjunction, if the parent node word is reserved, the child node word should also be reserved:

[0114]

[0115] wherein, LAD represents that the child node word and the parent node word are left attached relation, coolest represents a set of parallel relation conjunctions: coolest{and, with, follow, and, as well as};

[0116] When the child node word and the parent node word are right attached relation, and the child node word is "of" or "of", if the parent node word is reserved, the child node word should also be reserved:

[0117]

[0118] wherein, RAD represents that the child node word and the parent node word are right attached relation;

[0119] (4) Sentence length constraint

[0120] Because efforts should be made to compress the sentence to ensure that the optimized answer sentence is shorter than the original sentence, but if the compression rate of the optimized answer sentence is too large, the main information of the original sentence will be lost, so the sentence length constraint needs to be introduced to control the maximum and minimum length of the optimized answer sentence:

[0121]

[0122] wherein, m represents the number of words in the original sentence, and ω needs to be set as a positive parameter, 0 < ω < 1;

[0123] Step S5.3, under the constraints of syntactic structure integrity, semantic correctness, coherence and sentence length, maximize the objective function, convert the word sequence S = {w1, w2,..., w m} of the background material answer sentence into the label sequence y = {y1, y2,..., y m}, y i ∈{0,1}; if y i = 1, it means that the word w i is reserved; if y i = 0, it means that the word w i is deleted, and finally generate the answer sentence A = {a1, a2,..., a k} which matches the question well, has complete syntactic structure and summarizes the main points of the answer, (k < m).

[0124] Example 1

[0125] The background material selected for this example is the 2019 Beijing College Entrance Examination real question:

[0126] Material One

[0127] ①With the continuous growth of the global population and the rapid development of science and technology, mankind has created civilization while also creating a global ecosystem deeply affected by mankind. For a long time, the overuse of biological resources and land has led to a series of problems such as loss of animal and plant habitats, environmental pollution, and serious damage to the ecological environment and biological system. According to experts, due to human activities and climate change, the number of species on Earth is currently disappearing at a rate of 1000 times the normal level, and about 3.4 million plants and more than 5200 animals are at risk of extinction. The distribution of species has undergone significant changes, which has formed a global biodiversity crisis.

[0128] ②The biodiversity crisis is the result of a combination of factors, with urbanization being one of the important factors. Urbanization is a historical process that accompanies industrialization and modernization, and in turn promotes industrialization and modernization. The level of urbanization is an important indicator of modern civilization. However, the uncontrolled spread of urban development has led to the shrinking of wild animal and plant habitats, and some animals and plants have to live with humans in the city.

[0129] ③About 60-70% of the land surface in the city is covered by roads, artificial buildings, parking lots, and other hardening, making it difficult for water to penetrate and for plant seeds to take root. The population density of many cities around the world has reached tens of thousands per square kilometer, and the dense flow of people is a potential danger to many organisms; in addition, there are a large number of cars, motorcycles, and other vehicles speeding. High-rise buildings are everywhere, and heat is constantly accumulating, with the temperature in the city center sometimes even 10°C higher than the surrounding area. This heat island effect is also a major disturbance factor for the survival of organisms.

[0130] ④Biodiversity has brought great wealth to human development, but it is currently facing threats from urbanization and other factors. The impact of urbanization on biodiversity has become a focus of concern for ecologists.

[0131] Material Two

[0132] ⑤In recent years, biologists have found that some wild species in the city have an incredible adaptability to the man-made environment that is still very young in the long history of the Earth.

[0133] Scientists are trying to reveal the evolution of species in cities. “To be honest, some species have adapted to this extreme constraint in a way that is simply amazing,” said Pierre-Olivier Cheptou, director of the Centre d’écologie fonctionnelle et évolutive, University of Montpellier, France. “There is not only microevolution, but in some cases I think there is also macroevolution,” said Maano Hiltunen, an evolutionary biologist at the Netherlands Centre for Biodiversity Naturalis. “The speed of evolution we are observing in cities is close to the theoretical maximum, which is mind-boggling.”

[0134] The United Nations expects the 21st century to be a period of rapid urbanization: by 2030, the global urban area will reach 14 million square kilometers; by 2050, the urban population will increase by 2.5 billion. As a product of human civilization, cities first appeared on the Mesopotamian plain 5,000 years ago. Today, urbanization will become one of the most important evolutionary forces for life on Earth.

[0135] Indeed, various examples of biological evolution in cities are emerging, from the sides of concrete roads, the feet of buildings, the edges of balconies, and the staircases of subway stations. Underground house mosquitoes that breed in the tunnels of urban subways no longer hibernate in winter, and they now clearly prefer the blood of mammals over birds. They have also developed clear reproductive isolation from other mosquito populations. In addition to wild and domestic species, new urban species have emerged in some places, prompting a reevaluation of urban ecological environments and biodiversity. Scientists have found that cities often have more species than rural areas after comparing cities with rural areas that have been severely damaged by intensive agriculture. Compared with rural areas, cities provide diverse habitats for species. Animals and plants can live in all kinds of corners, edges, and cracks, in wastelands, cemeteries, and damp ditches, or in well-maintained, tasteful gardens. They can obtain water and food all year round, unlike in the wild, where water and food are seasonal. The urban heat island effect is also an advantage for urban habitats, as winters are no longer as cold.

[0136] In the era of intensive agriculture, cities will become new ecosystems that are conducive to the protection of biodiversity, and this potential will continue to be tapped.

[0137] Material Three

[0138] Studies have found that every time a species successfully adapts to urban life, multiple species disappear from the area; and if a species adapts to urban life too quickly, it means that many individuals have to make sacrifices. The rapid evolution of biology triggered by urbanization often comes at a cost.

[0139] Cities such as Toronto and Boston have white clover that has given up the ability to release cyanide to improve cold tolerance. Cyanide release can deter herbivores, but cold tolerance is reduced. In the city center, urban heat makes snow melt quickly, and without the cover of snow, plants are vulnerable to freezing at night. A new study shows that human activities, including hiking, are driving mammals around the world to become more active at night, showing a trend of increased nocturnality. Nocturnality enhancement will have a series of negative effects, including changes in habits and reduced reproductive capacity. Studies of white-footed mice in New York City parks found that urban white-footed mice overexpressed genes related to fatty acid digestion compared to rural white-footed mice. This evolutionary selection is likely related to the ease of eating human discarded oil paper, leftover pizza and cheese burgers in the city. Since 1940, the skull volume of Italian urban area bats has been increasing, which may be affected by streetlights. Streetlights attract and gather large numbers of large insects, and with the replacement of generations, bats with strong biting force become more dominant.

[0140] Urban biological evolution is closely related to biodiversity, and biological evolution is a difficult to control and low predictable issue. Assistant Professor Mark Johnson of the University of Toronto, Canada, emphasized that "we have observed that some species have shown convergent evolution in most cities around the world. In some cities, species have failed to adapt successfully, and the reasons are not yet known."

[0141] Question: What are the views of the above three materials on the relationship between urbanization and biodiversity?

[0142] Standard answer: (1) The first material, biodiversity is threatened by urbanization. (2) The second material, cities are conducive to protecting biodiversity. (3) The third material, the rapid evolution of urbanization has to pay a price.

[0143] The answer sentences corresponding to the standard answer in the background material are: (1) Biodiversity has brought great wealth to human development, but it is currently threatened by urbanization and other factors. (2) In the era of intensive agriculture, cities will become new ecosystems, which will help protect biodiversity and this potential will continue to be stimulated. (3) The rapid evolution triggered by urbanization often has to pay a price.

[0144] Step S1, build a summary corpus:

[0145] The corpus (1) includes the examination question and answer true questions and simulation questions of each province in the past years; (2) the examination reading comprehension question and answer questions crawled from the test paper setting network; (3) the Chinese single document summary evaluation task data set of NLPCC2017 conference. First, the sentences in the original text are divided into single sentences according to the punctuation marks, and then the similarity between the summary sentence and each sentence in the original text is calculated according to formula (1), the sentence with the highest similarity in the original text is selected as the corresponding sentence of the summary sentence, and a data set in the form of "original sentence - summary sentence" is formed.

[0146] Step S2, training and fine-tuning the CPT model using the summarization corpus:

[0147] The special marks "CLS" and "SEP" are added to the head and tail of the source sentence and the target sentence in the summarization corpus, respectively. The source sentence sequence is input into the shared encoder of the CPT model, and the target sentence sequence is input into the generation decoder. The CPT model is trained and fine-tuned so that the CPT model has the ability to remove redundant information and summarize. The fine-tuning mode of CPT is shown in the accompanying Figure 3 The left part is the shared encoder of the CPT model, and the right part is the generation decoder of the CPT model; w1w2...w m is the source sentence sequence, and t1t2...t n is the target sentence sequence.

[0148] Step S3, input the background material answer sentence into the CPT model for summarization:

[0149] The background material answer sentence S={w1, w2,..., w m} is input into the trained CPT model to generate an answer sentence T={t1, t2,..., t n} with important information. For example, the background material answer sentences "Biodiversity brings great wealth to human development, but it is currently facing threats from urbanization and other aspects." "In the era of intensive agriculture, cities will become a new ecosystem, which is conducive to the protection of biodiversity, and this potential will continue to be stimulated." and "The rapid evolution of biology caused by urbanization often comes at a cost." are input into the trained CPT model, and the summarized answer sentences are shown in Table 1.

[0150] Table 1

[0151]

[0152] Step S4, calculate the credibility of the words in the background material answer sentence by integrating the semantic authority, question matching degree and syntactic structure importance:

[0153] Take the first sentence of the background material answer in the 2019 Beijing exam as an example: "Biodiversity has brought great wealth to human development, but it is now facing threats from urbanization and other aspects." The segmented word sequence of the sentence is: S = {biological, diversity, for, human, development, bring, have, great, wealth,,, now, it, but, face, from, urbanization, and, other, aspects, of, threat,}.

[0154] Step S4.1. Calculate the semantic authority of the word a according to the text generated by the CPT model i The text generated by the CPT model expresses the main meaning of the original sentence, so if the word w m in the original sentence S = {w1, w2,..., w i N} is in the text T = {t1, t2,..., t n M} generated by the CPT model, set the semantic authority a i of the word w i to 1, otherwise set it to 0. The text generated by the CPT model is T = {face, urbanization, and, aspects, of, threat,.}, and the words "face", "urbanization", "and", "aspects", "of", "threat", "." in the background material answer sentence S have a semantic authority of 1, and the semantic authority of the remaining words is set to 0. Therefore, the semantic authority of the words in the background material answer sentence S is respectively: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 1, 1, 1, 1, 1, 1].

[0155] Step S4.2. Calculate the matching degree sim of the word and the question according to Word2Vec and HowNet i The word sequence of the question after removing stop words is Q * = {urbanization, biological, diversity, relationship, expression, point of view}. According to formulas (2)-(3), the matching degrees of the words in the background material answer sentence S and the question are respectively: [1.0, 1.0, 0.232, 0.61, 0.467, 0.36, 0.287, 0.356, 0.367, 0.15, 0.266, 0.319, 0.327, 0.306, 0.25, 0.268, 1.0, 0.224, 0.385, 0.318, 0.343, 0.15].

[0156] Step S4.3. Calculate the syntactic structure importance str of the word according to the depth of the word in the dependency syntax analysis tree iThe depths of the words in the dependency parse tree of the background material answer sentence S are [3, 2, 2, 4, 3, 1, 2, 3, 2, 2, 3, 3, 3, 2, 3, 4, 6, 7, 5, 6, 3, 2] respectively. The syntactic structure importance of the words are calculated according to formula (4) as [-3, -2, -2, -4, -3, -1, -2, -3, -2, -2, -3, -3, -3, -2, -3, -4, -6, -7, -5, -6, -3, -2] respectively.

[0157] Step S4.4, combining the semantic authority degree a i , the question matching degree sim i and the syntactic structure importance str i to calculate the credibility I of the words i . The credibility of the words in the background material answer sentence S are calculated according to formula (5) as [0.25, 0.5, -0.268, -0.39, -0.283, 0.11, -0.213, -0.394, -0.133, -0.35, -0.484, -0.431, -0.423, 0.806, -0.5, -0.732, 0.5, -0.526, 0.135, -0.182, 0.593, 0.65] respectively.

[0158] Step S5, optimizing the text output by the CPT model using an optimization model to generate a new answer sentence, as shown in the following figure: Figure 4

[0159] Step S5.1, constructing an optimization model objective function by the credibility of the words, which is used to identify authoritative information and delete redundant words to ensure the correctness of the generated answer sentence information, as shown in the following formula:

[0160]

[0161] wherein m represents the number of words in the background material answer sentence. y i represents the label of whether the ith word w i in the sentence should be deleted or retained. If y i = 1, the word w i is retained, and if y i = 0, the word w i is deleted. I i represents the credibility of the word w i .

[0162] ​Step S5.2, introduce the syntactic structure integrity, semantic correctness and coherence constraints to guarantee the fluency of the generated answer sentence, and introduce the sentence length constraint to control the length of the generated answer sentence. For example, the dependency syntactic analysis result of "Biodiversity brings great wealth to human development, but it is currently facing threats from urbanization and other aspects." is: [(1, 2, 'ATT'), (2, 6, 'SBV'), (3, 6, 'ADV'), (4, 5, 'SBV'), (5, 3, 'POB'), (6, 0, 'HED'), (7, 6, 'RAD'), (8, 9, 'ATT'), (9, 6, 'VOB'), (10, 6, 'WP'), (11, 14, 'ADV'), (12, 14, 'SBV'), (13, 14, 'ADV'), (14, 6, 'COO'), (15, 14, 'RAD'), (16, 21, 'ATT'), (17, 19, 'ATT'), (18, 17, 'RAD'), (19, 16, 'POB'), (20, 19, 'RAD'), (21, 14, 'VOB'), (22, 6, 'WP')]. The sentence length is 22. The deletion and retention of words are limited by the syntactic structure integrity, semantic correctness and coherence constraints, and the minimum length and maximum length of the optimized answer sentence are limited by the sentence length constraint.

[0163] Step S5.3, maximize the objective function under the constraints of syntactic structure integrity, semantic correctness, coherence and sentence length, and convert the word sequence S = {w1, w2,..., w m} of the background material answer sentence into a label sequence y = {y1, y2,..., y m}, y i ∈{0, 1}. If y i = 1, it means that the word w i is retained; if y i = 0, it means that the word w i is deleted, and finally generate an answer sentence A = {a1, a2,..., a k}, (k < m) that matches the question well, has a complete syntactic structure and summarizes the key points of the answer.

[0164] For example, the word sequence S = {biological, diversity, is, human, development, brought, has, great, wealth,, at present, it, but, faces, from, urbanization, and other, aspects, of, threat,.} of the sentence "Biodiversity brings great wealth to human development, and it is currently facing threats from urbanization and other aspects." is transformed into the label sequence y = {1, 1, 0, 0, 0, 1, 1, 1, 1, 1, 0, 1, 0, 1, 0, 1, 1, 0, 1, 1, 1, 1}, and finally a new answer sentence is generated: Biodiversity brings great wealth, and it faces threats from urbanization and other aspects.

[0165] Similarly, all the background material answer sentences are processed according to the above steps, and the optimized answer results are shown in Table 2 as follows:

[0166] Table 2

[0167]

[0168] The contents not described in detail in the specification of the present application belong to the prior art known to those skilled in the art. Although the above describes the specific embodiments of the present application for the purpose of facilitating the understanding of the present application by those skilled in the art, it should be clear that the present application is not limited to the scope of the specific embodiments, and for those skilled in the art, it is obvious that all the inventions utilizing the concept of the present application within the spirit and scope of the present application defined and determined by the appended claims are included.

Claims

1. A method for summarizing test reading comprehension answer sentences that integrates CPT and optimization models, characterized by: It includes the following steps: Step S1, constructing a general summary corpus; Step S2, training and fine-tuning the CPT model using the general summary corpus; Step S3, inputting the background material answer sentences into the CPT model trained and fine-tuned in Step 2 for general summarization; Step S4, comprehensively calculating the credibility of words in the background material answer sentences by semantic authority, question matching degree, and syntactic structure importance; Step S5, optimizing the text output by the CPT model using the optimization model, regenerating new answer sentences, and completing the summarization of the reading comprehension answer sentences in the exam; The specific steps for comprehensively calculating the credibility of words in the background material answer sentences by semantic authority, question matching degree, and syntactic structure importance in Step S4 include the following steps: Step S4.1: Calculate the semantic authority of the word based on the text generated by the CPT model in step 3 , the summary answer sentence text generated by the CPT model expresses the main meaning of the original sentence, that is, the background material answer sentence. If the original sentence Words in Text generated by the CPT model In Semantic authority Set to 1, otherwise set to 0; Step S4.2: Calculate the matching degree between words and questions by combining Word2Vec and HowNet , the matching degree between words and questions is conducive to recalling words in the original sentence that are more relevant to the question. The calculation formula is as follows: (2) (3) in Question The word list after removing stop words, Represents the words in the list, Indicates the use of Word2Vec calculation and The cosine similarity of the word vectors, and Respectively and The word vector of Indicates calculation by HowNet and The similarity of and is a positive parameter that needs to be set. ; Step S4.3, calculate the syntactic structure importance of the word according to the depth of the word in the dependency syntax analysis tree The closer the words are to the root node, the greater the probability that they will become the main component of the sentence. The farther away from the root node and the deeper the words are, the more likely they are to have no effect on the meaning of the original sentence. The calculation formula is as follows: (4) in, Expressive words The depth in the dependency parse tree is defined as 0 for the root node, 1 for the direct child nodes of the root node, and so on. Step S4.4, combining semantic authority , question matching and syntactic structure importance Three factors to calculate word credibility : (5) in 、 and is a positive parameter that needs to be set; The steps for optimizing the text output by the CPT model using the optimization model and regenerating new answer sentences in Step S5 include the following steps: Step S5.1, constructing an optimization model objective function through word credibility for the selection of authoritative information and redundant words to ensure the correctness of the information in the generated answer sentences: (6) in, Indicates the number of words in the answer sentence of the background material, Indicates the first words Whether the tag should be removed or retained if , words is retained if , words was deleted, Expressive words credibility; Step S5.2, introducing syntactic structure integrity, semantic correctness, and coherence constraints to ensure the smoothness of the generated answer sentences, and at the same time introducing sentence length constraints to control the length of the generated answer sentences; (1) Syntactic structure integrity constraint In the optimized answer sentence, if the child node word is retained, then its parent node word should also be retained: (7) in, yes The parent node word, and They are their corresponding labels respectively; For certain dependency relationships, such as the subject-predicate relationship, if the parent node word, that is, the predicate, is retained, then its child node word, that is, the subject, should also be retained: (8) in, Indicates a set of dependencies in which the child node words are often retained if the parent node word is retained in the optimized answer sentence: {SBV, VOB, IOB, FOB, POB, ATT, DBL, CMP, WP}, constructed based on the characteristics of the answer sentence summarization task in the reading comprehension test; Indicates child node words Its parent node word dependencies; (2) Semantic correctness constraint When the child node word and the parent node word are in a parallel relationship, and the parent node word is not the core word of the sentence, if the parent node word is retained, then the child node word should also be retained, otherwise the semantics expressed by the answer sentence is incomplete: (9) in, express The dependency relationship with its parent node word, COO means that the child node word and the parent node word are in a parallel relationship; HED means that the parent node word is not the core word of the sentence; Negative words affect the semantics of the answer sentence. When the child node word and the parent node word are in an adverbial-head relationship, and the child node word contains a negative word, if the parent node word is retained, then the child node word should also be retained: (10) Among them, ADV indicates that the child node word and the parent node word are in an adverbial-medial relationship. Represents the elements in the negation word list no_advlist{not, not, not, bye, don’t, no, not, no}; (3) Coherence constraint The coordinating conjunctions "and, with, together with, as well as, and" can ensure the connectivity of the optimized answer sentence. When the child node word and the parent node word are in a left-adjunct relationship, and the child node word is a coordinating conjunction, if the parent node word is retained, then the child node word should also be retained: (11) Among them, LAD means that the child node word and the parent node word are in a left-attached relationship. Conjunctions expressing a group of parallel relations: coolist {and, with, with, and, as well as}; When the child node word and the parent node word are in a right-adjunct relationship, and the child node word is "de" or "le", if the parent node word is retained, then the child node word should also be retained: (12) Among them, RAD represents that the child node word and the parent node word are in a right-adjunct relationship; (4) Sentence length constraint Because we need to try our best to compress the sentence to ensure that the optimized answer sentence is shorter than the original sentence, but if the compression rate of the optimized answer sentence is too large, the main information of the original sentence will be lost. Therefore, it is necessary to introduce sentence length constraints to control the maximum and minimum lengths of the optimized answer sentence: (13) in, Indicates the number of words in the original sentence. and Positive parameters that need to be set, , ; Step S5.3, under the constraints of syntactic structure integrity, semantic correctness, coherence and sentence length, maximize the objective function and convert the word sequence of the background material answer sentence into Convert to label sequence ;if , indicating words is retained; if , indicating words is deleted, and finally an answer sentence is generated that matches the question well, has a complete syntactic structure, and summarizes the key points of the answer. .

2. The method for summarizing test reading comprehension answer sentences by integrating CPT and optimization model according to claim 1, characterized in that: The step 1 is to construct the summary corpus as follows: using the corpus (1) the actual examination questions and simulation questions of each province in previous years; (2) the examination reading comprehension questions crawled from the examination website; (3) the Chinese single document summary evaluation task dataset of the NLPCC2017 conference; first, the sentences in the original text are divided into single sentences according to the punctuation marks, and then the similarity between the summary sentence and each sentence in the original text is calculated according to the following formula, and the sentence with the highest similarity in the original text is selected as the corresponding sentence of the summary sentence to construct the summary corpus; (1) in, Indicates the first Summary Sentence The word collection, Indicates the original text Sentences The word collection, Indicates words in a sentence.

3. The method for summarizing test reading comprehension answers that integrates CPT and optimization model according to claim 2, characterized in that: In step S2, the CPT model is trained and fine-tuned using the summary corpus as follows: special tags "CLS" and "SEP" are added to the beginning and end of the source and target sentences in the summary corpus, respectively; the source sentence sequence is input into the shared encoder of the CPT model, and the target sentence sequence is input into the generative decoder; the CPT model is trained and fine-tuned, so that the CPT model has the ability to remove redundant information and summarize.

4. The method for summarizing test reading comprehension answer sentences by integrating CPT and optimization model according to claim 3, characterized in that: In step S3, the background material answer sentence is input into the CPT model trained and fine-tuned in step S2 for summarization. Specifically, the background material answer sentence is input into the CPT model trained and fine-tuned in step S2 for summarization. Input into the trained CPT model, Indicates the number of words in the answer sentence of the background material, Indicates the first words, generate a more concise and authoritative answer sentence, that is, a summarized answer sentence .

Images